Project preparation
Project organisation
Project methods
Project content
Project impact
Project sustainability
4. The framework used by the project team to investigate the cost effectiveness and sustainability of technical and vocational education projects was derived from an initial literature survey, which sought to identify some factors leading to project success and failure. A number of these documents are themselves overviews, reviews and synopses of reports on vocational and technical education projects. The first part of this Report, therefore, attempts to summarise key issues raised from the analysis of a very wide range of documents, drawn substantially from the major donor agencies. Inevitably, in a survey on this scale, conflicting findings and opinions were encountered, and attention has been drawn to these where appropriate. The analysis is structured to examine five aspects of project management, which are reported on in turn:
- project preparation;
- project organisation;
- project methods;
- project content;
- project impact;
- project sustainability.
5. Underpinning the analysis is the problem that no single criterion or set of criteria of cost-effectiveness can be identified. The evidence emphasises the complexity of training provision. Although there is general agreement that the effectiveness of such training should be measured in terms of impact on the labour market, there is no such accord as to the measurement instruments which should be used. The complexity, variety and dynamism of both training provision and labour markets, and the importance of contextual social, cultural and political factors, make comparisons particularly difficult (King, 1988). Crude effectiveness measures such as the numbers of trainees obtaining employment and wage increases after training give few insights into the 'value-added' effects of the training itself - even where this basic information is available. Nor is there any consensus as to modes of training which are more or less efficient than others. While the importance of appropriate materials and equipment for technical and vocational training is of course generally acknowledged, access to, uses of and organisation for the practical elements of such training are all matters of dispute. Furthermore, despite the numbers of documents scanned, few detailed references were discovered to this important aspect of vocational and technical education in the initial literature survey. In consequence, these issues are examined through the field investigations reported on in Part B.
6. Policy papers and evaluation reports (Rosenthal et al, 1986; Hultin, 1987, Higginbottom, 1990) point to the significance of a number of pre-conditions necessary (but not sufficient) for the success of technical and vocational education programmes. These include macro-economic, social and political factors shaping the national or regional context in which a project is planned, together with the detailed technical, organisational and resource issues related to the project itself. This section examines four such pre-conditions:
- a sound basis in general education;
- firm links between training organisations and the labour market;
- needs analyses which involve customers as well as providers; and
- appropriate selection of trainees, staff and locations.
7. The significance of the educational base upon which technical or vocational programmes are built cannot be under-estimated. It determines the educational entry levels of those to whom the programme is directed, as shaped by the regional or national patterns of educational provision (Middleton & Demsky, 1989). This sound educational base is commonly identified as a necessary pre-condition for successful vocational training projects (Rosenthal et al, 1986), and is further defined as universal primary education with at least 50% take-up of secondary education (Hultin, 1987). The World Bank has recently emphasised strongly the importance of a strong basis of literacy, numeracy and problem-solving skills as a foundation for effective technical and vocational education (World Bank, 1991). It points to the importance of a sound - and completed - primary and secondary education, which focuses upon basic skills and knowledge developed at the primary level and the broad competencies developed through an academic secondary education.
8. However, there is no consensus when attempting to define the nature and structure of such provision. In particular, although the benefits of including both academic and vocational elements within a diversified secondary curriculum are emphasised by Hultin and others. The World Bank (1991) considers that diversified curricula and pre-vocational programmes at primary and secondary level dilute and reduce the effectiveness of this educational base. Production work in schools is seen as a means to improved teaching in Hultin's (1987) review for SIDA, but other evidence suggests that skills gained within a general education are lost if not practised immediately. Diversified schools are criticised particularly when found alongside specialised skill training centres, supported by private enterprise or by technical government ministries (e.g. labour, agriculture, industry), although Hultin pointed to the efficiencies obtained by parallel provision by Education and Labour Ministries. Some studies have found that educational performance in technical/vocational schools has exceeded that in general academic schools (Lockheed & Hanushek, 1987). There is more general agreement that the pre-vocational curriculum should include 'transition' elements which relate the general curriculum more closely to the world of work, and provide experience of working life, rather than trying to provide the specialist training found in the skill training centres (Coombe, 1988; World Bank, 1988 & 1991). A first question is, therefore, whether a programme investment of a given size might more usefully be devoted to the improvement of primary or general secondary education rather than to technical/vocational education. A second question is whether it is more cost-effective to invest in the development of technical and vocational elements within mainstream educational curricula rather than invest in the development of a discrete technical/vocational training sector with its separate institutions. These issues are returned to in Part B (paragraphs 33-35).
9. A second pre-condition concerns the economic environment within which technical and vocational education takes place. Where labour markets and unemployment levels are such as to make it difficult for the 'graduates' of technical/vocational education programmes to find work, it is unrealistic to expect that the skills developed will enable those graduates to achieve self-employment. Self-employment tends to be either low-skilled, or requiring the broader competencies acquired through general education. Governmental attempts to regulate or even predict their economy's manpower requirements have been largely unsuccessful (Carron, 1984; Psacharopoulos & Woodhall, 1985; Demsky, 1989). The notion that training should be geared to manpower requirement forecasts is now largely discredited (Hollister, 1983; Dougherty, 1989). Links between technical/vocational education programmes and the employers - especially private sector employers - likely to take advantage of the programme outputs are commonly weak, and it has been demonstrated that labour markets adjust to imbalances through capital-labour and task substitution rather than through training policies (Richter, 1986). Some sectors, particularly in agriculture and other rural industries, do not have an infra-structure to support training, while the weaknesses of private sector training (often reinforced by unnecessarily tight regulation) have meant that technical/vocational programmes have been concentrated in public sector institutions, whose growth is related more to short term political advantages rather than clearly identified economic or social needs. A further pre-condition for cost-effective vocational and technical education is, therefore, the existence of working links between training institutions (public and private sector) and sources of consequent employment. World Bank studies (Middleton & Demsky, 1989) have contrasted the achievements of middle-income developing countries, which have developed appropriate infra-structures, with the failures of low-income countries to set up appropriate vocational training. King (1988) however, points to the dangers of technological determination, which restricts the role of technical and vocational training in the industrially weaker developing countries.
10. Evidence points to some specific needs in designing and planning technical/vocational programmes. One is for the selection of programme objectives from an initial analysis of possible objectives. These need to be undertaken in such a way that the providing organisations and the potential beneficiaries appreciate both the selected objectives and those possible objectives excluded in this instance. In selecting objectives, initial needs analyses are required, involving simple data collection procedures which reach out to both the consumers (potential graduates etc.) and customers/clients (potential employers) as well as the government officials and institutional heads most likely to be involved. These have been criticised where they focus upon individual rather than organisational needs, (Akin-Ogundeji, 1987). This needs to be followed by an equally careful examination of the range of alternative ways by which agreed objectives might be achieved. This could well require some experimentation through pilot programmes, and Sahara (1991) has criticised aid agencies for omitting this crucial step. A third pre-condition is, therefore, the need for significant investigation and analysis of purpose and need, extending beyond the traditional governmental, political and institutional contacts to communities, employers and consumers.
11. A further pre-condition concerns the selection of those to be involved in the project. Timescales, as well as objectives, need to be realistic. The involvement of local staff in project design can identify potential blockages and the impact of local and national cultural influences on project timescales and objectives. The selection of external consultant support needs to give weight not only to expertise but also to commitment over a period of time. Investment in the selection, preparatory briefing and thorough induction of external consultants helps to alleviate 'culture shock' problems and build commitment. A particular problem has been the failure of training needs analyses to take note of the training needs of women, notably in agriculture and commercial areas (Dougherty, 1989; Demsky, 1989). The initial selection and involvement of counterpart staff with sufficient status to effect project change when necessary is also important. A further organisational need is the careful selection, not only of personnel but also of appropriate training sites (Lackey, 1981). Assumptions of achievement through existing public sector institutions, whether at home or overseas, need to be challenged. Investment in project staff selection, induction and development - including team-building - helps to reduce wastage later in the programme.
12. Higginbottom (1990a) points to the need for clear definition of objectives and recognition of social as well as economic issues. A more fundamental criticism of project preparation procedures comes from Sahara's (1991) comparison of donor cooperation and delivery styles. The limitations of approaches with tight project specifications according to donor-initiated blueprints are contrasted with the 'hands-off' approaches of agencies (such as SIDA), and the 'collaboration' approaches (typified by the Japan International Co-operation Agency) by which recipients request help and donors then examine their capacity to respond. Project preparation in the latter two styles is marked by the involvement of both donor and recipient in the target-setting processes. Sahara points to the value of small-scale initial interventions, learning about recipients' perceptions of problems, collecting information through in-depth surveys, and conducting small-scale investigation projects, in order to identify the most potentially successful areas of cooperation.
13. The involvement of local industry is generally seen as a central requirement for cost-effective technical/vocational education. Traditional apprenticeship systems have long been highly cost-effective means for transmitting skills (ILO, 1972; Fluitman, 1989). Programmes can take advantage of these, where they still exist, but must take care not to distort and thus destroy what is a low cost but fragile form of technical/vocational training (Bas, 1988). The more effective utilisation of existing employer training schemes can be stimulated by involving these in development projects, supporting them through technical assistance and some subsidy, and by encouraging the removal of artificial barriers designed to protect public sector training institutions. Cost reductions can be achieved when making use of skilled staff, materials and equipment as well as possible financial support from local industry. This usually requires effort to ensure that such industry recognises the consequential benefits of better skilled existing and future workers. This needs close and well-structured links between employers and institutions. Local advisory committees are identified as highly effective means for building these bridges. Technical/vocational education needs to have close links with local industries and employers and make full use of the cost economies possible in working with them, for example in using their equipment and staff.
14. The supply of equipment should complement rather than duplicate experiences which can be acquired through work-based learning. Obvious but important needs include an emphasis on simple, robust equipment for which spare parts and consumable materials can readily be obtained locally and, just as important, funding regimes which enable the regular purchase of these. There are a number of references to the wastefulness of large, elaborate and complex equipment where maintenance and repair demands resources beyond those of the recipient organisation (Cracknell et al, 1981). However, there is also criticism of policies which provide out-of-date equipment, which might be simple and robust but for which spare parts are no longer available. Centralised workshops have been identified as convenient means for maintaining effective supply lines, and a resource of competent maintenance staff (Austin, Mackintosh & Scott, 1987). But they have also been described as wasteful, inaccessible from training institutions, and governed by impenetrable bureaucracies. Further problems occur when equipment is received at a training centre which has neither the requisite installation nor maintenance skills. A maintenance culture (examined in more detail in paragraphs 74-76) may require its own training objectives, but these will promote project sustainability. Such a culture might incorporate systems for recycling materials, mobile service teams to repair broken equipment, and the encouragement of local entrepreneurs to fashion tools and spare parts. The establishment of a national or regional equipment manufacturing centre takes the process of centralised provision one stage further.
15. Evidence of premises provision points to the adequacy or otherwise of physical design, rather than providing specific cost reduction pointers. Adequacy measures include storage space, noise reduction, air pollution control, energy consumption and weather protection. Reports point to more and less effective forms of provision, without specifying the characteristics of either - except to point to situations where the technical advice of the donor agency has or has not been followed. The main cost reduction lessons come from the more intensive use of facilities by using the extended day, the extended year and double-shifts. The use of self-help methods in constructing workshops with community support, and the development of simple, cheap and locally designed facilities is noted.
16. Much criticism points to the inflexibility and rigidity of many externally-funded projects. The ability to change plans and vire funds as local needs or national economic or political requirements change is important, but this requires built-in monitoring systems to spot these changes, and also to identify unanticipated project consequences including social and environmental changes. The need has also been identified for flexible and responsive training systems, which can change as needs change; and in turn decentralised decision-making and close links with local industries seem necessary to ensure such responsiveness (World Bank, 1988; Dougherty, 1989).
17. This flexibility needs to extend to timescales and funding regimes. Resources are wasted when they are consumed hastily because of the impending end of a financial year, whether that of the donor or recipient organisation. Flexibility extends still further to include the use of staff. External consultant support can be very expensive, and, in order to be cost-effective, needs to be able to adapt to the changing needs of a programme and its environment. This points to, as indicated above, the long-term commitment to a programme and the need for multi-skilled consultant support which, once on site, can provide the support which is most appropriate at the time rather than that which was anticipated when the consultancy was first mooted. The training of local trainers can be a key element in reducing costs, particularly where they are then able to cascade skills. There are dangers, however, in becoming over-dependent both on external consultants and on their counterparts, as the project can then rely too heavily on key individuals (Higginbottom, 1990b).
18. The establishment and maintenance of effective planning, control and monitoring systems is an obvious but crucial aspect of project organisation. Lessons include the need to ensure the timescales of equipment and training provision coincide, that effective financial management systems are established, and that communication networks and information flows are not obstructed. The importance of consistent, well-organised donor supervision and monitoring is emphasised (Demsky, 1989; Harris, 1990). Performance monitoring and performance auditing are now being used to review the supply of equipment and materials and to test investment in these against programme objectives.
19. As already emphasised, the evidence points firmly towards closer integration of training and industry, rather than heavy investment in facilities which duplicate or simulate industrial experience. This requires a central focus on 'learning by doing'. This in turn requires careful organisation and effective supervision. The availability of placements for work-based learning and experience requires significant investment in the selection and development of industrial liaison/work placement specialists, whose tasks include explaining to local managers and supervisors the range of experiences sought through work placement and the benefits to the placement providers. Effectiveness is also enhanced by bringing in industrial managers to contribute to institution-based education and training. Barriers to the use of such staff in public sector institutions need to be overcome. This can require examination of timetabling systems as well as employment and payment systems.
20. Cultural issues need to be addressed, so that the off-job training experiences reinforce traditional hierarchical organisational structures and their managers, rather than attempting to challenge and criticise these. The emphasis needs to be on a partnership of training institutions and work places rather than on competing value systems.
21. In more detail, learning-by-doing strategies can most effectively be achieved by project approaches using small groups and team-work. Akin-Ogundeji (1987) has contrasted Western concerns with appropriate methodologies and their development with African training programmes focusing solely upon individual effectiveness. These need to be supported by appropriate educational technologies. In spite of supply and maintenance difficulties, the use of video and computer technology can substitute for the lack of skilled instructors, and/or replace some trainer costs to reduce the overall cost of training. It is important that these educational technologies are in tune with local consumer and work technologies, to facilitate maintenance and the provision of spare parts.
22. Curriculum frameworks which are based on the development of defined competencies, and are structured in modular formats, reduce the costs of technical and vocational education by:
- reducing the opportunity costs to trainees by providing short, intensive training modules which minimise absence from work;- reducing the volume of training received, by enabling trainees to undertake only those modules for which they have a defined training need;
- structuring the modular provision to maximise class sizes, relating the frequency of provision to demand; and
- encompassing forms of open learning and individualised instruction which can complement, and for some students replace, forms of traditional learning. Such approaches also relate more clearly to the outcomes than traditional technical/vocational courses, in terms of new competencies and their relevance to employment, thereby increasing effectiveness by increasing motivation.
23. The high cost of teachers in a labour intensive industry points to the need for cost reduction strategies by increasing teacher productivity. This is being achieved by requiring teachers to teach for longer hours per week, to teach larger classes, and, by operating double shifts, to make more effective utilisation of expensive premises. The rigidities of public institutions in these areas has encouraged a shift to private institutions, in order to reduce costs (Coombe, 1988). However, there is little evidence as yet of the impact of such changes on the quality of education (King, 1988).
24. The effectiveness of out-of-country training and education is questioned, not so much because of the irrelevance of such training, but because of the consequent success of graduates in moving either into more senior posts where the technical and vocational skills acquired through their training are no longer needed, or into a sector where their technical skills are lost to the project. The cost advantages of sending trainers to the host country rather than trainees overseas are spelled out by Baker et al (1984). The demand for out-of-country qualifications, particularly at Masters level, remains high. Legitimate expectations here can be met by making use of the more flexible routes to such qualifications, which incorporate in-country training, either on a franchise basis or through satellite campuses; and through development of individualised programmes and fellowships which lead to a portfolio against which the qualification is assessed. Such fellowships can benefit from short, sharply focused periods of attachment to carefully selected out-of-country industries as well as training institutions. More attention needs to be given to 'cascade' strategies and 'multiplier' or 'ripple' effects, whereby short overseas training experiences focus on helping the trainee to lead training activities on return. The need for a careful evaluation of the impact of overseas-based training compared with local provision is a high priority (Moock, 1984).
25. There is growing international recognition that output measures are more effective indicators of the performance of technical/vocational education than input measures. The importance of job placement at the end of a training programme is being demonstrated by new funding mechanisms which reward training institutions on the basis of the proportion of successful job placements. Although this is an obvious (though crude) efficiency measure, training institutions commonly have little commitment to job placement and even less responsibility for trainees in their professional development after the training period is over. Investment in career guidance and job placement will not reduce the costs of technical/vocational education. It is likely, however, to increase the efficiency of such education and training, particularly when one efficiency measure is the unit training cost per successful job placement.
26. An image of project success is important in providing its graduates with the confidence to make full use of their learning. This requires immediate evidence of beneficial results and changes. These are most immediately obvious in the employment success of graduates, and are used for investment in job placement as an integral part of a project. This in turn argues for the need to explain to local employers and their managers the skills which graduates will be able to bring - and the cost savings that those skills are likely to lead to.
27. Tracer studies and other short-term forms of manpower analysis are more useful measures of project success than rate of return calculations or measures of the extent to which manpower forecasts are fulfilled. As indicated earlier, there is general dissatisfaction with traditional approaches to manpower forecasting (Psacharopoulos & Woodhall, 1985; Demsky, 1989); and the need for developing more sophisticated techniques for manpower analysis is strongly emphasised.
28. The establishment of industry/education links, as with an Industrial Liaison Unit, has benefits beyond those of the initial intentions (Skelton and Faulkner, 1990). More generally the importance of incorporating formal mechanisms for beneficiary assessment is now being emphasised (Salmen, 1990, discussed below in paragraph 57). This is particularly important in view of the unanticipated benefits and costs not picked up through project objectives (Higginbottom, 1990).
29. In the longer term, the effectiveness of technical/vocational education projects depends in part upon the retention in employment of the trained staff, and their ability to contribute to their employers' success; and in part on the capacity of the workshops and training centres to maintain equipment with spares and consumable items well after the completion of the project, and the development of the technical support to maintain and repair equipment. These internal and external measures of sustainability come together with the growing inter-connectivity of teaching institutions and employers, so that responsibility for continuing the financial, material and expertise support initially derived through the project, is provided in the longer term by the local employers. This argues for the reduction of barriers between private and public sector institutions, and for the development of 'sustainability indicators' (World Bank, 1989).
30. The economic measures of sustainability need to be complemented by a 'social audit'. This would identify the consequences of project intervention in areas not readily definable in financial terms. The impact of potential cost reduction measures needs then to be judged not only economically but in terms of social consequences. The 'fit' between project outputs and the host area's needs is a key sustainability measure (Higginbottom, 1990). Project success measures need to include consideration of its capacity to adjust to changing local needs.
31. The issues examined in this initial analysis are cast at a high level of generality. There is little reference to detailed administrative and organisational issues - the building blocks of cost reduction within a project. This is a reflection of the literature initially examined. Reference to budgetary reductions are almost always in the context of governmental (donor or beneficiary) financial constraints and changes in local or global economies. Very little evidence is published concerning specific cost reduction measures, and even less evidence relates to the provision of equipment, materials and premises, which impact upon project effectiveness. Part B focuses upon these aspects of technical/vocational education projects, upon field analysis of the issues specified in the list below.
