4.1 Background to the projects
4.2 Starting up the new literacy programmes
4.3 Approaches to monitoring and evaluation of the pilot projects
4.4 Monitoring progress & problems in the pilot projects
4.5 Reading, writing and numeracy
4.6 Empowerment
4.7 Methodological learning
4.8 Cost effectiveness
A local ACTIONAID fieldworker keeping the records for a Shomiti on Bhola Island, Bangladesh.
The two year action-research project was designed to test the use of PRA in adult literacy programmes in three very different contexts. This would enable us to observe the flexibility of the approach and would lead to various adaptations and innovations. The three pilot projects could not have been more diverse.
In Bundibugyo, Uganda the pilot was in a multilingual area where the main local language had never previously been written. In Bangladesh the pilot was with women's savings and credit groups in a conservative Islamic area and in El Salvador the pilot was with a grassroots NGO, COMUS (The United Communities of Usulutan -Comunidades Unidas de Usulutan) which is led by ex-guerrillas converting to peaceful methods after ten years in arms.
Bangladesh is the most densely populated country in the world with 110 million people on a land mass the size of England and Wales. Over 65% of the population are illiterate (78% of females). The government aims to reduce this to 38% by the year 2000 and has placed a particular priority on women's education.
In 1995 a new Directorate of Non-Formal Education is being established with Asian Development Bank/ World Bank funding. An early priority for the DNFE is going to be targeting young adults (15-24), particularly females. Within this, support to the piloting of different implementation models is also to be emphasised. This provides a strong context for piloting new approaches to literacy such as REFLECT.
The project area for the REFLECT pilot was on Bhola, a delta island in the Bay of Bengal, in the extreme south of Bangladesh. It is a remote area (a 13 hour overnight launch journey from Dhaka) which has remained isolated from most processes of social change and development. The island is in a geographically vulnerable area under threat from severe cyclones. In a cyclone in 1971 it is estimated that hundreds of thousands of islanders died, though a similar cyclone in 1991 was less devastating.
The ACTIONAID programme was established on Char Fassion, Bhola Island in 1983. Over 12 years it has built up a large savings and credit programme targeted particularly at women. This programme is run through "Shomitis", community groups with about 15 members each. There are now 24, 000 members of this programme who access small loans (800 taka each, with up to three loans at any one time). The Shomitis meet regularly and act as a focus for discussion.
Establishing the shomiti programme was a major undertaking. At first women refused to leave their houses when ACTIONAID representatives entered the village. After some time they agreed to come out, but would do so only with a veil and two umbrellas to hide behind. Overtime the umbrellas were discarded and more recently the veils have been lifted as women gain more confidence through the regular contact with Shomiti assistants. The Shomiti members receive 12 week training courses on a range of health and development issues. Some Shomitis have decided to start up Children's Learning Centres - particularly in areas where the nearest primary school is too far away for their children to reach. They contribute towards paying for a teacher and receive some help from ACTIONAID. Adult Literacy Centres are also started up where Shomitis request support.
Although women are the main point of contact for the Shomiti programme it has been documented over some time that most women hand over the
credit they receive to their husbands and do not participate much in deciding how the loans should be spent. Indeed, it seems that some men only tolerate women's involvement because the Shomiti is a source of credit for themselves.
In 1993 ACTIONAID started to develop concrete plans to phase out of the Bhola project. It had never been ACTIONAID's intention to continue indefinitely, but the decision that phase out should be completed by 2000 raised a number of key issues. It was realised that if the Shomitis were to have a life after ACTIONAID's support they would need to become self-managing with each woman being able to keep her own Savings and Credit records. The literacy centres to date had not been very successful and there had been almost no focus on numeracy. Over 85% of women remained illiterate. In July 1993 a review of the education programme on Bhola Island recommended that the adult literacy programme needed to:
· focus more on numeracy skills
· be more effective in teaching literacy
· be used as a basis for empowerment.
In January 1995 a new Project Director started and in recent months has developed with local staff a new vision for the future. It is now intended that a Peoples Organisation should be established to continue in the area after ACTIONAID's departure. For this to succeed the shomiti programme needs to be put on a more rigorous financial footing. Shomitis will increase from the present size of 15 to a minimum of 30 members. Interest rates will rise to cover operational costs and larger loans are to be made available in various categories from general credit to seasonal credit, poultry credits, credit in kind (eg for a rickshaw or boat) and collective credit. The use of loans will be supervised with training and support offered. Shomiti member representatives will gradually be brought in to the management of the entire programme. Local marketing will be promoted.
Strengthening the Shomiti members' capacities to manage their own programme will be vital for the success of this strategy over the next five years.
El Salvador, on the Pacific Coast of Central America, is the most densely populated country in the Western hemisphere. Unequal distribution of land and wealth led to a brutal civil war in the 1980s in which over 75, 000 people died and millions were either forced into refuge or displaced internally. In 1992, Peace Accords were signed between the government and the guerrillas which included a commitment to land reform and democratic elections. However, progress with the land reform has been slow and there were many accusations of irregularity in the elections won in March 1994 by the extreme right wing ruling party, ARENA.
The economy is now gradually recovering from the war, helped temporarily by much international aid for reconstruction. However, the biggest single form of income is now remittances from relatives who fled El Salvador during the war and are now working in the United States.
The education system in El Salvador reflects the polarisation of society with most resources benefiting the elite who manage to reach higher education. During the 1980s the war disrupted the little education that was available to the poor. Hundreds of rural schools were permanently closed and the high levels of internal displacement meant schools in marginal urban areas were often seriously overcrowded. The rural schools which remained open suffered extremely high drop out rates.
Despite some courageous initiatives in the 1980s, adult literacy programmes were rarely stable enough to be successful, primarily because literacy teachers were regarded as subversives by the army. Indeed teachers in general were often a target (within just one year, 136 were assassinated). Some alternative models of popular education were developed by the FMLN guerrillas in areas of the country under their control - led by "popular teachers". However, many of these experiences also suffered from a lack of continuity.
By the end of the 1980s less than a third of the school-aged population of El Salvador was covered by State schools. Private schools have flourished but these have tended to serve only the middle and upper classes (about 20% of the population). Since the peace accords there has been some increase in expenditure on education but serious problems of under-resourcing remain, particularly in rural areas.
The pilot programme was conducted in the Department of Usulutan which was one of the most conflictive zones during the civil war, with control of the area fluctuating between the army and the guerrillas. The coffee plantations which dominated the landscape were abandoned for many years as it became impossible to harvest the coffee. Most people in the area sympathised with the guerrillas and indeed many of the young men and women (including boys and girls as young as 14) joined the guerrilla forces. They had little choice as their traditional form of employment as labourers on the coffee plantations had disappeared. In many areas, abandoned coffee was cut down and the poor grew maize and beans to feed themselves. In the late 1980s the communities in Central Usulutan, around the Taburete Hill, decided to organise themselves as civilians to try to improve their conditions. This would have led to army repression had it been open, so initially this organisation was clandestine. In the following years, with the wider movement for peace it was possible to come into the open and the "United Communities of Usulutan" (COMUS) was formed -as a grassroots organisation committed to improving the conditions of the poor in the area.
Gradually COMUS received external funding (initially from the Catholic Church) in order to run a credit programme for farmers. In 1992, with the support of ACTIONAID this diversified into an integrated development programme covering a range of activities. After the Peace Accords many demobilised guerrillas joined COMUS and the present director of COMUS was himself a guerrilla throughout the war.
One of the most important elements of the COMUS programme is now support to the land reform process - giving advice on legal rights and helping people to negotiate with central government to receive titles to land. Support is also given to people wishing to form cooperatives. The credit programme has gradually extended, backed up with technical advice from agronomists as many people who are now farmers have only ever worked as day labourers before and have little or no experience of growing food for themselves. Alongside this there is a primary health care programme with health promoters in each village - and a natural medicine programme developed to systematise the traditional knowledge of local committees.
Within COMUS's education programme the priority has been placed on adult literacy. For over a decade the primary schools scarcely functioned (and when they did they were not effective) and although the guerrillas ran some literacy classes most people missed any opportunity to receive any education. Adult literacy is thus a vital first step.
To run the literacy programme COMUS called on the expert support of a national NGO CIAZO (Inter-agency Committee for Literacy). CIAZO is a national NGO which specialises in popular education, particularly on adult literacy, providing training and technical support to some 30 grassroots organisations (covering a total of about 5,000 adults each year). The organisations which CIAZO supports sit on a management board of CIAZO itself, ensuring that CIAZO remains accountable to those who it serves. CIAZO plays the leading role in an umbrella group of education NGOs, has a joint programme with UNICEF and is widely regarded as the most established and professional NGO working in the area of adult literacy in El Salvador.
CIAZO has been running its literacy programme under the title "Teaching Literacy for Peace" for over three years. It has developed excellent primers and follow-up adult basic education materials. REFLECT was just one of three pilot programmes run by CIAZO in this period and CIAZO had no vested interest in making it work -particularly when the evaluation was to compare the REFLECT approach with ClAZO's own work developed carefully over several years.
Uganda has a population of 17 million people, 90% of whom live in rural areas. After two decades of conflict the country is experiencing a period of political and economic recovery under the National Resistance Movement. Agriculture is the main basis of the economy, with 80% of export earnings coming from coffee (though more than half of this currently disappears on debt servicing).
The literacy rate is officially 55% but is widely assumed to be lower. Women's literacy rates are even lower (officially 45%) reflecting the status of women in society. As in most countries women have lower incomes and reduced access to education and health care. The large number of households (approximately 40%) that are headed by women, are thus particularly vulnerable.
However, national policy has increasingly put women's issues on the agenda, and the essential contribution of women to the development of the country is widely discussed.
Uganda is a multi-lingual country; the three main language groups being Bantu, Nilotic, and Central Sudanic. The official languages are Lugandan, and English. Most people understand and speak languages other than their mother tongue.
The pilot project area, Bundibugyo, is situated in the extreme South West of Uganda along the border with Zaire, approximately 400 Kms from Kampala. It is separated from the rest of the country by the Rwenzori mountain chain, and is accessed by one steeply winding road. Physical isolation is one of the most striking features of the district.
The District is divided into Ntoroko and Bwamba counties, and ACTIONAID works in the latter. Bwamba has a population of 92, 300 divided between the highlands (40%) and the lowlands (60%) The area is thus divided by its topography, and even within the lowlands the land is crossed by steep ridges made by fast flowing mountain streams. Population density is very high (261 per sq.km) because most of the county is forest reserve which only the few remaining pygmies are allowed to inhabit.
Bwamba's economy almost wholly consists of a large number of self sufficient family farms, mainly operating at subsistence level. The increasing population density has led to land fragmentation, soil exhaustion and food insecurity. The area produces a great deal of Uganda's coffee, but prices paid to producers have always fluctuated -and usually not to the growers' advantage. Goods from outside are very expensive due to transport costs, and physical isolation makes profitable trading difficult.
The two main groups of people in Bundibugyo, the Bamba and the Bakonjo, have been dominated by the neighbouring Batoro people for a great part of their history. This has exacerbated economic problems as the development of the area was always neglected by the ruling Batoro. There are very few schools, roads or any other services.
Women have a very low status in Bwamba, are not allowed to speak before men (with no voice, therefore, in decision making), and are universally regarded as valuable human beings only in terms of their ability to produce crops and children. One of the ways in which this low status expresses itself is in a woman's powerless position in marriage negotiations (bride price is paid to male relatives who thus have a vested interest in the arrangements) and in marriage (particularly as there is widespread polygamy). In addition, women have no right to assets, and although most adult women have their own micro economic projects, they need access to land to grow food for their children. It is estimated locally that just 25% of women are literate (and in some parishes fewer than 10%) compared to about 50% of men. Initial research revealed that this was seen by both women and men as a reason for their exclusion from decision making.
Rutoro, the language of the dominant Batoro, has been used for all official purposes - in offices, hospital, churches, schools - and most salaried posts have been held by Rutoro speakers. Literacy and education and power have all been confined to the Rutoro language. In fact the languages of Lubwisi and Lukonjo had not been systematically written down until the REFLECT literacy programme started. The languages are therefore still developing in their written form, and are a focus for local identity.
ACTIONAID spent four months in early 1992 on intensive research at village level in Bundibugyo aiming to develop a relationship of trust; understand the nature of people's lives; and find out their priorities for ACTIONAID'S work over the following ten years of the programme. Participatory Rural Appraisal was one of the research methodologies used. The development approach recommended following this research was as follows:
· focus on vulnerable groups, especially women.· focus on capacity building for local structures. This led to support for elected Parish Councils for women only. Parish Councillors make all the key decisions about ACTIONAID supported activities in the parish and manage a devolved budget.
· a flexible planning and implementation cycle to respond to local demands. Integrated not sectoral approach.
· participatory approach to adult literacy (particularly for women) as an entry point activity.
In many respects Bundibugyo was an ideal area for piloting the REFLECT approach. Learners' enthusiasm for previously unwritten and underprivileged languages; the lack of disillusionment with literacy generally (quite different from people subjected to mass literacy campaigns); the political environment in Uganda, and the high calibre of ACTIONAID staff can all be counted as assets.
The less positive factors must not, however, be forgotten. These include the irregular attendance of learners (vulnerable due to their low income, poor health; heavy workload and extraordinarily difficult terrain); lack of external support for community actions from official agencies, the low status of women (majority of the learners) and the lack of reading materials in the mother tongue languages.
Origins
During the course of the research in 1992, the need for an interactive, practical and participatory literacy methodology as an entry strategy emerged. The aim was to target women (as the poorest, least powerful members of the community) with an activity that would empower them to take control of their own development. There was some experience of using PRA in the initial research and so, when the experimental use of PRA within adult literacy was suggested the Bundibugyo ACTIONAID Project (BAP) team agreed to be the first testing ground.
Designing the manual
In August 1993, over four weeks, the local ACTIONAID team (with support from the Education Policy Analyst of AAUK) used the results of the initial research to write units for the facilitators' manual. Different PRA techniques (maps, matrices, calendars etc) were identified which would bring out detailed discussion on all the key themes that had emerged from local research (eg isolation, gender inequity, prevalence of preventable illnesses etc). Reading, writing and numeracy work were then woven around these maps and matrices.
Three months into the life of the classes, the manual was revisited by the same team, who added an appendix with more complex agricultural units, units on savings and credit, and numeracy.
Language Issues
After consultation the BAP team decided that the main local language, Lubwisi, would be the language most widely demanded by learners. However, they also felt that the learners in each circle should retain the final choice over language (and they might chose other local languages like Lukonjo or Rutoro). This presented a serious challenge. If the team was developing primers, three primers would be needed. Moreover, the main language Lubwisi, was previously unwritten.
There were mother tongue speakers of all the local languages in the BAP team. To develop a written form for Lubwisi, those who spoke Lubwisi but were literate in other languages, each wrote a few pages concerning local development issues in Lubwisi - drawing on orthographic conventions from other local languages. These were then compared and analysed and where differences were identified a consensus was reached. These same texts were then subjected to syllabic surveys, to identify the most common syllables, so that the first key words which were taught would yield many new words (once broken into syllabic families and reconstructed by learners). The manual itself, once completed was translated into Lubwisi by local BAP staff and then shared with local Lubwisi leaders and educated people who had a strong interest in the transcription of their language. This led to various alterations before the manual was printed -but once it was printed it had the full support of all key players in the local community.
To enable participants in each circle to chose their own language, appendices to the manual were elaborated separately, with help from Lukonjo and Rutoro speakers - to identify suitable literacy work in their languages which would arise out of the same sequence of Units. It was recognised that most facilitators were likely to be bilingual (or trilingual) and would be able to read the manual in either Lubwisi or English. What they needed was a list of key words in their own languages. These languages were thus also subjected to a syllabic survey and key words were identified using the regular syllables first. Certain other changes were also necessary (eg in Rutoro most words start with a vowel - so inverted syllables had to be introduced from the start). The result was that, with two appendices (in Lukonjo and Rutoro) the same manual could be used, covering the same themes in the same sequence, for any of three local languages. For example, in the first Unit, a natural resource map, the word "kiti" (tree) is used in Lubwisi, "amikura" (mountains) in Bokonjo and "ekibira" (forest) in Rutoro.
Visual cards
Visual cards which could be used with each Unit were designed and produced with a local artist. The aim was to produce cards which could be easily copied by the facilitators as they transferred graphics from ground to paper.
Selecting Parishes
Enrolment of Learners
For the REFLECT pilot four parishes were selected: Hakitara, Bubukwanga, Busoru, and Bundimarangya. These parishes were chosen from the ten parishes in the sub-county because ACTIONAID research revealed them as relatively disadvantaged economically and socially.
Facilitator Selection and Training
Facilitators were recruited by the Parish Councillors using a criteria of "academic merit and commendable behaviour in public." They are usually residents of the village where they teach. Sixty five facilitators were selected, but only four were women. This unfortunate situation arose because of women's restricted access to education, and perhaps also due to anxiety about exactly what the job would involve (more women have applied to be facilitators for the second cycle of classes). The basic education level of most facilitators varied between P7 (seven years of primary schooling) and S4 (four years of secondary school). This is the same range as the facilitator's in the government's pilot adult literacy project.
The initial training lasted for ten days and consisted of PRA experience followed by practice with the units - especially the link with literacy and numeracy. The facilitators were imbued with the principles of respect for the learners, and that they were working for the Parish Councillors - not ACTIONAID. Most of the facilitators had no previous experience, which was a big advantage in absorbing a methodology so different from that of the top-down formal education system (and even many other adult literacy methods) in Uganda. Facilitators then formed Parish Groups for fortnightly training sessions, and exchange of experiences. Refresher courses of a week were run by ACTIONAID every six months. This also provided an opportunity to make up new units as needed. Other support from ACTIONAID consists of visits by field workers. Due to the difficulties of travel, field workers have tended to visit facilitators with problems frequently, and leave smooth running classes to their own devices. Parish Councillors visit classes to check on the attendance of facilitator and learners.
A pre-literacy campaign with a local "ngoma" group served to raise community awareness of the start of the programme. Women were both self selecting (as illiterate) and also positively encouraged to enrol. In fact there was some confusion at the beginning about whether men were allowed to enrol.
In the first three months of 1994, learners were enrolling in the four parishes - about thirty per class with the majority being women but over a wide age range (18 to 80). After an initial meeting, most of the sixty five circles which were formed, started to construct shelters out of local timber and spear grass. Other community members assisted as they saw the shelters as useful for everyone. To help protect the structures from the most voracious and ever-present threat, ACTIONAID made an input of termite-killer at this stage.
Support from ACTIONAID
ACTIONAID has provided (through elected parish councils) incentive payments to facilitators (equivalent to about $20 a month), a manual, a set of visual cards, a blackboard and some large pieces of manila paper for each facilitator. Learners received a exercise book and a pencil. Some have received a second exercise book but where these have not been distributed quickly enough, learners have bought their own (a fairly cheap item in Uganda in any case).
Reading Materials
In view of the lack of reading materials, ACTIONAID has been gradually feeding in the following: a pamphlet on micro projects; a book on civic life, and a health booklet. These have all been specially written and produced for the new literates by the BAP literacy co-ordinator James Kanyesigye - and are all written in Lubwisi. In addition, learners have shown themselves very keen to contribute to a newsletter, especially with articles from their oral traditions (previously unrecorded in writing) and local history. These are published by ACTIONAID in a bi-lingual local newsletter "Kukesa".
Starting Up
In November 1993 a team of four people (Oscar Garciaguierra and Marden Nochez Bonilla from CIAZO, David Archer from ACTIONAID UK and Abdon Machado Alvarez from COMUS) worked together over four weeks in order to develop a REFLECT manual adapted to the local context. In consultation with the leadership of COMUS the central objectives of the literacy programme were to promote participation, community development and local action.
The manual had the following:
|
UNITS |
THEMES |
KEY WORDS. |
|
1. MAP OF HUMAN RESOURCES |
water/ fuel |
"camino" |
|
2. HOUSEHOLD MAP |
population/ housing |
"teja" |
|
3. MAP OF HUMAN RESOURCES |
local knowledge |
sobadora |
|
4. AGRICULTURAL MAP |
soils/ planting |
café |
|
5. MAP OF LAND TENANCY |
agrarian reform |
parcela |
|
6. RAINFALL CALENDAR |
soil erosion/ drought |
lluvia |
|
7. AGRICULTURAL CALENDAR |
seed varieties |
semillas |
|
8. AGRIC. ANIMAL MATRIX |
productive animals |
ganado |
|
9. CROP MATRIX |
chemical/organic inputs |
plaga |
|
10. CALENDAR OF SHORTAGES |
survival strategies |
escases |
|
11. BUYING/SELLING CALENDAR |
intermediaries/cooperation |
coyote |
|
12. CALENDAR OF PURCHASES INCOME/ EXPENDITURE TREE |
inflation |
quintal |
|
13. SOURCES/USES OF CREDIT |
credit/loansharks |
credito |
|
14. REVISION/ PROJECTS MATRIX |
|
|
|
15. MATRIX ON CAUSES OF ILLNESS |
causes/prevention |
nino |
|
16. MAP OF HEALTH RISKS |
local preventive action |
basura |
|
17. CURATIVE MATRIX |
local curative practice |
examen |
|
18. HERB MATRIX |
documenting local knowledge |
jingebre |
|
19. NUTRITION MATRIX |
malnutrition |
desnutricion |
|
20. VACCINATION CARDS |
immunisation |
polio |
|
21. BODY MAPS |
pregnancy |
embarazada |
|
22. MATRIX OF VICES |
alcoholism |
vicio |
|
23. REVISION |
|
|
|
24. VENN DIAGRAM OF ORGANISATIONS |
organizacion |
COMUS |
|
25. EVALUATION MATRIX OF ORGANISATIONS |
aid |
organizacion |
|
26. TABLE OF PARTICIPATION |
self help |
participacion |
|
27. DAILY ROUTINE TABLE |
gender roles |
mujeres |
|
28. EDUCATION MATRIX |
children's education |
escuela |
|
29. HUMAN RIGHTS TABLE |
abuse/ violations |
reclutamiento |
|
30. DISPLACEMENT MAP |
effects of war |
refugiado |
|
31. PEACE ACCORDS LIST |
land reform |
acuerdos |
|
32. NATIONAL BUDGET PIE |
national priorities |
|
|
33. WELL BEING RANKING |
local causes of poverty |
|
|
34. IDEAL FUTURE MAP |
local priorities |
|
|
35. REVISION/ DESIGN OF POST-LITERACY |
|
|
|
36. SPECIAL UNIT - MATRIX OF POLITICAL PARTIES |
|
|
Visual cards were prepared by Alfredo Burgos in January 1994 - a total of 180 cards (which was too many). Although the cards were very well drawn and had some good comic detail to stimulate learners, they were in the end too detailed and almost impossible to use. They were not numbered or colour-coded and had no words written on them or any reference list. Given these shortcomings, despite being good pictures they were barely used.
Training in the use of graphics for facilitators in Usulatan.
The first training began in January/ February 1994 and the majority of the literacy circles opened in March 1994. Training was undertaken by the COMUS Promoter (initially only one) and the CIAZO adviser.
The Promoters and the Technical Adviser
The literacy programme was effectively organised by the education promoter in COMUS, Abdon Machado, a forty year old man who was illiterate himself until the age of 23. Two other literacy promoters joined during the year (Ovilio, who has just third grade primary education and Juan who started as a voluntary literacy teacher). Training and support in the process of documenting the experience was provided by the CIAZO adviser, Oscar. The contrast of styles between Oscar (who is from an urban, middle class educated family) and Abdon (who is a campesino) led to considerable tensions and personality clashes at times. Nevertheless, both had something important to offer to the programme.
Abdon Machado, the coordinator of the REFLECT programme for COMUS.
The Literacy facilitators
A total of 23 literacy facilitators received training. Some of them have left (eg there have been five changes of facilitators in Galingagua and four changes in EL Zungano). In some literacy circles there were two facilitators who rotated or who shared the work in various ways. All the facilitators came from the communities where they worked and were either nominated by their community or volunteered at a community assembly. The work was completely voluntary and so it was sometimes difficult to find willing people and there was little prospect of selecting those who were "suitable" -the promoters felt anyone who was willing to try should be offered training.
Each facilitator who volunteered was given ten days initial training and then attended two days a month follow-up training in which facilitators shared their experiences and planned for the coming weeks. The facilitators received travel costs and food for the training days which was considered essential to maintain their involvement. Some other small tokens of appreciation have been given such as small rucksacks and baseball caps.
There has been a lively debate about the nature of voluntary work and whether honorariums or stipends should be paid. Certainly payment for training days (when they miss whole days which could otherwise be spent productively in their fields) seemed essential and there were some tensions when this was nearly stopped. However, the motivation of the facilitators is impressively high. Over 80% of the facilitators say their primary aim has been to "help others". Other motivations that were mentioned were the desire to exchange experiences with others (mentioned by 47%), the desire to learn more themselves (40%) or to gain new experiences (27%).
Asked the question: "Who are you working with?" the majority (67%) mentioned their community (instead of ClAZO or COMUS). Asked "Who are you working for?" the answer was even stronger (87% said, "my community"). This was felt to be an important indicator that the facilitators were genuinely working out of commitment.
Most literacy facilitators had an education level around 6th grade primary. Some had received just three years of primary schooling and had difficulty reading and writing themselves (but wanted to share the little that they knew). Others had attended secondary school. This range of education levels was found to be very important in the evaluation.
One of the definite achievements of the literacy programme, mentioned by all observers, has been the development of a strong team feeling amongst the literacy facilitators. In general 18 arrive at each training day and they spend an evening together each month, often singing or telling stories. They have formed a formal committee to allocate responsibilities between them and they have even formed a musical group which tours villages, playing when new literacy classes are opened (with songs specially written for the occasions). In each village the literacy facilitators have also helped to form an education committee which addresses not just questions of adult literacy but also education provision for children.
The Learners
Most literacy circles started up with between 10 and 20 people Most of the learners were men (only 32% were women), a fact which may relate in part to the lack of gender sensitivity in the rest of the work of COMUS. All the learners were peasants who made a living from the land, whether working in cooperatives, being small-holders or scraping a living as landless labourers on large plantations. The main motivation of the learners was, of course, to learn to read and write. However, more than half also mentioned a desire to learn numeracy, 19% expected that they would discuss local issues and 10% said their aim was generally to help the cooperative. Joining the literacy circle was not always easy for the learners: almost a third of learners said their friends laughed at them and ten percent of learners even faced laughter within their own family.
Local musicians promoting REFLECT in the pre-literacy campaign.
Bangladesh was the last of the three pilot programmes to be set up, with the work on training and manual development taking place in March 1994. Over a four week period a team of seven people (two ACTIONAID staff from Bhola, two from Jamalpur, two from Tikkapara and one from AAUK) developed the Bhola manual. The programme was designed exclusively for women and was targeted at women who were already organised into the shomitis (savings and credit groups).
The Bangladeshi team which developed the REFLECT manual for Bhola Island.
The objectives of the programme were:
· to enable the women to develop sufficient literacy skills to read things encountered in their daily lives and write simple phrases about their lives.· to provide the women with the numeracy skills necessary to manage their own savings and credit books and deal with money, measurements, weights etc with more confidence.
· to enable the women to share their experiences and knowledge so as to increase each member's ability to participate more in decisions within their households and in the wider community.
The manual included a strong focus on Savings and Credit, including details of the Savings and Credit pass book, preference ranking on loan use, projections on loans, an intra-household decision making matrix, units about the Shomiti itself and a broad range of Units on health:
1. HABITATION/ HOUSING MAP
2. NATURAL RESOURCE MAP
3. AGRICULTURAL MAP
4. RAINFALL CALENDAR
5. WOMEN'S AND MEN'S WORK CALENDAR
6. FOOD AVAILABILITY CALENDAR
7. INCOME AND EXPENDITURE CALENDAR
8. PRICE OF BASIC COMMODITIES CALENDAR
9. SOURCES AND USES OF CREDIT MATRIX
10. SAVINGS AND CREDIT BOOKS - PART 1
11. PREFERENCE RANKING - USE OF LOANS/ USE OF SAVINGS
12. PROJECTIONS OF DIFFERENT LOAN USE, BENEFITS/ RISKS (VARIOUS)
13. SCHOOLING OF CHILDREN/ EDUCATION MATRIX
14. HEALTH CALENDAR and MATRIX OF CAUSES
15. HEALTH SEVERITY MATRIX
16. HEALTH CURATIVE MATRIX
17. HEALTH HERBAL MEDICINE MATRIX
18. HEALTH AND HYGIENE MAP
19. VACCINATION CARDS OF CHILDREN
20. RANKING OF FOODS BY VITAMIN VALUE/ HEALTHINESS
21. PROCESS ANALYSIS OF CAKES
22. WELL-BEING RANKING
23. OPPORTUNITIES AND SERVICES MAP
24. CHAPATI OF INFORMAL SOCIAL STRUCTURES
25. TIMELINE OF VILLAGE
26. PAST AND FUTURE OF SHOMITI
27. SAVINGS AND CREDIT BOOKS (PART 2.)
28. DECISION MAKING IN HOUSEHOLD MATRIX
29. PICTURE OF AN IDEAL FUTURE/ ACTION PLANNING
The initial manual was written in English with key words in Bengali. This was then translated and printed in Bengali in April/May. During this time 75 visual cards were also produced by an artist following a visit to Bhola. In May ten shomitis for inclusion within the initial project were selected.
Facilitators were selected at the same time. All facilitators were women and were local to the community where they teach. Most were young and had an education level of higher secondary (about 11 years education) - which is higher than the average education level of facilitators in El Salvador and Uganda. Three of the facilitators in Bhola were married.
The facilitators were given ten days initial training in June. The literacy circles were opened in early July. The sum of 650 taka a month was paid to facilitators as an honorarium. All facilitators attended ongoing exchange/ training workshops every fortnight. Supervision and field support was also provided.
The REFLECT literacy centres started with an average of 15 participants, mostly women from the local Shomiti (though in some villages other women also joined). The one or two Shomiti members who were already literate did not join. Most REFLECT circles chose to meet in the afternoons, often from 4-6pm, though this has varied with the seasons and some circles shifted their classes to earlier in the afternoon. In one case a circle chose to meet from 7-9am. Almost all circles committed themselves to meet six days a week and have maintained this momentum.
The circles rarely had a sheltered meeting place. The norm was for women to meet in the compound of one of their homesteads, laying rush mats on the floor and sitting. Other than the facilitator's manuals and the visual cards, the only equipment was a blackboard with chalk, learners notebooks and pencils, and lots of large sheets of paper with felt tip pens for drawing.
The REFLECT planning team select and sequence Units for the Bhola Island manual.
4.3.1 Bangladesh control groups
4.3.2 Uganda
4.3.3 El Salvador
The three pilot REFLECT programmes each required a slightly different approach to monitoring and evaluation given the contexts of the programmes and the different capacities of the organisations involved.
The evaluation was potentially very problematic as the original ODA funding was focussed on research and evaluation, but required only an internal evaluation by ACTIONAID to identify the strengths and weaknesses of the REFLECT approach in different settings. However, the level of interest in REFLECT (from other NGOs, government personnel, donors and academics) required us to do evaluations which would stand up to an increasingly large external audience (in over twenty countries). The potential "conflict of interests" could not be ignored. These were innovative literacy programmes that had been set up by ACTIONAID and which were to be evaluated by ACTIONAID. To satisfy this audience we involved neutral/ external evaluators in all three cases and sought to take a self-critical approach at all times.
A second key issue for the evaluations was the need to balance traditional methodologies with more innovative approaches which would be appropriate for the wider aims and participatory nature of the REFLECT process. All three evaluations thus included the testing of basic literacy and numeracy skills (using standard traditional tests) as well as the evaluation of the wider impact on empowerment and community development (eg using the construction of graphics by learners and semi-structured interviews of everyone directly and indirectly involved).
A third key issue concerned the comparability of the REFLECT circles with more traditional literacy classes. If we evaluated the REFLECT circles in the three pilots in isolation and found excellent results we would not be able to determine whether the results were due to the REFLECT methodology or were caused by other factors. In each of the pilots we had therefore identified Control Groups - other literacy programmes using more traditional, primer-based methods in neighbouring or comparable areas. These Control Groups were identified at the time of starting up the REFLECT circles and were monitored and evaluated over the same time period, using the same methods. By comparing the results we would thus be able to determine the relative effectiveness of the REFLECT approach compared to a primer-based approach.
A fourth issue that needs to be noted concerns the timeframe of the evaluations. The evaluations were done just 12 or 18 months after the literacy programmes had started. This was inevitable given the 2 year timeframe for the pilot project. To fully evaluate the sustained impact of these programmes would require a longer time frame (perhaps 3 years from the start up) and would still not address some of the broader questions concerning long term sustainability or how REFLECT would work on a larger scale.
In May 1994, at the same time as the REFLECT circles were identified, and ensuring that similar criteria were applied (eg Hindu/Moslem balance, distance from main road etc), ten control centres were also identified in Bhola which would be monitored closely but would continue using a primer-based approach. The primer used was devised by Friends in Village Development Bangladesh (FIVDB) and is widely regarded as one of the best in Bangladesh. These Centres received, from ACTIONAID, the normal support provided to literacy centres using FIVDB materials - in relation to training and materials. Baseline information was taken on these Control centres and the REFLECT centres to ensure that the starting points were consistent.
Specifically an initial test was done to determine whether any of the women had an initial level of literacy. Some were found who could write their name (evenly spread between REFLECT and Control groups) but they were unable to identify the letters that made up their name and could not read or write anything else.
Ongoing Monitoring
The ACTIONAID Bhola staff maintained close contact with all REFLECT and Control group
Literacy Centres. Fortnightly workshops were used for detailed monitoring purposes as well as for ongoing training. ACTIONAID's broader monitoring and evaluation system for Bhola was also used to monitor progress.
Evaluation Phase One: External evaluation
In February 1995 the ODA supported an external evaluation of the REFLECT pilot project led by Professor Rahman, Director of the Bangladesh Government's Non-Formal Education Expansion Programme, and Habibur Rahman, Coordinator of the Campaign for Popular Education CAMPE (an umbrella group of 200 Bangladeshi NGO's working in education). This evaluation focused on reading, writing and numeracy skills with only brief reference to life skills.
Evaluation Phase Two: Wider empowerment
The aim of phase two of the evaluation was to wrap up elements left over from the first phase and to focus on empowerment. It was undertaken by Ratindranath Pal (Deputy Director of the Centre for Studies in Sustainable Development) helped by David Archer, Rezaul Chowdhury, Shajahan and Nazrul. It involved:
· a review of secondary materials;
· construction of graphics with learners (70% sample)
· semi-structured interviews with learners (30% sample);
· an evaluation workshop with facilitators.
The issues covered included:
· content and quality of graphics/ learner's books;
· time use in the circles;
· learner's self evaluation of process/ impact;
· use and impact of each graphic;
· intra-household decision making;
· mobility of women;
· impact on children's education (enrolment and attendance)
· impact on the shomiti:
· impact on use of loans;
· local/ personal actions prompted by the literacy circle;
· attitudes to selected issues;
· new areas of knowledge;
· the impact on the facilitator's themselves (role/ perception in the community);
· attitudes of community leaders/ men.
It should be noted that by the time of this second evaluation the Control Group centres were closed and it was therefore not feasible to get comparative data.
Control Group
Unlike Bangladesh it was not possible in Uganda to identify another literacy programme in the same geographical area as the REFLECT pilot. In Bundibugyo there were no other adult literacy initiatives underway or planned. Instead, it was decided to use a literacy programme supported by ACTIONAID in another area of the country (Mityana in Mubende District) as a control group. This literacy programme is regarded as a positive model by many involved in adult literacy in Uganda - and the fact that it was an ACTIONAID programme would help to ensure that there was no vested interest in elevating one programme over the other.
Planning meeting for the evaluation team in Bundibugyo.
Evaluation
The evaluation was undertaken by a team of 16 people including Sara Cottingham of ACTIONAID, senior literacy experts from Uganda (including representatives of the umbrella group Uganda Joint Action for Adult Education and the Coordinator of "Literacy and Adult Basic Education", LABE) and other ACTIONAID staff (including representatives from the Mityana control group programme).
The evaluation was conducted during the period of the graduations, and it was therefore easier for the evaluation team to present the concept of evaluation to learners, and to focus on their views. The team's most important tools were PRA formats, to structure class discussions in an effective and participatory way, and to provide a permanent record of the evaluation for each literacy class. They covered:
· self-evaluation of progress in literacy and numeracy skills,· changes made in their own lives by every learner (gender disaggregated);
· whole group evaluation of the division of time between graphic construction, discussion and the three skills,
· the success of Action Points arising from each unit covered.
When the PRA formats were completed the class drew their own conclusions, and were thus able to get an instant feedback from the activity, as opposed to awaiting an ACTIONAID report.
This information was cross checked through intensive interviewing of:
· learners;
· learners who had decided to leave the class;
· Parish Councillors responsible for classes,
· Resistance Councillors (elected local leaders) from villages with a class;
· facilitators,
· some specialist personnel involved with carrying out Action Points (egs. water staff, District Education Officer).
This had the advantage of involving the broader community in a positive way, rather than merely informing them of what was in progress.
Literacy and Numeracy tests were also administered in order to allow different audiences to make comparisons. It was hoped that we could avoid establishing a culture of testing. All results were gender disaggregated.
Twenty-four classes (2/5 or 40% of the total number of classes) participated in the evaluation. This was a large sample in statistical terms, and it was felt that conclusions could be extrapolated for all sixty one classes with confidence. This sample was selected to include six classes from each parish, with a range of classes who had taken up the REFLECT methodology to varying degrees, but otherwise the process was entirely random.
The Control Groups
In El Salvador the evaluation was led by CIAZO, the national NGO specialising in adult literacy with whom the pilot project with COMUS was undertaken. Although CIAZO were involved in the pilot we felt they were also neutral in that the control group used was their own work in other areas which they have built up over five years with 30 different grassroots organisations. They had no vested interest in showing the REFLECT pilot (which is very small scale within their national programme) to have been more successful than the bulk of their own literacy work. Rather they wished to critically review the work in order to help them define their future methodology.
CIAZO chose two types of control group: firstly their other work with grassroots organizations similar to COMUS (one organization in Usulutan - Codecosta, and one in Conchagua) and secondly with their more recent work with local government at the municipal level with UNICEF. It proved difficult to collect all the data from these control groups but sufficient data was found to provide a clear comparison and this was then reinforced through discussions with CIAZO advisers from across the country.
Monitoring
Basic monitoring of the literacy circles took place on a permanent basis through support visits of the COMUS promoters and through the monthly workshops. Each literacy facilitator also maintained a notebook to document key events and debates in their circle.
The Evaluation
In March 1994 CIAZO, COMUS and ACTIONAID worked together to establish a detailed methodology for the evaluation and to field test some formats. Most of the evaluation work was then conducted between March and May 1995 by a team of 2 CIAZO staff (Luis and Oscar) and 3 COMUS staff (Juan, Ovilio, Abdon). Other CIAZO staff helped collect data from the control group (particularly Arantza). The evaluation data was consolidated and analysed in ten days of May with the help of other CIAZO personnel (including Nicolas Foroni, Arantza and Blanca), David Archer (ACTIONAID) and Sandrine Tiller (Latin American Coordinator of the World University Service).
The evaluation team reviewed basic information and statistics and conducted literacy and numeracy tests. A range of structured interviews were designed and various matrices were elaborated which could be completed with learners in a participatory way. All these formats were pre-tested in March and adapted in the light of this pre-testing. A particular effort was made to "close" open-ended questions during this pre-testing period so as to make the forms easier to complete.
The full scope of the evaluation involved:
|
|
REFLECT/COMUS |
CONTROL GROUP |
|
Basic Literacy and Numeracy Tests |
44 learners |
74 learners |
|
Matrix: Self-evaluation of Learning |
34 learners |
7 learners |
|
Matrix: Self-evaluation of Impact |
32 learners |
7 learners |
|
Table: Participation in Organisations |
36 learners |
- |
|
Table: Community Actions |
4 circles |
- |
|
Diagram: Time-use in circle |
7 circles |
1 circle |
|
Self-evaluation of facilitators |
15 facilitators |
- |
|
Evaluation of training |
15 facilitators |
- |
|
Structured Interviews: learners |
37 learners |
18 learners |
|
Structured Interviews: drop outs |
18 people |
5 people |
|
Structured Interviews: facilitators |
15 facilitators |
5 facils. |
|
Structured Interviews: comm leaders |
18 leaders |
5 leaders |
|
Structured Interviews: promoters |
3 promoters |
2 promoters |
|
Structured Interviews: leaders of COMUS |
5 people |
|
|
Structured Interviews: leaders of CIAZO |
2 people |
|
