The study confirmed that in primary schools in Malawi and Sri Lanka, children’s literacy learning and learning of mathematics were not being recorded systematically. Although teacher’s intuitive judgments of children’s capabilities were often good, it did not allow them to diagnose specific areas in which children were not achieving. Further, it was found that teachers did not have a ‘meta-language’ through which to communicate children’s achievements to parents or the educational authorities.
The study set out to address a number of important questions. First, it was designed to consider whether a profiling system capable of providing reliable and valid indicators of children’s learning could be developed in two countries, Malawi and Sri Lanka. Second, it was concerned to establish how far the capabilities of teachers to collect information about children as learners, evidence of their learning and to record the achievements of children, could be enhanced.
1. The findings show that teachers in Malawi and Sri Lanka were able, through a process of negotiation and development to establish profiling systems for primary education in which the achievements of children in literacy and mathematics could be charted.
The processes associated with developing the profiling systems, particularly the frequent meetings of teachers in each country, produced a number of important benefits which are worth highlighting here:
· Teachers were given the space to reflect upon their current professional practices and attitudes to identify those areas in the curriculum they wanted to change.2. Teachers involved in the study were able to collect information about children as learners, collect evidence of learning and to record the achievements of children.· During the process teachers dealt with theories of child development and literacy acquisition accepting some and rejecting others and in the process they developed their own theories about the education of young children.
· Teachers were highly motivated during the research process and they reported that developing and refining the profiling system was very satisfying. For many, they felt themselves to be part of a collaborative effort for learning about and improving primary education.
· Teachers were understandably dismayed when students levels of achievement failed to reach the desired norms that they had established. That 50% of children failed to achieve a level 3 in every aspect of literacy and mathematics assessed was however accepted by teachers and they understood that the profiling system gave them a base line for where children were at and what needed to be done in order to raise the children’s levels of achievement. Teachers recognized that using a profiling system made it possible to demonstrate the many different ways children can work towards mutually agreed standards of competence.
· Teachers reported that they had few problems in using the assessment tasks to collect information of children’s learning. Some tasks, as discussed in chapter 5, were not as well suited to the context of education as others and teachers found them more difficult to administer. But on the whole, all tasks were administered and were largely successful in eliciting from students the learning processes the researchers were interested in studying.3. A main question arising out of the research is the credibility of and potential use of the profiling system beyond individual classrooms. The issue here is the tension between the need to provide data for teachers, students and parents on the one hand and the needs of educational authorities for data reduction and for comparisons of schools. The question then was how could the rich qualitative evidence based on teacher records that had been generated be summarized to meet the needs of the educational system for indicators of ‘quality’ without trivializing or distorting the data or diminishing its value for making classroom decisions? Could the data be used effectively for these two quite distinct purposes.· Teachers reported that collecting information of children’s learning through the assessment tasks provided taught them a tremendous amount about what children knew and how they learnt.
· Teachers were also able to collect samples of children’s work and to produce test-like records such as running records, and observational notes which provided the actual evidence of children’s progress. This was a new and challenging activity for many teachers involved in the study and was only possible because dedicated record sheets were provided with every task.
· The real challenge for teacher-based assessment procedures lie in the collection of evidence of children’s work produced over the period of a school term or school year. In the case of Malawi, and perhaps less so in Sri Lanka, the level of resources available in schools would make this an impossible prospect. However, the study does demonstrate that the collection of evidence is possible even in poorly resourced educational contexts, when record sheets and clear guidelines on what constitutes evidence are provided.
· Teachers demonstrated that they were able to comment on and interpret children’s work samples to reach decisions about what they had achieved.
Certainly part of the answer would appear to lie in being able to assign scores or levels to the profiling system that can easily be interpreted at both a local, as well as a national level. The goal therefore was to provide a robust enough ‘language’ to describe the broad picture of students’ achievement at a given stage of schooling (Little and Wolf, 1997, Broadfoot and Gipps, 1997, Dougherty, 1998) for the English National Assessment system was designed explicitly to fulfil ‘diagnostic’, ‘formative’, ‘summative’ and ‘evaluative’ (ie system-wide) assessment purposes. However, it is doubtful whether these goals have all been achieved.
The four point scale was developed in keeping with other international examples of profiling systems. However, the research showed that the level descriptors describing each level may need review and revision in both countries as teachers become more familiar with a wider range of pupils’ work. In many other international examples where teachers have been involved in developing similar profiling systems, the development phases were not necessarily longer but more intensive. The levels of achievement in Malawi and Sri Lanka were adopted after only two drafts, but other examples show that it is often after 4 or 5 drafts that teachers find their anchors and are capable of writing descriptions of each level which are clear enough to be used comparably by teachers throughout the system.
To address the issue of the reliability of teacher ratings, the study provided for the samples of student work to be brought to the meetings by teachers, these were exchanged and blind read by their colleagues who then assigned a second score. Inter-rater reliability calculated on these scores were consistently high. Thus the finding suggests that the kind of methodology adopted in this project to generate both pedagogically useful and policy relevant data concerning the quality of learning and teaching has considerable potential.
Moreover, in addition to the high inter-rater reliability coefficients the moderation process itself is an important one. Teachers were asked to review samples of work achieved by students who were unfamiliar to them, to discuss their interpretations with colleagues an to arrive at a common understanding of the relative ‘value’ of the work. All these processes have been shown to contribute to the professional development of teachers (eg Haslen, 1994).
Recommendations
1. It is proposed that the research be adopted as a model for establishing profiling systems for the assessment of learning in developing countries. Shorter, more intensive cycles of work would be preferable.
2. The task development process in the research was time consuming and not always suited to teachers working on their own. It is recommended that nationally produced standardized tasks be developed with the assistance of curriculum developer in each country.
3. The administration of tasks proved to give teachers little difficulty. Nationally produced tasks could be administered by teachers in selected schools 3 times a year.
4. Large class sizes prevented the administration of tasks to whole classes in areas of assessing reading, oracy and in certain mathematics tasks. Sampling within classes for the administration of these tasks has proven to be a useful strategy. In such cases information on the learning of every individual child is not always possible but information gained of a random sample of children is a useful monitoring device.
5. The processes of gathering, recording and reporting such ‘quality’ information should be explicitly ‘low stakes’ and not used publicly in the form of league tables to compare schools, regions or countries. If this happens the formative potential for encouraging better teaching and learning is likely to be forfeit to more mechanistic approaches to ‘getting the scores up’.
