Evaluating a remedial learning program in Zambia

Since the launch of its National Policy on Education in 1996, Zambia has achieved near-universal primary school completion levels. National statistics indicate a completion rate of 91.8 per cent at Grade 7 and national assessments show increased levels of learning.

Challenges, however, remain in meeting the target of an average score of 40 per cent in literacy and numeracy in grades 5 and 9. The Education for All 2015 National Review Report highlights that despite successful expansion of access to education, the standardised national and regional assessments reflect some of the lowest academic achievements for those grades.

The Ministry of General Education (MoGE) has been striving to bring about a holistic development in education, with a continued focus on improving learning outcomes, particularly at the foundational level of grades 3, 4, and 5. It launched the Catch-Up program as a pilot in November 2016 to make improvements in the learning levels of primary school pupils. Based on the Teaching at the Right Level (TaRL) approach, the program ensured that learners are taught at the level of their ability.

The pilot targeted all learners in grades 3 to 5 in 80 selected schools over a period of one month for the Eastern Province in 2016 and six months in the Southern Province in 2017. The districts covered under the pilot were Monze and Pemba from the Southern Province and Katete and Chipata from the Eastern Province.

The MoGE piloted three models under the Catch-Up program and, based on initial results, decided to scale up the ‘One Hour a Day’ model in the two provinces. The ‘One Hour a Day’ model involves schools regrouping pupils by ability rather than grade for a concentrated one hour of literacy teaching during one school term, and by a mix of literacy and numeracy teaching on alternate days in the next school term of 100 days, either before or after normal classes.

To explore the opportunity of further scaling up the model to the entire country, the MoGE in association with UNICEF has commissioned an evaluation of the program to ACER for evidence-based decision making and data-driven action. 

The evaluation entails providing an external, independent assessment of the extent to which the Catch-Up program is meeting its intended objectives. The results of the evaluation will provide policymakers and other stakeholders, such as UNICEF, USAID, J-PAL, Pratham and VVOB, useful insights into the implementation of the program, its efficacy for improving foundational learning among target pupils, the challenges encountered and the measures taken to overcome them.

The evaluation exercise will measure:

• whether the program is achieving its objective of improving literacy and numeracy skills for all students, particularly those further behind on the learning trajectory
• the extent to which it is being implemented as designed, for instance, with regards to successful partnerships and sustainability mechanisms
• the minimum cost required to implement the Catch-Up Program, while  maintaining effectiveness
• the enablers or factors contributing to the positive and negative effects, if any, of the Catch-Up program.

The inception phase of the evaluation began in October 2019 and involved the refinement of the methodology, finalisation of the sample size, design of the study tools, and ethical approval of the methodology and tools from the University of Zambia Biomedical Research & Ethics Committee (UNZABREC).

The next phase of activities, comprising of training of field teams and data collection, will be initiated in March when the evaluation team from ACER travels to Zambia. Besides training the enumerators and supervisors on the administration of data collection tools, the team will also conduct field visits and surveys with the teachers, principals and other stakeholders of the program.

A validation workshop of the findings has been proposed for April and the final report will be submitted in May 2020. The report will provide the policymakers at MoGE with credible evidence on the effectiveness, efficiency, relevance, sustainability and equity aspects of the program in order to help them make an informed decision regarding further scale-up of the program. ■