Monitoring the impact of education investments in Timor-Leste

The Professional Learning and Mentoring Program (PLMP) is a joint project developed by the Australian Government Department of Foreign Affairs and Trade (DFAT) and the Timor-Leste Ministry of Education (MoE). Implemented in 2016, it was designed to build school leadership and teacher capacity, and to support the effective implementation of Timor-Leste’s new National Basic Education Curriculum.

DFAT commissioned the Australian Council for Educational Research (ACER) to undertake a multi-year study to investigate the effects of the PLMP on teacher knowledge, teaching practice and student learning, as part of the Teacher Development Multi-Year Study Series. Results from the first year of study in 2018 show a positive link between the PLMP and improved teaching quality and the effective implementation of the new curriculum.

The initial findings suggest:

• PLMP is effective in supporting improved teaching quality through building school leadership capacity
• PLMP supports the effective implementation of the National Basic Education Curriculum through facilitating peer learning and improving access to, and utilisation of, lesson plans and materials
• Student attendance and interest in lessons have improved in response to enhanced teacher confidence, preparation and classroom management skills learned through PLMP.

The connection between PLMP and improvements in student learning outcomes is unclear at this early stage, and will be investigated further in the subsequent years of this study. However, some initial indicative findings suggest:

• Overall scores on the Classroom Based Assessment (CBA) for students in Grade 2 and Grade 3 favour schools participating in the PLMP
• A higher percentage of Grade 2 and 3 students in PLMP-participating schools were able to respond to components on the Early Grade Reading Assessment (EGRA), compared to non-participating schools.

The PLMP aims to complement the Timor-Leste MoE’s teacher training and new curriculum material through strengthening school leadership, supporting continuous professional development of teachers, and supporting systems to monitor student learning outcomes and teacher performance. It includes these core components:

• Leaders of Learning Program: a series of training sessions aimed at strengthening school leader understanding of student-centred learning, and building school leadership capacity to support teachers to implement the curriculum
• Peer Learning Groups (PLGs): regular school-based meetings for teachers across a cluster of schools to learn from and support each other through sharing experiences and problem-solving together
• Local, national and international mentoring: ongoing support for school leaders to apply their learning from the leaders of learning program, and for teachers to implement the new curriculum
• Educational technology: access to information technology such as tablets with custom-built apps and a dashboard to enable efficient data collection, sharing and monitoring.

The three main questions guiding ACER’s research are:

• To what extent does the PLMP support improved teaching quality in Timor-Leste?
• To what extent does the PLMP support the effective implementation of Timor-Leste’s National Basic Education Curriculum?
• To what extent does teacher involvement in the PLMP lead to improved learning outcomes for Timor-Leste students?

The ACER study uses a mixed methods approach to data collection and analysis. Quantitative data was sourced from the CBA and EGRA data while qualitative data was collected across two participating municipalities through interviews with key stakeholders including PLMP mentors, school leaders, inspectors, municipal directors and teachers. Additionally, two PLG sessions were observed to gain insight into how these groups operate.

In addition to the findings outlined earlier, the study also identified a number of risks and opportunities for improvement, including:

• Program sustainability following the conclusion of the PLMP intervention is uncertain due to limited resourcing, capacity and motivation of school leaders and teachers to continue the activities
• School leadership transitions create a gap in institutional knowledge and capacity due to the absence of succession and handover plans.

The findings from this study are detailed further in the Timor-Leste interim report for the Teacher Development Multi-Year Study Series. Preparations for the second round of the study are currently underway, with the year 2 qualitative data collection planned to commence in May 2019.

The ACER team undertaking this research is led by Elizabeth Cassity, Jennie Chainey and Sheldon Rothman, with support from Debbie Wong and Hilary Hollingsworth.