Should we keep taking part in PISA?

Co-editor Kylie Hillman, who has worked with data from the Programme for International Student Assessment (PISA) with the Australian Council for Educational Research (ACER) for many years and who edited this special issue, responded by pointing out that poor results and declines in results were much more likely to be reported by the media than improved results – and at the same time, Australia’s results in other international assessments have contained several good news stories over the years.

Besides, Ms Hillman continued, the issues behind poor performance in reading, mathematics and science will not disappear just because we stop reporting them.

Her comment was echoed by Emma Medina, Senior Research Analyst with the New Zealand Ministry for Education. Her article with co-author Megan Chamberlain compared the reading performance of Australian and New Zealand students in PISA and the Progress in International Reading Literacy Study (PIRLS). As the New Zealand government’s PISA lead, Ms Medina says that results of large-scale student surveys like PISA push education policymakers to continue their work of improving teaching and learning.

Watch the webinar

What can two decades of data tell us?

With two decades of PISA results now available, Dr Lietz said secondary analysis of the data could bring valuable insights into underexplored areas that go beyond the league-table statistics often reported in the media. The special issue’s six articles illustrate how data can be examined in new ways, and authors from four articles joined the webinar to discuss their findings.

Dr Florence Gabriel’s article on maths anxiety, co-authored with her University of South Australia colleague Abhinava Barthakur and ACER’s Dr Sarah Buckley, looked at the results of more than 4000 students in PISA 2012. She said the study showed a strongly negative impact of maths anxiety on maths performance, and on other factors that affect maths performance. The result was a ‘domino effect’ in which any one factor can impact negatively upon others, leading to greater anxiety and lower maths performance. Her advice to teachers keen to break the cycle? Build students’ confidence by highlighting the tasks they can do and are good at before moving to the next challenge, Dr Gabriel said, and help them appreciate the control they have over their achievements in mathematics.

Dr John Ainley explained the genesis of his article with fellow ACER researchers Dr Dan Cloney and Jessica Thompson: a search for possible explanations for Australia’s declining performance. Their article examined the shifting distributions of 15-year-old students across year levels over time; in 2000, 18 per cent of 15-year-old students surveyed in PISA were in Year 11, compared with just 7 per cent in 2018. While their analysis found that changes in the year level distribution of students had only a small impact on performance at a national level, there were impacts in two states.  It is a fascinating example of how the data can be interrogated in new ways.

Can we effectively measure collaboration skills in an online assessment in which a student interacts with a computer? It’s an interesting question for post-pandemic education systems, said ACER’s Dr Claire Scoular. Her research with ACER colleagues Dr Dan Cloney and Dr Dara Ramalingam and University of Manchester’s Sofia Eleftheriadou mapped three assessments of collaborative problem-solving against a common framework, allowing for more valid and reliable comparisons of student achievement in this key capability.

Find out more: 

Dr Claire Scoular is presenting at our fully online Research Conference 2021. Register now!

The other two articles included in the AJE special issue also illustrate the insights that can be gained from a closer analysis of PISA data.  One article provides a particularly unique perspective; Australia is one of only two countries that link PISA data for 15-year-olds to later information about work and life from the Longitudinal Surveys of Australian Youth (LSAY), by relating teenagers’ science interest network characteristics to later science course uptake (the University of Amsterdam’s Maien S. M. Sachisthal, Brenda R. J. Jansen, Jonas Dalege and Maartje E. J. Raijmakers). The other article explores changes in attitudes of 15-year-old Australian students towards reading, mathematics, and science and their impact on student performance over the 20 years that PISA has been conducted (I Gusti Darmawan, the University of Adelaide). 

Thanks to SAGE Publications for making all articles in the AJE’s 20 Years of PISA special issue open access throughout November.

Read the articles.