Could students try harder on PISA?

Approximately every three years, when the results of the latest PISA test are released, media coverage is blanketed with league tables comparing the reading, mathematics and science achievement of participating countries.

Beyond these one-dimensional rankings, though, PISA provides a huge volume of nuanced information about 15-year-old students’ knowledge, skills and attitudes. This information has become increasingly important in education policy. In Australia, for example, results from PISA are used to measure progress towards the national goals for education, as outlined in The Alice Springs (Mparntwe) Education Declaration.

While the national significance of PISA in participating countries is high, the assessment itself is considered low-stakes for the participating schools and students. This is because, unlike other formal assessments, students’ results do not impact their grades and the data cannot be linked to the school (although schools do receive feedback on their students’ performance). And when the stakes are low, students may decide to moderate the amount of effort they invest.

To measure how much effort students put into PISA, at the end of the two-hour test, students are asked to rate their performance on an ‘effort thermometer’. The effort thermometer first guides students to think of a past situation of high personal importance in which they made a maximum effort. Students were then asked to use this experience to grade their effort in the PISA test on a scale ranging from ‘1’, minimum effort to ‘10’, maximum effort. Finally, students were asked to rate how much effort they would have invested if they knew their results would count towards their school marks.

In the most recent cycle of PISA, almost three quarters (73 per cent) of Australian students indicated that they would have invested more effort if the PISA test had counted towards their school marks. This was higher than the OECD average of 68 per cent but similar to New Zealand (73 per cent) and Singapore (74 per cent). This is quite interesting when you consider the strong performance of Singaporean students in the PISA reading literacy assessment. Singapore’s average score was 46 points higher than Australia’s, which is equivalent to more than a year of schooling.

Smaller proportions of students reporting they would have invested more effort were found in Korea (46 per cent), Chinese Taipei (45 per cent) and the participating Chinese provinces of Beijing, Shanghai, Jiangsu and Zhejiang (38 per cent). This corresponded with higher average reading literacy achievement compared to Australia in Korea and the four Chinese provinces. In Chinese Taipei, however, achievement was similar to Australia.

Within Australia, there were interesting differences between demographic groups. A larger proportion of female students, socioeconomically advantaged students, metropolitan students, non-Indigenous students and first-generation students indicated they would have invested more effort in the PISA test if it counted towards their school marks.

The next PISA study takes place in mid- to late-2022. Schools that are randomly selected to take part might like to consider how they can encourage students to give their best when completing the assessments. We can then see if any improvement in student effort corresponds with an increase in achievement.

Find out more:
Read the full analysis in Snapshots, Issue 16, October 2021 – How much effort are students putting into PISA? by Lisa De Bortoli.