PISA explained

The OECD will release the results from PISA (its Programme for International Student Assessment) 2018 tomorrow evening. This is the seventh time PISA has been administered and the results usually cause a barrage of attention from governments, and those interested in education, for years to come.

What many are wondering about the results tomorrow include

• where does Australia’s education system sit internationally?
• which countries are doing better than we are and which are doing worse?
• how are we doing internally – across states and territories, between girls and boys, or children from different socioeconomic and cultural backgrounds?

The last PISA 2015 report, published in 2016, showed Australia was behind countries such as Singapore, Canada and China in maths and science, and below Singapore, Canada, New Zealand and Ireland in reading.

The first PISA test was in 2000. The three assessment domains are rotated every three years, so one domain is the major focus (the major domain). A larger amount of the assessment time is devoted to this domain compared to the other two (the minor domains).

Reading was the major domain in 2000 and 2009, and again in PISA 2018. This means Australia will now have PISA reading results from three different time points. This allows us to investigate trends in average performance (up or down or flat) as well as look at reading in greater detail.

The last cycle showed Australia had slipped in each domain since 2012, which means we were getting worse at preparing students for the everyday challenges of adult life. All eyes with be on this report to see whether these declines have been arrested.

What is PISA?

PISA is a two-hour test to see how well students in secondary schools across all 36 OECD countries, and 43 other countries or economies, can apply reading, maths, science and other skills to real-life situations.

Rather than focusing on a particular grade or year level, PISA tests 15 year olds – or more specifically, students who are between 15 years and three months and 16 years and two months at the beginning of the testing period and are enrolled in an educational institution, either full-time or part-time.

This is because, in many of the countries that participated in PISA in the early years, students of this age are usually nearing the end of their compulsory schooling.

The assessment takes place every three years and students are tested in the three areas each cycle. Assessments areas such as financial and computer literacy, or collaborative problem-solving, change from cycle to cycle.

It would be hugely costly and time-consuming to test every 15 year old in the more than 70 participating countries. So a representative random sample of schools is drawn from all schools in each system being tested. Then, a representative sample of students is drawn from within each of those schools.

PISA was not designed to provide scores for individual students or schools – students don’t even complete the same test as the other students in the room. In PISA 2018, for example, there were more than 36 different test forms, covering different parts of the assessment.

In Australia, 740 schools and just over 14,200 students participated in PISA 2018.

What the test looks like

There is major difference between PISA and some other international student assessments such as the Trends in International Mathematics and Science Study (TIMSS). This focuses on how well students have learned the content of a defined curriculum.

PISA questions are designed to test students’ applied knowledge in reading, mathematics or science.

For instance, a science question (below) in the last cycle concerned bird migration.

Most migratory birds gather in one area and then migrate in large groups rather than individually. This behaviour is a result of evolution. Which of the following is the best scientific explanation for the evolution of this behaviour in most migratory birds?

• birds that migrated individually or in small groups were less likely to survive and have offspring

• birds that migrated individually or in small groups were more likely to find adequate food

• flying in large groups allowed other bird species to join the migration

• flying in large groups allowed each bird to have a better chance of finding a nesting site.

The next question asks students to identify a factor that might make the volunteers’ counts of migrating birds inaccurate, and explain how that will affect the count.

The type of testing has also changed in line with the way our societies have changed in the two decades PISA has been around. Students now take the assessments using computers, for instance.

The nature of reading has changed too, drastically. In the past, reading was mainly about extracting knowledge from linear texts in established sources. When students did not know the answer to a question, teachers could direct them to look in an encyclopaedia and the answer would be trustworthy.

Today, digital search engines provide students with millions of answers, and it is up to students to figure out what is accurate and what is misleading and potentially dangerous. PISA is now testing how students navigate multiple-source texts, deal with ambiguity, distinguish between fact, opinion and sensationalism, and triangulate different sources to construct knowledge.

Why we need PISA

There are many criticisms of PISA. From the fact the OECD allows some countries to test just regions – like B-S-J-Z (China) – to criticisms of the argument an increase in a country’s PISA scores will result in an increase in that country’s economic wealth.

Despite these criticisms, PISA is robust, and provides an idea of how countries are performing comparative to each other. It provides a birds-eye view of a country’s average student performance, or a state’s average student performance.

This allows us to look deeply and identify groups that might not be performing as well as we hope, or skill areas students aren’t grasping well.

And by the way, the answer to the bird question is option one.

Sue Thomson, Deputy CEO (Research), Australian Council for Educational Research and Kylie Hillman, Senior Research Fellow, Australian Council for Educational Research

This article is republished from The Conversation under a Creative Commons license. Read the original article.