Monday, 17 Oct 2022
In a recent webinar, we answered your questions about the updates to the PAT norms and how they will impact your reports. Clare Ozolins, ACER Psychometrics and Methodology Senior Research Fellow, spoke in depth with Daniel O’Loughlin, School Engagement Project Director, about understanding PAT norms and the updates published in October 2022.
Why compare students to PAT norms?
The norms provide a comparison set of data to measure where students’ achievement sits relative to the rest of the Australian student population. However, it is not recommended they be used to set learning targets for student progress or track learning needs and growth. ‘[The norms are] a distribution of achievement in the population, there's also a distribution of achievement in your classroom and you can compare that distribution in your classroom with what you see in the population,’ Ms Ozolins explained.
How were PAT norm data collected for the 2022 update?
The table summarises differences in methodology used to collect data for the 2022 update.
| 2016 update | 2022 update |
| Used results data collected between 2012 and 2014. | Used results data collected in 2019. |
| Sampled assessments from schools and students from almost every state and sector across Australia, combining the three-year span. | A significant increase in schools and students sitting PAT assessments meant a single year provided sufficient data. |
| Year level information optional, so assumptions were made, e.g. students taking test form 3 in Maths or Reading were in Year 3, as long as they were of the typical age range for Year 3 students. | Year level information is now recorded for every student. All appropriate results were included for the given year level, not just results from the test form aligned to the students’ year. |
| Used PAT Maths Plus assessments and PAT Reading Comprehension 4th Edition assessments. | Used PAT Maths 4th Edition assessments and PAT Reading 5th Edition assessments. |
‘We selected 2019 because we wanted data that were not affected by the impacts of changing school and learning arrangements due to COVID-19, most notably in Victoria, but right around the country,’ explained Ms Ozolins.
Samples continued to be limited to sittings that took place in October or November and assessments were limited to those that could be matched with school information in the ACER sampling frame. In both updates, samples were also restricted to students who were within the typical age range for each year level. You can read more about the methodology here.
How have the 2022 update norms been developed?
‘Once we have a sample for each year level, we compare those demographic distributions of schools and students in the sample with the national Australian population distribution. [We do that] to weight the data so that it reflects the demographic distributions in the population more closely,’ Ms Ozolins explained. Six demographic categories are used to create weight classes:
- jurisdiction – six states and two territories
- management – government and non-government schools
- sector – further breakdown to government, catholic and independent sectors
- school location – metropolitan and non-metropolitan
- school socioeconomic status – quintiles derived from the SEIFA Index or the ABS Education and Occupation Index
- gender distribution at the individual case level
The mean scale score for the year level and the standard deviation from the weighted data is then established. From there, assuming the distribution is normal, the scale score equivalents for each percentile rank can be estimated.
How do the new norms impact assessment and assessing?
‘The new norms shouldn't change anything for you other than the comparison of your student performance with the population performance at a particular year level,’ Ms Ozolins explained.
If comparing previously collected data using the 2016 update norms, the percentiles and stanine ranks may appear different, but the scale score achievement of students and what that means is exactly the same.
If you would like to hear more of the questions answered in the webinar, you can view a recording in the Help section of your online school account, or by emailing school.engagement@acer.org.