The value of international assessments of adults’ foundation skills

The value of international assessments of adults’ foundation skills

Educators and others in the vocational, adult and workplace education (VAWE) sector can learn plenty from empirically based research endeavours like the Adult Literacy and Life Skills (ALLS) survey, conducted in 2006, and the Program for the International Assessment of Adult Competencies (PIAAC) survey, conducted in 2011.

Literacy and numeracy, and the ALLS and PIAAC surveys

The value of the ALLS and PIAAC surveys lies not only in the evidence they deliver, but also in the way they describe what we mean by literacy and numeracy. Sitting behind the surveys are theoretical frameworks that attempt to describe what literacy and numeracy in the 21st century incorporates, and how this might be assessed in an international assessment.

There are extensive processes that sit behind international assessments that help to guarantee the quality of assessment that is developed. A major aspect is that test development and writing does not proceed without an agreed conceptual framework. Such frameworks are developed by teams of international experts from different countries with different cultural and language backgrounds.

The development of the assessment follows a comprehensive process to ensure the quality and validity of the test items being developed. Some of these processes include utilising a team approach to test development, comprehensive critiquing and reviewing of items by other test developers, feedback from participating countries and the use of focus groups and pilots with potential test-takers. Another crucial stage is trialling which is done with a sample of the target population in each participating country. The trial data is then collected and analysed psychometrically and, from these detailed analyses, ‘misbehaving’ items are rejected on a number of levels including for reliability, fairness and validity. Only successful items are used in the final assessment instrument.

In both ALLS and PIAAC, test-takers were also asked almost 300 background questions which were then used in analysis and research based on the results.

What do the results suggest?

The results from ALLS and PIAAC do not indicate, and do not claim to indicate, rates of illiteracy or innumeracy, but they do indicate that a significant number of Australians aged from 15 to 74 years do not have access to sufficient literacy and numeracy skills to be able to cope equitably with life in the 21st century.

Educators in the VAWE sector are aware that low literacy and numeracy skills can prevent people from being empowered and making considered decisions, whether they be on-the-spot decisions at a workplace or when out shopping, following written instructions about a medical or health matter, making decisions about financial matters and the like. Put simply, literacy and numeracy skills provide various pathways and options for success in life. Educators in the VAWE sector believe all people should have that access and choice, which is why many work in the sector.

How are the results influencing policy?

The evidence from ALLS and earlier similar studies indicates there is individual, social and economic value in supporting the improvement of the literacy and numeracy skills of individuals.

In Australia, evidence from the ALLS survey has seen the beginning of a set of significant responses and investments in adult literacy and numeracy. Both government and industry have begun to argue that the core skills of literacy and numeracy need to be supported on a national basis. In the last few years a considerable number of reports and initiatives have been published and introduced which contribute to doing this. These include:

• the Industry Skills Councils report, No More Excuses: An industry response to the language, literacy and numeracy challenge
•  research by the Australian Industry Group, reported in the National Workforce Literacy Project: Report on employers views on workplace literacy and numeracy skills
• the development of two new higher level language, literacy and numeracy qualifications for educators in the VAWE sector (Graduate Diploma of Adult Language, Literacy and Numeracy Practice and Graduate Diploma of Adult Language, Literacy and Numeracy Leadership), and
• the development of a new Foundation Skills Training Package.

How are the results influencing teaching and learning?

There is also much to be learned from international assessments in terms of the teaching and learning of literacy and numeracy skills. The meta-analyses of existing research and related literacy and numeracy issues to be found in international assessments can be a useful source of international research and theory, while the empirical data indicate much about what adults can and cannot do. International assessments like ALLS and PIAAC also provide schemes that describe factors affecting task, text and item difficulty that are of particular value for teachers.

There is some cause for optimism but still cause for concern. Numeracy remains the poor cousin in literacy and numeracy teaching and learning in the VAWE sector, despite research that indicates that numeracy may have a larger impact than literacy on individuals, society and the economy.

There is still much to be done, especially in relation to supporting and upskilling the language, literacy and numeracy skills and knowledge of educators in the VAWE sector, but at least the evidence from international assessments is prompting the kinds of support the sector deserves and needs.

A longer version of this article was published in Fine Print, the journal of the Victorian Adult Literacy and Basic Education Council.