Creating a global measure of digital and ICT literacy

Being competent in digital literacy and information and computer technology (ICT) has increasingly become important for full participation in a knowledge economy and an information society. Consequently, interest at the national and international level has emerged to explore the best ways of measuring the extent to which competence in this area is being achieved, and how equitable access to this knowledge is within and across countries.

As part of those explorations, ACER was commissioned by UNESCO to review possible future approaches to the measurement of digital and ICT literacy skills across a wide range of countries.

The findings from that review, published as a background paper for UNESCO’s 2016 Global Education Monitoring Report, reveal there are many challenges in developing comparable measures of digital and ICT literacy skills. While some of these challenges are similar to those in cross-national assessments of student learning of other subject areas, there are also those specific to measuring digital and ICT literacy. Challenges can be classified with regard to over-time comparisons, cross-country comparisons and cross-group comparisons.

Over-time comparisons

An assessment of digital and ICT literacy skills needs to react to changes in the patterns of use of ICT. Past decades have shown rapid changes in ICT-related technologies which have impacted on the way ICT is used and applied.

One possibility to deal with changes in the nature of ICT use is choosing a modular design where some test modules are replaced with new ones reflecting more recent developments, while retaining sufficient material from previous tests to enable comparability over time.

To assess digital and ICT skills, it is necessary to create authentic virtual environments, in which test-takers have to show their competencies regarding different applications of ICT. The assessment should therefore be administered on a delivery platform that is sufficiently flexible to cater for changing conditions in ICT use, both in terms of the newer aspects of ICT use as well as its use for delivery to test-takers, such as at schools where equipment may change over time.

Cross-country and cross-group comparisons

Assessing learning outcomes across a wide range of contexts is an ongoing challenge for all international studies.

Any performance-based assessment of digital and ICT literacy needs to be done on computers or equivalent electronic devices in order to measure skills in an authentic environment. The mode of a computer-based delivery must be chosen so that it can be applied across all countries participating in the assessment.

As in other assessments, measuring digital and ICT literacy should be done in a way that does not advantage or disadvantage certain sub-groups. Responses to specific test item should only be influenced by the ability of test-takers and not by their gender or socio-economic background.

While in many developed countries vast majorities of students have access to ICT at home as well as at school, and are expected to have developed familiarity with it, young people in developing, low-income countries may have little experience with and/or limited access to ICT at home or at school.

Given the diversity of familiarity with ICT across highly diverse population sub-groups, as well as across national boundaries, a global measure of digital and ICT skills would have to cover a wide range of competencies, from very basic skill levels to sophisticated knowledge about complex applications.

One way to address this is to allow countries to select from different modules or components that assess at appropriate levels for their populations but which are linked to measure a common construct that underpins the development of all test content.

Collecting meaningful data

In order to properly inform education policy, it is necessary to collect contextual data in addition to data about digital and ICT literacy skills. Contextual data provide information about factors influencing variation in skills and describe the contexts in which they are learned. It will be important to collect contextual information at the following levels:

The individual learner
The school/classroom environment
The home and peer environment
The contexts of the wider community.

Contextual data would be used to analyse within and across countries the extent to which socioeconomic factors, and school and home contexts influence variance in a person’s digital and ICT literacy.

Read the full report:
A global measure of digital and ICT literacy skills, by John Ainley, Wolfram Schulz and Julian Fraillon, (ACER, 2016). Paper commissioned for the UNESCO Global Education Monitoring Report 2016, Education for people and planet: Creating sustainable futures for all.

Creating a global measure of digital and ICT literacy