Measuring the engagement of online learners

In 2011, the Australian Council for Educational Research (ACER) was part of a consortium that was commissioned to develop a national survey to measure the quality of university students’ experience, to help ensure ongoing improvement in the quality of learning and teaching in Australian universities.

The annual survey, now known as the Student Experience Survey, focuses on aspects of the student experience that are measurable, are linked with learning and development outcomes, and for which higher education institutions can reasonably be assumed to have responsibility.

These aspects, including core areas of learner engagement, teaching and support, and educational development, are reported on the national Quality Indicators for Learning and Teaching website to help people make informed decisions about future study.

Since the survey was developed, the sector has experienced growth in the number of external mode students – those studying wholly online, by distance or off-campus.

According to the Commonwealth Department of Education and Training, results from the Student Experience Survey show consistent, significantly lower learner engagement levels for external students compared with internal students, in excess of the variation which would be expected between teaching modes.

Suspecting that the current learner engagement scale may have an inherent bias against external study modes, and may not fully capture important aspects of online teaching practice, the Department has therefore contracted ACER to review the Learner Engagement Scale of the Student Experience Survey and develop an alternate scale which is suitable for external mode students.

ACER’s review of the current Learner Engagement Scale involves stakeholder consultation, a literature review, statistical analysis of the current scale, and a review of current practice and trends in the delivery of external mode coursework in higher education.

An interim report on the findings of this review will include recommendations for the design and piloting of the new scale for external students. The design phase will include the development of an assessment framework and further stakeholder consultation, as well as focus group testing the scale with approximately eight external students.

The new scale will then be piloted among a sample of around 5000 external students and the results analysed for statistical validity. The new scale will also be evaluated against the other scales in the Student Experience Survey and analysed by respondent demographics before ACER delivers its final report and recommendations in mid-2017.

Have your say:
If you would like to contribute to the consultation process for this review, please email highereducation@acer.edu.au for more information.