Measuring the quality of early childhood education and care

Australian early childhood education and care recently entered an era of reform. In January 2012 new legislation covering long day care, preschool, family day care and outside school hours care services came into effect, introducing a National Quality Standard for Early Childhood Education and Care and School Age Care (NQS), against which services would be assessed.

Since 2010, ACER has worked with the Australian Government on the development of the processes used to assess whether a service was meeting the standards covered in the NQS and to assign a set of ratings to the service. For the last phase of this work, during 2012, ACER analysed the assessment and rating data collected on 491 services across Australia to determine if these processes were applied appropriately and consistently.

The National Quality Standard and the assessment and rating process

The NQS comprises seven quality areas that research has shown indicate a quality environment for young children’s learning and development:

1. Educational program and practice
2. Children’s health and safety
3. Physical environment
4. Staffing arrangements
5. Relationships with children
6. Collaborative partnerships with families and communities
7. Leadership and service management

Each quality area comprises two or three standards, and each standard comprises elements that describe the outcomes that contribute to the standard being achieved. There are 18 standards and 58 elements in total.

The assessment and rating process involves Authorised Officers in each State or Territory visiting a service to review its documentation, observe its practice with children and families, and undertake discussions with educators and staff, to determine whether the service meets each of the elements in the NQS.

Based on the extent that the elements have been met, each standard is assigned one of three rating levels: Working Towards NQS, Meeting NQS or Exceeding NQS. Services must meet all elements within a standard in order to meet that standard.

After all standards in a quality area have been rated, reviewers assign a rating on that quality area. All standards within a quality area must be met in order to meet the NQS for that quality area. In addition to the three rating levels used for standards, a fourth rating, Significant Improvement Required, may be assigned if the service does not meet that quality area or a relevant regulation in such a way that there is an unacceptable risk to the safety, health or wellbeing of children in the service.

Quality area ratings

During the evaluation period, State and Territory Authorised Officers assessed a total of 491 services, of which 224 were long day care centres, 137 were preschools, 81 were outside school hours care centres and 49 were family day care services. The ratings received by these services are not necessarily representative of the entire population.

Table 1 shows the ratings in the seven quality areas for all 491 services that were assessed.

Table 1: Distribution of ratings in each quality area (%)

The quality area of Educational program and practice relates to implementation of the relevant approved learning framework. For early childhood education and care this is Belonging, Being and Becoming, introduced in 2009, and for school age care it is My Time, Our Place, introduced in August 2011. This quality area had the lowest percentages of services that received ratings of Exceeding NQS (24 per cent) and Meeting NQS (33 per cent). Consequently, this quality area had the highest percentage of services rated at Working Towards NQS (43 per cent).

For many family day care and outside school hours care services, the learning frameworks are part of the new emphasis on quality standards. These services had not previously used a formal educational framework, so the majority of these services were rated at Working Towards NQS.

The quality area of Staffing arrangements had the highest percentage of services rated at Meeting or Exceeding NQS (88 per cent), while Relationships with children had the highest percentage of services rated at Exceeding NQS (39 per cent).

Overall ratings

Based on their quality area ratings, each service is then assigned an overall rating. Of the 491 services assessed, almost half (48 per cent) were rated as meeting or exceeding the NQS. Only two services—one preschool and one family day care—were rated as Significant Improvement Required, representing 0.4 per cent of all services assessed and rated as part of the evaluation.

Results varied by service type, as shown in Figure 1. One-half of preschools, which have had much more experience with more formal education programs, were rated at Exceeding NQS and another quarter of preschools were rated at Meeting NQS. By contrast, nearly three-quarters of outside school hours care services, which have not had a learning framework until 2011, were rated at Working Towards NQS.

Figure 1: Distribution of ratings by service type (%)

Because of differences in the number of services assessed, the evaluation was unable to determine why there are differences between the ratings achieved by the service types. There is, however, no evidence of bias against any service type or location, nor any bias as a result of variations in the judgements of Authorised Officers.

Participant feedback

Questionnaires were used to obtain comments from services and Authorised Officers relating to the assessment and rating process. More than 80 per cent of reviewers who responded to the survey said they had no difficulty collecting adequate evidence to assign a rating for a quality area, standard or element.

Regardless of the outcome of the assessment and rating process, services were generally satisfied that it was conducted in an appropriate manner, that there was adequate contact with the regulatory authority before and during the visit, and that the process was fair. In general, 90 per cent of services were satisfied with most aspects of the process. More than three-quarters of services were satisfied that they were fairly and accurately assessed and rated. However, there were some concerns expressed by services about procedures for the visit.

The main concern expressed by services related to how the visit was conducted, particularly whether the balance of ‘observe, document, discuss’ is correct. While services cannot compare the balance between visits, some suggested that there is too little observation and others suggested that there is too little discussion.

Nevertheless, 81 per cent of services responding to the online survey said that their experience of the assessment and rating process was positive, with more than one-third of these services calling it ‘very positive’.

Read the full report:
Evaluation of the assessment and rating process under the National Quality Standard for Early Childhood Education and Care and School Age Care, by a team of ACER researchers led by Sheldon Rothman, is available from <research.acer.edu.au/early_childhood_misc/8/>