Field Notes episode 4: Making sense of data at Mansfield State High School

Friday, 6 Dec 2024

With valuable and practical takeaways on collating useful data, building a picture of the whole student and supporting teachers to access evidence, Jeff's and Mark’s discussion is a candid look at the data environment within schools.   

We hope you enjoy this latest episode of Field Notes, a podcast that aims to share honest conversations with leading educational practitioners about how they use evidence to improve learning outcomes in a range of school settings.

You can stream the episode below, through Apple Podcasts, Spotify or Amazon Music, or keep reading for the transcript. 

 

 

Transcript

Alex: In the spirit of reconciliation, ACER acknowledges the Traditional Custodians of country throughout Australia and their connections to land, sea and community. We pay our respect to their Elders past and present, and extend that respect to all Aboriginal and Torres Strait Islander peoples today. ACER acknowledges the Aboriginal and Torres Strait Islander people who continue to contribute to our work to improve learning, education and research.   

In today's episode, Marc Kralj speaks with Jeff Broadway and Mark Redhead from Mansfield State High School in Queensland about how best to manage, plan and implement data in schools.

Now, in the school context, data is everywhere. And educators have an enormous amount of data that they collect every day and use to inform their decisions. This might include the simplest piece of data, like student attendance or informal observations about student behaviour. It could be academic results. It could be information relating to students who speak English as an additional language or students that come from a First Nations background. The data available in one classroom is endless.

The challenge for educators is to use it to find a story about what it all means in their school's own context. And that's a process that involves collecting that information, making those observations and crucially sharing those findings with colleagues and peers.

In this episode, you'll hear reference to PAT Maths and PAT Reading. PAT stands for Progressive Achievement, a form of assessment developed and provided by ACER that is available in a variety of learning areas including mathematics and reading.

With that in mind, let's join that conversation between Marc and 2 very experienced educators right now.

-

Marc: So with that, I'd like to introduce you to both of these gentlemen and they told me today that they're not going to answer questions individually, but they're going to have a discussion together, which I like that.

So I'm going to let them argue, first of all, over who's going to introduce themselves first, gentlemen.

Jeff: It'll start with me. My name is Jeff Broadway. I'm the Associate Principal at Mansfield State High School. I've been in this particular role for 2 years now and previous to that, I've been in administration for about 13 years at the Deputy Principal level. In the context of Mansfield State High School: high performing school, high expectations around academic performance. And there's been some strategic decisions made around how our particular school is structured, around 2 main concepts of learning and teaching.

With learning being the emphasis on what we do here at Mansfield, being learning is our work as part of the language of learning that we have, as opposed to the traditional way that's often presented in terms of papers and research is around teaching and learning. Learning is our work, so therefore, it's the priority around that.

My role in students and performance is around looking at student data sets that obviously help track their academic progress throughout their journey here at Mansfield State High School.

And the other facet of that is that my role is that I'm responsible for looking after the student services team, which is the deputy principals, the guidance officers, our heads of year, in order to be able to track those other subsets of data that are around about attendance, well-being and the different aspects of that role that we need to kind of focus on around making sure we're maximizing outcomes for students.

Marc: Thanks, Jeff. That takes us to Mark.

Mark: Thank you, Marc. My name is Mark Redhead. I was a school leader in several places and jurisdictions previous to coming to Mansfield as well. Here, I have been in the role of students and performance head of department for the past, I think it's getting close to about 11 years now. And I've enjoyed this role.

Previous roles included work with Northern Territory Department of Education and the University of Charles Darwin University, and also working for Central Office here in Queensland, where I was in strategic HR.

But the work that I'm doing these days, I'm finding particularly interesting because I can see very explicit and direct links to the outcomes of improved learning conditions, whether it's for our most struggling learners or those who are needing extension.

I can see some really strong links between our focus on data and how we use it, as Jeff said, to track student performance, but also to support them and help them do even better.

Marc: Thank you, Mark. To give some context, gentlemen, could you just give us an outline about the size of Mansfield State High School? I think context for listeners today in terms of the school itself, but I'm talking about size. Could you just give us a bit of an outline about that level of capacity?

Jeff: Mansfield State High School is an urban high school. Its current enrolment as of today, when we're recording this particular podcast, is 3,680 students. That in itself lends itself to various challenges we have around how we manage our cohorts. They're averaging at the moment anywhere between over 600 for most of them, except for grade 12. So there's still growth here at Mansfield for at least another 2 years. And then what we will then have is consistent cohorts of around about 630 students at our school. It’s a multicultural school, we have 75 different cultures represented at the school. It's a very diverse community that we operate with, and we have a teaching staff of about 253 and an ancillary staff of about 150.

So in itself, we run a small little city here at the school. One of the things, as I talked about in my intro, was that I've been at other schools for a while in my deputy principal role, I'm a relatively new to Mansfield. Despite the constraints around having a very, very large school, the work that we actually do here, there's a lot of really good things that occurred that makes that big school feel small. And that what we create here at Mansfield is focused on the learning and the teaching that happens here. But it's also too about creating some of that wellbeing space around being a sense of belonging at our school. As a newly arrived person, that's very evident at this school.

Mark: There is a great deal of pride in the school, the academic body and also the students themselves, of recognising that the relationships that we have with each other are the most important aspect of our work. This is a message that comes through the executive, but all the way through the entire place. It's absolutely, I think, cornerstone for the success of the place. As Jeff said, the know your learner approach is well-supported when we focus on relationships.

Marc: Thank you for that. I think that's important because no matter what size school we have, I think we try to endeavour to build those relationships, as you say, Marc, but also don't be too afraid by knowing that we work in large schools, or we work in complex and diverse schools as we do.

I'd like to begin with asking you this. We have data sets, we've talked about some data management planning, but what data sets do you currently use in the school? And this is in determining whether it's summative, formative, diagnostic. What are the sets that you currently use that are specific to Mansfield State High School?

Mark: So like all schools, our focus is on learning and teaching. And so the key data we focus on are the metrics of student performance, what we call their levels of achievement, so their A to E results in classes. But also the NAPLAN data, when that comes through for our juniors, is particularly useful.

Other information comes directly from the diagnostic tests of PAT for PAT-R and PAT-M, that you'd be very closely familiar with. And we use these 3 data sets to create periodic data called our ‘tri-data.’ So, we get those 3 together. We use those in a way that gives a very comprehensive picture on the current performance of a student but also when we compare that to previous reports, we can then start to get some sort of a longitudinal bead on those kids.

So we're able to see who's traveling particularly well, who is traveling where we expect them to be at, and also who may have fallen through the gaps, who needs a little bit more support. I think what's really important about the data that we produce here is that whilst we are a absolutely aspirant school community, so that the results are usually very, very high, so we certainly lead Queensland in most of the metrics that we use, and it is clear that our students are -

Jeff: We're comparable to our like schools. Obviously, our context, we talked about size and the type of students that we have. We hold our own in terms of those benchmarks that other schools are also using to indicate success and how they would support their students.

Mark: Yeah, so yeah, we have very healthy results in the student body, but we also are still cognisant that there are a number of kids who fall through those achievement gaps. So they are also a part of our interest and our concern in our datasets.

The datasets, the ‘tri-data’ reports, rather, are so useful to our staff because they are the starting point. Perhaps the most powerful use of them is not so much by the individual teacher, but when we get a collaborative process that Jeff manages through the heads of the year.

Jeff: As we talked about, the ‘Know Your Learner’ is one of our strategic priorities and is a consistent language that's used here. Obviously, those familiar relationships that you develop with students as you get to know them throughout the year, they develop quite strongly.

But that initial outset for the first time teachers see our students in that first week, there's an expectation here at our school that our teachers use the data sets that Marc's described through the triangulation of the NAPLAN, previous reporting data, whatever existing PAT data sets that we've got. To paint that picture around which particular students are going to be sitting in, for example, English 7A.

First time we see those students, what's the mix? Who are First Nation students? Who are ELD students? Who have current PLPs or up here, personalised learning plans in Queensland?

For us, it's really important that from day one, from when we start our school year, the conversations that I, in particular, are having with our staff is around the work that we do in terms of making success for our students and giving them the best opportunities for successful outcomes, is the fact that you are working and making the effort to look at the data to determine the type of learner or potentially the type of student that you've got in your class.

And that's very important work that exists here. And this is the work that's been going on for a number of years. That change around that building of relationships and understanding the students that sit in classes has been a very, very important focus in terms of how we best support students and also to around, you know, which particular data sets are important to give that quick access for our staff around who they're focusing their attention on.

Marc: It's really great to hear a couple of things, one is, particularly, diagnosing, analysing the strengths, weaknesses, and as you put it, what are the gaps?

Because as our students come in from different levels or different feeder schools or coming into an environment, we would hope that all students coming into Year 7, for example, were working at Year 7, but they're not.

We know that through analysing and diagnosing and looking at the data in so many ways, that we know our students are the same age and the same year level at very different points. So when you talk about the beginning of the year, that week zero and thinking teachers have a very good idea already what the data is telling them for those next steps in teaching and learning.

Here's my question to both of you. When we talk about gathering that information, more specifically, how do you support those teachers? What are the mechanisms that are put in place to say to teachers, this is the information, this is how you read the data, this is how you read the assessments.

What are those things that have been put in place to best support teachers understanding the data sets that are put in front of them, that are interrogated to be able to analyse and interpret for next steps?

Mark: Look, I think the most important thing is to have a centralised storage of all of your data. And we're very fortunate to use Teams and SharePoint as part of the Microsoft architecture of how we have all of our important documents. So, having really simple, recognisable data sets that are then seen once a semester is probably the first step.

When it becomes possible to identify those students, so you've got this instant gauge through ‘tri-data’ to work out, well, who are my target students, whether that be for students I'm looking to extend and develop or students who perhaps are needing extra support, you're then able to call upon the resources that are also put aside to those data sets.

My work is involved in meeting with staff and so constantly, I guess, beating the drum to make sure they recognise that these tools are there, and to create discrete how-to guides. Most recently, these have been video, and you have to gauge, because for some staff, they will prefer a PDF to read, you know, rather than a comprehensive five-minute video. But for as many people who have said, I really appreciate a simple document that I can read at my leisure, there are still others who like to follow along with the video.

So you need to take the existing how-to materials, whichever it is, I mean, we probably spend about a third of our time in this training environment, particularly with things as like PAT and NAPLAN, and showing staff, first of all, where the data is, how to use it, what it means. That in itself can be a particularly important bit of work.

Most recently, our focus has been on making sure that our staff appreciate what a scale score is for the PAT tests. And that simple element, that one number, which then opens up a world of possibility for the individual teacher, whether it's comparing kids or comparing cohorts or comparing the whole school, cannot be assumed that they will get it straight away.

So I make the point that really clear and deliberate, but also I think one of the things I appreciate about Jeff's wide experience here is constantly saying we need to keep this simple and accessible. So going back to the question, it is about making sure that it's easy to find, it doesn't change too much, like there isn't a new version or a new representation of how the data looks, and then also really clear how to guides on how to use it.

Jeff: The other thing around the mechanism around what we have in place at this school is the fact that the key component is actually allocating specific time to discuss data.

So we start our year as cramped a space as that can be for many schools and the constraints are around departmental compliance and your own kind of objectives, the new work you want to get across. We have dedicated time for our staff to meet with our leaders, our cohort leaders of deputies and heads of year, and we work through how to read our ‘tri-data’ sets.

So collectively, we will have our cohorts, our 7, 8, and 9 cohorts. We primarily use the data sets through the class dashboard kind of work that we do. It is around looking at that, having discussions.

The other part of that is it doesn't just stop around the student free day. We have also made some decisions, very difficult ones in terms of getting our core teachers in year 7, 8, and 9 opportunities throughout the year to actually have collaborative discussions around what they are facing with those common students, especially in the core areas around the science, maths and humanities.

Again, we value the work that discussion of data can do. It's one of those things that is not always loved by staff. We understand that. Mark's point about making it easy, showing how to access and creating that sense of purpose, that it's a useful tool as opposed to something to be feared, and providing the time for staff to actually interrogate that data.

Because we will talk about any kind of news story talks around about teachers' time pool. So anything we're asking them to do to take some away from that core business, we have to be able to simplify it, and we have to be able to give them the time and the space to be able to use it and have some of those discussions.

Marc: I hear consistency in your approach, how do we consistently make this operable for teachers so that they're not wasting or losing their time. I'm hearing we need to continue to learn, no matter what entry point we're at in terms of understanding, whether it's assessment data, whether it's curriculum, whether it's pedagogy, whether it's practices in the classroom or resourcing in the classroom.

And I'm also hearing that we were thinking about all our staff, and that's all staff because all staff, whether ancillary, whether they're teaching in the classroom, they are looking at some form of data set in their role.

-

Marc: Mark, you referred to the kinds of data sets. A lot of it's academic. We talk about the A to C to E. We talked about PAT-M, PAT-R. We talked about NAPLAN.

But that other evidence that you collect along the way to help identify student pathways, student interests, student well-being. I'd like you to probably just touch on some of those areas that you emphasise, because I often see them as parallels.

And I often see sometimes wellbeing and things such as student pathways and their academic as not being in different directions, but sort of moving in the same way, but maybe not at the same rate.

Mark: We have a key strategy here in the school called ‘My Path’. And this is intimately connected with student data, but then also the investigation from individual teachers to talk with a selected group of students about them.

And so it has 2 iterations. The first one is a period of time where students really focus on the data and start to self-reflect using metacognition to look at the engagement continuum and to really start to paint a picture for themselves about themselves as learners and start to ask the question, why do I excel in this particular area and what are some of the things that I might be able to export from that area into those subjects or topics that I'm not so good at?

So we put that challenge squarely on them with a lot of support from our heads of year and our deputies.

And then we get them to have a meeting with a coach. Some schools call this an academic coach. We call it the ‘My Path’ coach expressly because it isn't just about their academics. It's also about them as a 100% full person. And so the wellbeing factor really comes into that.

And it's enormously positive. In some of our surveys from staff responses, we've actually had some of them saying, can we extend the time? I mean, it's one of the difficult challenges in any school is just there's just no time.

But staff are actually seeing these, again, these relationships being built between students and staff members is key to positive outcomes for kids, whether that's academic or their wellbeing.

Then later on in the year, we actually have the formal introduction of this ‘My Path’ process with mum and dad and the student and their ‘My Path’ coach.

And so again, whilst in some cases it is certainly about subject selection, it is a really important part of our year 10 set plan, but it also involves your career aspiration, the features of your life and your habits, which support and contribute to learning, and perhaps identifying those sorts of behaviours that are not going to be very helpful if you're planning to do really well at school.

You know, in really general terms, this is a process that Jeff and I have worked really, really carefully to simplify, but also to make sure that it is really fiercely useful for the students and their families.

And in our second iteration together, we're finding that whilst there's still room for improvement, it is a very positive program that is well received.

Jeff: And it's a refinement in terms of that ‘My Path’ process. It's something that initially started in that senior space at our school, just to kind of give that student voice around, you know, what they are doing with their learning, how are they engaging?

And we reference in terms of the language, the commonality of language that we're using around student engagement in particular. It's a language not only for our staff to use, but our students and our parents. So it's the work that Amy Berry has researched and produced. And it fits our mantra really nicely for what we focus on here.

In terms of other data sets that we use, not just academic, as we know, there are other things that exist within our school. Again, our structure here is that we have student services meetings that are fortnightly, that is for each and every year level here.

At those particular meetings, we bring in a guidance officer, we bring in a head of year, we bring in the differentiated learning head of department, we have a deputy, we have myself attend every single meeting.

And what we do at those particular junctures is that is where we're looking at our behaviour data that comes out of our learning platform called One School here in Queensland. So we're looking at what is our behaviour doing?

We have a refined, quite a sophisticated attendance tracking spreadsheet that basically does 2-week increments around students' attendance. So, we can track in 2-week cycles, whether they're going up, going down. What that allows us to do is to actually get some interventions in place early, rather than suddenly numbers crash and then we're being reactive around some of that data.

So I suppose in terms of the traditional work that gets done in all schools around how they access the information. Obviously for us in the other sets of data really come down to attendance, engagement, behaviour, and all of those referral systems that we have in place that just feed into those regular student services meetings.

And we're fortunate here that we can resource that. And that's something that I think has been very keenly prioritised because hand-in-hand, we know if we're going to produce good outcomes at the end for students, the wellbeing aspect, the case management, the wraparound support that combines with that, that's going to support students in the best way possible to achieve their potential.

Marc: Mark, Jeff, wow. And I say wow simply because I'm thinking 3,600 students plus to be able to manage and monitor and track and work with students and have a process and a procedure in place that is able to enable those conversations and obviously connect with the parents, the caregivers of students across the school.

I'd like to finish with these things. I'm going to mention the word success. What has been a success that's really worked for you when you think about this deluge of data and information that you've got?

Mark: I'll go straight to a feature of a PAT-M report. One of our key elements of the success of the school is we're well known in the state as being a leader in the mathematics department. Our student results in math are extraordinarily high.

But when you are a student who's not good at math in that environment, this can be incredibly intimidating. So one of the wonderful features of the Data Explorer for ACER’s PAT platform in OARS is that you can produce a strand report that shows after a student has done their PAT-M test, where their strengths and weaknesses lie.

And one of the features of this sub-strand report is that I'm yet to see any report that I can generate from any low-performing student that doesn't show an area of success and growth.

So what you're able to do for that kid in a really granular sense and in a really successful sense is put a bit of really hard evidence and show them, here's a report about you. It shows me that you're perhaps not very strong in these areas of maths, but look at this over here.

And the way that that student face lights up, when I think about success, there's so many layers of success in our school, but a kid who has a bit of self-belief, whereas before they had none, is a very, very precious success.

Marc: You want to celebrate that, don't you? That's a great thing for a teacher to be able to see after they've spent that time understanding the starting point of that student and what they've done to impact their learning and those outcomes.

Jeff, you're about to say something.

Jeff: I would think that coming in from a school that didn't operate at the same level, totally different community, different contexts in terms of the outcomes that were produced, great school, hardworking staff, awesome kids, but just totally different factors that contributed to the way that school operated.

This school, I think the success is here, that you just have investment from our community here. If you talk to our students, when you look at our school opinion survey, our kids are happy here, our staff are happy here, our parents are happy with the school. We have a really nice community and people are invested in our school.

I think if you talked about to staff who've been here longer than 10 years, they would be able to say that what has been nice and successful around the growth of our school in terms of not only numbers, but in terms of the outcomes we produce for them, is that everyone's invested here at Mansfield.

And that's taken a lot of really hard work to get there. There's been a lot of hiccups, as everyone knows, schools are not even keel places, there's always challenges. But the success of, I think, this particular school is the fact that we have buy-in from all of our stakeholders due to previous work that's been done by wonderful staff and our administrators.

Marc: What a great way to summarise the community, and not just the school, not just the teachers and the students, but the community as a whole.

Challenges. What is still probably, maybe always will be or could be, or could be worked on, or is something that's been seen at the moment. But what's a challenge in terms of use of data, use of evidence, and the information, that vast information you have?

Mark: I suppose once we start thinking about a staff member's ability to generate reports or to understand data, and to actually create something that is useful, you start to see quite different levels of ability for different staff.

It's necessarily focused on faculties or even perhaps keenness or interest.

It's again about noting that the investment of time in really understanding what this data explorer can do, for example, is worth your time.  Because if you're already time poor, you need to be persuaded, and the act of persuading someone in itself takes time. So there is a challenge.

That is one of the great challenges. What is most useful and what I've seen work best is when we can really lift up a couple of key players within faculties. I'll go back to the point Jeff said before, over 250 academic staff. So the idea of getting them all into one room to train them up and to do something is just impossible. We don't even bother.

But getting some champions and getting them to the point where they agree and they feel comfortable about sharing and that confidence around is the current piece of work that I'm looking at and it is proving to be challenging.

I think we're going to get there. But for the moment, it is really, really important to find the time. And again, I'll go back to what Jeff said earlier with regards to prioritising opportunities for staff to have conversations about kids and about the data of those kids.

That's expensive, because again it's time related, but the school has prioritised that with what appears to be very good outcomes for that investment.

So yeah, perhaps it's just a sales job. It's about impressing upon many staff that this is worth your time because the benefits for you are going to be quite strong and you will enjoy it.

Jeff: Yeah. And I would absolutely agree with that. And I think it's the work that we've got to expand on is around making sure that in our roles, that we are making ourselves available throughout the year, through our professional learning communities, offering those sessions to staff, accessibility to information.

We know our teachers are such hard working people. We know the amount of time they put into their development, their kids’ marking, all the deadlines they have to meet, that are constraints put on by our outside organisations, whatever it is that is takes up their time, that they don't have a lot to give in terms of extras.

And so we have to provide those opportunities and we build those into our professional learning communities at regular junctions throughout the school year. We make a financial decision around that.

We support our teachers to come off class, to have collaboration with their peers because those professional collaborations probably produce more interesting, useful ideas in the time that they get together.

Then if we're saying, you've got to be able to go through 20 pages of data and pull out these things and struggle through yourself because we know when we get with other professionals who are teaching the same students that are sitting in front of you, that collectivisation of ideas cuts down the amount of thinking and the amount of time they have to spend on that.

And I think that's a key part of it is that whatever data sets you use, with the wealth that's out there, schools have to kind of tailor it to what suits their specific context and make it as simple as possible and as accessible as possible for staff.

Marc: A thing I'd like to ask you both, and that is, if Mansfield State High School had a crystal ball and we're thinking maybe this time next year, maybe the end of 2025, in terms of data management planning as a whole, where would you want to target yourself?

Where would you want to be? Maybe one, 2, 3 things that would really emphasise improvement or we got to that point or we're heading to that finishing line.

Jeff: I would say at the moment that I think next year is going to be around, you used the term refinement. I think that the work that's actually happened here has generated success for a number of years.

I'm fairly confident that the processes, the systems that we have in place that support students, processes to support learning, they're there. I would like to see next year's work especially from my level is terms around about again, how do we refine those processes to make them more streamlined, easier to access, easier to understand.

And I would say that if it ain't broke, that you don't need to fix it, but you can definitely tweak it. And I think that that's my work in the oversight of all of those particular processes that I manage under my kind of portfolio for this year.

Mark: Yeah, I agree. It's a difficult question to answer.

The National Tool for School Improvement comes to mind immediately as we never accept that we're done, we never accept that we're finished and we've got it right, that there's always another piece of work to go to either improve the scope or just get that extra kid across the line who wasn't quite there last time. So, I think that one of the additional things is that the pace of change is going to be ramped up.

But I think another one is some of the work around, dare I say it, AI. I think there's some enormous impact that's going to happen. And who knows, we might actually very quickly find really fine-grained data on what kids are thinking because they'll start scraping and mining their social media accounts.

But I think what Jeff's really alluded to is that we have plotted a pretty steady course and it is about having a really close look at where are those pain points.

So I think maybe, yeah, the surveys, the staff surveys, asking them to complete very short surveys, that's probably the most important messages. You cannot give staff too many surveys. There is death by survey that you must be aware of, and so, or beware of because you want to, you're gently coaching them and getting them to come along and to get better. But you don't want to push them too hard because you don't want to lose them.

Marc: I think you probably made a great point, Mark, in terms of where to and the things that impact through society such as AI, and I think that's probably another conversation that we may even have 12 months down the track.

With that, I'd like to thank both Mark Redhead and Jeff Broadway just for joining us today.  But thank you for your honesty. Thank you for your transparency. You're willing to share what has been a journey and continues to be a journey over time. And I think that that helps people understand that in their own school, they're not alone.

You may have heard today some of those things that resonate with you, some of the challenges, and certainly some of the successes. So once again, thank you, Jeff, and thank you, Mark.

Alex: Thank you again to Jeff Broadway and Mark Redhead for taking the time to share their experiences with us. If you'd like to learn more about PAT, please contact ACER's School Support team at school.support.org. Thank you so much for listening. We'll be back with more insights in education soon.