Field Notes episode 5: A system approach to data with Catholic Education South Australia

CESA, a network of 101 schools educating more than 52,000 students across South Australia, generates a wealth of data daily. The challenge lies in making that data accessible, actionable, and aligned with CESA's strategic vision. Jarrod and Lisa share how CESA has established a ‘data-literate school culture’ by implementing a systematic approach to engage with data at all levels — empowering leaders, teachers, and system coaches to use evidence effectively.

Throughout the episode, you’ll hear about the practical strategies CESA has employed, including:

• building data dashboards to provide schools with clear, actionable insights
• developing a system-wide framework for data efficacy that aligns with their strategic goals
• using tools, such as PAT, to support evidence-based decision-making and track student growth.

The conversation offers valuable takeaways for educators, providing a roadmap for schools aiming to embed effective use of evidence into their school’s culture.

We hope you enjoy this latest episode of Field Notes, where we share honest conversations with educational leaders using evidence to improve learning outcomes across a variety of settings

You can stream episode 5 below, through Apple Podcasts, Spotify or Amazon Music, or keep reading for the transcript. 

Learn more about transforming systems at Research Conference 2025 in February!

Transcript

Alex: In the spirit of reconciliation, ACER acknowledges the traditional custodians of country throughout Australia and their connections to land, sea and community. We pay our respect to the elders past and present and extend that respect to all Aboriginal and Torres Strait Islander peoples today. ACER acknowledges the Aboriginal and Torres Strait Islander people who continue to contribute to our work to improve learning, education and research.

Welcome to Field Notes, a podcast from Australian Council for Educational Research. In today's episode, our education consultant, Marc Kralj, speaks with Jarrod Carter and Lisa Karanicolou from Catholic Education South Australia (CESA).

Catholic Education South Australia is a network of 101 schools across South Australia that educate and support more than 52,000 students. This produces a vast amount of data for schools to utilise efficiently and successfully with the support of CESA coaches and system management.

The challenge for Jarrod and Lisa is in establishing a systematic approach to engage with this data, ensuring that leaders and teachers in all schools work collaboratively with system coaches and leaders to fulfill CESA's strategic vision.

From data dashboards to a system-wide data efficacy approach, you will hear some key actions that Catholic Education South Australia have strategically implemented over several years to establish what they call a data literate school culture.

You will also hear reference to PAT, an assessment tool developed and offered by ACER that has assisted them in this journey. With that in mind, let's join Marc for that conversation now.

Marc: It is with great pleasure that I introduce two Catholic Education South Australia leaders who have a vast experience in education. And I'd like to begin by introducing Jarrod, a senior education data analyst, as well as Lisa, a system coach specialising in standardised assessments. Now, to begin with, I'd like to have our guests today introduce themselves. So, as always, ladies before gentlemen, I'd like to ask Lisa just to give a bit about her background and her current role. Thank you, Lisa.

Lisa: Thank you, Marc. My current role is actually System Coach for School and System Improvement. And I really enjoy this role as it really focuses on learning curriculum assessments. And I think I've been in this space for quite some time. So previous to this role as the leader of learning at a school, which pretty much did this work also. Worked alongside teachers and leaders to build capacity, using informed data and evidence-based practices. I've actually been teaching for over 25 years in Catholic education, and I'm really passionate in this space.

Marc: Thank you, Lisa I like that word passionate because I think to be able to work with students, with schools, with teachers as a coach, and especially with parents and broader community, passion is what it's all about. Jarrod Carter, Senior Education Data Analyst. Can you please give us a bit of background about your experiences and your current role, please?

Jarrod: Thanks, Marc. Yes, so my role is, as you said, Senior Educational Data Analyst, and I'm responsible for managing the system assessments and reporting across our network of schools.

So that can include anything from managing and implementing the NAPLAN assessments, down to PAT testing and then smaller school level assessments, and then reporting that information back to system leaders, so they can get an understanding of how our system is tracking, also to school leaders so they can understand better where their school sits within the network, and then also providing information to classroom teachers so they can understand how their students are going within their classroom.
My background was in secondary teaching, and before that I had a brief career in design engineering, and brought those skills across to my current role here at CESA.

Marc: Thank you very much. Now, Lisa, for our listeners today in this podcast, can you give us a bit of a background about, I suppose, who we are as Catholic Education South Australia?

Lisa: So we are 101 schools, and we're committed to the education excellence and holistic development of students to meet academic, spiritual and personal growth goals.

Marc: And as a system, quality and performance group, Jarrod, can you explain what we mean by quality and performance within Catholic Education South Australia?

Jarrod: Yes. So Lisa and I both work in the SQP team, School Quality and Performance, and this is a CESA section. We're dedicated to partnering with school leaders and schools to drive that measurable improvement for students across the sector. We do this through a number of structured ways, but it really involves collaborating with principals and school leaders and classroom teachers. We do this within set regions across the state. We do this to drive evidence-based practices aligned with our key targets, and we also have a clear focus on improvement as well.
So we have a number of improvement agendas happening across our dioceses schools.

Marc: Well, thank you, Jarrod. Thank you, Lisa. Catholic Education South Australia has several key focus areas and strategies to support schools' teaching and learning, aiming to enhance your students' outcomes, including quality teaching and learning, faith development and continuous improvement. What are CESA's focus areas and what strategies have been developed to support the schools' teaching and learning and outcomes for student learning?

Jarrod: Yeah, that's right, Marc. We probably should start with our strategic goals as a system and how that relates to how we work with our schools. When you are 101 schools, I think it is so important to have that strategic vision that binds everything that we do.

So starting at the top, we have our Towards 2027 strategy, Expanding Horizons and Deepening Practices. This was developed by our commission, establishes the system-wide mission for excellence in terms of Lisa and our work, has a real strong focus on literacy and numeracy as one of those pillars to drive excellence within the system.
Marc: Some of those key components that you talk about, Lisa, what can you share with us about those specific things?

Lisa: Catholic Education South Australia really does strive hard to a shared commitment. And that commitment is about ensuring students are literate and numerate, while supporting our teachers with the tools to implement effective instructional practices. And that comes about through some data utilisation, which includes looking at data-driven strategies to assist with identification and the monitoring of at-risk students. And we do that through a process of case management. We uphold quite strongly to identify students, the school and system progress over time, to ensure we're watching and monitoring what's happening there, and establishing a recommended set of standardised early years assessments.

Another thing that we do, Marc, is professional learning for teachers and school leaders in relation to those standardized assessments, ensuring there's provision of targeted support for schools and students based on their literacy and numeracy data, as one of those pillars Jarrod had mentioned.

Marc: All educators have their own starting points, and I think that's very reflective of what you've just mentioned about developing and supporting professional development, that continued learning of educators. But they're also determined by their experiences. Their own skills developed over time, many based on their positions. They have varied positions in a school, the year levels taught and the context of the schools they have been teaching in. Well, these experiences come from their understanding of data, as you mentioned already. All evidence to make informed decisions for their school, year levels and individual students.

So what approaches have CESA put in place to determine and support the development of staff and schools understanding that term data literacies?

Jarrod: Yeah, probably a good point now, Marc, to bring in a term that Lisa and I have come up with, and that's system data efficacy. So that is the term that we developed when defining the practices we are trying to embed just in individual schools, but across our entire system.

I really want to stress that we're a system of schools. System data efficacy, it refers to our system's capacity to effectively manage, interpret and apply data to improve educational outcomes, and most importantly, guide instructional decisions. When we were developing this term, we looked at five system pillars to how we can build system data efficacy.

And if I could just go through them now, the first one is just the data literacy of our system. So I'm talking about not just the data literacy of, say, a school principal or a school leader, I'm talking about everyone in that system, from the staff in the SQP section, the principal, the school leaders, the leader of learning, all the way down to the classroom teacher. Just what are their skills when it comes to interpreting data? Because it can't be given that they have those statistical understandings when either looking at assessment data in academic grades or NAPLAN assessments, we really need to make sure we're building that literacy.

Another key pillar is just data access. Providing educators with access to their data, it sounds like a really simple thing to do, but we know that access to data is one of those key barriers to being able to really interrogate where students and schools are at a particular point in time.

Just going back to that system approach, another key pillar is systematic data insights. We really ensure that data is collected and analysed systemically and regularly to make meaningful insights actionable. One way we do that is when we collect data, we're making sure the schools can access it and see their results. But we want them to benefit from system trends within the broader network as well. We want to leverage a central way of collecting data to reduce that administration burden within schools.

The last two pillars are around training and support. We need to make sure that, yes, we say data is important. We have these strategies, but unless we have that training and support, and data literacy is an important one, that was pillar one, that's not going to improve unless we make sure that we have adequate training and access to that training.

And the final one, the final pillar that glues this definition together for system data efficacy is just around shared expectations and culture. To build a data literature culture, we need to set high achievement goals in the system, schools and classrooms, to drive data being a key part of practice of everyone within the system.

Marc, I really want to stress issues of data literacy, data access, training and support, cultural expectation. This is not just us making this up. The research really does back this up, that these are barriers for building data capacity within a school system. But what Lisa and I really excited about, the good news is the research also says when you get those supports right, you can really build the capacity and overcome those obstacles and being able to use data in everyday practice of the school system.

Marc: You've mentioned a couple of things, and I think it's coming through with both you and Lisa. Professional development, training and development. And they're two different things. Now, we talk about professional learning and understanding to support and understand and build upon knowledge. We also talk about training, and we think about the new things that we need to learn. And I often think about the term unlearn. So some of the misconceptions we have around the use of information, the use of evidence, the use of data. So I'm glad, I'm really glad that I'm hearing this continual consistency across your conversation today. I'm going to ask a bit about where have you come from in terms of the work that you've done so far?

Jarrod: Marc, when I first arrived, and I've been in the office now for six years, education data, particularly at a system level, school, but also at the school level, was relatively new and probably best to say, an unknown science. That was certainly my experience.

So where we came from, when I first arrived, we were mainly relying on the availability of some really large-scale data sets. So I'm talking about the obvious ones like such as NAPLAN. And because these collections were often few and far between, it would often lead to really reactive uses of data. Something would go unknown for a long time, and then you would see this piece of data and it's like suddenly there's this big reaction. What's been happening at this school or classroom or whatever that may be. That's certainly where we've come from and we've come a long way in being able to try and fill the gaps between those big reactionary pieces of data collections to then have a consistent feedback loop of assessments within the system.

Lisa: And to avoid some of those misinterpretations, we really want to ensure a cohesive approach across the CESA schools and student assessments and ensuring that we have an alignment around progress monitoring.

One thing that we really identified quite early on was that we as a system needed to develop shared language around when we're talking about data and data interpretation and understanding of what we're looking at to be able to talk with schools about how progress monitoring is occurring.

Marc: History tells us a lot about what we've done and I think it's also almost like a foundation of how did we actually get there? What did we actually change? So Jarrod, could you just talk a bit further about some of the things that had changed to get to this point of understanding, developing and even the thinking that's reflected in this area?

Jarrod: Yeah, well, I'll talk about what we did to and the journey is still going, but to build that system data efficacy. So going back to what I said before, we had these big broad collections that would often lead to these reactionary decisions. I'm not saying that they're not quality assessments.

In fact, they're often some of our better assessments and more reliable assessments we have in the system. But what we've done to fill those gaps is just broaden those data sets. So we've expanded our collections to make better collections and more reliable collections of academic grades.

We've improved our PAT testing, Marc. When I first got here, there was not a lot of direction or support around that. We now have 100 schools all doing that and set timeframes with set expectations around that. And we also brought in things like well-being metrics as well.

So we could balance what we could see in the student background and the well-being, the attendance, for example. And now we can balance that and triangulate that with those larger assessments that I was talking about before.

Going back to that literacy and numeracy strategy, it demanded that we have a more focused approach on intervention. So that for students at risk. So we had to better support that strategy. We had to bring in new assessments like the Phonics Screening Check at year one and also the year one number check as well. And to do that as a system, we now build our capacity that we have a measure that was very new just a few years ago to one that each school is now looking for. They're waiting to see their improvement in that. And when they see scores of concern, they know what the step A, step B, step C is from that. So that's been a really good example where we've enacted the strategy through new assessments.

And Lisa, did you want to talk perhaps about your own role as a system coach and how it feeds into that as well?

Lisa: Yes. Thank you, Jarrod. So as a systems coach, we really have the privilege of walking into schools and we're walking alongside leaders of learners and sometimes other extended leadership members. And what we will do is we'll talk about their data story. So we walk through that together.

We will analyse the data that's building the system data efficacy right across CESA because our whole team has got the same shared understandings when we are having these conversations out in schools. And I think that, you know, it was exciting about this.

The whole catalyst was our initial work with Lyn Sharratt, where CESA, Catholic Education, partnered with her and we introduced this idea of shared beliefs and actually a case management approach. And through that case management approach and working with schools, we had student monitoring how that was happening, where is the spots for capacity building. And often that would lead to the development of data walls within schools, you know, to visualise and understand what that progress actually looks like.

And it's really quite context specific too. We can really nuance what was happening for each school in this way. This further then assisted teachers in understanding that all students are capable learners, they could be thriving learners. And they can learn, fostering that collective belief that all students have their potential.

Marc: You know, I like how you've referred to a couple of things here. More specifically that you don't just do it on your own. We look for outside sources of support. And I think when I hear Lyn Sharratt introduced to this conversation, I think of putting data faces on the wall, in particular with Lyn and Michael Fullan as well. And I think of Clarity and I think of the support that you look for outside. And I suppose that's our best way of learning. What research has been done, what evidence is being found through other researchers, such as Lyn Sharratt as well.

Jarrod: Yeah, I have a saying there, Marc, that no school is an island. Yeah, you can't work in isolation. And I guess you need to extend that to say that no system of schools is an island as well. And we really do lean on our friends like yourself to be able to bring in those external evidence-based perspectives on the directions that we're taking.

Marc: Having described a developmental history and program within CESA to support the schools, where to now? Considering the successes and changes that have emerged, what can CESA as a system do to now think about future projects? I suppose building on the current learning and perhaps consider what's been learnt to develop improved learning using reliable assessment tools and resources to support teacher schools to deliver the curriculum.

Lisa: With our conversations we've already been having around shared beliefs and understandings, we are also looking towards what does it actually mean to be proficient?

Proficient in literacy and efficient in numeracy and how we might utilise the data that we have before us to help us make those decisions, but also to continue the emphasis with very clear expectations and guidelines. But that is where the shared beliefs and understandings really bring true, that we're all on the same page and having the same conversations.

The access and support to curriculum materials and assessments to support best practice is really paramount in Catholic Education. And our focus to continue to build structures to support our educators, to understand where learners are at in their learning, as we said already, with the case management, walking alongside system coaches to better inform decision making.

And we have a whole range of tools which we use for that. So we might use our data dashboards, professional learning. And sometimes its system coaches working with leaders within the school to actually run their own professional learning sessions with their staff. And I think then, Jarrod, we can talk about how that consistent, reliable data is really important in that space also.

Jarrod: Yeah, I think from my perspective, the future is about how can we continue to build those consistent and reliable data systems.  So, yeah, let's look at the work we do with ACER. Marc, PAT remains one of those foundational assessments where if we can build consistent use, consistent understanding, shared understanding, it really does serve as a reliable benchmark to validate the practices that we're implementing within schools.

And for what purpose Lisa there was suggesting around proficient students and what does that mean? And that's a big conversation topic that we have in the system all the time. What's the purpose of what we're doing? And if it's not to produce proficient students, I'm not sure what we're doing. And that's a really easy thing to say. OK, we want to lift our level of proficient students in literacy and numeracy. But to enact those goals, you really do need those reliable structures. And our consistent and reliable assessments are a part of, not the only thing, but a part of those structures about driving consistency. So we do know when students are presenting as proficient or not proficient in literacy and numeracy.

Probably the other thing I'm excited about is new assessments as well. So this is just an ever-changing space since I've been working in this field. We don't want to change our assessments all the time. There's value in doing the one assessment for a long time. You build that longitudinal understanding.

But in saying that new tools are consistently provided to us, I'm finding that is a really exciting piece as well. Where it's heading at the moment is, I think over a decade plus, it's been a top-down approach where we built the system capacity in those large-scale assessments like PAT or NAPLAN. And now we're moving further down to say, those more diagnostic curriculum-based assessment types.

So it's been really, really exciting to work in that space as well. Probably a really good example is the work Lisa and I have done and the broader SQP team in the Phonics screening check. So four or five years ago, it was an assessment we knew nothing about. We had to build the capacity because it was part of one of our strategies that involved training schools and educators. We had to build the digital infrastructures as well. So that can't go underestimated what a big challenge that was to build the digital infrastructure. So we knew where every school we're at. So every school leader could see where their school was at. So every educator could see where their students were as well.

And the exciting thing is now we can now see, we've been doing it for enough years now to see what do these Phonics screening check results mean when that child goes and does their E3 NAP plan. And we're starting to build our own intellectual property around benchmarking and identifying students much earlier. So that's what I'm really excited about.

Rather than the larger, big accountability type assessments that we've been doing for decades, moving into that more diagnostic type assessments. So rather than being the reactionary data points that I talked about earlier, it's more about those self-correcting data points so we can get these children back on track and build towards that proficiency in the literacy and numeracy that we spoke about before.

Marc: It's exciting because I can also see as you're developing these things over the last two years, you're also developing teacher and leader capability across the schools as well. Now, you've mentioned data dashboards. Now, I've got to ask, when we reflect on data dashboards, that could tell us a lot of different things. Jarrod, could you or Lisa or both of you just explain a bit more about what we refer to as a data dashboard?

Jarrod: Yeah, I probably should take that one Lisa, given that it takes up a large part of my time. But I'll go back to the strategy. What are we trying to deliver in our strategy? Like we said, thriving students, and a part of that is proficiency in literacy and numeracy. To enact that, we collect assessments, be it academic grades or a well-being check, collecting attendance, or final screen check, number check, all these things.

Then we start collecting that, but then it gets to the point where we then need to put a narrative around that data. And that's the digital data dashboards that we take that data from our schools and present back to them so they can see what that meaningfully means. And when we present the information, when we talk about data literacy, it needs to be a way that enacts those strategies.

So when we talk about proficient students, when the average educator or school leader logs into these data dashboards, it needs to be quite clear to them, putting faces to the data, like we often speak about, which students are exceeding, which students are proficient on track, which students do we need to be concerned about.

So we spend a lot of time with those dashboards. I need to bring in our own ICT and BI reporting team here. They give me a lot of excellent service helping me deliver those dashboards. And it's really been probably one of the more exciting parts of working in education in the last six, eight years to see these tools come in. But around that is a lot of professional learning support as well. Lisa, did you want to talk about how you then support leaders of learning in schools to understand this data?

Lisa: We're very lucky to have those dashboards, Jarrod. And one thing that's really great about these is we can actually really nuance down to the individual child. So we can actually look at cohort data, we could look at a whole school data, but we can actually also look at individuals. And that's really great to inform teaching and learning or interventions as needed.

Another really great benefit of the dashboard is actually the ability now to pull more than one data set piece together or to use that alongside some school based data to find trends that are happening within the school and to make really informed decisions about next steps to improve literacy and numeracy outcomes.

And this actually comes back to the original term we started off today, which is building the system data efficacy. And this is a really great way that we are stepping forward with that, because underpinning all that is our shared beliefs and understandings.

But going out to schools as a systems coach and the team of coaches here, we walk alongside, we sit with the leaders. And sometimes, like I said earlier, groups of leaders sometimes, and we can actually itemise and drill down on aspects that make us wonder or curious about some things. It's actually things we're really excited about and being able to determine what those aspects are can really inform the school improvement plans.

Marc: Look, thank you both for expanding on that, because I think that's important because sometimes people will hear something like data dashboards, but it could be different in different school or a different sector or a different organisation. But to be able to achieve this, we also need systems to be provided with quality materials. So in this way, what are some of those quality materials that you would like to be able to focus on?

Jarrod: Yeah, this is probably a favourite topic of mine. We wouldn't send the tradesmen out unless they had the right tools. So why would we expect our educators and our school leaders to not also be expected to have quality materials out there?
And that's where I think our role at the system level and advising our 101 schools can really play a part in vetting those materials and making sure we're getting the right resources into our schools. So when we look at new materials, it's not done.

When we have a set criteria of what we're looking for, and let's take some of the PAT testing or our PAT test or our OARS account as an example, Marc, and yeah, I'm not meaning to talk up your product here, but it makes a good example.

We want to make sure it's aligned to the curriculum. Does the OARS account and the PAT testing provide that? Yes. We want to make sure it's accessible to our teachers. Yes, we can provide that through the OARS account.

We want to make sure that it's backed up with quality evidence. Well, when I see the students on the 65th percentile on the PAT-M test, I can do that with confidence because I know that ACER has done the work and the research and has the evidence to provide that.

And then just simple, tangible things. Does it have a safe digital ecosystem where we're happy to put our student and teacher data? Those things are really important to us. And I know it's great to talk about things like strategy, proficiencies and things like that. But at the end of the day, we also need quality tools that are able to deliver that as well.

Marc: Part of that too is, I think we have a reference to professional learning and support, Lisa.

Lisa: It's very important that we do this. And what's really great about Catholic Education is that we do provide that on different levels because we understand how busy schools can be. So sometimes we might offer that professional learning online or on demand. We have leaders of learning days and we have principal learning days as well with leaders forums where we all get together and have these conversations.

But these structured support systems help educators effectively use resources and know how to use those digital tools in schools, fostering a culture of continuous improvement, shared understanding and overall using the best practices for literacy and numeracy improvement.

Marc: Before we finish today, if you have the opportunity, which you probably do now, what are some of the things, maybe one, two or three things that you would provide as a priority if you were talking to another system, another group of schools and organisation in a reflection of the journey that CESA has currently taken?

Jarrod: If you take it back a step, from my point of view, probably the first thing you need in place is just that strong vision.

So earlier, we talked about our strategies, that literacy and numeracy strategy, that really does provide us a clear goals and vision of what we meant to do in our day-to-day work. So that'd be my first one. Make sure you have that, those visions and goals. They're articulated and well understood.

And then secondly, it's probably about, okay, now that we have that, now we know where we're heading. What are the key structures and pillars, those tangible things that we need to deliver those strategies? So for me, every time I do build a dashboard or organise some professional learning for schools, I'm making sure it's aligned to those strategies. So I'm making sure that strategy is being enacted.

And if I could give one bit of advice to systems out there, and even schools is, once you have those first two things in place, that change will come. It doesn't always come as quick as you would like. And sometimes it comes quicker than you expected. But if you have those first two things in place, that positive change will eventually come.

One thing I've noticed, which gives me a lot of feedback that we are moving forward, is just the questions that I now get from schools. If I go back two or three or four years, many of the questions I would get, and my phone rings every day from schools, I get emails every day, and often they'll be shallow type level questions.

How do I log into this? Where do I get my access for this? What does this mean? Now, when I get questions, I've got to be ready for these schools. I know they've already looked at the data. So we've solved that access. We've solved that data literacy bit. When they've done those two things before they ring me and then they're interrogating the data and challenging me on the observations that we have provided back to schools.

Even at a system level, Lisa gave a definition before about data dashboards. I would have died three years ago if I'd heard a colleague say that. So I can see that that capacity is growing, which is just fantastic.

Marc: You know change has happened when the conversations change, when the questions change, when they become more sophisticated, when they become more challenging, and it puts you on the spot. And I think that tells us a lot about what you've done, what Catholic Education South Australia has been able to achieve and do. And I'm sure over the next few years, we're going to keep looking out to see what the next step is. Where have you grown, where have you progressed, and where have you developed?

I'd like again to thank Lisa and Jarrod from Catholic Education South Australia. And I thank them for their experience, their transparency, and their ability to be able to share some of the things that have helped them in their journey.

Alex: Thank you again to Jarrod and Lisa from Catholic Education South Australia.

If you'd like to know more about PAT, please contact school.support@acer.org. Thank you for listening. We'll be back for more insights into education soon.