School audits lead to improvement

Despite the current global preoccupation with whole-system reform, in many school settings the improvement agenda is too general. To address that, ACER developed a tool that provides individual schools with a way to narrow and sharpen the focus of their school improvement plan. In the two years following the first use of the tool there are signs that it is having a positive impact on the quality of teaching and learning.

Development of the audit tool

Following the release of results from the 2008 National Assessment Program – Literacy and Numeracy (NAPLAN), the Queensland Government commissioned ACER to undertake an independent review of literacy, numeracy and science standards in Queensland primary schools.

Conducted by ACER Chief Executive Professor Geoff Masters between December 2008 and April 2009, the review concluded that the way to raise achievement levels in primary schools is to increase the resources and support available to schools, with individual schools best placed to determine the details of the resources and support they require to improve student outcomes and to meet targets. The review also identified forms of support that are likely to be of general benefit to schools in their efforts to improve literacy, numeracy and science learning.

ACER was then commissioned to develop an audit instrument for use in reviewing teaching and learning practices in all Queensland government schools, based on international research on school improvement. Working with the Queensland Department of Education, ACER developed an audit framework that enabled school leaders to identify the systems and processes needed to ensure that effective teaching and learning occurred in every classroom in every school.

Individual schools [are] best placed to determine the details of the resources and support they require to improve student outcomes and to meet targets.

The framework comprised eight domains of effective practice: an explicit improvement agenda; analysis and discussion of data; a culture that promotes learning; targeted use of school resources; an expert teaching team; systematic curriculum delivery; differential classroom learning; and effective teaching practices. A ninth domain, school-community partnerships, was added in late 2012 when the re-named National School Improvement Tool was endorsed by Australia’s Ministers for Education through the Standing Council on School Education and Early Childhood for the use of all Australian schools in their school improvement planning.

The school audit process

The audit tool was piloted in late 2009 before being implemented across Queensland in 2010. In total, 1257 schools were audited.

Depending on the size of the school, one or more auditors spent two days in a school gathering evidence as it relates to the domains within the tool, obtained from relevant documents such as school policies, curriculum plans, appraisal processes and reporting templates. Members of the school community, including the leadership team, teachers, students and parents were interviewed and asked questions relevant to the audit tool.

The audit process has been about conversations rather than interrogations.

The audits were conducted by school principals seconded into the role following rigorous selection and training. As such, the school audits were in effect a process of peer review. Principals and teachers in Queensland schools had great confidence in the fact that a seconded principal coming into their school would be more likely to understand the specific context of the school and be far more likely to demonstrate empathy for the teachers being interviewed during the evidence collection process.

The importance of the school principal’s role in school improvement was emphasised throughout the audit process. At the conclusion of the audit, the auditor provided the principal with an eight-page profile and a report, which also addressed instances where evidence was unavailable. The auditor did not present the findings to staff of the school – this was the task of the principal.

It is up to the school leaders to develop a school improvement plan informed by the findings of the report.

In Queensland, the audit process has been about conversations rather than interrogations. The audit report describes the current situation in a school but deliberately does not specify the future strategies a school should pursue. It is up to the school leaders to develop a school improvement plan informed by the findings of the report. And the evidence suggests that improvement is taking place.

Positive outcomes

The 2013 NAPLAN National Summary Report indicates that, although Queensland is not the highest performing state in Australia, it has recorded significant growth at the primary level in recent years. For example, at Year 3, Queensland was the only state or territory to record a moderate increase in mean numeracy achievement between 2008 and 2013, and was one of only four jurisdictions to record moderate increases in mean reading achievement over the same period.

Queensland also produced the largest increase in Year 5 mean numeracy achievement between 2008 and 2013. During this time, the largest increases in Year 5 mean reading achievement were found in Queensland and the Northern Territory – where Professor Masters has also been working alongside the education department on school improvement reforms. In addition, only Queensland and the Northern Territory recorded an increase in Year 5 mean reading achievement between 2012 and 2013.

The NAPLAN report also indicates that there was little change in Queensland’s Year 5 mean reading achievement between 2008 and 2011, with the increases that did occur taking place from 2011 to 2013, after the school audit process and subsequent implementation of school improvement plans.

Ninety-seven per cent of principals were either satisfied or very satisfied with the audit process.

Possibly the greatest indicator of the Queensland audit program’s success, however, has been the widespread support from the principals and teachers. In the final report to the Queensland Department of Education, ACER Principal Research Fellow Dr Gabrielle Matters and her co-writers note that 97 per cent of principals were either satisfied or very satisfied with the audit process. Principals and teachers indicated that they want the process to continue and it has been widely regarded as a high quality professional learning experience for them.

Future directions

There are a number of lessons that can be taken from Queensland’s experience of a system-wide auditing process. As Dr Matters and her co-writers observed:

• The audits were well within the bounds of understood practice, so the intellectual load on principals and auditors was not overpowering.
• The development of criteria and the use of standards-based assessment are the strengths of the tool, giving school leaders confidence that the process is transparent.
• The research base behind the tool means that school leaders feel confident that there is a level of objectivity in ratings that they do not normally associate with others types of inspections of school practice.

Importantly, the process does not end with the implementation of the resulting school improvement plan. As indicated in the principal feedback, audit processes can and should be ongoing. The audit tool is now being used as part of Queensland’s ongoing school review cycle.

While Queensland is an example of a system-wide implementation of the audit process, a benefit of the National School Improvement Tool is that it is also suitable for self-review. The audit tool is being used as a basis for school self-reviews in Tasmanian government schools. The tool has also been used extensively by the Association of Independent Schools Western Australia on a diverse range of schools, and is being used in government school reviews in the Northern Territory.

Ultimately, the experience of the National School Improvement Tool in schools to date highlights what can be achieved by trusting educators to drive their own school improvement.

Find out more:
ACER works with systems and schools in a number of ways to support school and system improvement based on the National School Improvement Tool. To find out more about these services visit < www.acer.edu.au/school-improvement/ >

Read the full report:
Implementing a generic school improvement framework: Experience of the National School Improvement Tool in Australian settings, by Elizabeth Hartnell-Young, Robert Marshall and Robert Hassell, was presented at the 2014 International Congress for School Effectiveness and Improvement.