Understanding equity through research data

Understanding equity through research data

Large-scale assessments play an important role in education policy and planning in many countries. They typically use a common assessment tool administered to a large sample or whole population of students under uniform conditions. Most use methods that enable the measurement of change over time. Large-scale assessments also typically measure aspects of student background that enable analyses of the distribution of achievement.

Both the National Assessment Program – Literacy and Numeracy (NAPLAN) for Years 3 and 5 students and the Programme for International Student Assessment (PISA) usefully identify trends in student achievement. NAPLAN has assessed the population of Australian students in Years 3, 5, 7 and 9 annually since 2008. PISA has assessed an international sample of 15-year-old students, including Australian students, every three years since 2000.

Primary reading interventions boost results

There has been steady improvement in reading achievement in Years 3 and 5 students from 2008 to 2013. The improvements have been greatest where there have been the strongest interventions. This gives cause for optimism in terms of the efforts that have been made in the preschool years, the early years of schooling and primary school in general.

The counterpoint is that there have been only isolated instances of improvement in other curriculum areas such as numeracy or writing.

This appears to be consistent with an emphasis on reading in reform initiatives in preschool, and the early and middle primary years. Improvements have occurred in the areas at which most reform initiatives have been targeted.

While there have been substantial initiatives in preschool and early primary school in most states and territories, in Queensland there have been structural changes with the introduction of Year K (or preparatory year) in schools prior to and at this time. This appears to have been associated with the improvement in Year 3 reading achievement in Queensland from 2008 to 2012 and in Year 5 reading achievement from 2011 to 2013. There were smaller improvements in numeracy at Years 3 and 5 in Queensland, suggesting that the impact of the structural change was not confined to reading.

In the Northern Territory, there were also improvements in reading achievement at Years 3 and 5. In the Northern Territory, there had been substantial reform initiatives focused on reading achievement, especially Indigenous student achievement. It is also notable that the improvements in reading achievement among Indigenous students in Years 3 and 5 reflect a number of reform initiatives at both the national, and state and territory levels.

Investigating declines in secondary reading and maths

Reading and mathematics achievement in the middle secondary years has seen declines over the past nine to 12 years. These declines vary among states and territories, and have been associated with increased differentiation among schools.

There were no significant changes from 2000 to 2009 for reading or from 2003 to 2012 for mathematics in the differences between girls and boys, between Indigenous and non-Indigenous students, or between students in metropolitan and non-metropolitan locations.

For both reading and mathematics there was a reduction in the difference between students with an immigrant background and those with a non-immigrant background, and between students with a language background other than English (LBOTE) and other students, arising mainly from the fact there was no decline for students with an immigrant background or LBOTE students, whereas there had been a decline for other students.

There were differences in the change in mean reading scores among the states and territories, with significant declines in Tasmania, South Australia, New South Wales, the Australian Capital Territory and the Northern Territory. There were no significant changes in Western Australia, Victoria or Queensland. The variations among Australian states and territories in the extent of the declines suggest that there may be some systemic factors associated with curricula or school organisation that may be linked to these declines in reading achievement.

Correlations between the declines in reading and mathematics among the states and territories suggest that it is unlikely that changes in curricula or teaching would provide the main explanation for those declines, although they could be associated with more general changes in approaches to teaching.

The declines in achievement of 15-year-olds in reading and mathematics in PISA over the past nine years do not appear to be associated with changes in the personal, social and demographic characteristics of students. The magnitude of the declines differed among the states, and those declines appeared to be similar for reading and mathematics. The declines in achievement were associated with increased differentiation among schools, suggesting care should be taken with initiatives that may exacerbate differences among schools in intake characteristics or effectiveness and support for measures that provide quality assurance.

Further information:
This article is based on the conference paper, ‘Perspectives on quality and equity from large-scale assessment studies', delivered by John Ainley at ACER’s Research Conference 2014 on the theme ‘Quality and Equity: What does research tell us?’ on 4 August.

The conference proceedings, including full conference papers, are available from the ACER research repository.