HSC exam review recommends further improvements

9 April 2002
HSC exam review recommends further improvements

Further improvements to NSW HSC examinations are recommended in a report released today by Professor Geoff Masters, Executive Director of the Australian Council for Educational Research (ACER).

The 83-page report, Fair and Meaningful Measures?, presents the results of a review of the New Higher School Certificate Examination program.

Professor Masters said that his overall conclusion was that the 2001 examination program had run relatively smoothly. He had found no serious flaws in its implementation.

Among the report’s 20 recommendations are proposals that the Board of Studies:

• links more closely its processes for developing, piloting and revising guidelines for marking examinations;
• explores the feasibility of appointing a Chief Examiner to oversee the development and marking of each HSC paper;
• investigates why some courses had unusually small (and other courses had unusually large) numbers of students scoring 80 or better;
• provides additional details of the marks achieved in each HSC course (including, for example, the mark achieved by the top 5% of students in each course);
• removes references to ‘bands’; and
• continues to work with tertiary institutions to ensure that students, parents and teachers understand the relationship between new HSC marks and the Universities Admission Index (UAI).

"In general, HSC examination processes are thorough and consistent with best practice. However it is clear there were some difficulties in implementing the new procedures in their first year and there were misunderstandings about aspects of the new assessment and reporting approach," Professor Masters said.

He said that further improvements could be made to address these issues.

"Many of the issues raised with the review can be dealt with through refinements to existing procedures and better communication to address misunderstandings," Professor Masters said.

"At the same time some significant challenges remain for the Board of Studies in order to consolidate the reforms."

Most concerns raised in submissions and discussions with education groups centred on:

• the adequacy of marking guidelines for some courses and the consistency of their application;
• differences among courses in the percentage of students scoring 80 or better and, in particular, the low percentage scoring above 80 in Standard English;
• a widespread lack of understanding of the relationship between HSC marks and the UAI, and the processes for calculating the UAI; and
• the adequacy with which some examination papers reflected the syllabus.

"In 2001 there was an undue focus on ‘bands’ in reporting achievement," Professor Masters added.

"This has detracted from the important relationship between marks and standards."

The review received 181 submissions. Consultations were held with peak education bodies and representatives of school systems and independent schools.

Download Report:

Fair and meaningful measures? A review of examination procedures in the NSW Higher School Certificate by Prof Geoff Masters
available to download from the Board of Studies NSW website.

ENDS

REVIEW OF THE HIGHER SCHOOL CERTIFICATE EXAMINATION PROGRAM

RECOMMENDATIONS

Developing examinations

That the Board of Studies

• ensure better dissemination of the reasons for, and nature of, sampling of content and outcomes in HSC examinations;
• include in the Board’s examination development process explicit checks on the adequacy of sampling and on the level of detail assessed in each examination;
• further clarify its rationale for the Glossary of Terms and its intentions and processes for the use of the Glossary in developing examination questions and marking student work; and
• provide an explanation of the ‘question scaling ’ process designed to adjust for unintended differences in the difficulties of optional questions.

Marking examinations

That the Board of Studies

• explore the feasibility of appointing a ‘Chief Examiner’ responsible for both
  the setting and oversight of the marking of the examination paper in each course;
• develop still closer integration of the examination development process and
  the process for revising marking guidelines, developing marking schemes and identifying benchmark scripts; and
• continue to explore ways of enhancing the processes of pilot testing and adjusting marking guidelines, developing more detailed marking schemes

(where appropriate) and identifying benchmark scripts.

Setting standards

That the Board of Studies

• cease using the term ‘bands ’ and refer instead to HSC mark ranges;
• for courses with unusually low or unusually high percentages of students in Bands 5 and 6 in 2001, investigate possible explanations (such as examination, marking and judging processes and the standards themselves) and make changes to future processes if appropriate;
• investigate methods currently used to place Standard and Advanced English results on the same scale with a view to using only one method for HSC reporting and UAI purposes from 2002;
• investigate the feasibility of using the judging process to establish one set of cut-off marks on the common scale for Standard and Advanced English;
• investigate whether there are subjects in which it might be desirable to report results in an Extension course on the same scale as the 2-unit course; and
• explore the possibility of using direct comparisons of examination questions in different years to enhance year-to-year comparability of HSC marks.

Reporting HSC results

That the Board of Studies

• remove references to ‘bands’ from reports (eg, Statement of Results and Course Reports) and not report percentages in bands;
• continue efforts to explain the nature of the scale on which new HSC results are reported and to show how differences between past and current HSC marks can be understood in terms of the changed scale;
• monitor the possibility that the number of students achieving low standards in a course will be increased by the decision to assign a mark of 50 to students achieving the minimum standard expected;
• continue efforts to ensure that described standards in all courses can be used with both examinations and school assessments;
• continue to consider how standards packages, professional development activities and consensus moderation exercises can be used to enhance understandings of achievement standards and the comparability of school assessment marks; and
• from 2002, provide a report of the marks in each course corresponding to some selected percentile points.

Using HSC results

That the Board of Studies

• continue to work with tertiary institutions and authorities to ensure that students, parents and teachers understand the relationship between new HSC marks and the Universities Admission Index.

General observations

• There is widespread support for the reforms underlying the new HSC
• There is widespread support for the HSC examination system
• There was a general expectation that refinement would be required
• The 2001 examinations ran relatively smoothly
• There is strong support for the move to standards-referenced assessment