What is AHELO?
Is it possible to undertake an international assessment of final-year students’ capacity to use, apply and act on the knowledge and reasoning they have gained from their degrees? Is it possible to assess these outcomes in an efficient and internationally comparable way? Can policymakers, institutional leaders, faculty and students be convinced that the assessment of higher education learning outcomes as an essential checkpoint in the educational process? These questions lie at the core of the OECD’s Assessment of Higher Education Learning Outcomes (AHELO) Feasibility Study.
AHELO involves the development and validation of assessments in three core areas – Generic Skills, Economics and Civil Engineering, as well as the development of contextual instruments to aid with the interpretation of assessment data. The assessments are targeted at students in the final year of bachelor degrees and aim to assess their capacity to apply their skills and knowledge to real-world problems. AHELO is an ambitious project which is attempting to develop new methodologies and technical standards for the assessment of higher education learning outcomes. It is taking place on a global scale, with 16 countries participating in the development and validation of assessments, and engagement from experts, institutions, governments, and key higher education bodies from around the world. The project is run by a consortium of international organisations, led by the Australian Council for Educational Research.
AHELO responds to a critical information gap. Efforts to improve the quality of teaching and to enhance students’ learning outcomes are stymied by the absence of reliable information which enables comparative judgments to be made about the capabilities of students in different institutions and in different countries, or about the quality of teaching. In the absence of such data on core higher education activities of learning and teaching, the standing of a higher education institution is based largely on reputation and research performance. AHELO’s objective is to create a richer source of information through designing and testing measures which give due weight to teaching practices and learning outcomes.
The developmental phase of AHELO is drawing to a close. Assessments have been developed through the collaboration of experts from around the world and subsequently translated and adapted for use in a number of countries. Planning for validation of the assessments with students in participating countries is underway. Despite the relatively early stage in its evolution, initial indications suggest that AHELO has the potential to reshape the higher education landscape in important ways. First, discussions among some stakeholders appear to have moved on from whether learning outcomes should be measured to how they can be measured. At the same time, the engagement of national systems and institutions (both within and outside the OECD) has increased significantly throughout the life of the Feasibility Study, with indications that this trend is continuing. Taken together, these indicate the existence of a desire for data on the quality of teaching and learning, both to inform improvements and also to demonstrate quality.
Students in a number of countries have been involved in qualitative testing of the assessments as part of their development. During focus groups and cognitive labs, students have had a chance to provide feedback on the assessments. While not all data has yet been analysed, initial findings suggest that students perceive the assessments positively. Individual students have commented on their appreciation of tasks which call on them to “apply knowledge in real-world ways”. Others have referred to the nature of the tasks and their inclusion of “interesting day to day problems you may experience as a project manager”.
As AHELO moves forward, questions about the validity, delivery and scoring of assessments, as well as the engagement of university teaching staff and the perception of institutions about the usefulness of data generated by AHELO will be answered. Thousands of students in numerous countries will sit the assessments, and their results will be subject to intensive psychometric analysis. Hundreds of university teaching staff will be trained in the scoring of student responses and will be asked to provide detailed input on the relevance of assessment tasks. Numerous institutions will be provided with detailed reports on the performance of their students and then asked to comment on the ways in which this information could be used to inform curriculum and teaching improvements.
