Evaluation for innovation and improvement

Evaluation for innovation and improvement

A three-year research project investigating the impact of philanthropy in education has culminated in the publication of a guide to determining whether positive change is occurring.

The Leading Learning in Education and Philanthropy (LLEAP) project aimed to build knowledge and improve outcomes for schools and philanthropic supporters. Findings from annuals surveys informed the development of the LLEAP Dialogue Series guides and case studies.

The LLEAP guides are designed as flexible, user-friendly tools to help different groups within education and philanthropy to form and foster partnerships focused on growing educational projects to maximise impact. The third and final guide in the series, titled Growing Ideas Through Evidence, officially launched in January, focuses on evaluation of outcomes.

According to the guide’s authors, Dr Michelle Anderson and Dr Emma Curtin, the prospect of evaluating a program or project is often seen as daunting, with many people not knowing what or how to evaluate.

The issue, Drs Anderson and Curtin note, is that failing to identify and articulate whether improvements are being made can be the undoing of a project or program even before it gets off the ground, and can also hamper bringing on board new supporters, or communicating the impact of the work to others.

The guide explains that evaluation – the process of identifying an issue, gathering information and then making decisions based on that information – happens every day in all kinds of ways.

‘A big part of evaluating is ‘noticing’,’ Drs Anderson and Curtin write.

‘The adages, “what gets measured, gets done” and “what gets measured, gets noticed” are being challenged in education. Improving outcomes for students means we need to rethink what we notice, how we notice and whose noticing counts.’

Noticing improvement can come from many different sources and perspectives: summative, formative, quantitative and qualitative. Growing Ideas Through Evidence includes an Evidence and Approaches Stimulus Tool to show examples of evidence and how approaches can provide information for evaluation and improvement.

Drawing on cases of practice featured in the guide, the Evidence and Approaches Stimulus Tool is designed to help:

• inspire and expand thinking and creativity;
• promote discussion in relation to specific outcomes;
• invite ‘noticing’ at the individual, classroom, organisational and community level; and
• keep the focus on outcomes.

The tool is designed to help people identify specific ‘ways of noticing’ improvement, through evidence and various program approaches, and stops short of specifying how, how much and how often the information might be gathered.

Drs Anderson and Curtin acknowledge that evaluations are rarely straightforward: knowledge will only ever be partial and compromises will be made, often governed by the amount of time and resources available.

‘An evaluation, typically, is a juggling act; it is more an art than a science,’ they write.

‘At some point evaluators must decide what can best be done, and what that might imply for the quality of the information gathered, and the decisions that will flow from the use of this information.’

Further information:
LLEAP is an initiative of the Australian Council for Educational Research in partnership with The Ian Potter Foundation, Origin Foundation, Scanlon Foundation, the Vincent Fairfax Family Foundation and the Myer Foundation and Sidney Myer Fund.

To download the LLEAP Dialogue Series Guide: Growing Ideas Through Evidence, visit the Leading Learning in Education and Philanthropy website.