Championing what works

Lee Elliot Major describes an evidence-led initiative that is gaining widespread support, and ruffling a few feathers

“SHOW ME THE EVIDENCE!” It is an exhortation far too rarely heard, let alone considered, in education. Every time I say it to teachers, I am reminded of the catch phrase made famous by the film Jerry Maguire – “Show me the money!” The currency I am interested in of course is not cash, but hard research evidence. Yet in keeping with the film’s central character, I am calling on a profession to challenge its long held beliefs and practices.

What we know
● The £200 million Education Endowment Foundation will adopt an evidence-based approach to its work to improve the achievement of the poorest children in underperforming schools in England.
● Robust evaluation of interventions proposed by schools and other organizations will be at the heart of the EEF’s work, with at least 10% of funds devoted to evaluations.
● The EEF aims to produce broader lessons on how best to boost achievement for all schools, with an interactive guide showing what works best, and also what doesn’t work, in the classroom.

For too long, the benchmark for teaching has been set too low. We need to challenge teachers and ask how what they are doing in the classroom compares to the very best teaching approaches that are underpinned by research showing demonstrable impacts on the learning and achievement of children. For too long, education debates have been preoccupied solely with how much money ends up in our schools. Of course money is important; but we must also ask what that money is actually spent on. We need to know what education bang we are getting for our buck. We want to know what works best.

Such questions are particularly important for our poorest children. For as long as data has been collected, they have sat on the wrong side of a stark and persistent achievement gap. And as education becomes ever more important in determining prospects in later life, their more privileged peers have moved further ahead in the social mobility race. It is why “Show me the evidence” is one of the central rallying cries of the newly created Education Endowment Foundation (EEF).

The Education Endowment Foundation

The EEF is an education first in England. Set up by the Sutton Trust as the lead charity in partnership with Impetus, a venture philanthropy charity, it is funded by an endowment of £125 million from the UK Government. With extra income from endowment returns and fundraising, the aim is to spend £200 million over the next 15 years. Its mission is to improve the achievement of around 160,000 children (those on free school meals) in underperforming schools in England.

What makes the EEF so distinctive and exciting is its potential to develop lessons not only for the country’s poorest children, but the millions of children across all schools. The EEF will aspire to the highest scientific standards when evaluating the impact of the innovative projects proposed by schools, charities, universities, and other organizations to demonstrate what works best in boosting achievement. These will include randomized trials and quasi-experimental methods, up to now conspicuous only for their absence in the educational world. The EEF is “a gigantic do-tank” says the EEF’s chairman, Sir Peter Lampl. Overall, at least 10% of the EEF’s funds will support evaluation.

Importantly, the EEF is an independent body, acting without fear or favor when evaluating what works best in schools. And with a 15-year horizon, it is in it for the long haul. It is open to any ideas that have the potential to boost achievement. Much to the delight of researchers, evaluations will be designed before the projects actually begin, not as an afterthought, as is too often the case.

Ten leading research groups have been selected as EEF evaluators (including the Institute for Effective Education). These teams will bid to undertake the evaluations of EEF projects. Before the end of the year, the first wave of evaluations will start to be formulated for grants under consideration in the first EEF funding round. The task for researchers will be to measure as far as possible the quantitative impact of interventions, as well as their costs, and how well interventions are deployed in schools.

Each evaluation will be decided on a caseby- case basis. Projects will range from small pilot studies, to fully blown multi-million pound trials. Evaluations will adopt standardized tests for students to measure their achievement or, where available, the official school results of children. And all the results will be made public. An overarching review will compare the impacts of all the projects funded, and track students over a number of years, to gauge the long-term effects.

The pupil premium toolkit

The ambitious EEF plans have been widely praised – and are part of an evidence-based movement gathering momentum in the UK. But the evidence-based approach is not without its detractors. They argue that complex research findings will be cherry-picked by teachers to back their own prejudices; that project evaluations will capture only a partial reflection of the reality of school life; that results averaged over entire school systems will produce crude approximations with little relevance for an individual classroom that is its own unique learning environment; and that it is unethical to offer some children an educational “treatment” and not others who form a control group for comparison.

Some of the answers to these questions lie with the pupil premium toolkit published by the Sutton Trust last year (see educationendowmentfoundation. This promises to be the living, breathing document at the heart of the EEF’s research efforts, as it adds hard evidence to what we know works best in the classroom. An attempt to help UK schools to decide how to deploy their “pupil premium” funds for poorer children, the toolkit is an accessible guide summarizing research from across the world to compare the impact (in students’ extra months of development) and cost effectiveness of different approaches in schools. Next year an updated version will be launched on the EEF website, as an interactive resource for teachers.

The toolkit’s main author, Professor Steve Higgins from Durham University, stresses that the kit is not intended to provide “off-the-shelf ” solutions that are the latest to guarantee overnight success; it is simply outlining the best bets – the approaches that consistently show greater impact on achievement.

The hope is to empower teachers to embrace an evidence-based approach so they can make decisions for themselves and monitor the impact of interventions in their own context. The dream is to establish “research leaders” among senior staff in every school.

The toolkit’s findings have already ruffled a few feathers in the educational establishment – challenging some of the assumptions that small reductions in class sizes, for example, or employing teaching assistants are good ways of raising achievement. The evidence suggests not. It should surprise few teachers that the highest-ranked approaches in the toolkit concern the interaction between teacher and student in the classroom; the impact of school structural reforms pale into insignificance in comparison. Amid the ever-growing market of educational initiatives being promoted to principals, this could be one of the great legacies of the EEF: to help debunk some of the unsubstantiated claims that plague the teaching profession.

A final thought on evidence-based education

The skepticism toward the evidence-based approach was met with a compelling riposte from the EEF’s chief executive, Dr. Kevan Collins, during the foundation’s inaugural conference in October. Is it not a waste of our public money and even unethical to keep pursuing approaches in schools that could be having little, or even a negative impact on the learning of our children, particularly those from poorer backgrounds? That is the question that every teacher needs to ponder on. Indeed one day I hope it will not only be researchers and educationalists asking these questions. It will be parents and children demanding: “Show me the evidence!”

About the author

Dr. Lee Elliot Major is director for research and policy at the Sutton Trust and chair of the EEF’s evaluation advisory group.


February 2012