New methods of economic evaluation are being developed to assess the lifelong impacts of educational interventions on individual well-being and social inequality. Richard Cookson describes a promising new approach based on the “quality adjusted life year” concept routinely used in the health sector
Economic evaluation is an essential and inevitable component of evidence-informed decision-making in the education sector. Effectiveness evidence alone is not enough. Introducing new educational interventions almost always requires an investment of time, money, and other scarce resources. These scarce resources could be deployed in other ways, to deliver other important benefits. It is therefore not enough to know that an intervention is likely to have beneficial educational effects. Decision makers also need to assess whether the benefits are worth the costs. Implicitly or explicitly, they need to tot up the costs, estimate any long-run savings, and compare these costs and savings with the benefits to children, parents, and wider society. Economic evaluation is a way of carrying out this kind of assessment carefully and transparently, using a standard set of methods and based on evidence and assumptions that can be subjected to independent scrutiny.
|What we know|
|● Standard methods of economic evaluation do not provide information about lifelong impacts on individual well-being and social inequality.
● New methods are being developed based on the “quality adjusted life year” (QALY) from health economics.
● The idea is to value interventions by how many years of good life they produce, rather than how much people are willing to pay for them.
Recent advances in cost–benefit analysis
The standard approach to economic evaluation in the education sector is known as cost-benefit analysis. In recent years, practitioners of cost-benefit analysis have made important strides in addressing two key weaknesses in the evidence base:
- Bias and uncertainty in effectiveness estimates; and
- Lack of evidence about long-term effects after the effectiveness study has finished.
The first weakness is addressed using modern methods of evidence synthesis, such as meta-analysis, which combine evidence from multiple different effectiveness studies to form a more balanced, overall estimate. Effectiveness estimates are also sometimes weighted to allow for bias. For example, evidence from studies conducted by intervention developers and other “enthusiasts” tends to be more favourable than studies conducted by independent teams. The second weakness is addressed using modern methods of simulation modelling, which take short-term outcomes from trials and then extrapolate them into the future based on a variety of further assumptions and evidence. Both of these recent advances draw on methods for evidence-informed decision-making developed in the health care sector.
Two unsolved problems
However, cost-benefit analysis still suffers from two major limitations. First, it values outcomes in terms of individual consumption rather than individual well-being. Second, it provides no information about the impacts of interventions on social inequalities. In standard cost-benefit analysis, non-monetary outcomes are given a monetary value based on how much people are willing to pay for them in terms of foregone economic consumption. These monetary amounts are then added up to create an overall measure of benefit. One concern about this is that focusing on consumption as the ultimate unit of value may distract attention away from what really matters from a policy perspective, ie, improving human well-being. More importantly, the conventional approach fails to measure well-being impacts accurately.
There is substantial variation between people in how far changes in consumption impact upon their well-being. Two particularly important sources of variation are that an extra dollar:
- Does more to improve the well-being of a poor person than a rich person; and
- Is no use to anyone after their death.
Failing to allow for these variations between individuals in the conversion rate from consumption to well-being can generate controversial ethical implications. For instance, the standard approach implies that end-of-life medical care to extend a wealthy older person’s life by a few months is a far more worthwhile use of public funding than preschool education yielding lifelong well-being gains to a child from a low-income family. This is because the wealthy older person is able and willing to pay far more than the low-income family – both because they are wealthier and because they cannot enjoy their wealth after their death – even though the child stands to gain many more years of good life.
The well-being QALY
To address these two limitations, methods from the health field – suitably adapted – may again prove helpful. In particular, the concept of the “quality adjusted life year” (QALY) could be adapted for use in the education sector. The QALY measure of individual health is now routinely used in the health sector to make comparisons between many different kinds of health outcomes. Instead of measuring years of healthy life – the health QALY – one could measure years of good life – the well-being QALY. The basic idea is the same: to measure years of life, adjusted for quality of life. The difference is that to evaluate educational interventions one would need to adjust for a wider range of dimensions of quality of life. This approach would then allow the costs and benefits of social policies to be compared in terms of well-being, rather than just in terms of money.
This proposal has been made recently by the Commission on Well-being and Policy, chaired by a former head of the UK civil service, Gus O’Donnell. According to O’Donnell and colleagues, “the UK governmental body NICE (the National Institute for Health and Care Excellence)…has performed a signal service. It has shown to the world that the well-being approach can become an acceptable basis for public policy.” The basic idea is to measure the impacts of educational interventions on lifelong well-being, in terms of additional years of good life. This requires micro-simulation modelling of the entire individual life course, extrapolating short-run educational outcomes to longer-term life events of interest. An important benefit of such models is that they allow one to estimate social distributions of outcomes by income, ethnicity or other individual characteristics of interest. This makes it possible to evaluate long-term impacts on social inequalities, and to evaluate trade-offs between efficiency (ie, doing the most good) and equity (ie, reducing social inequality).
This approach has not yet been applied to evaluate educational interventions. However, researchers are actively working in this area and in the next few years we can expect prototype analyses to be performed. In the health sector, it took approximately 30 years for the QALY to move from the stage of “interesting idea” to “routinely used”. It is to be hoped that acceptance of the wellbeing QALY will not take quite so long in the education sector.
About the author
Richard Cookson is a Professor and NIHR Senior Research Fellow in the Centre for Health Economics, University of York. His research focuses on equity in health and health care, and he has served in various advisory roles in the health sector.
Asaria M, Cookson R, and Griffin S (2014), Incorporating Health Inequality Impacts into Cost-Effectiveness Analysis. In: Anthony J. Culyer (ed.), Encyclopedia of Health Economics, Vol 2. San Diego: Elsevier. pp. 22–26.
Cookson R et al (forthcoming), Wellbeing QALYs: A Practical Measure of Well-being for Monitoring Social Progress and Evaluating Social Policies. In Nir Eyal, Samia Hurst, Drew Schroeder and Dan Wikler, Ethical Issues in Measurement of the Global Burden of Disease, Oxford University Press.
Cookson R and Culyer A (2010), Measuring Overall Population Health: The Use and Abuse of QALYs. In Evidence-based Public Health: Effectiveness and Efficiency, edited by A. Killoran and M. Kelly, pp. 148–68. Oxford: Oxford University Press.