In recent years, lots of efforts have been put into coming to grips with measuring the outcomes and impact of development interventions. Meanwhile, the efficiency question (whether results are proportionate to the costs incurred) often remains inadequately addressed in evaluations. There are however some practical guidelines that could help you with your efficiency analyses!
By Heinz Greijn
There is a lot of confusion about the concept of efficiency. This limits what we can learn from evaluations. Without efficiency analysis it is not possible to select the projects that generate most impact with the budget available, it is hard to make statements about the scalability of interventions, and there is little incentive to innovate and develop ways do more with less.
Limited understanding of efficiency
Last week I wrote that one of the main problems with understanding efficiency, is that it is often reduced to prudent procurement and contacting of services. Efficiency is limited to the relation between costs of inputs and outputs. Even the OECD/DAC whose glossary is often referred to as the standard in the sector, is doing this. According to these definitions even a project that has no, or even negative, outcomes or impact, can still be efficient. A definition that can lead to such conclusions is not helpful for innovation and the improvement of interventions. A useful definition must be based on the premise that effectiveness is a prerequisite for efficiency. In other words, without effectiveness there can be no efficiency.
Practical guidelines for analysing efficiency
In his paper ‘Efficiency analysis: Balancing between rigour and realism’ Antonie de Kemp (IOB evaluator and “chief efficiency”) elaborates on the definition challenge. He also explains why efficiency analysis in the areas of development cooperation and foreign policy does not receive much attention. Antonie recognizes that in our field it is not always possible to apply the most rigorous methods for analysing efficiency. Therefore, he concludes with some practical guidelines that can help practitioners and evaluators improving their efficiency analysis such as:
- Effectiveness, effectiveness, effectiveness first: no measurement of efficiency without evaluating effectiveness.
- Realism and a thorough understanding of the program and what it may accomplish.
- Need to take into account differences when comparing groups.
- An analysis who benefits.
- Need to devote more resources to monitoring and evaluation.
- Focus: efficiency of the efficiency analysis: do not devote 80% of your evaluation resources on trying to estimate a 20% impact.
- Keep it as simple as possible: You do not need to publish your results in an A-Journal, but it is important to get a better understanding of the relationship between the costs and (potential) benefits of your project.
Antonie has been evaluator at Policy and Operations Evaluation Department (IOB) since 2005. Previously he worked as a researcher for the Netherlands Court of Audit, the Netherlands Institute for Social Research (SCP) and the Institute for Research on Public Expenditure (IOO). He also advised on these ten practical cases of efficiency, published by The Spindle earlier.
Learn how to analyse efficiency in this 2-day training!
Only few practitioners and evaluators are familiar with the methods and tools to assess efficiency. Therefore, Partos will organise on 19 and 21 June a training on efficiency analysis. Trainers are Markus Palenberg and Pol de Greve. The target group of this training are evaluators, M&E managers and other professionals for whom the efficiency question is relevant in the design, planning, implementation and evaluation stage of projects.
In this two-day training participants will learn when and how to apply a variety of methods and tools including: cost effectiveness analysis (CEA), cost benefit analysis (CBA), multiple attribute decision making (MADM), benchmarking unit costs, partial efficiency indicators, follow the money, financial analysis and comparative rating by stakeholders. Here you can find more information about the training and register.
Questions? Contact Heinz Greijn – facilitator of The Efficiency Lab powered by Partos /The Spindle.