There are many theories on how to analyse the efficiency of a development intervention. Then again, these are just theories. What happens in the real world? In practice, it can be quite a struggle to analyse the efficiency of a certain project in difficult circumstances. That is why on 23 November 2017, Partos / The Spindle organised a conference on efficiency. The Efficiency Lab challenged three experts on to recommend what methods to use in different, practical cases. One of the experts is Markus Palenberg. Why are efficiency analyses of development projects often of poor quality? Below, Markus Palenberg tries to explain and offers a conceptual framework that can help to solve this.
by Markus Palenberg
Why even bother doing efficiency analysis? There are compelling reasons why professionals working in international aid cannot get around efficiency analysis. Namely, efficiency analysis helps choosing the “right” interventions. It can help selecting interventions that are 1) overall beneficial and 2) maximize overall net benefits to society. Next to this, efficiency analysis helps improving interventions. It can inform operational and strategic improvement of interventions, through minimizing costs and maximizing yields. Also, efficiency is a standard evaluation criterion. For example, OECD DAC lists efficiency as one of five recommended criteria for evaluating development assistance. And last but not least, efficiency analysis is required by law. Budget codes in several countries require efficiency analysis for measures affecting national budgets.
Why is it so difficult?
Efficiency remains an elusive or ambiguous concept, and efficiency analysis is often considered somewhat of a challenge. Again, there are several reasons for that. One is efficiency blabber jabber: efficiency-related terminology and concepts are confusing and ambiguous. For example, OECD DAC offers two conflicting definitions and UK’s “value for money” approach uses a narrow definition. Then there’s efficiency analysis without a clear purpose, resulting in lack of direction for analysis and evaluation findings of limited usefulness. Lastly you need proper skills and tools. Some types of efficiency analysis are rather demanding and require expert skills and experience.
Our terminology and concepts
We adopt a broad understanding of the term “efficiency”. We include, for example:
- Output-level efficiency (production efficiency): conversion of inputs into outputs;
- Impact/outcome-level efficiency (allocation efficiency): conversion of inputs into outcomes or impacts; or
- Net benefit, utility, and other economic efficiency measures.
We differentiate two principal types of efficiency analysis based on their purposes:
- Level 2 analysis compares the efficiency of entire aid interventions with alternatives or benchmarks with the purpose of maximizing total welfare (by selecting those interventions that produce the largest net benefit to society with available resources).
- Level 1 analysis has the purpose of improving the efficiency of individual interventions. It does this by investigating the intervention at hand and/or by benchmarking partial efficiency indicators across several interventions. In contrast to level 2 analysis, level 1 analysis does not assess or compare outcome/impact-level efficiency of different interventions.
- In addition, a third type of efficiency analysis exists: descriptive (level 0) analysis describes or provides an opinion on efficiency without a clearly identified purpose.
Along these three types, we have collected the following list of tools and methods for assessing the efficiency of aid interventions:
|Type||Level 2 Analysis||Level 1 Analysis||Descriptive Analysis|
|Description||Compare the efficiency of entire aid interventions with alternatives and benchmarks||Identify efficiency improvement potential in an aid intervention||Describe or provide an opinion on efficiency|
Select the most efficient interventions from many
Improve the efficiency of one intervention
(Méthode des Effets)
Social Return on Invest
Multiple-Attribute Decision-Making (MADM)
|Unit Cost Benchmarking
Follow the Money
Efficiency Ratings by Stakeholders
Specific Evaluation Questions on Efficiency
Data Envelopment Analysis
Stochastic Frontier Analysis