Efficiency analysis: Why bother? … and why is it so difficult?

By Markus Palenberg, Institute for Development Strategy

There are many theories on how to analyse the efficiency of a development intervention. Then again, these are just theories. What happens in the real world? In practice, it can be quite a struggle to analyse the efficiency of a certain project in difficult circumstances. That is why on 23 November 2017, Partos / The Spindle organised a conference on efficiency. The Efficiency Lab challenged three experts on to recommend what methods to use in different, practical cases. One of the experts is Markus Palenberg. Why are efficiency analyses of development projects often of poor quality? Below, Markus Palenberg tries to explain and offers a conceptual framework that can help to solve this.

by Markus Palenberg

Why even bother doing efficiency analysis? There are compelling reasons why professionals working in international aid cannot get around efficiency analysis. Namely, efficiency analysis helps choosing the “right” interventions. It can help selecting interventions that are 1) overall beneficial and 2) maximize overall net benefits to society. Next to this, efficiency analysis helps improving interventions. It can inform operational and strategic improvement of interventions, through minimizing costs and maximizing yields. Also, efficiency is a standard evaluation criterion. For example, OECD DAC lists efficiency as one of five recommended criteria for evaluating development assistance. And last but not least, efficiency analysis is required by law. Budget codes in several countries require efficiency analysis for measures affecting national budgets.

Why is it so difficult?
Efficiency remains an elusive or ambiguous concept, and efficiency analysis is often considered somewhat of a challenge. Again, there are several reasons for that. One is efficiency blabber jabber: efficiency-related terminology and concepts are confusing and ambiguous. For example, OECD DAC offers two conflicting definitions and UK’s “value for money” approach uses a narrow definition. Then there’s efficiency analysis without a clear purpose, resulting in lack of direction for analysis and evaluation findings of limited usefulness. Lastly you need proper skills and tools. Some types of efficiency analysis are rather demanding and require expert skills and experience.

Our terminology and concepts
We adopt a broad understanding of the term “efficiency”. We include, for example:

  • Output-level efficiency (production efficiency): conversion of inputs into outputs;
  • Impact/outcome-level efficiency (allocation efficiency): conversion of inputs into outcomes or impacts; or
  • Net benefit, utility, and other economic efficiency measures.

We differentiate two principal types of efficiency analysis based on their purposes:

  • Level 2 analysis compares the efficiency of entire aid interventions with alternatives or benchmarks with the purpose of maximizing total welfare (by selecting those interventions that produce the largest net benefit to society with available resources).
  • Level 1 analysis has the purpose of improving the efficiency of individual interventions. It does this by investigating the intervention at hand and/or by benchmarking partial efficiency indicators across several interventions. In contrast to level 2 analysis, level 1 analysis does not assess or compare outcome/impact-level efficiency of different interventions.
  • In addition, a third type of efficiency analysis exists: descriptive (level 0) analysis describes or provides an opinion on efficiency without a clearly identified purpose.

Along these three types, we have collected the following list of tools and methods for assessing the efficiency of aid interventions:

 

Type Level 2 Analysis Level 1 Analysis Descriptive Analysis
Description Compare the efficiency of entire aid interventions with alternatives and benchmarks Identify efficiency improvement potential in an aid intervention Describe or provide an opinion on efficiency
 

Purpose

 

Select the most efficient interventions from many

 

Improve the efficiency of one intervention

 

(Unclear)

Examples  

Cost-Benefit Analysis

(Méthode des Effets)

Cost-Effectiveness Analysis

Cost-Utility Analysis

Social Return on Invest

Multiple-Attribute Decision-Making (MADM)

Unit Cost Benchmarking

Follow the Money

Efficiency Ratings by Stakeholders

Financial Analysis

 

Expert Judgement

Specific Evaluation Questions on Efficiency

 

Data Envelopment Analysis

Stochastic Frontier Analysis

Crazy solutions for complex problems

Read more

Do you know new civil society partnerships?

Read more

Blockchain: Ideals and Technology hand in hand

Read more

Sign up to our newsletter and stay up to date