Learning from AgResults

AgResults learning takes two forms:

  1. Learning generated from the experience of AgResults stakeholders in designing and implementing projects;
  2. Learning generated from the External Evaluator in its independent ongoing review of AgResults projects.

AgResults has generated a wealth of learning, results of which can be found in the Learning Library


bloc 1
bloc 2
bloc 3

External Evaluation

The AgResults Steering Committee appointed Abt Associates to serve as the External Impact Evaluator for the AgResults pilots. The Evaluator’s role is to determine on a scientific basis if the AgResults pilots’ pull mechanisms achieve their objectives. In that role, the Evaluator does the following:

  • Uses a mix of quantitative and qualitative methods
  • Assesses whether pilots produce private sector behaviors or social outcomes that are better or different than without PfR incentives
  • When possible, compares PfR mechanism outcomes against traditional “push” outcomes
  • Analyzes cost effectiveness of PfR projects and sustainability of results

Evaluator Questions

What has been the impact of the AgResults project/pilot on private sector involvement in the development and uptake of agricultural innovations?
What has been the impact of the AgResults project/pilot on smallholders’ update of innovative technologies?
What has been the impact of the AgResults project/pilot on smallholders’ incomes?
What evidence exists that the AgResults pilots are scalable and that their effects will be sustainable in the medium to long term?
What has been the impact of the AgResults project/pilot on poor customers’ demand for derivative food products?
What is the evidence on the scale of any effect on private sector investment and uptake, and on the cost-effectiveness of AgResults as an approach?
What lessons can be learned about the best practices in the design and implementation of agricultural pull mechanisms?

AgResults Theory of Change

PfR incentives seek to overcome market failures for particular agricultural technologies. They incentivize the private sector to work towards pre-defined results through rewards that are paid out only if the results are achieved. AgResults projects use PfR incentives to encourage key private sector actors’ involvement in the market for a technology, either by developing the technology or encouraging its adoption by poor households.

The failure of a market to develop can be caused by low supply, low demand, a weak policy environment, or a combination of these. This diagram shows some examples:

Constraints Limiting Market for Agriculture Technology

Evaluation Framework

The Evaluator is charged with assessing whether the PfR incentives address the key underlying market failures and engage the private sector in the market while leading to smallholder impact as implied in the evaluation questions.

The Evaluator employs a common evaluation across all the AgResults projects, applying economic theory to examine how each incentivizes the private sector to overcome the market failure and improve smallholder welfare. It also identifies behaviors of market actors that might impede or aid development impact on smallholders.

The common evaluation framework allows for comparison across pilots and generalization of results and lessons learned. In all AgResults project evaluations, the Evaluator applies mixed methods using rigorous qualitative and quantitative analytic methods to test economic theory-driven hypotheses. The diagram below displays how the Evaluator answers all evaluation questions in its charge.

Evaluation Stages

1. Initial Qualitative Assessment

To inform each project-specific evaluation design, conduct an initial qualitative assessment to assess the pilot context, form a hypothesis of expected impact, examine the expected heterogeneity of impacts, identify external factors that might inhibit or boost impact, and meet with key stakeholders to secure their cooperation and buy-in.

2. Baseline

During the baseline stage, gather information on market for the technology, its adoption, initial engagement of potential competitors, and smallholder livelihoods and agricultural practices.

3. Endline

During the endline stage, gather information on competitor activities, development of the technology market, and smallholder adoption and income gains

4. Cost-Effectiveness Analysis/Synthesis

At the end of AgResults, synthesize the findings across pilots to draw lessons on the design and implementation of pull mechanisms. Assess AgResults’ cost effectiveness in achieving key outcomes across projects and also as compared to traditional approaches.

5. Sustainability Analysis

For key projects, evaluate the sustainability of AgResults’ impact by assessing private sector engagement in technology provision and the continuation of favorable smallholder outcomes two years after the project ends.