Quantile methods
- Evaluation
- Evaluation
- Evaluation Methods
- Data Management
- Performance Monitoring and Evaluation Framework (PMEF)
- Monitoring and evaluation framework
Quantile methods (quantile dose-response function (QDRF) and quantile conditional treatment effect (QCTE)) help CAP evaluators understand how support affects different types of farms (e.g. low income, middle income, and high income farms), revealing patterns that are hidden behind average effects. This makes them especially valuable for assessing equity and targeting dimensions of CAP measures.
Page contents
Basics
In a nutshell
Quantile methods allow evaluators to look beyond the average effect of CAP support and examine how effects differ across the entire distribution of farms (e.g. low‑income, middle‑income, and high‑income farms).
Instead of estimating a single impact of CAP support on an outcome (e.g. income), quantile methods estimate effects at multiple points of the distribution (such as the 10th, 50th, and 90th percentiles).
This helps identify patterns, such as whether an additional euro of support has a stronger impact on lower‑income farms or whether higher‑performing farms benefit more.
For example, quantile analysis can reveal whether:
- an additional euro of support has a stronger stabilising effect on farms at the bottom of the income distribution (lower quantiles) compared to high-income farms (upper quantiles);
- productivity gains from investment subsidies are concentrated among already high-performing farms or are more evenly distributed; and
- direct payments reduce income volatility primarily for smaller or lower-income farms.
The two main tools are:
-
QDRF examines how different levels (‘doses’) of support influence different types of farms along the distribution (e.g. low‑income vs. high‑income). The focus is on the variation in the amount of support.
Example: Imagine farms receive different amounts of investment support: EUR 0, EUR 5 000, EUR 10 000, EUR 20 000. The QDRF tells you, for example:
- At the 10th percentile (poorest farms): increasing support from EUR 5 000 to EUR 10 000 increases income by X%.
- At the 90th percentile (wealthiest farms): the same increase in support has almost no effect.
-
QCTE estimates how the treatment effect varies across quantiles, conditional on receiving the treatment. It compares treated vs. untreated farms at each quantile. The focus is on the variation in the effect of being in the treatment group (binary treatment).
Example: Suppose some farms receive investment support (treatment = 1) and others do not (treatment = 0). The QCTE estimates:
- At the 25th percentile: receiving support increases income by EUR 2 000.
- At the 75th percentile: receiving support increases income by EUR 400.
These methods show how CAP support affects farms with very different characteristics or starting conditions, rather than only the ‘typical’ or average farm.
Key assumptions include:
- The relationship between CAP support and outcomes may differ across the distribution (heterogeneous effects).
- The statistical model must be correctly specified for each quantile.
- Sufficient data must be available across all relevant parts of the distribution to produce reliable estimates.
Pros and cons
| Advantages | Disadvantages |
|---|---|
| Reveal heterogeneous effects: show who benefits more (e.g. small vs large, low‑income vs high‑income farms). | Technically demanding, requiring advanced econometric knowledge. |
| Go beyond averages to assess equity and distributional impacts of CAP support. | Need large samples; results can be unstable in the tails (very low or very high quantiles) if there are few observations. |
| More complex to explain to non‑technical audiences. |
When to use?
Use quantile methods when:
- The policy question is explicitly about distributional impacts (e.g. does support reduce income inequality? who gains most from a measure?).
- Support levels vary across farms, and you suspect that effects differ across farm types or income levels.
- Support is broadly provided to (almost) the entire farm population (e.g. decoupled direct payments), so there is no clear untreated control group, and you want to exploit variation in the level of support rather than a treated/untreated comparison;
- There are large micro‑datasets (e.g. FADN/FSDN) and the assessment wants to go beyond ‘one average effect’.
- They are a good complement to average‑effect methods (e.g. DiD) when equity or targeting issues are central.
Preconditions
- Large cross‑section or panel datasets (many farms, several years) to obtain reliable estimates at different quantiles.
- Continuous or at least multi‑valued treatment (e.g. varying levels of support, not only 0/1).
- Access to strong econometric skills and software, as quantile methods are more complex than standard methods.
When to use quantile methods in the context of CAP Strategic Plan assessments
Quantile methods can be used to analyse how CAP support affects not only the average farm but also the entire distribution of outcomes.
Here, the ‘distribution of outcomes’ refers to the distribution of the dependent variable (e.g. farm income), but quantile methods can be applied separately or interactively for different groups of farms (e.g. by farm size, type or region) to explore whether distributional effects differ across these categories.
They are suitable for studying how support influences:
- Farm income or entrepreneurial income to see whether lower‑income farms gain proportionally more (or less) than higher‑income ones.
- Agricultural factor income to examine distributional changes across different types of farms.
- Productivity indicators (e.g. total factor productivity) to assess whether support primarily benefits already high‑performing farms or helps lower‑performing farms catch up.
- Environmental or social indicators, provided they are measured at farm level and distributional patterns are relevant, e.g. how support affects low‑ vs. high‑emission farms, or farms with low vs. high employment intensity.
These methods are especially valuable when it is important to understand how outcomes differ across farms, not just the average effect. This is particularly relevant to policy objectives such as fairness (equity), the stability of vulnerable farms (resilience) and targeted support. They help evaluators understand whether CAP measures reduce disparities, reinforce existing gaps or produce heterogeneous effects that may be hidden when looking only at average impacts.
Step-by-step
Step 1 – Define the outcome and treatment, e.g. farm income as outcome, level of CAP support as (continuous) treatment.
Step 2 – Choose quantiles of interest, e.g. 10th, 25th, 50th, 75th, 90th percentiles,
Step 3 – Estimate quantile models (QDRF or QCTE) to obtain effects of support at each quantile, controlling for relevant farm characteristics (e.g. farm size, production type, location, input prices, weather conditions), so that the estimated effects of support are not confounded by systematic differences between farms.
Step 4 – Compare results across quantiles, identifying whether support has stronger effects for lower‑income farms, median farms or top performers.
Step 5 – Interpret in policy terms in relation to target, redistribution and equity objectives. In interpreting the results, evaluators should translate distributional patterns into clear policy messages.
For example, if the support generates relatively higher effects for low-income farms, this can be considered evidence that the measure contributes to equity objectives and better targeting of vulnerable farms, supporting the need to maintain or strengthen objectives aimed at these groups.
If, on the other hand, the support generates relatively higher effects for higher-income or larger farms, this may indicate that the current design reinforces existing disparities, suggesting the need to adjust eligibility rules, payment formulas or modulation/limitation to improve redistribution.
Similar reasoning applies to other outcomes such as productivity or environmental performance. For example, if productivity gains are concentrated among already high-performing farms, or if environmental improvements are limited to a small group of farms, this can motivate recommendations to refine targeting criteria or incentive levels so that support better reaches underperforming or high-priority groups.
Main takeaway points
- Quantile methods reveal how CAP support affects different types of farms across the entire distribution – not only the average farm. They help uncover whether low‑income, low‑productivity or otherwise vulnerable farms benefit differently from support compared with higher‑performing farms. This provides a much richer picture than average effects alone.
- They are especially valuable for evaluating equity, redistribution, and targeting aspects of the CAP. These methods directly answer questions such as ‘Who benefits the most?’, ‘Does support reduce disparities among farms?’ or ‘Are vulnerable groups catching up or falling behind?’ – questions that are central to many CAP objectives.
- They require large datasets and advanced statistical expertise, so they are best used by experienced analysts as a complement to simpler methods. In practice, quantile analysis is most effective when evaluators have access to detailed micro‑data and the capacity to estimate models separately across the distribution. When used alongside more conventional approaches, they add important insights that would otherwise remain hidden.
Learning from practice
Ciliberti, S., Severini, S., Ranalli, M.G., Biagini, L., Frascarelli, A. (2022), Do direct payments efficiently support incomes of small and large farms?, European Review of Agricultural Economics, 49(4), 796–831.
Ciliberti et al. (2022) apply a quantile continuous treatment effect model to Italian FADN data to assess how CAP direct payments improve farm incomes across the entire farm size distribution. Their results show that income responses to direct payments are higher on large farms than on small ones, and lower on farms receiving a higher level of support. This suggests that direct payments are not particularly effective in supporting small farm incomes and reducing disparities within the farming population.
Further reading
Assessing the effectiveness and efficiency of CAP income support instruments
The guidelines on how to assess CAP income support instruments offer a comprehensive evaluation framework, including judgment criteria and technical guidance for quantitative analyses.
- Evaluation