Lernportal

Synthetic control method

The synthetic control method (SCM) is a quasi‑experimental approach that offers a robust way to estimate what would have happened in the absence of a CAP intervention by building a synthetic comparison unit from similar regions or countries. It is particularly useful for assessing the impact of policy interventions implemented at the regional or national level.

Page contents

Young male agronomist holding up rain gauge

Basics

In a nutshell

SCM is a method to estimate the impact of a policy that applies to one (or very few) countries, regions, or sectors. It builds an ‘artificial twin’ of the treated unit by combining several unaffected units (with weights) so that this synthetic twin behaves like the treated unit before the policy. Any clear gap that appears after the policy is interpreted as the policy’s effect.

SCM chooses weights for a group of untreated units to build an artificial twin of the treated unit so that their weighted average closely matches the treated unit before the intervention, in terms of outcomes (e.g. agricultural production) and key characteristics (e.g. farm structure).

After the policy starts (e.g. a CAP reform), the treated unit’s trajectory is compared with that of its synthetic (artificial) twin. The core assumption is that, without the policy, the treated unit would have continued to evolve like this synthetic combination. Under this assumption, the difference between the two paths after the intervention is taken as the effect of the policy.

Example: Using SCM to assess the impact of a CAP reform in Country A

Suppose we want to evaluate the impact of a CAP reform introduced in 2014 on agricultural value added in Country A.

We begin by selecting a group of untreated countries (for example, Countries B, C, D and E that did not implement the same reform).

The SCM algorithm chooses a set of non‑negative weights for these donor countries so that their pre‑reform trends and characteristics resemble those of Country A as closely as possible. Countries with pre‑reform trends and characteristics closer to Country A receive higher weights. Countries that don’t resemble A receive weights close to zero. For example, the optimal weights might be: B = 0.4, C = 0.3, D = 0.3, E = 0.

Using these weights, we build ‘Synthetic Country A’ as a weighted average of the donor countries.

The same set of weights is applied to:

pre‑reform agricultural value added (e.g., 2000–2013), and
covariates, such as farm size distribution, input use per hectare or livestock density.

This means every indicator of Synthetic A is the weighted combination of the corresponding indicators in B, C and D.

After 2014, we compare the evolution of agricultural value added in:

actual Country A, and
synthetic Country A (what A’s trajectory would have looked like without the reform).

If, after the 2014 reform, actual Country A’s agricultural value-added rises clearly above the synthetic trajectory, this gap is interpreted as the effect of the CAP reform under the SCM assumptions (mainly that no other major shocks affected Country A but not its donor group).

Pros and cons

Advantages	Disadvantages
Useful when only one or a few units are treated, and no clear control group exists.	Strongly data‑dependent: needs long, high‑quality pre‑policy data and several suitable donor units
More flexible than standard DiD when parallel trends are doubtful, because SCM chooses weights so that the synthetic unit closely reproduces the results of the treated unit in all pre-treatment years.	Results are context‑specific and may not generalise to other Member States, regions or periods. The same caveat applies to EU-level, national or regional SCM studies, where estimated effects are local to the treated unit, the donor pool and the time window used in the analysis.
Provides a visually intuitive comparison (treated vs. synthetic path)	Results may vary depending on how we set up the method. In particular, they depend on the countries or regions we use for comparison, the characteristics we choose to match (e.g. farm structure or production levels) and the years examined before the reform. If the synthetic ’twin’ does not closely follow the unit treated before the start of the policy, the results are much less convincing.
	Statistical inference is less standard than in regression models and can be harder to explain to non‑specialists.

When to use?

SCM is most appropriate when a large‑scale CAP change affects one (or very few) countries, regions, or sectors (e.g. major reform, specific national implementation). SCM is usually applied to such aggregate units (countries, regions, sectors), but in principle, similar ideas can be used at more detailed levels when suitable data and comparison units exist.

It is particularly suitable when:

other broadly similar units that have not undergone the same change in CAP, so they can be used to build a credible synthetic twin;
a long pre-intervention period with consistent data is available to document trends, and
the goal is to construct a credible counterfactual trajectory for an aggregate unit (country, region, sector).

SCM is not intended for routine farm‑level evaluation or for measures implemented everywhere at the same time, because there is no untreated comparison group to build a synthetic control.

Preconditions

To use SCM credibly and obtain meaningful results, evaluators need:

A clear treated unit (e.g. one Member State or region subject to a specific reform).
A pool of comparable untreated units to build the synthetic control (artificial twin).
A long and reliable pre‑policy time series (ideally greater than 8-10 years) for the outcome and key covariates.
A deep knowledge of the policy timing and context to define the intervention date and donor pool appropriately.

These elements ensure that SCM can match pre‑reform trends accurately and generate a synthetic counterfactual that is credible.

When to use SCM in the context of CAP Strategic Plan assessments

SCM is mainly suited to aggregate indicators at country or large‑region level, such as:

Agricultural production indexes, sectoral value added, or agricultural GDP
Indicators of agricultural support levels.
Macro‑level environmental or structural indicators, where good historical series exist.

As mentioned earlier, it is less suited to detailed farm‑level indicators, where microdata methods are preferable.

Step-by-step

Step 1 – Define the treated unit and policy change (e.g. a Member State at the time of a CAP‑related reform).

Step 2 – Select the donor pool of untreated units (countries/regions not subject to the same change).

Step 3 – Choose what to match and over which past years:

First, decide which key characteristics (predictors) of the country or region should be similar in the synthetic twin, for example, farm structure, input use or past levels of agricultural production.
Then select a pre‑intervention period (a run of years before the policy starts) over which the synthetic unit should closely follow the treated unit’s outcome path.

The SCM procedure will then assign weights to the comparison units so that, in those years, the synthetic twin matches both the chosen characteristics and the outcome trajectory as well as possible.

Step 4 – Compare post‑intervention paths of the treated unit and its synthetic twin.

For example, suppose that before the 2014 CAP reform, a region's agricultural added value always closely tracked that of its synthetic twin. After the reform, between 2015 and 2020, the actual region grew to around 110, while the synthetic region remained close to 100. In each post-reform year, we examine the gap between these two lines (e.g., 110 minus 100 = 10 index points). If the pre-reform correspondence was good and no other major shock affected this region alone, then this post-reform gap is interpreted as the estimated effect of the CAP reform on agricultural value added.

Step 5 – Conduct placebo and robustness checks

A placebo check is a ‘fake treatment’ test used to evaluate whether the research design would still detect an effect when it should not.

For example, we repeat the same SCM procedure but pretend that other untreated countries or regions received the policy (e.g. Country C did not adopt the reform, but we treat it as if it had been treated by setting a fake intervention date, e.g. 2014, even though nothing changed in C in 2014 and then we build a synthetic control for Country C, using the same method as for Country A).
If many of these placebo units show gaps as large as the treated unit, the estimated effect may simply reflect random noise.
If the treated unit displays a much larger gap than any placebo unit, this strengthens the credibility of the finding and supports the interpretation that the policy had an effect.

Main takeaway points

SCM builds a data‑driven ‘artificial twin’ of the treated unit by optimally combining untreated units to approximate what would have happened without the policy.
It is especially useful for cases where there are only a few treated units, such as large CAP reforms or national implementation choices
Its reliability depends on good pre‑policy fit, appropriate donor selection and expert implementation.
It is a specialised tool, designed for specific evaluation questions at country, regional or sector level, and not intended for routine, farm‑level or universally applied measures.

Learning from practice

Olper, A., Valenti, D., Raimondi, V., & Curzi, D., (2023), The EU enlargements treatment effect on agricultural policy, Applied Economic Perspectives and Policy, 45(2), 1134-1153.

The authors use SCM to examine how EU membership has affected the level of agricultural protection in acceding countries. Each enlargement is treated as a ‘political shock’ for the incoming Member State, and a synthetic control is constructed from non‑acceding countries that match agricultural protection indicators and pre‑accession macroeconomic conditions. Comparing post‑accession trajectories, the authors find that in earlier enlargements (1973 and 1985), accession significantly increased agricultural assistance relative to the synthetic counterfactual.