Randomised Controlled Trials

Two farmers in a field controling a machine

Basics

In a nutshell

In an RCT, evaluators first define a target group (for example, eligible farmers in certain regions) and then randomly split them into:

a treatment group, which receives a new version of the policy (e.g. a redesigned contract, a higher payment, or a specific advisory service); and
a control group, which receives the current or no version of the policy.

Outcomes (such as adoption of practices or changes in environmental or economic indicators e.g. income) are then measured in both groups and compared.

The key assumption is that, thanks to randomisation, the only systematic difference between the groups is the policy variant itself. Under this condition, the difference in outcomes can be interpreted as the causal effect of that policy design.

Pros and cons

Advantages	Disadvantages
Provide high‑quality causal evidence on whether a specific policy variant works, because randomisation strongly reduces selection bias and unobserved confounding.	Very difficult to implement for political, ethical, or legal reasons, especially when withholding or delaying support is considered unacceptable.
Allow clear comparisons between alternative designs (e.g. different payment levels or contract formats) in real‑world conditions	Results are typically context‑specific (particular regions, sectors, or groups), so generalisation to other regions/sectors/contexts or future periods must be done cautiously.
	Require careful planning, strong coordination, and additional resources for design, communication, data collection, and monitoring.

When to use?

RCTs cannot be introduced retrospectively during the evaluation phase. They must be planned in advance. The random allocation of units to treatment and control groups has to be embedded in the design of the measure from the outset, with groups defined and randomly assigned before the policy (or policy variant) is implemented. If groups are not established ex ante through a random process, an RCT design cannot be reconstructed ex post.

RCTs are most appropriate when:

The evaluation question concerns policy design choices (for example, payment format, contract type or the provision of advisory support), typically within a new or existing voluntary intervention. In the context of the CAP, it is generally neither feasible nor acceptable to randomise access to a whole CAP measure. However, randomisation may be possible when comparing a newly introduced intervention (e.g. a pilot agri-environment scheme) with the status quo, or when testing alternative variants within that new intervention.
The measure is voluntary and subject to budget or capacity constraints, such as agri‑environment‑climate schemes or eco‑schemes, meaning that not all eligible farmers can or must receive the same version immediately. This makes it possible, in principle, to introduce controlled variation in who receives which policy variant without denying access to an entire measure.
It is possible to pilot the measure before full rollout, for example, by testing it in selected areas or among a subset of eligible farmers. In such cases, random assignment within the eligible group can be used to compare different design options before scaling the measure up nationally.

Preconditions

Sufficient numbers of eligible farms to form treatment and control groups. There is no fixed ‘magic number’, but each group must be large enough to ensure adequate statistical power. In practice, the minimum required sample size depends on several factors: the expected size of the policy effect, the variability of the outcome, the chosen significance level (e.g. 5%), the desired statistical power (often set at 80%) and whether randomisation occurs at the individual farm level or at a cluster level (e.g. groups of farms or geographic areas). These elements are typically combined through a formal power calculation to determine the number of farms needed in the treatment and control groups.
Capacity to implement, monitor and track the different policy variants over time.
Agreement from managing authorities and stakeholders on using randomisation for evaluation purposes.

Because the assignment is random, the groups are, on average, similar in all relevant characteristics before the intervention. This means that any systematic differences observed afterwards between the two groups, the ‘treatment group’ and the ‘control group’, can be attributed to the policy itself and not to pre‑existing differences between farms. For this reason, RCTs are considered the ‘gold standard’ for identifying causal effects in evaluation.

However, it is important to note that this approach is rarely used in policy evaluation, as the policy measures introduced allow farmers to decide whether to apply based on their own expectations, making the assignment non-random by nature.

Step-by-step

Step 1 – Identify which policy design element (e.g payment level) should be evaluated.

Step 2 – Decide which farmers or areas are eligible for participation in the experiment.

Step 3 – Specify clearly what the treatment group receives and what the control group receives (status quo or alternative).

Step 4 – Use a transparent, documented random process to assign farms (or other units) to treatment and control groups.

Step 5 – Roll out the policy variants as planned, ensure adherence to the design, and collect data on outcomes and relevant covariates over time.

Step 6 – Compare average outcomes between groups and interpret differences as causal effects of the tested policy variant, while discussing context and any implementation issues.

Main takeaway points

RCTs are the clearest tool for identifying causal effects of specific policy designs but are hardly ever used in CAP evaluation questions. RCTs have so far been used only exceptionally in CAP evaluations because the feasibility, ethical, political and administrative constraints do not allow a randomised assignment of treated and non-treated beneficiaries.
They are most useful for testing and fine‑tuning design and implementation options (e.g. contracts, incentives, information) within voluntary and pilot settings.
Successful use of RCTs in CAP requires strong institutional support, careful planning and clear communication with farmers and stakeholders.

Learning from practice

Behaghel, L., Macours, K., & Subervie, J., (2019), How can randomised controlled trials help improve the design of the common agricultural policy?, European Review of Agricultural Economics, 46(3), 473-493.

Behaghel, Macours and Subervie (2019) discuss how randomised nudges and contract variants could be embedded in the design of agri‑environmental measures and other CAP instruments to test different specific policy options. Their work illustrates how experimental designs can be used to evaluate, for example, the uptake of agri‑environmental-climate contracts, the coordination among farmers in providing environmental public goods, equity-efficiency trade‑offs in decoupled payments and different contract design features. In these designs, farmers are randomly assigned to alternative information framings, incentive levels or contract conditions. This random allocation allows the resulting RCTs to generate robust causal evidence on which variant performs better in terms of participation and environmental outcomes.