Learning portal - Regression discontinuity design

• Available dataset containing the eligibility index and observations on eligible and non-eligible units.
• Time series of cross-sectional data.

The technique can be applied to assess the effect of CAP support on the evolution of the programme effects listed in the following table.

• Step 1 – Ensure that treatment is assigned exclusively on the basis of a cut-off value on an eligibility index.
• Step 2 – RDD analysis should begin with a graphical presentation in which the value of the outcome indicator for each data point is plotted on the vertical axis, and the corresponding value of the eligibility index is plotted on the horizontal axis. The graphical presentation provides a powerful visualisation of discontinuity (‘jump’) in the outcome indicator value at the cut-off point.
  • It is recommended to proceed with the graphical analysis in four steps:
    • Divide the eligibility index into a number of equal-sized intervals, which are often referred to as ‘bins’;
    • Calculate the average value of the outcome indicator and the midpoint value of the eligibility index for each bin and count the number of observations in each bin;
    • Plot the average outcome indicator values for each bin on the Y-axis against the midpoint values of the eligibility index for each bin on the X-axis, using the number of observations in each bin as the weight. The  size of a plotted data point should reflect the number of observations contained in the corresponding bin;
    • To help readers better visualise whatever patterns exist in the data, the evaluation can superimpose flexible regression lines on top of the plotted data. This also provides a visual sense of the amount of noise in the data.
  • A challenge of the graphical assessment is to determine the bin width. There are some recommended formal tests which can be found in Jacob et al. (2012).
• Step 3 – Next, the evaluator can formally estimate the treatment effects using an RDD. There are two types of strategies for correctly specifying the functional form between an eligibility index and outcome indicator:
  • Discontinuity at the cut-off point: This parametric strategy uses every observation in the sample to model the value of the outcome indicator as a function of the eligibility index and treatment status. This method considers all available observations, including those far from the cut-off value, to estimate the average value of the outcome indicator for the observations near the cut-off value. To minimise bias, different functional forms for the eligibility index (linear, quadratic, cubic), as well as interactions with treatment, are tested by inspecting the residuals and conducting F-tests on interaction terms.
  • Local randomisation: This nonparametric/local strategy adopts local randomisation for the estimation of treatment effects and limits the analysis to observations that lie within the bandwidth of the cut-off value, where the functional form is more likely to be linear. The main challenge here is selecting the right bandwidth. Once the bandwidth is selected, a linear (or polynomial) regression is estimated, using observations within one bandwidth on either side of the cut-off value.
• Step 4 – Assess the internal validity of RDD impact estimates. If the cut-off value is to be chosen using information about an observed eligibility index, evaluators can set this value in a way that includes or excludes specific observations. On the other hand, if an eligibility index of each observation is determined based on knowledge about the corresponding cut-off value, it can be manipulated to include or exclude specific observations. The methods that researchers can use to determine whether the eligibility indexes or the cut-off values could have been manipulated (that is, whether or not an RDD is internally valid) include:
  • Examination of the implementation process;
  • Plotting the probability of receiving treatment as a function of the eligibility index. For a valid RDD, there should be a discontinuity (or ‘jump’) at the cut-off value in the probability of receiving treatment;
  • Plotting the relationship between non-outcome variables and an eligibility index. Non-outcome variables here refer mainly to potential covariates that, according to the theory of action, should not be affected by a treatment.
• Step 5 – Assess the precision of the estimates obtained from an RDD. This is something that is particularly relevant when using an existing data set. The precision of estimated treatment effects is typically expressed in terms of a minimum detectable effect (MDE) or a minimum detectable effect size.
• Step 6 – The results of RDD should be accompanied by a critical discussion of the obtained evidence including triangulation with other quantitative and qualitative findings.

• RDD is distinct due to its use of eligibility cut-offs for programme evaluation.
• By comparing units just above and below the cut-off, RDD minimises selection bias.
• Essential conditions for RDD include a continuous eligibility index and a clear cut-off point.
• RDD differs from other methods by focusing on localised impacts rather than generalising effects.
• It offers valuable insights into the causal effects of specific interventions or programmes.

• Varacca, A., Arata, L., Castellari, E., & Sckokai, P. (2023)
  Does CAP greening affect farms’ economic and environmental performances? A regression discontinuity design analysis. European Review of Agricultural Economics, 50(2), 272–303
• Wuepper, D., & Finger, R. (2023)
  Regression discontinuity designs in agricultural and environmental economics. European Review of Agricultural Economics, 50(1), 1–28
• Jacob, Robin Tepper, Pei Zhu, Marie-Andrée Somers, and Howard Bloom. 2012
  ‘A Practical Guide to Regression Discontinuity.’ New York: MDRC

• European Evaluation Network for Rural Development for the CAP (2014)
  Capturing the success of your RDP: Guidelines for the ex post evaluation of 2007-2013 RDPs