Learning portal - Case studies

Case studies are a vital tool for the detailed exploration of the 'how' and 'why' behind outcomes. Their main strength lies in providing a comprehensive view of an intervention in real-world settings, which is particularly beneficial when data is scarce or complex phenomena are involved.

Basics

In a nutshell

Understanding the how and why

Case studies are the preferred empirical evaluation tool when answering ‘how’ and ‘why’ questions. They allow for a detailed examination of an intervention or programme in a real-world context. In situations of data scarcity, case studies can often provide more reliable insights as they can rely on multiple data-collection methods.

For example, the ‘success case method’ identifies individual cases that have been particularly successful (or unsuccessful) and uses case study analytical methods to develop credible arguments about the contribution of the intervention to these.

Case studies draw from several established traditions including theory-based evaluation, organisational development, appreciative inquiry, narrative analysis and quantitative statistical analysis of effects.

Exemplifying results

They can underpin or discard hypotheses and, due to their idiographic character, are useful for illustrating or exemplifying results that already seem sufficiently confirmed. In this case, it is recommended to select and launch the case studies once the quantitative analysis has been executed and the first results are available. Indeed, the first quantitative results may raise questions about specific cases for which additional qualitative information is necessary. With their help, intrinsic driving factors and cause/effect relationships can be made explicit and accessible for discussion.

Pros and cons

Advantages	Disadvantages
Provides an in depth illustration of the situation in a real-world context. Helps understand complex phenomena and their influence on policy effects. Triangulates evaluation findings from various methods. Helps understanding of causality beyond numbers. Overcomes data scarcity by producing specific case information.	Limited number of observations possible. Resource intensive. Challenge of selecting representative cases. Difficulty to generalise the findings and establish overall judgments (validity).

When to use?

Case studies are most often used to collect information on the effectiveness of an intervention or programme and the causal mechanisms between the intervention(s) implemented and the effects achieved. Hence, as they reflect the relative influence of factors hindering or supporting the achievement of the expected results, case studies are valuable tools for assessing the efficiency and relevance of CAP Strategic Plans.

Case studies are particularly recommended:

To understand the variety of effects generated by one specific intervention and the determining factors enhancing or hindering the expected (theoretical) effects in different contexts. This is very useful to revise a theory-based analysis by highlighting the different causal processes affecting other types of beneficiaries. In this case, it is recommended to select case studies that are representative of the various contexts and can reflect the results achieved by a CAP Strategic Plan in different situations e.g. results might be of a different nature and/or magnitude according to the characteristics of the beneficiaries (location, farm type, size, etc.). A classification analysis is thus recommended to establish representative groups of beneficiaries, in which (single or multiple) case(s) will be selected for a detailed evaluation of policy outcomes.
To consider the overall effects achieved by an entire CAP Strategic Plan (i.e. several interventions) over one specific objective or element. Here, the approach can consist of selecting various examples of projects or practices supported by different interventions, e.g. assessing the effects achieved by a CAP Strategic Plan on animal welfare. This can illustrate how different CAP interventions supporting improved housing and husbandry conditions were implemented by farms in other targeted sectors. For each intervention, one or more examples can describe the farm context, the implementation of the practice or investment supported, the amount delivered to the farm, the expected effect on animal welfare, the difficulties encountered, any unexpected consequences of the policy, etc.

Another approach would consist of comparing territories or sectors with different issues and distinguishing the effects of a CAP Strategic Plan on specific objectives in particular contexts. For example, to assess the effects of a CAP Strategic Plan on water, River Basin Districts could be considered to examine how the CAP interventions implemented contributed to alleviating pressure off water from agricultural sources. Additional information can be of specific interest (e.g., water analysis results on different time periods) to complement the whole picture.

Preconditions

Good knowledge of the programme beneficiaries, their geographical distribution and economic situation.
Existence of good examples to be elaborated as representative cases.
Some statistical skills to employ specific techniques to select representative cases based on the processing of quantitative data.
Good case study writing technique.

Step-by-step

Step 1 – Defining the case study approach: When using case studies, it is important to determine the appropriate levels of analysis that are in line with the objective of the evaluation. The level of analysis may vary according to the objective of the evaluation and whether case studies aim to:
- illustrate examples of achievements of a CAP Strategic Plan;
- compare the performance of different CAP interventions;
- assess the effectiveness achieved at a certain scale;
- collect specific information on a given topic.
The following framework can help design the case study approach according to the relevant scale, analysis and scope of an evaluation.

Text version: The figure shows how to approach the design of a case study by depicting the scale, which can either be beneficiaries/projects or sectors/territories, and second, the scope, that is either a single intervention or several interventions/programme/policy.

The framework is broken down into four quadrants formed by the intersections of these categories:

First, beneficiaries/projects at the level of a single intervention; second, beneficiaries/projects at the level of several interventions/programme/policy; third, sectors/territories at the level of a single intervention; and fourth, sectors/territories at the level of several interventions/programme/policy.

The number and variety of cases may vary according to the evaluation objective(s). The number of case studies depends on the diversity of situations that need to be illustrated to better understand the phenomenon. However, the number of case studies will also result from the available resources (e.g. budget and time) and the extent to which a comprehensive analysis is required to reflect the complexity of a situation. This enables decisions to focus only on a few representative situations rather than all existing situations.

Aside from the number and type of cases selected, the timeline for their execution should be defined according to the methodological approach. Case studies can have an exploratory role and be launched at the beginning of an evaluation process or a confirming/explanatory role. In this latter case, they can be planned once the information on observed effects (outputs, outcomes or impacts) has been processed to further examine and complement the outcomes of the quantitative analysis.

Step 2 – Selecting the cases: The choice of case studies plays a decisive role in an evaluation. Therefore, cases must be selected with the greatest care, according to the methodological approach and expected contribution to an evaluation.

For instance, selecting case studies representative of reality (and its inherent diversity) enables a comprehensive picture of a situation. It also favours generalising findings from case studies. On the other hand, choosing individual cases that have been particularly successful and/or unsuccessful can help identify factors supporting/hindering a policy's outcomes.

Independently of the case studies’ unit (e.g. beneficiaries, projects, territories, etc.), establishing a classification helps identify the relevant criteria or indicators for the selection of case studies. However, the selection also reflects the information of specific interest for the evaluation, which can be spread into different categories. Examples of criteria for the selection of the case studies are level of execution (output), level of performance (outcomes), geographical location, sectors and contextual information.
Step 3 – Considering the study scope and content: Case studies can include different types of information and analyses depending on the objectives of an evaluation. For instance, the descriptive background of a case study can mobilise quantitative information sources and methods. In contrast, the policy's rationale, the reasons for the implementation choices, the observed effects, and the identification of external factors influencing the policy's effectiveness can be addressed through in-depth interviews with key stakeholders, complemented by a literature review.
Step 4 – Generating the findings: Case studies will serve as a source of information for the evaluation report. Hence, the facts collected through case studies will be extracted, assembled, and used for analysis in the main report.

It is, however, necessary to assess the internal and external validity of case study findings and to what extent they can (or cannot) be extrapolated or generalised for other situations. Useful questions to ask include:
- Do the case studies’ empirical results tend to validate the theoretical assumptions defined at the outset of the evaluation (theory-based analysis)?
- Were there several case studies that demonstrated similar results and confirmed the logical reasoning?

Main takeaway points

Case studies are important in situations where complex socio-economic effects and causality need to be understood.
They provide an in-depth, reliable snapshot in scenarios where data are limited.
The success case method is crucial in isolating and analysing particularly successful or unsuccessful cases.
Case studies are versatile and useful in both qualitative and quantitative evaluations.
They offer unique insights into the effectiveness, efficiency and relevance of interventions, particularly under the CAP Strategic Plan.

Learning from practice

Publication: Lattanzio KIBS SpA (Virgilio Buscemi, Paola Paris, Silvia De Matthaeis and others), 2022
Third Thematic Evaluation Report "Innovation in Agriculture - The Strategic Plans of the Operational Groups".
Arfini, F., Brasili, C., Fanfani, R. et al. (2001)
Tools for evaluating EU agricultural policies: An integrated approach. Statistical Methods & Applications 10, 191–210.
Gagnon, Y.-C. (2012)
L’étude de cas comme méthode de recherche, 2e édition (2nd ed.). Presses de l’Université du Québec.
Yin, R. K. (2018)
Case study research and applications: Design and methods (6th ed.). Sage Publications.