Interviews

• Structured interviews rely on a questionnaire with standardised questions, all of which must be posed to an interviewee. Each individual interview includes the same series of questions asked in a fixed order. Structured interviews are generally used for survey interviews, aiming to gather data on the opinions of a high number of individuals on a specific topic. Possible answers collected in structured interviews can be predefined to facilitate their use in quantitative analyses.

• Semi-structured interviews offer more flexibility, allowing an interviewer to adapt the set of questions according to an interviewee’s answers, knowledge, or willingness to discuss certain topics in the context of individual sessions. Therefore, it is possible to leave specific questions out, mix the order of questions, or rephrase standard questions differently depending on the context. Semi-structured interviews can be used as an exploratory tool, allowing the investigation of different facets of the issues with various stakeholders.

• Unstructured interviews take the form of an open conversation. Instead of predefined questions, unstructured interviews rely on a list of topics to be discussed with interviewees. The interviewer will steer

It is important to note that the information collected from stakeholders interviewed can be partial, which is why it is important to collect information from different types of stakeholders with different interests and perspectives.

• Approach a complex topic and understand underlying mechanisms (e.g., understand farmers’ behaviour and potential reasons for the low uptake of certain interventions) or the relevance of one intervention (e.g., by assessing farmers’ needs and potential improvements needed to the policy).
• Get the opinions of various stakeholders on the implemented CAP intervention(s) and the effects achieved. Interviews will help collect information on the effectiveness of the implemented interventions and their relevance. They may also serve to collect opinions on the coherence and efficiency of the policy. In these cases, interviews can reveal deep-seated antagonisms and major power struggles between various players (e.g., Managing Authority, farmers, NGOs, etc.) and their respective interests. The positioning of the different stakeholders can be important to consider when designing the intervention to overcome potential barriers (e.g., how to enable smaller farms to qualify for investment support when their project is not as ambitious as the ones from more competitive farmers).
• Compensate for the absence of quantitative data by collecting opinions of a large number of interviewees (e.g., farmers) on the reason(s) behind their decision to apply for the interventions, the actual changes implemented with the support received, the administrative and technical difficulties encountered by the beneficiaries of the measures, etc. For that purpose, it is important to prepare a standardised questionnaire with a majority of close-ended questions. It is also recommended to establish a large, representative sample. In cases where interviews are a major source of primary data for the evaluation, the interviewee replies must always be taken cautiously, as they can be partial, biased, erroneous, etc.
• Triangulate the results from a quantitative analysis. The interviews can shed light on the reasons behind certain phenomena observed by a quantitative analysis.

Interviews can also be conducted at different stages of the evaluation process. For instance, interviews conducted before the structuring phase can help understand the context and different aspects to be considered by the evaluation. Some evaluators also like to wait for the outcomes of the literature research and the quantitative analyses before starting the interviews. This allows them to discuss with the different stakeholders the observed results and the reasons for them.

• Focus on a few respondents with significant knowledge and understanding of the context and effects of the measure(s)/policy you want to assess, e.g., researchers. These persons are supposed to have objective views about the topic and will describe the situation factually and the mechanisms at stake, backing their statements with findings from academic sources. Interviews with a few knowledgeable persons can be particularly useful when starting an evaluation to understand the context of the evaluation and subsequently adjust the methodological approach.
• Contact all types of stakeholders concerned by the measure(s)/policy you want to assess, e.g., Managing Authority, Paying Agency, farmers’ representatives, farmers, researchers, NGOs, etc. When quantitative data are scarce, or to triangulate the results from the quantitative analysis, it is recommended to carry out interviews with a sample of stakeholders representative of the agricultural sector.

Step 3 – Conduct the interviews

The interviewer asks the interviewees questions while keeping notes of their answers by hand. Interviews can also be recorded, although this is not recommended for evaluation purposes. Indeed, it’s important that interviewers feel free to share their opinions and criticise policy choices if necessary. Guaranteeing anonymity can help to remove barriers and encourage interviewees to share their views freely.

Once interviews have been conducted, they need to be transcribed before the analysis. In general, it is preferable to aim for a verbatim transcription, i.e., one that reports exactly word for word what was said during the recorded interview.

Step 4 – Analyse the results

The interview analysis aims to condense large amounts of data into a set of key themes, more or less common to the interviews conducted. This summarising process enables conclusions to be drawn and highlights potential patterns, e.g., the logical connections between different issues or ideas identified from interviews. Special computer programs can organise, manage, sort, and analyse large amounts of qualitative data. In the case of structured interviews with close-ended questions, it is possible to carry out a statistical analysis of the responses, as in the case of a survey. During this stage, the verification of the data and collected findings from the interviews is also required.

Step 5 – Report results

The main ideas and issues identified through the interviews are brought back to document the evaluation’s corresponding factors of success. The use of verbatim as examples in the evaluation report can help illustrate specific aspects. However, respect for confidentiality and anonymity is essential if already agreed with the interviewee.

• Interviews are very useful for collecting information, but their success will depend on the care dedicated to the preparatory phase and the relevance of the elaborated questionnaire.
• Different approaches exist to conduct interviews, which vary according to the objectives of the interviews:
  • Structured interviews are generally used for survey interviews, aiming to gather data on the opinions of a high number of individuals on a specific topic.
  • Semi-structured interviews can be used as an exploratory tool, allowing the investigation of different facets of issues with the various stakeholders.
  • Unstructured interviews with a few stakeholders can be particularly useful before the launching of an evaluation to understand the context, objectives, and motivations behind the evaluation.
• Interviews can shed light on the context of how CAP interventions are implemented, which is to be considered when designing or assessing an intervention.
• Guaranteeing anonymity can encourage interviewees to share their views freely.
• The information collected from stakeholders can be biased, inaccurate, erroneous, etc., and should be verified or confronted with other sources.

• EU CAP Network (2024) CAP measures played a major role in supporting the development of outermost regions

• European Commission: Directorate-General for Agriculture and Rural Development (2024). Evaluation support study of the EU support schemes for agriculture in the outermost regions (POSEI) and smaller Aegean islands (SAI): executive summary. Publications Office of the European Union.