Assessing CAP income support instruments
- Evaluation
- Evaluation
- Evaluation Methods
- Data Management
- Performance Monitoring and Evaluation Framework (PMEF)
- Monitoring and evaluation framework
Learn more about how to assess the effect of CAP income support instruments on the level and stability of farms’ income and to what extent the delivered support was effectively transferred into additional income for beneficiaries.
Page contents
Basics
In a nutshell
Farm income is considered viable when it helps farmers face the risks inherent to their business. Hence, farm income level, distribution and volatility can affect farm economic viability in a significant manner.
Farm income is the portion of agricultural output value that can be used to remunerate the fixed factors of production (labour, land and capital), whether they are external or family-owned. It can be measured through different indicators (‘Farm Net Value Added’, ‘Family Farm Income’, with/without wages, etc.) and is generally examined by type of farming, region, farm size and area.
Farm Net Value Added and Family Farm Income are generally calculated from data collected under the EU’s Farm Accountancy Data Network (FADN). However, Eurostat also provides indicators related to measuring agricultural income with data collected under the EU’s Economic Accounts of Agriculture (EAA).
Relation to the CAP
CAP income support interventions shall contribute to “supporting viable farm income and the resilience of the agricultural sector across the EU, in order to enhance long-term food security and agricultural diversity, as well as to ensure the economic sustainability of agricultural production” (Specific Objective 1 (SO) of the CAP under the 2023-2027 programming period).
Following several reforms, CAP income support is nowadays mostly granted in the form of a decoupled support per hectare. In the 2023-2027 programming period, basic income support for sustainability (BISS) has replaced the basic payment scheme (BPS) and the single area payment scheme (SAPS). However, CAP decoupled payments have generally remained stable over time and thus provide a basic guarantee to farmers in the context of growing price volatility and climatic risks.
In addition, the provision of coupled income support (CIS) can address difficulties specific to a particular sector, production type or farming method, while the complementary redistributive income support for sustainability (CRISS) can target smaller farms.
The CAP also provides additional income support for farms in areas with specific needs, notably areas facing natural or other specific constraints (ANC).
The level of support granted to farms varies greatly both among and within Member States, depending on farm sector characteristics, farm sizes and locations, Member States’ allocations (as established under Annex V of Regulation (EU) 2021/2115) as well as Member States’ choices within the flexibilities offered by the legal framework (e.g. regarding capping and degressivity of income support, internal convergence, etc.).
What to evaluate?
With the new delivery model established by Regulation (EU) 2021/2115, CAP income support interventions, including direct payments, are now included in Member States’ CAP Strategic Plans. The evaluation of their effectiveness and efficiency over the programming period is an integral component of Member States’ evaluation activities.
Annex 1 of Implementing Regulation (EU) 2022/1475 introduces ‘viable farm income’ as one of the key elements to be considered when evaluating the effectiveness of the CAP Strategic Plan in regard to SO1. With respect to the assessment of the efficiency of CAP income support instruments, Article 1(3) indicates that Member States must analyse whether the effects or benefits are achieved at a reasonable cost.
As stated in Article 6, evaluations should be based on the Performance Monitoring and Evaluation Framework (PMEF) and “Member States shall quantify the contribution of the CAP Strategic Plans to the development of at least the common impact indicators set out in Annex III to this Regulation”. Out of the impact indicators set out in Annex III, four refer to farm income level, variability and distribution.
Income-related impacts indicators referred to in Article 6(5) of Commission Implementing Regulation (EU) 2022/1475 are listed in the following table.
Indicator code | Indicator name |
---|---|
I.2 | Reducing income disparities: Evolution of agricultural income compared to the general economy |
I.3 | Reducing farm income variability: Evolution of agricultural income |
I.4 | Supporting viable farm income: Evolution of agricultural income level by type of farming (compared to the average in agriculture) |
I.5 | Contributing to territorial balance: Evolution of agricultural income in areas with natural constraints (compared to the average) |
The guidelines ‘Assessing the effectiveness and efficiency of CAP income support instruments’ further develop an evaluation framework with recommended factors of success/judgement criteria and possible indicators to assess the effectiveness and efficiency of CAP income support instruments in relation to income level, stability and distribution. Member States may choose among the factors of success depending on the intervention logic of their CAP Strategic Plans and their evaluation needs.
In terms of timing evaluations, CAP income support instruments can be assessed throughout the 2023-2027 programming period or ex post, depending on the intended use of the evaluation findings. However, the evaluation of CAP income support distribution can only be implemented when output data are reported and available at the Member State level. As demonstrated in the guidelines, one possibility to overcome issues associated with data availability is to examine data from previous programming periods.
Step-by-step
Typical evaluation phases when assessing the CAP income support instruments
The evaluation of the CAP income instruments can be organised into six phases, as shown in the figure.
-
The figure shows the step-by-step approach of organising an evaluation, involving six phases.
The first phase is planning, which encompasses tasks such as:
- deciding on resources;
- defining stakeholders that are to be involved; and
- establishing coordination and quality control arrangements.
The second phase is preparing, characterised by;
- reviewing of the intervention logic; and
- 2) defining evaluation elements.
The third phase is called structuring, and contains tasks such as;
- setting up a consistent evaluation approach;
- identifying and selecting evaluation methods and tools; and
- reviewing data requirements and data collection needs.
The fourth phase is observing, containing tasks such as;
- managing data collection; and
- implementing evaluation methods for data and information collection.
The fifth phase is centred around analysing, wherein;
- indicators are calculated; and
- the collected information is analysed with selected evaluation methods.
Lastly, judging is the sixth and final phase, distinguished by interpreting evaluation findings, answering evaluation questions and drawing conclusions and recommendations.
Methodological approach to assess the net effects of CAP income support instruments
The assessment of the net effects of CAP income support instruments on farm income requires describing the actual situation and comparing it with a counterfactual situation without CAP income support. However, such a counterfactual situation cannot be found, as CAP direct payments are delivered to most eligible farmers. Therefore, the approach recommended in the guidelines considers a hypothetical situation where CAP income support is not in place. One step of the approach therefore consists of simulating the hypothetical situation without CAP support.
Step 1 – Describe the context and CAP instruments implemented
It is important to depict the overall context in which farm income can be assessed and the effects of CAP income support interventions can be evaluated by describing e.g. the farm population and the average farm income by type of farming, region and farm size in ANC.
The CAP interventions and instruments implemented should also be described e.g. the average amount granted under each intervention by farm type, as well as the share of farms benefiting from the different CAP income support interventions.
Step 2 – Observe indicators with CAP support
Relevant sources with information on farm income and the CAP amount distributed to farms should be considered when computing the different indicators suggested in the guidelines (e.g. FADN, national statistics, DME and Eurostat).
When possible, the analysis should consider individual farm data to distinguish the impact by CAP instruments, farm size and different regions. The comparison of income levels across farms is made possible by standardising the absolute income level according to the Annual Working Units (i.e. FNVA/AWU).
At the national level, the macro-economic analysis is based on national aggregated Eurostat data (EAA), in particular data on the agricultural factor income.
Step 3 – Simulate a hypothetical situation without CAP support
Although the simulation could simply consist of subtracting the overall amount of CAP income support from the current farm income level, this would not acknowledge that generally, not all support is transferred into additional income for farmers, because of transaction and opportunity costs or because of distributive leakages (e.g. increases in input prices). Hence, a simulation of the hypothetical situation ‘without CAP support’ must consider that the current income is reduced by only a share of the support provided by ISI. This concept is referred to as ‘Income Transfer Efficiency’ (ITE).
Only econometric methods are adapted for considering the impact of CAP interventions on farm income when applied to the whole population. Methods recommended in the guidelines to calculate the ITE are:
- Cross-section model
- System Generalised Method of Moments (SYS-GMM)
- Quantile Dose-Response Function (QDRF) and Quantile Continuous Treatment Effect (QCTE)
Once the ITE is known, it is possible to recalculate the farm income indicators by subtracting the share of support effectively transferred into farm income.
Step 4 – Compare the two indicators to identify the net effect of the CAP
The comparison of income related indicators with and without CAP support enables the demonstration to which extent these CAP payments have contributed to enhancing farmers’ income or reducing its volatility.
Step 5 – Assess the efficiency of CAP income support instruments
ITE calculation reveals the actual increase in income due to an additional Euro spent on income support instruments. Additionally, a cost-effectiveness analysis investigates the different costs associated with the policy and compares them to the effects achieved.
Learning from experience
Bewertung von Direktzahlungsmaßnahmen in der neuen GAP
- Biagini, L., Antonioli, F., & Severini, S. (2020). The role of the common agricultural policy in enhancing farm income: A dynamic panel analysis accounting for farm size in Italy, Journal of Agricultural Economics, 71(3), 652–675.
- Ciliberti, S., Severini, S., Ranalli, M. G., Biagini, L., & Frascarelli, A. (2022). Do direct payments efficiently support incomes of small and large farms?, European Review of Agricultural Economics, 49(4), 796–831.
- European Commission, Directorate-General for Agriculture and Rural Development, (2020). Evaluation study of the impact of the CAP measures towards the general objective “viable food production”: final report, Publications Office.
- Severini, S., Tantari, A., & Di Tommaso, G. (2016). Do CAP direct payments stabilise farm income? Empirical evidence from a constant sample of Italian farms, Agricultural and Food Economics, 4(1).
Training Material
The presentation on ‘How to assess the effectiveness and efficiency of CAP income support instruments’ outlines legislative requirements and methodological challenges of assessing CAP income support interventions. It provides methodological guidance for Managing Authorities on structuring evaluations and ensuring implementation follow up. Additionally, it demonstrates how to quantify the CAP’s contribution to agricultural income related impact indicators listed in Annex III of Commission Implementing Regulation (EU) 2022/1475.
The presentation is divided into five presentations:
- Introduction of the training’s objectives and main topic
- Background information relevant for preparing the evaluation of CAP income support instruments
- Approaches for assessing the effectiveness of CAP income support instruments
- Approaches for assessing the efficiency of CAP income support instruments
- Quantitative methods for evaluating Income Transfer Efficiency and netting out CAP effects on farm income
Further reading
Bewertung der Wirksamkeit und Effizienz der GAP-Instrumente zur Einkommensstützung
Ressourcen
Introduction of the training’s objectives and main topic
(PPTX – 1.81 MB)
Background information relevant for preparing the evaluation of CAP income support instruments
(PPTX – 5.15 MB)
Approaches for assessing the effectiveness of CAP income support instruments
(PPTX – 2.62 MB)
Approaches for assessing the efficiency of CAP income support instruments
(PPTX – 1.83 MB)
Quantitative methods for evaluating Income Transfer Efficiency and netting out CAP effects on farm income
(PPTX – 2.04 MB)