Learning Portal - Assessing AKIS
- Evaluation
- Evaluation
- Digitalisation
- Fostering Knowledge & Innovation
- Performance Monitoring and Evaluation Framework (PMEF)
- Monitoring and evaluation framework
Evaluating the Agricultural Knowledge and Innovation System (AKIS) strategic approach provides insights into how the Common Agricultural Policy (CAP) supports the enhancement of knowledge flows and exchanges between individuals and agricultural organisations and institutions involved in using and producing knowledge.
Page contents
Basics
What is AKIS?
AKIS is a concept that reflects the need for a systemic approach to enhance knowledge flows and innovation in agriculture and rural areas. It has evolved into what is currently used in EU policy as a concept that describes the different actors and sources of knowledge interacting to allow for the continuous production of new knowledge and innovation.
The definition of AKIS in Regulation (EU) No 2021/2115 describes it as “the combined organisation and knowledge flows between individuals, organisations, and institutions who use and produce knowledge for agriculture and interrelated fields”.
This definition simplifies a more comprehensive undestanding of AKIS that encompasses all actors from agriculture and interrelated fields and organisations. These actors include farmers, foresters, farmers’ and foresters’ organisations and cooperatives, advisors, researchers, trainers, rural entrepreneurs, non-governmental organisations (NGOs), public authorities, etc. Together they generate, share and use knowledge and innovation for agriculture and interrelated fields such as rural areas, value chains, landscape, environment, climate, biodiversity, consumers and citizens, food and non-food systems including transformation and distribution chains, etc.
Relation to the CAP
In the 2023-2027 CAP, the AKIS strategic approach has been enshrined in the policy architecture for the first time by contributing to the Cross-Cutting Objective (CCO) of modernisation, knowledge sharing, innovation and digitalisation. In CAP Strategic Plans, Member States show their AKIS strategic approach, which will contribute to the achievement of the CCO. They describe the organisational set-up of AKIS and how advisory services, research and the national CAP Networks will cooperate to provide advice, knowledge flows and innovation services.
The AKIS strategic approach is translated into practice by using and combining three types of interventions, which together contribute to the well-functioning of AKIS. These are:
- Knowledge exchange and dissemination of information;
- Farm advisory services;
- Operational Groups’ (OG) innovation projects under the EIP.
Additionally, digitalisation helps to further improve knowledge flows by providing innovative solutions in terms of collection (e.g. knowledge reservoirs), management (e.g. advisory tools) and sharing. The National CAP Networks’ fostering innovation and knowledge exchange activities also support these interventions.
The combination of the above types of interventions should be tailored to national circumstances to foster systemic collaboration. This collaboration involves researchers and other experts, advisors, farmers, foresters and their organisations. Additionally, it should curate an enabling environment for all actors to meet and collaborate regularly around practical needs and innovative solutions.
What to evaluate?
The AKIS strategic approach, in line with Article 114 (a) of Regulation (EU) 2021/2115, is one of the cornerstones of the CCO and legal requirements stem from those texts applicable to the evaluation of this CCO. For instance, Annex 1 of Implementing Regulation (EU) 2022/1475 considers AKIS as one of the key elements to be considered when evaluating the effectiveness of the CCO. Notably, Member States are expected to assess AKIS related interventions and their impact on innovation uptake by farmers based on the support for their AKIS strategic actions.
Furthermore, as stated in Article 2 (d) of Implementing Regulation (EU) 2022/1475, based on Member States’ evaluation needs and considering the CAP Strategic Plan’s intervention logic and implementation, AKIS can be considered as a specific topic to be assessed within the CAP Strategic Plans evaluation.
The guidelines ‘Evaluating the AKIS Strategic Approach in CAP Strategic Plans’ propose an evaluation framework with several key elements to assess the contribution of the AKIS strategic approach to achieving the CCO relating to the modernization of agriculture and rural areas. For each of the recommended key elements in the guidelines, relevant factors of success and possible indicators are also proposed. Member States may choose the factors depending on the intervention logic of the CCO in their CAP Strategic Plans and evaluation needs.
In terms of timing of evaluations, the AKIS strategic approach needs to be evaluated during the 2023-2027 implementation period and ex post. In principle, based on the general intervention logic of AKIS within CAP Strategic Plans, the recommendation is to evaluate the AKIS strategic approach continuously to capture different aspects of its overall design, implementation and contribution to the CCO and Specific Objectives of CAP Strategic Plans.
Step-by-step
Evaluation phases when assessing the AKIS strategic approach
The evaluation of the AKIS strategic approach can be organised into six phases, as shown in the figure.
-
The figure shows the phases of organising an evaluation of the AKIS strategic approach.
The first phase is planning, which encompasses factors such as: first, deciding on resources; second, defining stakeholders that are to be involved; and third, establishing coordination and quality control arrangements.
The second phase is preparing, characterised by first reviewing the intervention logic and then defining evaluation elements.
The third phase is called structuring and contains tasks such as: first setting up a consistent evaluation approach; second, identifying and selecting evaluation methods and tools; and third, reviewing data requirements and data collection needs.
The fourth phase is observing, containing tasks such as: first, managing data collection; and second, implementing evaluation methods for data and information collection.
The fifth phase is centred around analysing, wherein indicators are calculated and collected information is analysed with selected evaluation methods.
Lastly, judging is the sixth and final phase, distinguished by interpreting of evaluation findings, answering evaluation questions, and drawing conclusions and recommendations.
Theory of change approach to evaluate AKIS
The AKIS strategic approach is a complex system related to each Member State's context. For this reason, the theory of change approach is recommended for evaluating AKIS due to its ability to capture the interconnectedness of various interventions and their effects. It allows the systematic production of specific evidence based knowledge that will facilitate collective learning of relevant AKIS actors, including Managing Authorities. This will also allow reflections on the different features of the AKIS in Member States.
The starting point of a theory-based evaluation design is always a causal chain or theory of change which explains how and why the intervention will work and is expected to lead to the intended outcomes. (Learn more about theory-based approaches)
The theory of change is a good approach to assessing the effectiveness of interventions. The assessment of effectiveness explores the extent to which the interventions meet policy objectives. The theory of change evaluation approach answers a key question related to effectiveness: how and to what extent have the stated objectives been achieved? (e.g. to what extent does the AKIS strategic approach contribute to the achievement of the CCO? To what extent did the AKIS strategic approach contribute to capacity development?)
Applying the theory of change to evaluating the AKIS strategic approach involves the following steps:
Step 1 – Orientation phase:
- Selecting the evaluator early to help them understand AKIS interventions and their combination.
- Identifying interventions related to AKIS, such as cooperation measures, advisory services and knowledge exchange.
Step 2 – Identifying and engaging relevant AKIS actors:
- Creating a list of AKIS actors, including institutional bodies, potential beneficiaries and influential actors.
- Involving key actors throughout the evaluation process to implement the theory of change approach.
Step 3 – Introductory qualitative analysis:
- Co-reviewing the pathway of change of the AKIS strategic approach.
- Reviewing planned interventions and identifying potential problems, constraints and influential factors.
- Analysing the expected implementation of interventions and mapping pre-conditions, interim outcomes and ultimate goals.
- Estimating the magnitude of change by linking expected effects with the intervention logic of the AKIS strategic approach.
- Considering factors like the intensity of interventions and the budget allocation to interventions.
Step 4 – Co-reviewing variables of observation:
- Selecting indicators proposed in the evaluation elements.
- Assessing indicators against a target or baseline to determine the degree of performance of AKIS interventions.
Step 5 – Identification of methods, tools and data sources:
- Choosing methods based on AKIS key elements to assess evaluation questions and related factors and indicators.
- Developing evaluation criteria and indicators for every element of an expected change.
The theory of change approach to evaluating the AKIS strategic approach could be applied through several methods and tools that adequately fit the purpose of system analysis, review the theory of change and support collective learning towards action.
Main takeaway points
- The evaluation of the AKIS strategic approach measures its effectiveness in achieving the CAP’s objectives related to modernisation, knowledge sharing, and innovation in agriculture.
- It assesses how well the various components of AKIS—such as advisory services, knowledge exchanges, and innovation projects—are integrated and contribute to the CAP's goals.
- The evaluation focuses on the adaptability and responsiveness of AKIS to the changing needs and conditions within agricultural sectors across Member States.
- By evaluating AKIS, policymakers and stakeholders can identify successful practices and pinpoint areas that require more focused interventions or enhancements.
- The outcomes of AKIS evaluations help in understanding the impact of digital tools and innovative strategies on improving the overall agricultural knowledge system within the CAP framework.