Learning Portal

Innovation capacity scoring tool

Find out more about FAO’s innovation capacity scoring tool and learn how to assess agricultural innovation system (AIS) capacities via 24 indicators across individual, organisational and policy dimensions, identifying gaps and measuring progress to guide innovation in agriculture effectively.

Page contents

Bioengineer, scientist collects data for research on new variety pepper in greenhouse.

Basics

In a nutshell

What is the innovation capacity scoring tool?

Innovation, be it technological, institutional, or social, emerges from collective thinking and action. It is a process by which multiple stakeholders put knowledge to use. Networks of research, extension, producers, agribusiness, and other actors, as well as the policies, attitudes, and behaviours affecting them, act as catalysts for innovation. They play a critical role in shaping food systems by generating, documenting, blending, sharing, and applying local and scientific knowledge and stimulating learning. For all these diverse actors to connect, collaborate, and learn together effectively within a system, a range of soft skills is essential. The availability of these skills combined with an enabling environment may determine the capacity for innovation of a system of actors.

A specific innovation capacity scoring tool based on the Capacity Development – Agricultural Innovation System (CDAIS) Open link in new windowapproach has been developed by the Food and Agriculture Organization (FAO) to systematically assess capacity development needs and progress made over time due to collaborative innovation processes. This innovation capacity scoring tool provides scores about the level of innovation capacities of a system based on 24 indicators related to different domains of individual capacities (to navigate complexity, collaborate, reflect and learn, and engage in strategic and political processes) along with technical skills and enabling environments.

The indicators address three capacity development dimensions, namely:

Individual level (i.e., focus on skills, knowledge, and changes in behaviour through training and participation).
Organisational level (i.e., emphasis on tools, systems, and processes necessary for organisational improvements).
Enabling environment (i.e., targeting policy frameworks, institutions, and political processes).

The method follows a participatory approach, which requires effective facilitation to allow participants to understand the context and the questions used to calculate the indicators.

This tool can be adapted to EU evaluations and help assess ex ante the innovation capacity development needs by identifying strengths and weaknesses. It can also be used ex post to assess the progress and performance towards increasing innovation capacities of CAP-supported interventions.

Pros and cons

Advantages	Disadvantages
A multi-criteria approach can generate robust evidence for developing agricultural innovation strategies. The scoring tool can be used concurrently or as part of a larger multi-stage assessment involving other tools (e.g. stakeholder mapping or visualised AKIS mapping).	The tool relies on stakeholder participation and requires a relatively small group. In large systems, a robust sampling process is needed to select participants. Effective facilitation is crucial for participant understanding and accurate data collection. Changes in participant composition between assessments can affect results. Without rigorous designs (e.g., control groups), attributing capacity changes to interventions is challenging.

When to use?

The method is ideally applied to the assessment of the innovation capacity of systems with a relatively small number of actors (e.g., at the level of an EIP Operational Group). It can potentially strengthen innovation capacity and develop concrete capacity development intervention plans with stakeholders.

Evaluating capacity development interventions requires reliable baseline data for comparisons with mid-term or post-intervention results. The scoring tool, using a 0-3 scale for capacity levels, enables before-and-after assessments to gauge changes over time. Results can be aggregated by topic or disaggregated by factors such as gender, illustrating changes quantitatively or qualitatively.

While causality is difficult to establish due to the absence of control groups, correlations between inputs and outcomes, as well as comparisons between assessed systems, can guide future interventions. Visualisation tools (e.g., pre- and post-assessment graphs) and qualitative data provide context, with a theory of change helping validate results and understand impacts.

The scoring tool’s indicators require tailoring to fit specific contexts. This necessitates the adjustment of questions or the number of indicators, particularly for the enabling environment (e.g., governance, policies, and collaboration).

Preconditions

Identification and mapping of the stakeholders that will participate in the scoring process
Availability of trained facilitators familiar with the FAO guide on this method.
Tailoring the indicators to fit specific contexts, particularly for the enabling environment (e.g., governance, policies, collaboration).

Organisation of focus groups and structured interviews to collect scores.

Step-by-step

Below is a detailed step-by-step approach to how the methodology is applied.

Step 1 – Preparation

Understand the context, the stakeholders, and the specific case where the scoring tool will be applied.
Identify the sample of stakeholders participating in the session, considering that the sample must support external validity (obtaining representative results and generalising back to the population).
Customise the scoring tool to align with the local conditions and specificities of the agricultural innovation system (AIS) or project.

Step 2 – Primer (participatory simulation game/role play)

Organise a participatory session involving all relevant stakeholders or a sample of them as determined in Step 1.
Conduct a simulation game or role play:

Participants are presented with an outline of the current situation from a particular perspective. Each ‘player’ is assigned an objective but is not told how to achieve it and is therefore free to follow his or her instincts. The objective might conflict with the objectives of other players. The essential purpose of the game is to see how each player handles the situation. During the game, the players have to react in real time and in accordance with the role or perspective assigned to them. Through this game, the method:

Fosters a shared understanding of challenges within the innovation system.
Highlights the capacities needed to address these challenges collaboratively.

Find out more examples of these simulation or role play gamesOpen link in new window.

Step 3: Data collection

To obtain most observations within a limited time, individual scoring can be done in groups of three to five respondents with one facilitator rather than in one-to-one interviews.
Facilitators ask questions that should be illustrated with examples obtained through the game/role play, and the assessment should follow without much delay.
Facilitators need to provide some quality control, assuring complete and realistic responses.

Indicators for capacity scoring questionnaire
Key innovation capacities (soft skills)	Additional information
Capacity to navigate complexities (nine indicators) Skills to understand and solve problems. Management skills Access to and mobilisation of resources. Sharing of information within the group. Utilisation of local knowledge. Informed decision-making. Idea where the group wants to be in the future. Strategy plan to achieve an idea for the future.	Technical skills (one indicator) Availability of required technical skills.
Capacity to collaborate (three indicators) Cooperation among actors in the group. Representation of stakeholders in group coordination. Incentives for networking and partnering.
Capacity to learn and reflect (four indicators) Joint learning and experimentation. Training covering multi-stakeholder innovation processes. Understanding of knowledge flows. Documentation and monitoring processes.	Enabling environment (two indicators) Favourable socio-economic circumstances for linking producers to markets. Efficiency of registration/certification processes in agriculture.
Capacity to engage in strategic and political processes (five indicators) Role and responsibilities of the leadership. Awareness of agricultural development issues. Awareness of opportunities for policy change. Influence on decision-making processes by stakeholders. Effectiveness of communication channels.

Examples of questionnaires can be found in Annex 1 of ‘Assessing innovation capacities: a scoring tool’.

Step 4 – Data entry

Input the data collected from the questionnaires into a pre-configured spreadsheet.
Visualise the data for easier interpretation (e.g. graphs, tables etc.).

Step 5 – Data analysis

Analyse the data to profile capacities under each heading.
Identify areas of strength and gaps in functional and technical capacities.

Step 6 – Feedback and roadmap definition

Share the results with stakeholders through visualised capacity profiles (e.g. graphs).
Facilitate discussions to agree on the next steps and actions.
Develop a roadmap outlining strategies and interventions to address capacity gaps.

Main takeaway points

Innovation capacity scoring is a diagnostic tool that can help explore data limitations but without the possibility of offering definitive answers.
A thorough diagnostic of an AIS can help identify policy and investment gaps and highlight innovation opportunities.
National actors and stakeholders must articulate their information and knowledge needs to guide the scoring process. Active involvement from the beginning ensures the scoring aligns with laid-out requirements and priorities.
A well-chosen set of indicators, actors, linkages, and relationships of the observed AIS supports a meaningful evaluation.

Learning from experience

Grovermann, C. (2017). Assessing innovation capacities: a scoring toolOpen link in new window, Food and Agriculture Organization of the United Nations (FAO).

FAO Practical tool:Open link in new window Monitoring capacity developmentOpen link in new window.