RAAIS Rapid Appraisal of Agricultural Innovation Systems (Part I)A diagnostic tool for integrated analysis of complex problems andinnovation capacityMarc Schut ab Laurens Klerkx a Jonne Rodenburg c Juma Kayeke d Leacuteonard C Hinnou eCara M Raboanarielina f Patrice Y Adegbola e Aad van Ast g Lammert Bastiaans g

a Knowledge Technology and Innovation Group Wageningen University PO Box 8130 6700 EW Wageningen The Netherlandsb International Institute of Tropical Agriculture (IITA) Quartier INSS Rohoro Avenue drsquoItalie 16 BP 1893 Bujumbura Burundic Africa Rice Center (AfricaRice) East and Southern Africa PO Box 33581 Dar es Salaam Tanzaniad Mikocheni Agricultural Research Institute (MARI) PO Box 6226 Dar es Salaam Tanzaniae Institut National des Recherches Agricoles du Beacutenin (INRAB) PO Box 02 BP 238 Porto-Novo Beninf Africa Rice Center (AfricaRice) PO Box 01 BP 2031 Cotonou Bening Crop Systems Analysis Group Wageningen University PO Box 430 6700 AK Wageningen The Netherlands

A R T I C L E I N F O

Article historyReceived 25 February 2014Received in revised form 13 August 2014Accepted 19 August 2014Available online 22 October 2014

KeywordsAgricultural research for development(AR4D)Farming systems researchIntegrated assessment(Participatory) research methodsSystem diagnosticsWicked problems

A B S T R A C T

This paper introduces Rapid Appraisal of Agricultural Innovation Systems (RAAIS) RAAIS is a diagnostictool that can guide the analysis of complex agricultural problems and innovation capacity of the agri-cultural system in which the complex agricultural problem is embedded RAAIS focuses on the integratedanalysis of different dimensions of problems (eg biophysical technological socio-cultural economicinstitutional and political) interactions across different levels (eg national regional local) and the con-straints and interests of different stakeholder groups (farmers government researchers etc) Innovationcapacity in the agricultural system is studied by analysing (1) constraints within the institutional sectoraland technological subsystems of the agricultural system and (2) the existence and performance of theagricultural innovation support system RAAIS combines multiple qualitative and quantitative methodsand insider (stakeholders) and outsider (researchers) analyses which allow for critical triangulation andvalidation of the gathered data Such an analysis can provide specific entry points for innovations to addressthe complex agricultural problem under study and generic entry points for innovation related to strength-ening the innovation capacity of agricultural system and the functioning of the agricultural innovationsupport system The application of RAAIS to analyse parasitic weed problems in the rice sector con-ducted in Tanzania and Benin demonstrates the potential of the diagnostic tool and providesrecommendations for its further development and use

copy 2014 The Authors Published by Elsevier Ltd This is an open access article under the CC BY-NC-NDlicense (httpcreativecommonsorglicensesby-nc-nd30)

1 Introduction

The Agricultural Innovation System (AIS) approach has becomeincreasingly popular as a framework to analyse and explore solu-tions to complex agricultural problems (eg Hall et al 2003 WorldBank 2006) The AIS approach evolved from a transition fromtechnology-oriented approaches to more systems-oriented ap-proaches to agricultural innovation (eg Klerkx et al 2012a) Withinthe AIS approach innovation is perceived as a process of com-bined technological (eg cultivars fertilizer agronomic practices)and non-technological (eg social practices such as labour organi-

zation or institutional settings such as land-tenure arrangements)changes (Hounkonnou et al 2012 Leeuwis 2004) Such changesoccur across different levels (eg field farm region) and are shapedby interactions between stakeholders and organisations insideand outside the agricultural sector (Kilelu et al 2013 Klerkx et al2010)

Adopting an AIS approach to study complex agricultural prob-lems has important implications for research First it requires ananalysis that acknowledges and integrates the different dimen-sions levels and stakeholdersrsquo interests associated with the problemunder review Second it necessitates a holistic understanding of theinnovation capacity of the agricultural system in which the complexproblem is embedded (Hall 2005) Third it requires insight in thestructural conditions provided by the agricultural innovation supportsystem that can enable or constrain innovation in the agricultural

Corresponding author Tel +257 720 787 40E-mail address mschutcgiarorg marcschutwurnl (M Schut)

httpdxdoiorg101016jagsy2014080090308-521Xcopy 2014 The Authors Published by Elsevier Ltd This is an open access article under the CC BY-NC-ND license (httpcreativecommonsorglicensesby-nc-nd30)

Agricultural Systems 132 (2015) 1ndash11

Contents lists available at ScienceDirect

Agricultural Systems

journal homepage wwwelseviercom locate agsy

system (Klerkx et al 2012b World Bank 2006) Fourth it re-quires a thorough understanding of the interactions betweencomplex agricultural problems innovation capacity in the agricul-tural system and the agricultural innovation support system

Despite the recent development and application of a variety ofmethods that can support AIS analyses (eg World Bank 2012) thepotential of the AIS approach to address complex agricultural prob-lems remains underutilized in many fields of study (eg Schut etal 2014a) Four main reasons for this were identified First methodsused for the analysis of complex agricultural problems generally havea narrow focus rather than a holistic view They support the anal-ysis of a specific dimension (eg the economic dimension in Beintemaet al 2012) level (eg the national level in Temel et al 2003) orstakeholder group (eg farmers in Amankwah et al 2012 Totin et al2012) Second studies that do include analysis of multiple dimen-sions of problems (eg Singh et al 2009) interactions across differentlevels (eg Douthwaite et al 2003) or multi-stakeholder dynam-ics (eg Hermans et al 2013) often have limited attention for theintegrated analysis of these features of complex agricultural prob-lems Third studies that integrate the analysis of multiple dimensionsof problems interactions across different levels and multi-stakeholderdynamics (eg Lundy et al 2005 van Ittersum et al 2008) havelimited attention for understanding innovation capacity in the ag-ricultural system and the functioning of the agricultural innovationsupport system A fourth reason is that the majority of AIS studiesare conducted ex-post (eg Basu and Leeuwis 2012) lack a clearstructure to delineate systemrsquos boundaries (Klerkx et al 2012b) orare based on comprehensive studies which take considerable time(eg Jiggins 2012) Although such studies provide a better under-standing of the drivers of innovation in agricultural systems theirdiagnostic ability to identify entry points for innovation to over-come complex agricultural problems is limited

Based on the above review of the availability scope and use ofmethods for AIS analyses we have developed and tested a diag-nostic tool that can support the Rapid Appraisal of AgriculturalInnovation Systems (RAAIS) RAAIS fits within a tradition of lsquorapidappraisal approachesrsquo used in the field of agriculture including theParticipatory (Rapid) Rural Appraisal (Chambers 1994) Rapid Ap-praisal of Agricultural Knowledge Systems (RAAKS Engel 1995) andthe Rapid Appraisal of Potato Innovation Systems (Ortiz et al 2013)RAAIS integrates and builds upon existing (agricultural) innova-tion system concepts and combines multiple methods of datacollection The objectives of RAAIS are to provide a coherent set of(1) specific entry points for innovation to address complex agri-cultural problems and (2) generic entry points that can enhanceinnovation capacity of the agricultural system and the perfor-mance of the agricultural innovation support system The aim ofthis paper is to provide a conceptual framework (Section 2) and amethodological framework (Section 3) for RAAIS Based on its ap-plication in a study on parasitic weeds in rice production in Tanzaniaand Benin we reflect on the extent to which RAAIS is able to meetits objectives and provide recommendations for further develop-ment and use of RAAIS (Section 4) followed by the main conclusions(Section 5)

2 Conceptual framework for RAAIS

The agricultural innovation system ndash including both the agri-cultural system and its innovation capacity and the agriculturalinnovation support system ndash may be very good at tackling somecomplex agricultural problems but may be incapable to deal withothers (Hung and Whittington 2011 Markard and Truffer 2008)It underlines that understanding complex agricultural problems in-novation capacity in the agricultural system and the functioningof the agricultural innovation support system requires integrativeanalysis Despite of their interrelated character we deem it

useful for analytical purposes to first address them separately(Sections 21 22 and 23) before showing their embeddedness(Section 24)

21 Complex agricultural problems

Complex agricultural problems are defined as problems (1) thathave multiple dimensions (Schut et al 2014b) (2) that are em-bedded in interactions across different levels (Giller et al 2008)and (3) where a multiplicity of actors and stakeholders are in-volved (Funtowicz and Ravetz 1993) Regarding the first complexagricultural problems are an interplay of biophysical technologi-cal social-cultural economic institutional and political dimensionsTo exemplify this we use a case by Sims et al (2012) who analyseconstraints for the upscaling of conservation agriculture in sub-Saharan Africa They demonstrate how import taxes on steel butnot on imported agricultural machinery (institutional dimension)disadvantage manufacturers in developing locally adapted agricul-tural equipment such as no till planters (technological dimension)for effective soil conservation for sustainable crop management (bio-physical dimension) Concerning the second the dimensions ofcomplex agricultural problems often have different implicationsacross different levels Mitigating the impact of agro-industrial biofuelproduction on food security for instance will require different strat-egies when approached at the national level (eg policies avoidingagro-industrial biofuel production in regions where pressure on ag-ricultural land is already high) or at the farm household level (egbalancing the allocation of household labour to on-farm crop pro-duction and off-farm plantation work) (Schut and Florinunder review) Nevertheless the different levels are interrelated andconsequently coherent multi-level strategies are required Regard-ing the third complex agricultural problems are characterised bythe involvement of a multitude of actors stakeholders and theorganisations they represent (Hounkonnou et al 2012 Ortiz et al2013) (Table 1) Actors include about anyone that can be related di-rectly or indirectly to a problem or the potential solutionStakeholders are those actors or actor groups with a vested inter-est in addressing the problem (McNie 2007) and their participationin exploring solutions to complex agricultural problems is per-ceived as a critical success factor (eg Giller et al 2011) Stakeholderparticipation can provide enhanced insights in the different dimen-sions of the problem and the types of solutions that are bothtechnically feasible and socio-culturally and economicallyacceptable (Faysse 2006) However stakeholder groups are no ho-mogeneous entities and often focus on their own rather than acommon interest (Leeuwis 2000)

Table 1Examples of stakeholder groups and diversity within stakeholder groups

Stakeholder groups Diversity within stakeholder group

1 Farmers Smallholder farmers agro-industrialfarmers

2 Non-governmentalorganisations (NGO) andcivil society organisations

(Inter)national agricultural networks andassociations cooperatives developmentorganisations donors

3 Private sector Input and service providers (eg seedand agro-dealers private extensionservices) agricultural entrepreneurs (egprocessors traders retailers transportcompanies)

4 Government Politicians policymakers extension andcrop protection officers

5 Research and training National agricultural research institutesagricultural education and traininginstitutes universities internationalresearch institutes

2 M Schut et alAgricultural Systems 132 (2015) 1ndash11

22 Innovation capacity in the agricultural system

The agricultural system is defined as the ldquooperational unit of ag-riculturerdquo including all actors and organisations at local regionaland national levels involved in the production processing and com-mercialization of agricultural commodities (Spedding 1988)Consequently innovation capacity in the agricultural system isdefined as the ability of these actors and organisations to developnew and mobilise existing competences (including knowledge skillsand experiences) to continuously identify and prioritise con-straints and opportunities for innovation in a dynamic systemscontext (Leeuwis et al 2014)

Following the typical system boundaries used in generic (ie non-agricultural) studies of innovation systems (Carlsson et al 2002Papaioannou et al 2009 Wieczorek and Hekkert 2012) we con-ceptualise the agricultural system as a combination of interrelatedinstitutional sectoral and technological subsystems The institu-tional subsystem comprises different types of institutions whichare the formal and informal rules and structures that shape per-spectives and practices (Leeuwis 2004) In this paper we examinesix types of institutions policy research education and trainingextension markets and politics across different aggregation levels(eg national regional or district) (eg Cooke et al 1997 Freeman1988 1995) The sectoral subsystem is defined around a commod-ity or segments of a value chain (eg rice or cocoa) (eg Blay-Palmer2005 Gildemacher et al 2009) The analysis of the sectoral sub-system seeks to understand interactions between for instance accessto credit inputs and services agricultural production post-harvest activities trade marketing and consumption related to thefunctioning of that value chain (eg Thitinunsomboon et al 2008)Within the agricultural system different sectoral subsystems canexist and interact Technological subsystems are defined around anexisting or novel technology (eg irrigation mechanised weeding)or field of knowledge (eg integrated pest management) to addressa particular problem that may well cut across different sectoral sub-systems (Carlsson and Stankiewicz 1991 Chung 2012 Hekkert et al2007)

23 The agricultural innovation support system

The agricultural innovation support system provides the struc-tural conditions that can enable (when present) or constrain (whenabsent or malfunctioning) innovation within the agricultural systemand its subsystems (Klein Woolthuis et al 2005 van Mierlo et al2010 Wieczorek and Hekkert 2012) (Table 2) Structural condi-tions include (1) adequate knowledge infrastructure in the form ofresearch education and extension physical infrastructure and assetssuch as roads and vehicles and functional communication andfinance structures (2) institutions comprise clear regulatory frame-works and their proper implementation and enforcement (3)interaction and collaboration between multiple stakeholders in theagricultural system and (4) stakeholder capacities (eg literacy andentrepreneurship) and adequate human and financial resources (egnumber of extension officers and funds for their backstopping) Theanalysis of the presence and functioning of these structural con-ditions contributes to a better understanding of what constraintsor enables innovation capacity in the agricultural system (eg limitedmulti-stakeholder collaboration) as well as how the structural con-ditions provided by the agricultural innovation support systemstimulate or hamper this (eg incentive structures for different stake-holder groups to collaborate)

The set-up of the agricultural innovation support system maybe good at supporting incremental lsquosystem optimisationrsquo that re-produce the current state of affairs but less good at supportinglsquosystem transformationrsquo that can lead to radical innovations Forexample the presence of an effective top-down technology-

oriented agricultural extension system can enable the disseminationof crop protection solutions through a technology transfer ap-proach However the existence of this system can form a constraintfor the promotion of agro-ecological approaches through partici-patory farmer-led experiments Consequently to achieve systemtransformation both the agricultural system and the agriculturalinnovation support system should undergo continuous adapta-tion (Hall et al 2004 Spielman 2005)

24 Interactions between complex agricultural problems innovationcapacity in the agricultural system and the agricultural innovationsupport system

The integrated analysis of complex agricultural problems in-novation capacity of the agricultural system and the performanceof the agricultural innovation support system can provide a coher-ent set of specific and generic entry points for innovation (Fig 1)Specific entry points for innovations relate to those innovations thatdirectly contribute to addressing the complex agricultural problemunder study Generic entry points for innovation related to strength-ening the innovation capacity of agricultural system and thefunctioning of the agricultural innovation support system Forexample to reduce fruit waste in developing countries existing tech-nologies for conserving fruits can be adapted to fit the local context(specific entry point for innovation of the technological subsys-tem) This may trigger access to export markets (specific entry pointfor innovation of the sectoral subsystem) and require certificationpolicies to supply such fruit export markets (specific entry point forinnovation of the institutional subsystem) To support the devel-opment implementation and enforcement of certification policiesthe establishment of a national agricultural certification bureau maybe required (generic entry point for innovation) The existence ofsuch a bureau can provide an incentive for investing in the exportof other agricultural products for instance vegetables that in turncan trigger the development or adaptation of conservation tech-nologies to reduce vegetable waste The above example shows howstructural adaptations of the agricultural innovation support systemcan enhance innovation capacity to addressing the complex agri-cultural problem under review (fruit waste) but can also have a spill-over effect on addressing other complex agricultural problems(vegetable waste) Furthermore the agricultural innovation supportsystem can provide conditions that support innovation in the ag-ricultural sector more generally for instance through innovation

Table 2Structural conditions that enable or constrain innovation in systems (based on KleinWoolthuis et al 2005 van Mierlo et al 2010 Wieczorek and Hekkert 2012)

Structuralconditions forinnovation

Description

Infrastructureand assets

Knowledge research and development infrastructurephysical infrastructure including roads irrigation schemesand agricultural inputs distribution communication andfinancial infrastructure

Institutions Formal institutions including agricultural policies lawsregulations (food) quality standards agriculturalsubsidies Monitoring and Evaluation (MampE) structuresorganisational mandates market (access) and tradeagreements informal institutions such as social-culturalnorms and values

Interaction andcollaboration

Multi-stakeholder interaction for learning and problem-solving development and sharing of knowledge andinformation public-private partnerships networksrepresentative bodies (eg farmers association) power-dynamics

Capabilities andresources

Agricultural entrepreneurship labour qualificationshuman resources (quality and quantity) education andliteracy rates financial resources

3M Schut et alAgricultural Systems 132 (2015) 1ndash11

policy or funding schemes that affect multiple institutional sectoraland technological subsystems

3 Methodological framework for RAAIS

31 Selection criteria for methods

RAAIS is a diagnostic tool that combines multiple methods of datacollection Building on existing experiences with rapid appraisal ap-proaches and (participatory) innovation systems analysis five criteriafor the selection of methods have been identified

1 Methods should be diverse rigorous and be able to generateboth qualitative and quantitative data This enhances thecredibility and strength of the analysis (Spielman 2005)Qualitative data provide the basis for the identificationand analysis of the different dimensions of complex agricultur-al problems and structural conditions enabling or constrainingthe innovation capacity Such data may also provide narrativesregarding the underlying causes and historical evolutionof constraints Quantitative data analysis can build on this by pro-viding (descriptive) statistics and trends on for instance thedistribution of constraints across different levels stakeholdergroups or study sites

2 Methods should facilitate both lsquoinsiderrsquo and lsquooutsiderrsquo analysisInsider analysis implies data analysis by stakeholders who can

provide highly detailed explanations of specific phenomena basedon their knowledge and experiences However insiders such asfarmers or policymakers often have an incomplete or insuffi-cient critical view of the broader agricultural system or theagricultural innovation support system Consequently it is im-portant to complement insider analysis by outsider analysis ofdata by researchers (van Mierlo et al 2010) By combining insiderand outsider analysis the delineation of the systems boundar-ies is done in a participatory way by stakeholders and researchers

3 Methods should be able to target different stakeholder groupsacross different levels When studying complex agricultural prob-lems it is essential to include different groups of stakeholderstheir perceptions on what constitutes the problem and what areperceived feasible or desirable solutions (Faysse 2006 Ortiz et al2013)

4 Methods should be able to target stakeholders individually inhomogeneous groups and in heterogeneous groups so as tocapture individual group and multi-stakeholder perceptions onproblems and solutions Discussion and debate in both homo-geneous and heterogeneous stakeholder groups generally providea rich analysis of complex problems and potential solutions Fur-thermore multi-stakeholder interaction may reveal asymmetricpower-relationships that are necessary to understand innova-tion capacity in the agricultural system On the other handpower-relationships group pressure or mutual dependenciesbetween stakeholders may result in situations where sensitive

Fig 1 Schematic representation of the dynamic interactions between complex agricultural problems (multiple dimensions multi-level interactions and multi-stakeholderdynamics) innovation capacity of the agricultural system (including its institutional sectoral and technological subsystems) and the structural conditions within the ag-ricultural innovation support system that can enable or constrain innovation capacity in the agricultural system (infrastructure and assets institutions interaction and collaborationand capabilities and resources) RAAIS provides insight into the current state of the system (on the left) RAAIS provides specific and generic entry points for innovationthat can guide a transition towards the desirable state of the system (on the right) in which the complex agricultural problem is addressed and the innovation capacity inthe agricultural system has increased Generic entry points for innovation can have a spill-over effect in terms of addressing other complex agricultural problems than theone under review

4 M Schut et alAgricultural Systems 132 (2015) 1ndash11

questions are avoided or receive socially desirable responsesMethods that target stakeholders individually are more likely toprovide insights in such questions (International Institute forSustainable Development 2014)

5 Methods together should provide sufficient detail on the complexagricultural problem under review the innovation capacity inthe agricultural system and the functioning of the agriculturalinnovation support system (World Bank 2012)

Combining different types of methods and data collection tech-niques provides an opportunity to triangulate and validate dataDepending on the nature of the agricultural problem under reviewand the available resources and time different types of data col-lection methods can be used for RAAIS taking into account the abovecriteria for method selection

32 Methods of data collection

Based on the five criteria four complementary methods for datacollection were selected to be part of RAAIS (Table 3)

321 Multi-stakeholder workshopsMulti-stakeholder workshops mainly focused on the insider anal-

yses of innovation capacity in the agricultural system and thestructural conditions provided by the agricultural innovation supportsystem A participatory workshop methodology facilitates differ-ent groups of stakeholders to ndash individually and in homogeneousand heterogeneous groups ndash identify categorise and analyse con-straints for innovation in the agricultural system Depending on thetype of problem workshops can be organised with stakeholders rep-resenting national regional andor district levels or for instanceacross different study sites where a specific problem is eminent Tokeep the workshops manageable and to stimulate interaction anddebate the participation of a maximum of 25 participants per work-shop is proposed for instance consisting of five representatives ofthe five different stakeholder groups in Table 1 As much as possi-ble each group should be a representative sample with respect tofor instance gender age income or ethnic groups The work-shops should be held in the language that all participants speakand be facilitated by someone who is familiar with the culturalnorms has affinity with the problem and understands the reali-ties of the different stakeholder groups The proposed workshopmethodology consists of 13 Sessions subdivided into three catego-ries with each their own focus (1) identifying constraints (2)categorising constraints and (3) exploring specific and generic entrypoints for innovation Figure 2 and Table 4 provide an overview ofthe 13 Sessions their sequence and relations and their specific ob-jective in RAAIS

Workshops are designed to take approximately 1 day Besidesthe facilitator a note-taker documents the outcome of the differ-ent sessions and captures discussions among participants Workshopfacilitation and note-taking protocols ensure that the workshoporganisation facilitation and documentation is standardized whichis essential for comparing or aggregating the outcomes for in-stance across different study sites

A crucial element in the workshops is the use of coloured cardsAt the start of the workshop (Session 1) each of the stakeholdergroups is assigned a different colour During Session 2 each par-ticipant individually lists five constraints or challenges they face intheir work and writes them down on their coloured cards If fivestakeholder groups are equally represented this results in 125 cardsDuring Session 3 the participants discuss within their stakehold-er groups the listed constraints explore overlapping issues and jointlydevelop a stakeholder group top-5 If necessary constraints canbe reformulated based on discussions within the group Each of thestakeholder groups use their top-5 throughout the rest of the Ta

ble

3M

eth

odol

ogic

alfr

amew

ork

indi

cati

ng

how

diffe

ren

tm

eth

ods

for

data

coll

ecti

onco

rres

pon

dw

ith

the

sele

ctio

ncr

iter

iafo

rm

eth

ods

Sele

ctio

ncr

iter

iafo

rm

eth

ods

Met

hod

sfo

rda

taco

llect

ion

Typ

eof

data

Insi

der

outs

ider

Stak

ehol

der

grou

pSt

akeh

olde

rin

volv

emen

tA

ISan

alys

is

Qualitative

Quantitative

Insider

Outsider

Farmers

NGOcivilsociety

Privatesector

Government

Researchandtraining

Individual

Homogeneousgroups

Heterogeneousgroups

Complexagriculturalproblems

Innovationcapacityintheagriculturalsystem

Agriculturalinnovationsupportsystem

Mu

lti-

stak

ehol

der

wor

ksh

ops

Sem

i-st

ruct

ure

din

-dep

thin

terv

iew

s

Qu

esti

onn

aire

s

Seco

nda

ryda

taan

alys

is

N

aN

a

5M Schut et alAgricultural Systems 132 (2015) 1ndash11

sessions during the workshop hence 25 cards (five cards per stake-holder group) (Photo 1ndash4)

The use of the coloured cards facilitates the analysis of differ-ent sessions during and after the workshops As the cards are codedand recycled throughout the successive sessions photographs canbe taken to capture the results (for example Photo 1 and 4) Suchphotographs can be analysed after the workshop and can also beused to validate the note-takerrsquos data Furthermore the cards provideinsight into the relations between constraints identified by differ-ent stakeholder groups (Photo 2 and 3) Combining the results fromdifferent sessions can stimulate integrative analyses for instancecombining data resulting from Sessions 5 and 6 provides insight inthe structural conditions for innovation across different levels Sim-ilarly the outcome of Sessions 7 and 11 can be compared totriangulate the data as both seek to identify key constraints for in-novation in the agricultural system

322 Semi-structured in-depth interviewsTo guide the semi-structured interviews a topic list is pre-

pared and fine-tuned for each interview Using a topic list providesa degree of flexibility to identify and to anticipate interestingstorylines related to the problem under review and allows valida-tion of data that was gathered during previous interviews or duringthe workshops Interviews should take a maximum of 1 hour en-suring a high level of attentiveness of both the respondent as wellas the interviewer Sampling of interview respondents should followa stratified approach to ensure that stakeholders representing dif-ferent study sites different stakeholder groups and differentadministrative levels are included Within those strata respon-dents can be selected purposive or based on snowball samplingwhere interview respondents make suggestions for who else shouldbe included in the sample (Russell Bernard 2006) The sample sizecan be based on the concept of ldquosaturationrdquo or the point at which

no new information or themes are observed in the interview data(Guest et al 2006) Interviews can be recorded and transcribed elec-tronically From an ethical point of view interviewees should givepermission for interviews to be recorded and researchers shouldensure confidentiality of all interview data Recording may not alwaysbe desirable as the voice recorder can create a barrier between theresearcher and the respondent especially when it comes to dis-cussing politically sensitive issues Instead of recording detailed notescan be taken and transcribed electronically The transcribed inter-views can be coded Ideally interviews are conducted and codedby two researchers which will enhance the quality of theanalysis

323 SurveysBased on the workshops and the interviews some of the con-

straints may be eligible for broader study among specific groupsof stakeholders through the use of surveys Such surveys mayprovide more insights in for example the socio-economic impactsof climate change on smallholder agriculture in specific regionsthe quality of agricultural extension received by farmers in ad-dressing complex agricultural problems or access to agriculturalinputs for male or female headed households Surveys are not nec-essarily limited to farmers but can also be conducted with any ofthe other stakeholder groups involved For the data to be comple-mentary surveys should be completed in the same study sites aswhere the workshops were organised and among a representativesample of the targeted stakeholder group To achieve that a strati-fied random sampling strategy can be used to identify respondentsacross different study sites levels or stakeholder groups A (effi-cient) sampling method that allows for optimal allocation ofresources can be used to determine the sample size (eg Whitleyand Ball 2002)

324 Secondary data collectionSecondary data are written data with relevance for the analy-

sis of the complex agricultural problem innovation capacity of theagricultural system or the functioning of the agricultural innova-tion support system Examples are policy documents projectproposals and reports laws or legal procedures project evalua-tions curricula for agricultural education and training (agricultural)census and organisational records such as charts and budgets overa period of time The sampling of secondary data is not clear cutKey agricultural documents such as agricultural policies or agri-cultural research priorities should be included These documentscan refer to other relevant data Furthermore secondary data is oftenprovided during or following interviews Insights from secondarydata can be verified in interviews with stakeholders (eg the extentto which policy is implemented and enforced)

4 RAAISrsquo ability to provide specific and generic entry pointsfor innovation and lessons learnt from its application

We tested RAAIS through a case study aimed at analysing con-straints and opportunities for innovation to effectively addressparasitic weeds in rain-fed rice production systems in Tanzania(AprilndashOctober 2012) and Benin (JunendashAugust 2013) The results fromRAAIS in Tanzania are elaborated in Schut et al (2014c) Data weregathered across national zonal regional and district levels Multi-stakeholder workshops (with 68 participants in Tanzania and 66participants in Benin) were organised in three study sites (dis-tricts) in Tanzania and Benin where parasitic weeds are eminentIn-depth interviews were held with representatives of national-zonal- regional- and district-level representatives of farmer coop-eratives and associations NGO civil society private sectorgovernment and research and training institutes (42 in Tanzania65 in Benin) Across the three study sites in the countries a

Session 2

Session3

Session1

Categorisingconstraints

Exploring specific and generic entry points for innovation

Identifying constraints

Fig 2 The relation between the 13 Workshop Sessions and their sequence sub-divided over the three categories The dotted arrows indicate relations between thedifferent sessions in terms of triangulation and validation of data

6 M Schut et alAgricultural Systems 132 (2015) 1ndash11

socio-economic farmer survey (152 in Tanzania 182 in Benin) washeld to study the impact of parasitic weeds on rain-fed rice farming(see Nrsquocho et al 2014 for more information) In Tanzania a farmer-extensionist survey (120 farmers 30 agricultural extension officers)was held to explore the effectiveness of the national agricultural ex-tension policy across the three study sites (see Daniel 2013 for more

information) Additionally for both countries secondary data in-cluding crop protection extension and general agricultural policynational research priorities agricultural census and agricultural train-ing curricula were analysed Data gathering and initial analysis tookaround three months for each of the countries and involved tworesearchers We first conducted the in-depth interviews followed

Table 4The 13 Workshop Sessions subdivided over the three categories and their specific activities and objectives in the RAAIS

Categories Sessions Activities Objective(s)

Identifyingconstraints

1 Opening andparticipantintroduction

Participants (1) introduce themselves andreceive information about the workshopmethodology and (2) are subdivided overdifferent stakeholder groups (eg groupsidentified in Table 1)

bull To ensure an equal representation of participants over thedifferent stakeholder groups

2 Individualbrainstorming aboutconstraints

Participants individually identify fiveconstraints they face in their work

bull To make an inventory of general constraints in the agriculturalsystem faced by stakeholders

3 Developing a top-5 ofconstraints instakeholder groups

Participants (1) discuss constraints withinrespective stakeholder group (2) develop anstakeholder group top-5 of constraints (3)present the top-5 to other stakeholder groupsand (4) discuss within and betweenstakeholder group(s)

bull To gain insights in the key constraints in the agricultural system asfaced by different stakeholder groups

bull To create awareness and stimulate learning among stakeholders

Categorisingconstraints

4 Categorisingconstraints alongdifferent types ofinstitutions

Participants (1) categorise top-5 constraints aspolicy- research- education and training-extension- markets- and or politics-related(2) present results to the other groups and (3)discuss within and between the stakeholdergroup(s)

bull To gain insights in how key constraints relate to the different typesof institutions (institutional subsystem)

bull To create awareness and stimulate learning between stakeholders

5 Categorisingconstraints alongstructural conditionsthat can enable orconstrain innovation

Participants (1) categorise top-5 constraintsalong the structural conditions drivers ofinnovation (Table 2) and (2) discuss withinand between the stakeholder group(s)

bull To gain insights in how the stakeholder constraints relate tostructural conditions provided agricultural innovation supportsystem and whether these enable or constrain innovation capacity

bull To create awareness and stimulate learning between stakeholders

6 Categorisingconstraints acrossdifferent(administrative) levelswithin the institutionalsubsystems

Participants (1) categorise top-5 constraintsacross different administrative levels (egnational regional district) (2) discuss resultswith other stakeholder groups and (3) discusswithin and between the stakeholder group(s)

bull To gain insights in how key constraints relate to differentinstitutional (administrative) levels

bull To identify and analyse interactions between different levelsbull To create awareness and stimulate learning between stakeholders

7 Identifyingrelationships betweenconstraints andidentifying keyconstraints

Participants (1) jointly discuss and identifyrelations between the different constraints (2)identify constraints or challenges that arecentral in the analysis and (3) discuss withinand between the stakeholder group(s)

bull To analyse relationships between different constraintsbull To identify key constraintsbull To create awareness and stimulate learning between stakeholdersbull Identify generic entry points for enhancing the innovation

capacity in the agricultural system8 Categorising

constraints along thesectoral subsystem

Participants (1) categorise stakeholder grouptop-5 constraints along the segments of thevalue chain and (2) discuss within andbetween the stakeholder group(s)

bull To analyse constraints along the sectoral subsystembull To create awareness and stimulate learning between stakeholders

9 Categorisingconstraints alongdifferent technologicalsubsystems

Participants (1) categorise top-5 constraintsalong different technological or knowledgefields and (2) discuss within and between thestakeholder group(s)

bull To analyse constraints along different technological subsystemsbull To create awareness and stimulate learning between stakeholders

Exploringentrypoints forinnovation

10 Exploring constraintsstakeholder groups cansolve themselvesversus problems thatcan only be solvedwith or by others

Participants (1) categorise top-5 constraints aslsquocan be solved within the stakeholder grouprsquoor lsquocan only be solved in collaboration withother stakeholder groupsrsquo and (2) discussionwithin and between the stakeholder group(s)

bull To identify constraints that require collaboration betweenstakeholder groups

bull To create awareness and stimulate learning between stakeholdersbull Identify entry points for innovation in the agricultural system

11 Exploring constraintsthat are easy difficultto solve

Participants (1) categorise top-5 constraints asrelatively lsquoeasyrsquo or lsquodifficultrsquo to address and (2)discuss within and between the stakeholdergroup(s)

bull To explore which constraints require system optimisation (easy toaddress) and those that require system transformation (difficult toaddress)

bull To create awareness and stimulate learning between stakeholdersbull To triangulate data with Session 7 (are key constraints perceived

to be easy difficult to address)bull Identify entry points for enhancing the innovation capacity in the

agricultural system12 Exploring constraints

that are structuraloperational

Participants categorise top-5 constraints alonga four-step gradient ranging from lsquoverystructuralrsquo lsquostructuralrsquo lsquooperationalrsquo and lsquoveryoperationalrsquo challenges and constraints

bull To distinguish between structural constraints that require specificinnovation and more structural problems that require genericinnovation

bull To create awareness and stimulate learning between stakeholdersbull To triangulate data with Sessions 7 and 11 (relation between key

constraints how these are perceived by stakeholders)bull Identify generic entry points for enhancing the innovation

capacity in the agricultural system13 Identifying priorities

and solution strategiesParticipants (1) jointly discuss and develop anoverall top-5 of constraints and (2) jointlyidentify potential strategies to address theseconstraints

bull To explore opportunities for addressing systems constraintsthrough multi-stakeholder collaboration

bull To explore similarities and differences with the key systemsconstraints identified in Session 7

bull Identify key entry points for innovation

7M Schut et alAgricultural Systems 132 (2015) 1ndash11

by the multi-stakeholder workshops In Tanzania both the socio-economic farmer survey and the farmer-extensionist survey wereheld after the interviews and workshops In Benin the socio-economic farmer survey was held preceding the in-depth interviewsand workshops Secondary data collection occurred throughout thefieldwork Below we will further reflect on the main objectives ofRAAIS as well as provide recommendations for further improve-ments and use of RAAIS using our experiences from Tanzania andBenin

41 RAAISrsquo ability to provide specific entry points for innovation toaddress complex agricultural problems

RAAIS contributed to an integrated understanding of differentproblem dimensions multi-level interactions and multi-stakeholderdynamics related to parasitic weed problems With regard to thedifferent problem dimensions interviews demonstrated a poten-tial relation between for example the preference for growing localaromatic rice varieties (social-cultural dimension) the low capac-ity of farmer to purchase certified seeds (economic dimension) andthe spread of parasitic weed seeds through the local rice seed system(technological dimension) Additionally analysis of workshop datarevealed how the untimely and insufficient availability of agricul-tural inputs provided by the government (institutional dimension)and limited interaction and collaboration among networks of key

stakeholders (political dimensions) form additional bottlenecks foraddressing such problems It created awareness that describing andexplaining complex agricultural problems and exploring and de-signing solutions is unlikely to be successful if the different problemdimensions are analysed and treated separately (Hall and Clark 2010Spielman et al 2009)

Data gathering across different levels (national region and dis-trict level) enabled the analysis of the interactions and (mis)matchesbetween different levels (Cash et al 2006) An example that emergedduring the workshops and the interviews is Tanzaniarsquos nationalexport ban that prohibits export of agricultural produce (eg of rice)as long as the country has not been declared lsquofood securersquo This na-tional export ban influences local market prices and consequentlyalso farmersrsquo willingness and ability to invest in for example pur-chasing agricultural inputs such as fertilizers and seeds (eg Poultonet al 2010) This in turn provided an opportunity to identify entrypoints for innovation across different levels which has been iden-tified as a critical factor for addressing complex agricultural problems(eg Giller et al 2008 2011) As expected and confirming previ-ous reports (eg van Mierlo et al 2010) the participatory analysisof multi-level interactions showed that stakeholders (insiders) oftenidentify constraints at the level they represent (Schut et al 2014c)This was complemented by our analysis as researchers (outsiders)of the multi-level interactions regarding the parasitic weedproblems

Photo 1ndash4 Photo 1 (top left) Top-5 of constraints of NGO civil society representatives and their categorisation under the different components of the institutional sub-system (Session 4) Photo 2 (top right) The categorisation of the top-5 of the different stakeholder groups along different structural conditions that can enable or constraininnovation (Session 5) Photo 3 (bottom left) The identification of relationships between different constraints (arrows) and key problem (circled cards) (Session 7) Photo4 (bottom right) The categorisation of the top-5 of the different stakeholder groups along a four-step gradient ranging from structural to operational constraints (Session12) Photos were taken by M Schut during multi-stakeholder workshops in Tanzania held in October 2012

8 M Schut et alAgricultural Systems 132 (2015) 1ndash11

The involvement of different groups of stakeholders was essen-tial for enhancing the credibility validity and quality of RAAIS aswell as for delineating the boundaries of the agricultural system andthe agricultural innovation support system which is considered akey challenge when using AIS approaches to analyse complex ag-ricultural problems (Klerkx et al 2012b) Furthermore stakeholderparticipation provided a better understanding of the feasibility andacceptability of solutions for stakeholder groups Although we believethat the stakeholder groups included in the testing of RAAIS (Table 1)provide a good starting point other stakeholder groups (for in-stance the media) may be included in the sample (eg Ortiz et al2013) depending on the type of complex agricultural problem underreview The triangulation of data resulting from the different methodsenabled us to validate findings and to verify strategic communi-cation by stakeholders for instance to verify how the extensionsystem as described by policymakers in interviews functioned inreality according to surveyed farmers

42 RAAISrsquo ability to provide generic entry points for innovation

RAAIS demonstrates interactions between complex agricultur-al problems innovation capacity of the agricultural system ndashconsisting of institutional sectoral and technological subsystemsndash and the agricultural innovation support system For example ap-plying fertilizer (technological subsystem) in rain-fed rice productionis seen as a promising management strategy to reduce infection levelsof Rhamphicarpa one of the parasitic weeds involved in the studyand mitigate negative effects of the parasite on rice yields (Rodenburget al 2011) However as was highlighted during the RAAIS work-shops in both in Benin and in Tanzania fertilizers are difficult toaccess in rural areas In Benin there is no well-developed privateagro-dealer network and distribution infrastructure to support thesupply of agricultural inputs Furthermore interviews showed thatthe public extension and input supply systems in Benin focus onthe cotton sector rather than on cereal crops (sectoral subsys-tems) In Tanzania a private agro-dealer network and distributioninfrastructure exists but structures controlling the quality of fer-tilizers (institutional subsystem) are functioning sub-optimallyaccording to interviewed government officials In some areas fakeagro inputs are dominating the market resulting in a limited trustand willingness to invest in applying fertilizer according to farmerrepresentatives who participated in the workshops The exampleshows how the absence or poor performance of fertilizer distribu-tion infrastructure limited farmer-extensionist interaction and lackof functional institutions for quality control (being structural con-ditions for innovation) constrain the innovation capacity in theagricultural systems and its technological (in this case fertilizer) andsectoral (the rice value chain) subsystems Another example is basedon secondary data analyses that demonstrated the lack of an op-erational strategy to address parasitic weeds in Tanzania and BeninIn both the interviews and workshops stakeholders highlighted thegeneral lack of interaction and collaboration between stakehold-ers in the agricultural sector (being a structural condition forinnovation) as one of the main reasons for the absence or poor im-plementation of parasitic weed and other agricultural policies andstrategies

The aforementioned examples demonstrate how RAAIS cansupport the identification of generic entry points for innovation Suchinnovations can directly contribute to addressing the complex ag-ricultural problem under review but can also have a spill-over effectin terms of addressing broader constraints that hamper the inno-vation capacity in the agricultural system For example the lack ofstakeholder interactions and collaboration in the agricultural systemcan provide an entry point for the adaptation of the structuralconditions in the broader agricultural innovation support systemfor example through investments in innovation brokers or

multi-stakeholder platforms (Kilelu et al 2013 Klerkx et al 2010)Such structural adjustments can facilitate multi-stakeholder col-laboration in tackling parasitic weed as well as other complexagricultural problems

43 Lessons learnt from applying RAAIS and recommendations forfurther improvement

Based on our experiences in Tanzania and Benin we recom-mend conducting RAAIS in an interdisciplinary team of researcherswith expertise on different dimensions of complex agricultural prob-lems and on different data collection methods (Hulsebosch 2001)Other suggestions include the experimentation with other combi-nations of methods and on different types of complex agriculturalproblems The workshop methodology could be made more inter-active in the sense of directly feeding back results of the sessionsto participants to stimulate reflection and validate analyses duringthe workshops Post-workshop surveys could provide additionalinsight into whether stakeholders felt they could freely raise anddiscuss their ideas and needs

The multi-stakeholder workshops but also the surveys pre-sented a rather static picture of the complex agricultural problemunder review and the innovation capacity of the agricultural systemin which the problem is embedded However initial workshops andsurveys could function as a baseline to which future workshops andsurveys can be compared Other methods such as secondary dataanalysis or in-depth interviews present a more dynamic image ofhow for example collaborations between stakeholders evolve overthe years Our experiences in Tanzania and Benin show that ensur-ing social differentiation among workshop participants intervieweesand survey respondents (eg of different gender of age) was chal-lenging as for example the majority of workshop participants weremale Specific Workshop Sessions could have more attention for cat-egorisation and priority setting by different gender or age groupsThe facilitation of the multi-stakeholder workshops ensured thatdifferent stakeholder groups could raise and discuss their ideas(Hulsebosch 2001) Despite such efforts unequal power relationsand differences in the ability to debate and negotiate that inher-ently exist between groups may have played a role In line with ourexpectations politically sensitive issues were more freely dis-cussed in individual interviews as compared to multi-stakeholdersetting

The combination of different methods of data collection wasessential In terms of the sequence of data collection we recom-mend to first conduct and analyse the RAAIS multi-stakeholderworkshops to identify constraints and subsequently conduct thein-depth interviews and surveys that can provide more insight inthe distribution and underlying root causes of these constraintsThe workshops then provide a lsquofast-trackrsquo approach to identifyingentry points for innovation that can subsequently be validated andexplored in more detail using the in-depth interviews and stake-holder surveys This would furthermore increase the lsquorapidnessrsquoof RAAIS as a diagnostic tool

An updated version of the RAAIS multi-stakeholder workshopshas been used to identify constraints challenges and entry pointsfor innovations related to the lsquosustainable intensification of agri-cultural systemsrsquo in Burundi the Democratic Republic of CongoRwanda Nigeria and Cameroon under the CGIAR Research Pro-gramme for the Humid Tropics (Humidtropics) (Schut and Hinnou2014) Several of the recommendations made in this paper includ-ing the revised sequence of methods for data collection and the useof post-workshop participant questionnaires have been imple-mented and tested successfully Some of the bottlenecks identifiedsuch as social differentiation (eg gender and age groups) amongworkshop participants remained problematic and require furtherattention At the end of the Humidtropics RAAIS workshops

9M Schut et alAgricultural Systems 132 (2015) 1ndash11

participants developed action plans to address the prioritised con-straints (Workshop Session 13) This required an extension of theworkshops of half a day The development and implementation ofthe action plans forms an important element for continued stake-holder collaboration in multi-stakeholder platforms

5 Conclusions

This paper demonstrates the potential of RAAIS as a diagnostictool that can support and guide the integrated analysis of complexagricultural problems innovation capacity in the agricultural systemand the performance of the agricultural innovation support systemRAAIS combines multiple qualitative and quantitative methods andinsider (stakeholders) and outsider (researchers) analyses whichallow for critical triangulation and validation of the gathered dataSuch an analysis can provide specific entry points for innovationsto address the complex agricultural problem under study and genericentry points for innovation related to strengthening the innova-tion capacity of agricultural system and the functioning of theagricultural innovation support system

Recommendations for further improvement include using RAAISfor the analysis of other types of complex agricultural problems usingother combinations of methods of data collection and providing di-rectly feedback to workshop participants to stimulate reflection andvalidate workshop outcomes An adapted sequence of data collec-tion methods in which workshops provide a lsquofast-trackrsquo approachto identifying entry points for innovation followed up by more in-depth interviews and stakeholder surveys would increase the RAAISrsquodiagnostic capacity The participatory development of concrete actionplans based on RAAIS can provide a basis for continued multi-stakeholder collaboration to operationalise and implement specificand generic entry points for innovation

Acknowledgements

This research forms part of the PARASITE programme fundedby the Integrated Programmes Scheme of the NetherlandsOrganisation for Scientific Research ndash Science for Global Develop-ment (NWO-WOTRO) (wwwparasite-projectorg) Additional supportis provided by the CGIAR Research Program on Climate Change Ag-riculture and Food Security (CCAFS) The CGIAR Research Programmefor the Humid Tropics (httphumidtropicscgiarorg) provided theopportunity to further adapt apply and improve RAAIS across dif-ferent Humidtropics Action Areas Special thanks to Elifadhili Danielwho conducted the farmer-extensionist survey

References

Amankwah K Klerkx L Oosting SJ Sakyi-Dawson O van der Zijpp AJ MillarD 2012 Diagnosing constraints to market participation of small ruminantproducers in northern Ghana an innovation systems analysis NJAS-Wagen JLife Sc 60ndash63 37ndash47

Basu S Leeuwis C 2012 Understanding the rapid spread of System of RiceIntensification (SRI) in Andhra Pradesh exploring the building of supportnetworks and media representation Agr Syst 111 34ndash44

Beintema N Stads G-J Fuglie K Heisey P 2012 ASTI global assessment ofagricultural RampD spending International Food Policy Research InstituteWashington DC Agricultural Science and Technology Indicators Rome Italy andthe Global Forum on Agricultural Research Rome Italy

Blay-Palmer A 2005 Growing innovation policy the case of organic agriculturein Ontario Canada Environ Plann C Gov Policy 23 557ndash581

Carlsson B Stankiewicz R 1991 On the nature function and composition oftechnological systems J Evol Econ 1 93ndash118

Carlsson B Jacobsson S Holmeacuten M Rickne A 2002 Innovation systems analyticaland methodological issues Res Policy 31 233ndash245

Cash DW Adger WN Berkes F Garden P Lebel L Olsson P et al 2006 Scaleand cross-scale dynamics governance and information in a multilevel worldEcol Soc 11 8 Available from lthttpwwwecologyandsocietyorgvol11iss12art18gt [online]

Chambers R 1994 The origins and practice of participatory rural appraisal WorldDev 22 953ndash969

Chung CC 2012 National sectoral and technological innovation systems the caseof Taiwanese pharmaceutical biotechnology and agricultural biotechnologyinnovation systems (1945ndash2000) Sci Public Policy 39 271ndash281

Cooke P Uranga MG Etxebarria G 1997 Regional innovation systems Institutionaland organisational dimensions Res Policy 26 475ndash491

Daniel E 2013 Assessment of agricultural extension services in Tanzania A casestudy of Kyela Songea Rural and Morogoro Rural Districts Wageningen Universityand Research CentreAfrica Rice Center Wageningen the NetherlandsDar esSalaam Tanzania p 45

Douthwaite B Kuby T van de Fliert E Schulz S 2003 Impact pathway evaluationan approach for achieving and attributing impact in complex systems Agr Syst78 243ndash265

Engel PGH 1995 Facilitating Innovation An Action-Oriented Approach andParticipatory Methodology to Improve Innovative Social Practice in AgricultureWageningen University Wageningen the Netherlands p 300

Faysse N 2006 Troubles on the way an analysis of the challenges faced bymulti-stakeholder platforms Nat Resour Forum 30 219ndash229

Freeman C 1988 Japan a new national innovation system In Dosi G FreemanC Nelson R Soete T (Eds) Technology and Economic Theory London pp330ndash348

Freeman C 1995 The national system of innovation in historical perspective CambJ Econ 19 5ndash24

Funtowicz SO Ravetz JR 1993 Science for the post-normal age Futures 25739ndash755

Gildemacher PR Kaguongo W Ortiz O Tesfaye A Woldegiorgis G Wagoire WWet al 2009 Improving potato production in Kenya Uganda and Ethiopia a systemdiagnosis Potato Res 52 173ndash205

Giller KE Leeuwis C Andersson JA Andriesse W Brouwer A Frost P et al2008 Competing claims on natural resources what role for science Ecol Soc13 34 Available from lthttpwwwecologyandsocietyorgvol13iss32art34gt[online]

Giller KE Tittonell P Rufino MC van Wijk MT Zingore S Mapfumo P et al2011 Communicating complexity integrated assessment of trade-offs concerningsoil fertility management within African farming systems to support innovationand development Agr Syst 104 191ndash203

Guest G Bunce A Johnson L 2006 How many interviews are enough Anexperiment with data saturation and variability Field Methods 18 59ndash82

Hall A 2005 Capacity development for agricultural biotechnology in developingcountries an innovation systems view of what it is and how to develop it J IntDev 17 611ndash630

Hall A Clark N 2010 What do complex adaptive systems look like and what arethe implications for innovation policy J Int Dev 22 308ndash324

Hall A Rasheed Sulaiman V Clark N Yoganand B 2003 From measuring impactto learning institutional lessons an innovation systems perspective on improvingthe management of international agricultural research Agr Syst 78 213ndash241

Hall AJ Yoganand B Sulaiman RV Raina RS Prasad CS Naik GC et al 2004Innovations in innovation reflections on partnership institutions and learningICRISAT Andra Pradesh

Hekkert MP Suurs RAA Negro SO Kuhlmann S Smits REHM 2007 Functionsof innovation systems a new approach for analysing technological changeTechnol Forecast Soc Change 74 413ndash432

Hermans F Stuiver M Beers PJ Kok K 2013 The distribution of roles andfunctions for upscaling and outscaling innovations in agricultural innovationsystems Agr Syst 115 117ndash128

Hounkonnou D Kossou D Kuyper TW Leeuwis C Nederlof ES Roumlling N et al2012 An innovation systems approach to institutional change smallholderdevelopment in West Africa Agr Syst 108 74ndash83

Hulsebosch J 2001 The use of RAAKS for strengthening community-basedorganisations in Mali Dev Pract 11 622ndash632

Hung S-C Whittington R 2011 Agency in national innovation systems institutionalentrepreneurship and the professionalization of Taiwanese IT Res Policy 40526ndash538

International Institute for Sustainable Development 2014 Rapid Rural Appraisal(RRA)

Jiggins J 2012 Diagnosing the scope for innovation linking smallholder practicesand institutional context NJAS-Wagen J Life Sc 60ndash63 1ndash122

Kilelu CW Klerkx L Leeuwis C 2013 Unravelling the role of innovation platformsin supporting co-evolution of innovation contributions and tensions in asmallholder dairy development programme Agr Syst 118 65ndash77

Klein Woolthuis R Lankhuizen M Gilsing V 2005 A system failure frameworkfor innovation policy design Technovation 25 609ndash619

Klerkx L Aarts N Leeuwis C 2010 Adaptive management in agricultural innovationsystems the interactions between innovation networks and their environmentAgr Syst 103 390ndash400

Klerkx L Schut M Leeuwis C Kilelu C 2012a Advances in knowledge brokeringin the agricultural sector towards innovation system facilitation IDS Bull 4353ndash60

Klerkx L van Mierlo B Leeuwis C 2012b Evolution of systems approaches toagricultural innovation Concepts analysis and interventions In Darnhofer IGibbon D Dedieu B (Eds) Farming Systems Research into the 21st CenturyThe New Dynamic Springer Dordrecht pp 457ndash483

Leeuwis C 2000 Reconceptualizing participation for sustainable rural developmenttowards a negotiation approach Dev Change 31 931ndash959

Leeuwis C 2004 Communication for Rural Innovation Rethinking AgriculturalExtension (with Contributions of Anne van den Ban) Blackwell Science Oxford

10 M Schut et alAgricultural Systems 132 (2015) 1ndash11

22 Innovation capacity in the agricultural system

The agricultural system is defined as the ldquooperational unit of ag-riculturerdquo including all actors and organisations at local regionaland national levels involved in the production processing and com-mercialization of agricultural commodities (Spedding 1988)Consequently innovation capacity in the agricultural system isdefined as the ability of these actors and organisations to developnew and mobilise existing competences (including knowledge skillsand experiences) to continuously identify and prioritise con-straints and opportunities for innovation in a dynamic systemscontext (Leeuwis et al 2014)

Following the typical system boundaries used in generic (ie non-agricultural) studies of innovation systems (Carlsson et al 2002Papaioannou et al 2009 Wieczorek and Hekkert 2012) we con-ceptualise the agricultural system as a combination of interrelatedinstitutional sectoral and technological subsystems The institu-tional subsystem comprises different types of institutions whichare the formal and informal rules and structures that shape per-spectives and practices (Leeuwis 2004) In this paper we examinesix types of institutions policy research education and trainingextension markets and politics across different aggregation levels(eg national regional or district) (eg Cooke et al 1997 Freeman1988 1995) The sectoral subsystem is defined around a commod-ity or segments of a value chain (eg rice or cocoa) (eg Blay-Palmer2005 Gildemacher et al 2009) The analysis of the sectoral sub-system seeks to understand interactions between for instance accessto credit inputs and services agricultural production post-harvest activities trade marketing and consumption related to thefunctioning of that value chain (eg Thitinunsomboon et al 2008)Within the agricultural system different sectoral subsystems canexist and interact Technological subsystems are defined around anexisting or novel technology (eg irrigation mechanised weeding)or field of knowledge (eg integrated pest management) to addressa particular problem that may well cut across different sectoral sub-systems (Carlsson and Stankiewicz 1991 Chung 2012 Hekkert et al2007)

23 The agricultural innovation support system

The agricultural innovation support system provides the struc-tural conditions that can enable (when present) or constrain (whenabsent or malfunctioning) innovation within the agricultural systemand its subsystems (Klein Woolthuis et al 2005 van Mierlo et al2010 Wieczorek and Hekkert 2012) (Table 2) Structural condi-tions include (1) adequate knowledge infrastructure in the form ofresearch education and extension physical infrastructure and assetssuch as roads and vehicles and functional communication andfinance structures (2) institutions comprise clear regulatory frame-works and their proper implementation and enforcement (3)interaction and collaboration between multiple stakeholders in theagricultural system and (4) stakeholder capacities (eg literacy andentrepreneurship) and adequate human and financial resources (egnumber of extension officers and funds for their backstopping) Theanalysis of the presence and functioning of these structural con-ditions contributes to a better understanding of what constraintsor enables innovation capacity in the agricultural system (eg limitedmulti-stakeholder collaboration) as well as how the structural con-ditions provided by the agricultural innovation support systemstimulate or hamper this (eg incentive structures for different stake-holder groups to collaborate)

The set-up of the agricultural innovation support system maybe good at supporting incremental lsquosystem optimisationrsquo that re-produce the current state of affairs but less good at supportinglsquosystem transformationrsquo that can lead to radical innovations Forexample the presence of an effective top-down technology-

oriented agricultural extension system can enable the disseminationof crop protection solutions through a technology transfer ap-proach However the existence of this system can form a constraintfor the promotion of agro-ecological approaches through partici-patory farmer-led experiments Consequently to achieve systemtransformation both the agricultural system and the agriculturalinnovation support system should undergo continuous adapta-tion (Hall et al 2004 Spielman 2005)

24 Interactions between complex agricultural problems innovationcapacity in the agricultural system and the agricultural innovationsupport system

The integrated analysis of complex agricultural problems in-novation capacity of the agricultural system and the performanceof the agricultural innovation support system can provide a coher-ent set of specific and generic entry points for innovation (Fig 1)Specific entry points for innovations relate to those innovations thatdirectly contribute to addressing the complex agricultural problemunder study Generic entry points for innovation related to strength-ening the innovation capacity of agricultural system and thefunctioning of the agricultural innovation support system Forexample to reduce fruit waste in developing countries existing tech-nologies for conserving fruits can be adapted to fit the local context(specific entry point for innovation of the technological subsys-tem) This may trigger access to export markets (specific entry pointfor innovation of the sectoral subsystem) and require certificationpolicies to supply such fruit export markets (specific entry point forinnovation of the institutional subsystem) To support the devel-opment implementation and enforcement of certification policiesthe establishment of a national agricultural certification bureau maybe required (generic entry point for innovation) The existence ofsuch a bureau can provide an incentive for investing in the exportof other agricultural products for instance vegetables that in turncan trigger the development or adaptation of conservation tech-nologies to reduce vegetable waste The above example shows howstructural adaptations of the agricultural innovation support systemcan enhance innovation capacity to addressing the complex agri-cultural problem under review (fruit waste) but can also have a spill-over effect on addressing other complex agricultural problems(vegetable waste) Furthermore the agricultural innovation supportsystem can provide conditions that support innovation in the ag-ricultural sector more generally for instance through innovation

Table 2Structural conditions that enable or constrain innovation in systems (based on KleinWoolthuis et al 2005 van Mierlo et al 2010 Wieczorek and Hekkert 2012)

Structuralconditions forinnovation

Description

Infrastructureand assets

Knowledge research and development infrastructurephysical infrastructure including roads irrigation schemesand agricultural inputs distribution communication andfinancial infrastructure

Institutions Formal institutions including agricultural policies lawsregulations (food) quality standards agriculturalsubsidies Monitoring and Evaluation (MampE) structuresorganisational mandates market (access) and tradeagreements informal institutions such as social-culturalnorms and values

Interaction andcollaboration

Multi-stakeholder interaction for learning and problem-solving development and sharing of knowledge andinformation public-private partnerships networksrepresentative bodies (eg farmers association) power-dynamics

Capabilities andresources

Agricultural entrepreneurship labour qualificationshuman resources (quality and quantity) education andliteracy rates financial resources

3M Schut et alAgricultural Systems 132 (2015) 1ndash11

policy or funding schemes that affect multiple institutional sectoraland technological subsystems

3 Methodological framework for RAAIS

31 Selection criteria for methods

RAAIS is a diagnostic tool that combines multiple methods of datacollection Building on existing experiences with rapid appraisal ap-proaches and (participatory) innovation systems analysis five criteriafor the selection of methods have been identified

1 Methods should be diverse rigorous and be able to generateboth qualitative and quantitative data This enhances thecredibility and strength of the analysis (Spielman 2005)Qualitative data provide the basis for the identificationand analysis of the different dimensions of complex agricultur-al problems and structural conditions enabling or constrainingthe innovation capacity Such data may also provide narrativesregarding the underlying causes and historical evolutionof constraints Quantitative data analysis can build on this by pro-viding (descriptive) statistics and trends on for instance thedistribution of constraints across different levels stakeholdergroups or study sites

2 Methods should facilitate both lsquoinsiderrsquo and lsquooutsiderrsquo analysisInsider analysis implies data analysis by stakeholders who can

provide highly detailed explanations of specific phenomena basedon their knowledge and experiences However insiders such asfarmers or policymakers often have an incomplete or insuffi-cient critical view of the broader agricultural system or theagricultural innovation support system Consequently it is im-portant to complement insider analysis by outsider analysis ofdata by researchers (van Mierlo et al 2010) By combining insiderand outsider analysis the delineation of the systems boundar-ies is done in a participatory way by stakeholders and researchers

3 Methods should be able to target different stakeholder groupsacross different levels When studying complex agricultural prob-lems it is essential to include different groups of stakeholderstheir perceptions on what constitutes the problem and what areperceived feasible or desirable solutions (Faysse 2006 Ortiz et al2013)

4 Methods should be able to target stakeholders individually inhomogeneous groups and in heterogeneous groups so as tocapture individual group and multi-stakeholder perceptions onproblems and solutions Discussion and debate in both homo-geneous and heterogeneous stakeholder groups generally providea rich analysis of complex problems and potential solutions Fur-thermore multi-stakeholder interaction may reveal asymmetricpower-relationships that are necessary to understand innova-tion capacity in the agricultural system On the other handpower-relationships group pressure or mutual dependenciesbetween stakeholders may result in situations where sensitive

Fig 1 Schematic representation of the dynamic interactions between complex agricultural problems (multiple dimensions multi-level interactions and multi-stakeholderdynamics) innovation capacity of the agricultural system (including its institutional sectoral and technological subsystems) and the structural conditions within the ag-ricultural innovation support system that can enable or constrain innovation capacity in the agricultural system (infrastructure and assets institutions interaction and collaborationand capabilities and resources) RAAIS provides insight into the current state of the system (on the left) RAAIS provides specific and generic entry points for innovationthat can guide a transition towards the desirable state of the system (on the right) in which the complex agricultural problem is addressed and the innovation capacity inthe agricultural system has increased Generic entry points for innovation can have a spill-over effect in terms of addressing other complex agricultural problems than theone under review

4 M Schut et alAgricultural Systems 132 (2015) 1ndash11

questions are avoided or receive socially desirable responsesMethods that target stakeholders individually are more likely toprovide insights in such questions (International Institute forSustainable Development 2014)

5 Methods together should provide sufficient detail on the complexagricultural problem under review the innovation capacity inthe agricultural system and the functioning of the agriculturalinnovation support system (World Bank 2012)

Combining different types of methods and data collection tech-niques provides an opportunity to triangulate and validate dataDepending on the nature of the agricultural problem under reviewand the available resources and time different types of data col-lection methods can be used for RAAIS taking into account the abovecriteria for method selection

32 Methods of data collection

Based on the five criteria four complementary methods for datacollection were selected to be part of RAAIS (Table 3)

321 Multi-stakeholder workshopsMulti-stakeholder workshops mainly focused on the insider anal-

yses of innovation capacity in the agricultural system and thestructural conditions provided by the agricultural innovation supportsystem A participatory workshop methodology facilitates differ-ent groups of stakeholders to ndash individually and in homogeneousand heterogeneous groups ndash identify categorise and analyse con-straints for innovation in the agricultural system Depending on thetype of problem workshops can be organised with stakeholders rep-resenting national regional andor district levels or for instanceacross different study sites where a specific problem is eminent Tokeep the workshops manageable and to stimulate interaction anddebate the participation of a maximum of 25 participants per work-shop is proposed for instance consisting of five representatives ofthe five different stakeholder groups in Table 1 As much as possi-ble each group should be a representative sample with respect tofor instance gender age income or ethnic groups The work-shops should be held in the language that all participants speakand be facilitated by someone who is familiar with the culturalnorms has affinity with the problem and understands the reali-ties of the different stakeholder groups The proposed workshopmethodology consists of 13 Sessions subdivided into three catego-ries with each their own focus (1) identifying constraints (2)categorising constraints and (3) exploring specific and generic entrypoints for innovation Figure 2 and Table 4 provide an overview ofthe 13 Sessions their sequence and relations and their specific ob-jective in RAAIS

Workshops are designed to take approximately 1 day Besidesthe facilitator a note-taker documents the outcome of the differ-ent sessions and captures discussions among participants Workshopfacilitation and note-taking protocols ensure that the workshoporganisation facilitation and documentation is standardized whichis essential for comparing or aggregating the outcomes for in-stance across different study sites

A crucial element in the workshops is the use of coloured cardsAt the start of the workshop (Session 1) each of the stakeholdergroups is assigned a different colour During Session 2 each par-ticipant individually lists five constraints or challenges they face intheir work and writes them down on their coloured cards If fivestakeholder groups are equally represented this results in 125 cardsDuring Session 3 the participants discuss within their stakehold-er groups the listed constraints explore overlapping issues and jointlydevelop a stakeholder group top-5 If necessary constraints canbe reformulated based on discussions within the group Each of thestakeholder groups use their top-5 throughout the rest of the Ta

ble

3M

eth

odol

ogic

alfr

amew

ork

indi

cati

ng

how

diffe

ren

tm

eth

ods

for

data

coll

ecti

onco

rres

pon

dw

ith

the

sele

ctio

ncr

iter

iafo

rm

eth

ods

Sele

ctio

ncr

iter

iafo

rm

eth

ods

Met

hod

sfo

rda

taco

llect

ion

Typ

eof

data

Insi

der

outs

ider

Stak

ehol

der

grou

pSt

akeh

olde

rin

volv

emen

tA

ISan

alys

is

Qualitative

Quantitative

Insider

Outsider

Farmers

NGOcivilsociety

Privatesector

Government

Researchandtraining

Individual

Homogeneousgroups

Heterogeneousgroups

Complexagriculturalproblems

Innovationcapacityintheagriculturalsystem

Agriculturalinnovationsupportsystem

Mu

lti-

stak

ehol

der

wor

ksh

ops

Sem

i-st

ruct

ure

din

-dep

thin

terv

iew

s

Qu

esti

onn

aire

s

Seco

nda

ryda

taan

alys

is

N

aN

a

5M Schut et alAgricultural Systems 132 (2015) 1ndash11

sessions during the workshop hence 25 cards (five cards per stake-holder group) (Photo 1ndash4)

The use of the coloured cards facilitates the analysis of differ-ent sessions during and after the workshops As the cards are codedand recycled throughout the successive sessions photographs canbe taken to capture the results (for example Photo 1 and 4) Suchphotographs can be analysed after the workshop and can also beused to validate the note-takerrsquos data Furthermore the cards provideinsight into the relations between constraints identified by differ-ent stakeholder groups (Photo 2 and 3) Combining the results fromdifferent sessions can stimulate integrative analyses for instancecombining data resulting from Sessions 5 and 6 provides insight inthe structural conditions for innovation across different levels Sim-ilarly the outcome of Sessions 7 and 11 can be compared totriangulate the data as both seek to identify key constraints for in-novation in the agricultural system

322 Semi-structured in-depth interviewsTo guide the semi-structured interviews a topic list is pre-

pared and fine-tuned for each interview Using a topic list providesa degree of flexibility to identify and to anticipate interestingstorylines related to the problem under review and allows valida-tion of data that was gathered during previous interviews or duringthe workshops Interviews should take a maximum of 1 hour en-suring a high level of attentiveness of both the respondent as wellas the interviewer Sampling of interview respondents should followa stratified approach to ensure that stakeholders representing dif-ferent study sites different stakeholder groups and differentadministrative levels are included Within those strata respon-dents can be selected purposive or based on snowball samplingwhere interview respondents make suggestions for who else shouldbe included in the sample (Russell Bernard 2006) The sample sizecan be based on the concept of ldquosaturationrdquo or the point at which

no new information or themes are observed in the interview data(Guest et al 2006) Interviews can be recorded and transcribed elec-tronically From an ethical point of view interviewees should givepermission for interviews to be recorded and researchers shouldensure confidentiality of all interview data Recording may not alwaysbe desirable as the voice recorder can create a barrier between theresearcher and the respondent especially when it comes to dis-cussing politically sensitive issues Instead of recording detailed notescan be taken and transcribed electronically The transcribed inter-views can be coded Ideally interviews are conducted and codedby two researchers which will enhance the quality of theanalysis

323 SurveysBased on the workshops and the interviews some of the con-

straints may be eligible for broader study among specific groupsof stakeholders through the use of surveys Such surveys mayprovide more insights in for example the socio-economic impactsof climate change on smallholder agriculture in specific regionsthe quality of agricultural extension received by farmers in ad-dressing complex agricultural problems or access to agriculturalinputs for male or female headed households Surveys are not nec-essarily limited to farmers but can also be conducted with any ofthe other stakeholder groups involved For the data to be comple-mentary surveys should be completed in the same study sites aswhere the workshops were organised and among a representativesample of the targeted stakeholder group To achieve that a strati-fied random sampling strategy can be used to identify respondentsacross different study sites levels or stakeholder groups A (effi-cient) sampling method that allows for optimal allocation ofresources can be used to determine the sample size (eg Whitleyand Ball 2002)

324 Secondary data collectionSecondary data are written data with relevance for the analy-

sis of the complex agricultural problem innovation capacity of theagricultural system or the functioning of the agricultural innova-tion support system Examples are policy documents projectproposals and reports laws or legal procedures project evalua-tions curricula for agricultural education and training (agricultural)census and organisational records such as charts and budgets overa period of time The sampling of secondary data is not clear cutKey agricultural documents such as agricultural policies or agri-cultural research priorities should be included These documentscan refer to other relevant data Furthermore secondary data is oftenprovided during or following interviews Insights from secondarydata can be verified in interviews with stakeholders (eg the extentto which policy is implemented and enforced)

4 RAAISrsquo ability to provide specific and generic entry pointsfor innovation and lessons learnt from its application

We tested RAAIS through a case study aimed at analysing con-straints and opportunities for innovation to effectively addressparasitic weeds in rain-fed rice production systems in Tanzania(AprilndashOctober 2012) and Benin (JunendashAugust 2013) The results fromRAAIS in Tanzania are elaborated in Schut et al (2014c) Data weregathered across national zonal regional and district levels Multi-stakeholder workshops (with 68 participants in Tanzania and 66participants in Benin) were organised in three study sites (dis-tricts) in Tanzania and Benin where parasitic weeds are eminentIn-depth interviews were held with representatives of national-zonal- regional- and district-level representatives of farmer coop-eratives and associations NGO civil society private sectorgovernment and research and training institutes (42 in Tanzania65 in Benin) Across the three study sites in the countries a

Session 2

Session3

Session1

Categorisingconstraints

Exploring specific and generic entry points for innovation

Identifying constraints

Fig 2 The relation between the 13 Workshop Sessions and their sequence sub-divided over the three categories The dotted arrows indicate relations between thedifferent sessions in terms of triangulation and validation of data

6 M Schut et alAgricultural Systems 132 (2015) 1ndash11

socio-economic farmer survey (152 in Tanzania 182 in Benin) washeld to study the impact of parasitic weeds on rain-fed rice farming(see Nrsquocho et al 2014 for more information) In Tanzania a farmer-extensionist survey (120 farmers 30 agricultural extension officers)was held to explore the effectiveness of the national agricultural ex-tension policy across the three study sites (see Daniel 2013 for more

information) Additionally for both countries secondary data in-cluding crop protection extension and general agricultural policynational research priorities agricultural census and agricultural train-ing curricula were analysed Data gathering and initial analysis tookaround three months for each of the countries and involved tworesearchers We first conducted the in-depth interviews followed

Table 4The 13 Workshop Sessions subdivided over the three categories and their specific activities and objectives in the RAAIS

Categories Sessions Activities Objective(s)

Identifyingconstraints

1 Opening andparticipantintroduction

Participants (1) introduce themselves andreceive information about the workshopmethodology and (2) are subdivided overdifferent stakeholder groups (eg groupsidentified in Table 1)

bull To ensure an equal representation of participants over thedifferent stakeholder groups

2 Individualbrainstorming aboutconstraints

Participants individually identify fiveconstraints they face in their work

bull To make an inventory of general constraints in the agriculturalsystem faced by stakeholders

3 Developing a top-5 ofconstraints instakeholder groups

Participants (1) discuss constraints withinrespective stakeholder group (2) develop anstakeholder group top-5 of constraints (3)present the top-5 to other stakeholder groupsand (4) discuss within and betweenstakeholder group(s)

bull To gain insights in the key constraints in the agricultural system asfaced by different stakeholder groups

bull To create awareness and stimulate learning among stakeholders

Categorisingconstraints

4 Categorisingconstraints alongdifferent types ofinstitutions

Participants (1) categorise top-5 constraints aspolicy- research- education and training-extension- markets- and or politics-related(2) present results to the other groups and (3)discuss within and between the stakeholdergroup(s)

bull To gain insights in how key constraints relate to the different typesof institutions (institutional subsystem)

bull To create awareness and stimulate learning between stakeholders

5 Categorisingconstraints alongstructural conditionsthat can enable orconstrain innovation

Participants (1) categorise top-5 constraintsalong the structural conditions drivers ofinnovation (Table 2) and (2) discuss withinand between the stakeholder group(s)

bull To gain insights in how the stakeholder constraints relate tostructural conditions provided agricultural innovation supportsystem and whether these enable or constrain innovation capacity

bull To create awareness and stimulate learning between stakeholders

6 Categorisingconstraints acrossdifferent(administrative) levelswithin the institutionalsubsystems

Participants (1) categorise top-5 constraintsacross different administrative levels (egnational regional district) (2) discuss resultswith other stakeholder groups and (3) discusswithin and between the stakeholder group(s)

bull To gain insights in how key constraints relate to differentinstitutional (administrative) levels

bull To identify and analyse interactions between different levelsbull To create awareness and stimulate learning between stakeholders

7 Identifyingrelationships betweenconstraints andidentifying keyconstraints

Participants (1) jointly discuss and identifyrelations between the different constraints (2)identify constraints or challenges that arecentral in the analysis and (3) discuss withinand between the stakeholder group(s)

bull To analyse relationships between different constraintsbull To identify key constraintsbull To create awareness and stimulate learning between stakeholdersbull Identify generic entry points for enhancing the innovation

capacity in the agricultural system8 Categorising

constraints along thesectoral subsystem

Participants (1) categorise stakeholder grouptop-5 constraints along the segments of thevalue chain and (2) discuss within andbetween the stakeholder group(s)

bull To analyse constraints along the sectoral subsystembull To create awareness and stimulate learning between stakeholders

9 Categorisingconstraints alongdifferent technologicalsubsystems

Participants (1) categorise top-5 constraintsalong different technological or knowledgefields and (2) discuss within and between thestakeholder group(s)

bull To analyse constraints along different technological subsystemsbull To create awareness and stimulate learning between stakeholders

Exploringentrypoints forinnovation

10 Exploring constraintsstakeholder groups cansolve themselvesversus problems thatcan only be solvedwith or by others

Participants (1) categorise top-5 constraints aslsquocan be solved within the stakeholder grouprsquoor lsquocan only be solved in collaboration withother stakeholder groupsrsquo and (2) discussionwithin and between the stakeholder group(s)

bull To identify constraints that require collaboration betweenstakeholder groups

bull To create awareness and stimulate learning between stakeholdersbull Identify entry points for innovation in the agricultural system

11 Exploring constraintsthat are easy difficultto solve

Participants (1) categorise top-5 constraints asrelatively lsquoeasyrsquo or lsquodifficultrsquo to address and (2)discuss within and between the stakeholdergroup(s)

bull To explore which constraints require system optimisation (easy toaddress) and those that require system transformation (difficult toaddress)

bull To create awareness and stimulate learning between stakeholdersbull To triangulate data with Session 7 (are key constraints perceived

to be easy difficult to address)bull Identify entry points for enhancing the innovation capacity in the

agricultural system12 Exploring constraints

that are structuraloperational

Participants categorise top-5 constraints alonga four-step gradient ranging from lsquoverystructuralrsquo lsquostructuralrsquo lsquooperationalrsquo and lsquoveryoperationalrsquo challenges and constraints

bull To distinguish between structural constraints that require specificinnovation and more structural problems that require genericinnovation

bull To create awareness and stimulate learning between stakeholdersbull To triangulate data with Sessions 7 and 11 (relation between key

constraints how these are perceived by stakeholders)bull Identify generic entry points for enhancing the innovation

capacity in the agricultural system13 Identifying priorities

and solution strategiesParticipants (1) jointly discuss and develop anoverall top-5 of constraints and (2) jointlyidentify potential strategies to address theseconstraints

bull To explore opportunities for addressing systems constraintsthrough multi-stakeholder collaboration

bull To explore similarities and differences with the key systemsconstraints identified in Session 7

bull Identify key entry points for innovation

7M Schut et alAgricultural Systems 132 (2015) 1ndash11

by the multi-stakeholder workshops In Tanzania both the socio-economic farmer survey and the farmer-extensionist survey wereheld after the interviews and workshops In Benin the socio-economic farmer survey was held preceding the in-depth interviewsand workshops Secondary data collection occurred throughout thefieldwork Below we will further reflect on the main objectives ofRAAIS as well as provide recommendations for further improve-ments and use of RAAIS using our experiences from Tanzania andBenin

41 RAAISrsquo ability to provide specific entry points for innovation toaddress complex agricultural problems

RAAIS contributed to an integrated understanding of differentproblem dimensions multi-level interactions and multi-stakeholderdynamics related to parasitic weed problems With regard to thedifferent problem dimensions interviews demonstrated a poten-tial relation between for example the preference for growing localaromatic rice varieties (social-cultural dimension) the low capac-ity of farmer to purchase certified seeds (economic dimension) andthe spread of parasitic weed seeds through the local rice seed system(technological dimension) Additionally analysis of workshop datarevealed how the untimely and insufficient availability of agricul-tural inputs provided by the government (institutional dimension)and limited interaction and collaboration among networks of key

stakeholders (political dimensions) form additional bottlenecks foraddressing such problems It created awareness that describing andexplaining complex agricultural problems and exploring and de-signing solutions is unlikely to be successful if the different problemdimensions are analysed and treated separately (Hall and Clark 2010Spielman et al 2009)

Data gathering across different levels (national region and dis-trict level) enabled the analysis of the interactions and (mis)matchesbetween different levels (Cash et al 2006) An example that emergedduring the workshops and the interviews is Tanzaniarsquos nationalexport ban that prohibits export of agricultural produce (eg of rice)as long as the country has not been declared lsquofood securersquo This na-tional export ban influences local market prices and consequentlyalso farmersrsquo willingness and ability to invest in for example pur-chasing agricultural inputs such as fertilizers and seeds (eg Poultonet al 2010) This in turn provided an opportunity to identify entrypoints for innovation across different levels which has been iden-tified as a critical factor for addressing complex agricultural problems(eg Giller et al 2008 2011) As expected and confirming previ-ous reports (eg van Mierlo et al 2010) the participatory analysisof multi-level interactions showed that stakeholders (insiders) oftenidentify constraints at the level they represent (Schut et al 2014c)This was complemented by our analysis as researchers (outsiders)of the multi-level interactions regarding the parasitic weedproblems

Photo 1ndash4 Photo 1 (top left) Top-5 of constraints of NGO civil society representatives and their categorisation under the different components of the institutional sub-system (Session 4) Photo 2 (top right) The categorisation of the top-5 of the different stakeholder groups along different structural conditions that can enable or constraininnovation (Session 5) Photo 3 (bottom left) The identification of relationships between different constraints (arrows) and key problem (circled cards) (Session 7) Photo4 (bottom right) The categorisation of the top-5 of the different stakeholder groups along a four-step gradient ranging from structural to operational constraints (Session12) Photos were taken by M Schut during multi-stakeholder workshops in Tanzania held in October 2012

8 M Schut et alAgricultural Systems 132 (2015) 1ndash11

The involvement of different groups of stakeholders was essen-tial for enhancing the credibility validity and quality of RAAIS aswell as for delineating the boundaries of the agricultural system andthe agricultural innovation support system which is considered akey challenge when using AIS approaches to analyse complex ag-ricultural problems (Klerkx et al 2012b) Furthermore stakeholderparticipation provided a better understanding of the feasibility andacceptability of solutions for stakeholder groups Although we believethat the stakeholder groups included in the testing of RAAIS (Table 1)provide a good starting point other stakeholder groups (for in-stance the media) may be included in the sample (eg Ortiz et al2013) depending on the type of complex agricultural problem underreview The triangulation of data resulting from the different methodsenabled us to validate findings and to verify strategic communi-cation by stakeholders for instance to verify how the extensionsystem as described by policymakers in interviews functioned inreality according to surveyed farmers

42 RAAISrsquo ability to provide generic entry points for innovation

RAAIS demonstrates interactions between complex agricultur-al problems innovation capacity of the agricultural system ndashconsisting of institutional sectoral and technological subsystemsndash and the agricultural innovation support system For example ap-plying fertilizer (technological subsystem) in rain-fed rice productionis seen as a promising management strategy to reduce infection levelsof Rhamphicarpa one of the parasitic weeds involved in the studyand mitigate negative effects of the parasite on rice yields (Rodenburget al 2011) However as was highlighted during the RAAIS work-shops in both in Benin and in Tanzania fertilizers are difficult toaccess in rural areas In Benin there is no well-developed privateagro-dealer network and distribution infrastructure to support thesupply of agricultural inputs Furthermore interviews showed thatthe public extension and input supply systems in Benin focus onthe cotton sector rather than on cereal crops (sectoral subsys-tems) In Tanzania a private agro-dealer network and distributioninfrastructure exists but structures controlling the quality of fer-tilizers (institutional subsystem) are functioning sub-optimallyaccording to interviewed government officials In some areas fakeagro inputs are dominating the market resulting in a limited trustand willingness to invest in applying fertilizer according to farmerrepresentatives who participated in the workshops The exampleshows how the absence or poor performance of fertilizer distribu-tion infrastructure limited farmer-extensionist interaction and lackof functional institutions for quality control (being structural con-ditions for innovation) constrain the innovation capacity in theagricultural systems and its technological (in this case fertilizer) andsectoral (the rice value chain) subsystems Another example is basedon secondary data analyses that demonstrated the lack of an op-erational strategy to address parasitic weeds in Tanzania and BeninIn both the interviews and workshops stakeholders highlighted thegeneral lack of interaction and collaboration between stakehold-ers in the agricultural sector (being a structural condition forinnovation) as one of the main reasons for the absence or poor im-plementation of parasitic weed and other agricultural policies andstrategies

The aforementioned examples demonstrate how RAAIS cansupport the identification of generic entry points for innovation Suchinnovations can directly contribute to addressing the complex ag-ricultural problem under review but can also have a spill-over effectin terms of addressing broader constraints that hamper the inno-vation capacity in the agricultural system For example the lack ofstakeholder interactions and collaboration in the agricultural systemcan provide an entry point for the adaptation of the structuralconditions in the broader agricultural innovation support systemfor example through investments in innovation brokers or

multi-stakeholder platforms (Kilelu et al 2013 Klerkx et al 2010)Such structural adjustments can facilitate multi-stakeholder col-laboration in tackling parasitic weed as well as other complexagricultural problems

43 Lessons learnt from applying RAAIS and recommendations forfurther improvement

Based on our experiences in Tanzania and Benin we recom-mend conducting RAAIS in an interdisciplinary team of researcherswith expertise on different dimensions of complex agricultural prob-lems and on different data collection methods (Hulsebosch 2001)Other suggestions include the experimentation with other combi-nations of methods and on different types of complex agriculturalproblems The workshop methodology could be made more inter-active in the sense of directly feeding back results of the sessionsto participants to stimulate reflection and validate analyses duringthe workshops Post-workshop surveys could provide additionalinsight into whether stakeholders felt they could freely raise anddiscuss their ideas and needs

The multi-stakeholder workshops but also the surveys pre-sented a rather static picture of the complex agricultural problemunder review and the innovation capacity of the agricultural systemin which the problem is embedded However initial workshops andsurveys could function as a baseline to which future workshops andsurveys can be compared Other methods such as secondary dataanalysis or in-depth interviews present a more dynamic image ofhow for example collaborations between stakeholders evolve overthe years Our experiences in Tanzania and Benin show that ensur-ing social differentiation among workshop participants intervieweesand survey respondents (eg of different gender of age) was chal-lenging as for example the majority of workshop participants weremale Specific Workshop Sessions could have more attention for cat-egorisation and priority setting by different gender or age groupsThe facilitation of the multi-stakeholder workshops ensured thatdifferent stakeholder groups could raise and discuss their ideas(Hulsebosch 2001) Despite such efforts unequal power relationsand differences in the ability to debate and negotiate that inher-ently exist between groups may have played a role In line with ourexpectations politically sensitive issues were more freely dis-cussed in individual interviews as compared to multi-stakeholdersetting

The combination of different methods of data collection wasessential In terms of the sequence of data collection we recom-mend to first conduct and analyse the RAAIS multi-stakeholderworkshops to identify constraints and subsequently conduct thein-depth interviews and surveys that can provide more insight inthe distribution and underlying root causes of these constraintsThe workshops then provide a lsquofast-trackrsquo approach to identifyingentry points for innovation that can subsequently be validated andexplored in more detail using the in-depth interviews and stake-holder surveys This would furthermore increase the lsquorapidnessrsquoof RAAIS as a diagnostic tool

An updated version of the RAAIS multi-stakeholder workshopshas been used to identify constraints challenges and entry pointsfor innovations related to the lsquosustainable intensification of agri-cultural systemsrsquo in Burundi the Democratic Republic of CongoRwanda Nigeria and Cameroon under the CGIAR Research Pro-gramme for the Humid Tropics (Humidtropics) (Schut and Hinnou2014) Several of the recommendations made in this paper includ-ing the revised sequence of methods for data collection and the useof post-workshop participant questionnaires have been imple-mented and tested successfully Some of the bottlenecks identifiedsuch as social differentiation (eg gender and age groups) amongworkshop participants remained problematic and require furtherattention At the end of the Humidtropics RAAIS workshops

9M Schut et alAgricultural Systems 132 (2015) 1ndash11

participants developed action plans to address the prioritised con-straints (Workshop Session 13) This required an extension of theworkshops of half a day The development and implementation ofthe action plans forms an important element for continued stake-holder collaboration in multi-stakeholder platforms

5 Conclusions

This paper demonstrates the potential of RAAIS as a diagnostictool that can support and guide the integrated analysis of complexagricultural problems innovation capacity in the agricultural systemand the performance of the agricultural innovation support systemRAAIS combines multiple qualitative and quantitative methods andinsider (stakeholders) and outsider (researchers) analyses whichallow for critical triangulation and validation of the gathered dataSuch an analysis can provide specific entry points for innovationsto address the complex agricultural problem under study and genericentry points for innovation related to strengthening the innova-tion capacity of agricultural system and the functioning of theagricultural innovation support system

Recommendations for further improvement include using RAAISfor the analysis of other types of complex agricultural problems usingother combinations of methods of data collection and providing di-rectly feedback to workshop participants to stimulate reflection andvalidate workshop outcomes An adapted sequence of data collec-tion methods in which workshops provide a lsquofast-trackrsquo approachto identifying entry points for innovation followed up by more in-depth interviews and stakeholder surveys would increase the RAAISrsquodiagnostic capacity The participatory development of concrete actionplans based on RAAIS can provide a basis for continued multi-stakeholder collaboration to operationalise and implement specificand generic entry points for innovation

Acknowledgements

This research forms part of the PARASITE programme fundedby the Integrated Programmes Scheme of the NetherlandsOrganisation for Scientific Research ndash Science for Global Develop-ment (NWO-WOTRO) (wwwparasite-projectorg) Additional supportis provided by the CGIAR Research Program on Climate Change Ag-riculture and Food Security (CCAFS) The CGIAR Research Programmefor the Humid Tropics (httphumidtropicscgiarorg) provided theopportunity to further adapt apply and improve RAAIS across dif-ferent Humidtropics Action Areas Special thanks to Elifadhili Danielwho conducted the farmer-extensionist survey

References

Amankwah K Klerkx L Oosting SJ Sakyi-Dawson O van der Zijpp AJ MillarD 2012 Diagnosing constraints to market participation of small ruminantproducers in northern Ghana an innovation systems analysis NJAS-Wagen JLife Sc 60ndash63 37ndash47

Basu S Leeuwis C 2012 Understanding the rapid spread of System of RiceIntensification (SRI) in Andhra Pradesh exploring the building of supportnetworks and media representation Agr Syst 111 34ndash44

Beintema N Stads G-J Fuglie K Heisey P 2012 ASTI global assessment ofagricultural RampD spending International Food Policy Research InstituteWashington DC Agricultural Science and Technology Indicators Rome Italy andthe Global Forum on Agricultural Research Rome Italy

Blay-Palmer A 2005 Growing innovation policy the case of organic agriculturein Ontario Canada Environ Plann C Gov Policy 23 557ndash581

Carlsson B Stankiewicz R 1991 On the nature function and composition oftechnological systems J Evol Econ 1 93ndash118

Carlsson B Jacobsson S Holmeacuten M Rickne A 2002 Innovation systems analyticaland methodological issues Res Policy 31 233ndash245

Cash DW Adger WN Berkes F Garden P Lebel L Olsson P et al 2006 Scaleand cross-scale dynamics governance and information in a multilevel worldEcol Soc 11 8 Available from lthttpwwwecologyandsocietyorgvol11iss12art18gt [online]

Chambers R 1994 The origins and practice of participatory rural appraisal WorldDev 22 953ndash969

Chung CC 2012 National sectoral and technological innovation systems the caseof Taiwanese pharmaceutical biotechnology and agricultural biotechnologyinnovation systems (1945ndash2000) Sci Public Policy 39 271ndash281

Cooke P Uranga MG Etxebarria G 1997 Regional innovation systems Institutionaland organisational dimensions Res Policy 26 475ndash491

Daniel E 2013 Assessment of agricultural extension services in Tanzania A casestudy of Kyela Songea Rural and Morogoro Rural Districts Wageningen Universityand Research CentreAfrica Rice Center Wageningen the NetherlandsDar esSalaam Tanzania p 45

Douthwaite B Kuby T van de Fliert E Schulz S 2003 Impact pathway evaluationan approach for achieving and attributing impact in complex systems Agr Syst78 243ndash265

Engel PGH 1995 Facilitating Innovation An Action-Oriented Approach andParticipatory Methodology to Improve Innovative Social Practice in AgricultureWageningen University Wageningen the Netherlands p 300

Faysse N 2006 Troubles on the way an analysis of the challenges faced bymulti-stakeholder platforms Nat Resour Forum 30 219ndash229

Freeman C 1988 Japan a new national innovation system In Dosi G FreemanC Nelson R Soete T (Eds) Technology and Economic Theory London pp330ndash348

Freeman C 1995 The national system of innovation in historical perspective CambJ Econ 19 5ndash24

Funtowicz SO Ravetz JR 1993 Science for the post-normal age Futures 25739ndash755

Gildemacher PR Kaguongo W Ortiz O Tesfaye A Woldegiorgis G Wagoire WWet al 2009 Improving potato production in Kenya Uganda and Ethiopia a systemdiagnosis Potato Res 52 173ndash205

Giller KE Leeuwis C Andersson JA Andriesse W Brouwer A Frost P et al2008 Competing claims on natural resources what role for science Ecol Soc13 34 Available from lthttpwwwecologyandsocietyorgvol13iss32art34gt[online]

Giller KE Tittonell P Rufino MC van Wijk MT Zingore S Mapfumo P et al2011 Communicating complexity integrated assessment of trade-offs concerningsoil fertility management within African farming systems to support innovationand development Agr Syst 104 191ndash203

Guest G Bunce A Johnson L 2006 How many interviews are enough Anexperiment with data saturation and variability Field Methods 18 59ndash82

Hall A 2005 Capacity development for agricultural biotechnology in developingcountries an innovation systems view of what it is and how to develop it J IntDev 17 611ndash630

Hall A Clark N 2010 What do complex adaptive systems look like and what arethe implications for innovation policy J Int Dev 22 308ndash324

Hall A Rasheed Sulaiman V Clark N Yoganand B 2003 From measuring impactto learning institutional lessons an innovation systems perspective on improvingthe management of international agricultural research Agr Syst 78 213ndash241

Hall AJ Yoganand B Sulaiman RV Raina RS Prasad CS Naik GC et al 2004Innovations in innovation reflections on partnership institutions and learningICRISAT Andra Pradesh

Hekkert MP Suurs RAA Negro SO Kuhlmann S Smits REHM 2007 Functionsof innovation systems a new approach for analysing technological changeTechnol Forecast Soc Change 74 413ndash432

Hermans F Stuiver M Beers PJ Kok K 2013 The distribution of roles andfunctions for upscaling and outscaling innovations in agricultural innovationsystems Agr Syst 115 117ndash128

Hounkonnou D Kossou D Kuyper TW Leeuwis C Nederlof ES Roumlling N et al2012 An innovation systems approach to institutional change smallholderdevelopment in West Africa Agr Syst 108 74ndash83

Hulsebosch J 2001 The use of RAAKS for strengthening community-basedorganisations in Mali Dev Pract 11 622ndash632

Hung S-C Whittington R 2011 Agency in national innovation systems institutionalentrepreneurship and the professionalization of Taiwanese IT Res Policy 40526ndash538

International Institute for Sustainable Development 2014 Rapid Rural Appraisal(RRA)

Jiggins J 2012 Diagnosing the scope for innovation linking smallholder practicesand institutional context NJAS-Wagen J Life Sc 60ndash63 1ndash122

Kilelu CW Klerkx L Leeuwis C 2013 Unravelling the role of innovation platformsin supporting co-evolution of innovation contributions and tensions in asmallholder dairy development programme Agr Syst 118 65ndash77

Klein Woolthuis R Lankhuizen M Gilsing V 2005 A system failure frameworkfor innovation policy design Technovation 25 609ndash619

Klerkx L Aarts N Leeuwis C 2010 Adaptive management in agricultural innovationsystems the interactions between innovation networks and their environmentAgr Syst 103 390ndash400

Klerkx L Schut M Leeuwis C Kilelu C 2012a Advances in knowledge brokeringin the agricultural sector towards innovation system facilitation IDS Bull 4353ndash60

Klerkx L van Mierlo B Leeuwis C 2012b Evolution of systems approaches toagricultural innovation Concepts analysis and interventions In Darnhofer IGibbon D Dedieu B (Eds) Farming Systems Research into the 21st CenturyThe New Dynamic Springer Dordrecht pp 457ndash483

Leeuwis C 2000 Reconceptualizing participation for sustainable rural developmenttowards a negotiation approach Dev Change 31 931ndash959

Leeuwis C 2004 Communication for Rural Innovation Rethinking AgriculturalExtension (with Contributions of Anne van den Ban) Blackwell Science Oxford

10 M Schut et alAgricultural Systems 132 (2015) 1ndash11

policy or funding schemes that affect multiple institutional sectoraland technological subsystems

3 Methodological framework for RAAIS

31 Selection criteria for methods

RAAIS is a diagnostic tool that combines multiple methods of datacollection Building on existing experiences with rapid appraisal ap-proaches and (participatory) innovation systems analysis five criteriafor the selection of methods have been identified

1 Methods should be diverse rigorous and be able to generateboth qualitative and quantitative data This enhances thecredibility and strength of the analysis (Spielman 2005)Qualitative data provide the basis for the identificationand analysis of the different dimensions of complex agricultur-al problems and structural conditions enabling or constrainingthe innovation capacity Such data may also provide narrativesregarding the underlying causes and historical evolutionof constraints Quantitative data analysis can build on this by pro-viding (descriptive) statistics and trends on for instance thedistribution of constraints across different levels stakeholdergroups or study sites

2 Methods should facilitate both lsquoinsiderrsquo and lsquooutsiderrsquo analysisInsider analysis implies data analysis by stakeholders who can

provide highly detailed explanations of specific phenomena basedon their knowledge and experiences However insiders such asfarmers or policymakers often have an incomplete or insuffi-cient critical view of the broader agricultural system or theagricultural innovation support system Consequently it is im-portant to complement insider analysis by outsider analysis ofdata by researchers (van Mierlo et al 2010) By combining insiderand outsider analysis the delineation of the systems boundar-ies is done in a participatory way by stakeholders and researchers

3 Methods should be able to target different stakeholder groupsacross different levels When studying complex agricultural prob-lems it is essential to include different groups of stakeholderstheir perceptions on what constitutes the problem and what areperceived feasible or desirable solutions (Faysse 2006 Ortiz et al2013)

4 Methods should be able to target stakeholders individually inhomogeneous groups and in heterogeneous groups so as tocapture individual group and multi-stakeholder perceptions onproblems and solutions Discussion and debate in both homo-geneous and heterogeneous stakeholder groups generally providea rich analysis of complex problems and potential solutions Fur-thermore multi-stakeholder interaction may reveal asymmetricpower-relationships that are necessary to understand innova-tion capacity in the agricultural system On the other handpower-relationships group pressure or mutual dependenciesbetween stakeholders may result in situations where sensitive

Fig 1 Schematic representation of the dynamic interactions between complex agricultural problems (multiple dimensions multi-level interactions and multi-stakeholderdynamics) innovation capacity of the agricultural system (including its institutional sectoral and technological subsystems) and the structural conditions within the ag-ricultural innovation support system that can enable or constrain innovation capacity in the agricultural system (infrastructure and assets institutions interaction and collaborationand capabilities and resources) RAAIS provides insight into the current state of the system (on the left) RAAIS provides specific and generic entry points for innovationthat can guide a transition towards the desirable state of the system (on the right) in which the complex agricultural problem is addressed and the innovation capacity inthe agricultural system has increased Generic entry points for innovation can have a spill-over effect in terms of addressing other complex agricultural problems than theone under review

4 M Schut et alAgricultural Systems 132 (2015) 1ndash11

questions are avoided or receive socially desirable responsesMethods that target stakeholders individually are more likely toprovide insights in such questions (International Institute forSustainable Development 2014)

5 Methods together should provide sufficient detail on the complexagricultural problem under review the innovation capacity inthe agricultural system and the functioning of the agriculturalinnovation support system (World Bank 2012)

Combining different types of methods and data collection tech-niques provides an opportunity to triangulate and validate dataDepending on the nature of the agricultural problem under reviewand the available resources and time different types of data col-lection methods can be used for RAAIS taking into account the abovecriteria for method selection

32 Methods of data collection

Based on the five criteria four complementary methods for datacollection were selected to be part of RAAIS (Table 3)

321 Multi-stakeholder workshopsMulti-stakeholder workshops mainly focused on the insider anal-

yses of innovation capacity in the agricultural system and thestructural conditions provided by the agricultural innovation supportsystem A participatory workshop methodology facilitates differ-ent groups of stakeholders to ndash individually and in homogeneousand heterogeneous groups ndash identify categorise and analyse con-straints for innovation in the agricultural system Depending on thetype of problem workshops can be organised with stakeholders rep-resenting national regional andor district levels or for instanceacross different study sites where a specific problem is eminent Tokeep the workshops manageable and to stimulate interaction anddebate the participation of a maximum of 25 participants per work-shop is proposed for instance consisting of five representatives ofthe five different stakeholder groups in Table 1 As much as possi-ble each group should be a representative sample with respect tofor instance gender age income or ethnic groups The work-shops should be held in the language that all participants speakand be facilitated by someone who is familiar with the culturalnorms has affinity with the problem and understands the reali-ties of the different stakeholder groups The proposed workshopmethodology consists of 13 Sessions subdivided into three catego-ries with each their own focus (1) identifying constraints (2)categorising constraints and (3) exploring specific and generic entrypoints for innovation Figure 2 and Table 4 provide an overview ofthe 13 Sessions their sequence and relations and their specific ob-jective in RAAIS

Workshops are designed to take approximately 1 day Besidesthe facilitator a note-taker documents the outcome of the differ-ent sessions and captures discussions among participants Workshopfacilitation and note-taking protocols ensure that the workshoporganisation facilitation and documentation is standardized whichis essential for comparing or aggregating the outcomes for in-stance across different study sites

A crucial element in the workshops is the use of coloured cardsAt the start of the workshop (Session 1) each of the stakeholdergroups is assigned a different colour During Session 2 each par-ticipant individually lists five constraints or challenges they face intheir work and writes them down on their coloured cards If fivestakeholder groups are equally represented this results in 125 cardsDuring Session 3 the participants discuss within their stakehold-er groups the listed constraints explore overlapping issues and jointlydevelop a stakeholder group top-5 If necessary constraints canbe reformulated based on discussions within the group Each of thestakeholder groups use their top-5 throughout the rest of the Ta

ble

3M

eth

odol

ogic

alfr

amew

ork

indi

cati

ng

how

diffe

ren

tm

eth

ods

for

data

coll

ecti

onco

rres

pon

dw

ith

the

sele

ctio

ncr

iter

iafo

rm

eth

ods

Sele

ctio

ncr

iter

iafo

rm

eth

ods

Met

hod

sfo

rda

taco

llect

ion

Typ

eof

data

Insi

der

outs

ider

Stak

ehol

der

grou

pSt

akeh

olde

rin

volv

emen

tA

ISan

alys

is

Qualitative

Quantitative

Insider

Outsider

Farmers

NGOcivilsociety

Privatesector

Government

Researchandtraining

Individual

Homogeneousgroups

Heterogeneousgroups

Complexagriculturalproblems

Innovationcapacityintheagriculturalsystem

Agriculturalinnovationsupportsystem

Mu

lti-

stak

ehol

der

wor

ksh

ops

Sem

i-st

ruct

ure

din

-dep

thin

terv

iew

s

Qu

esti

onn

aire

s

Seco

nda

ryda

taan

alys

is

N

aN

a

5M Schut et alAgricultural Systems 132 (2015) 1ndash11

sessions during the workshop hence 25 cards (five cards per stake-holder group) (Photo 1ndash4)

The use of the coloured cards facilitates the analysis of differ-ent sessions during and after the workshops As the cards are codedand recycled throughout the successive sessions photographs canbe taken to capture the results (for example Photo 1 and 4) Suchphotographs can be analysed after the workshop and can also beused to validate the note-takerrsquos data Furthermore the cards provideinsight into the relations between constraints identified by differ-ent stakeholder groups (Photo 2 and 3) Combining the results fromdifferent sessions can stimulate integrative analyses for instancecombining data resulting from Sessions 5 and 6 provides insight inthe structural conditions for innovation across different levels Sim-ilarly the outcome of Sessions 7 and 11 can be compared totriangulate the data as both seek to identify key constraints for in-novation in the agricultural system

322 Semi-structured in-depth interviewsTo guide the semi-structured interviews a topic list is pre-

pared and fine-tuned for each interview Using a topic list providesa degree of flexibility to identify and to anticipate interestingstorylines related to the problem under review and allows valida-tion of data that was gathered during previous interviews or duringthe workshops Interviews should take a maximum of 1 hour en-suring a high level of attentiveness of both the respondent as wellas the interviewer Sampling of interview respondents should followa stratified approach to ensure that stakeholders representing dif-ferent study sites different stakeholder groups and differentadministrative levels are included Within those strata respon-dents can be selected purposive or based on snowball samplingwhere interview respondents make suggestions for who else shouldbe included in the sample (Russell Bernard 2006) The sample sizecan be based on the concept of ldquosaturationrdquo or the point at which

no new information or themes are observed in the interview data(Guest et al 2006) Interviews can be recorded and transcribed elec-tronically From an ethical point of view interviewees should givepermission for interviews to be recorded and researchers shouldensure confidentiality of all interview data Recording may not alwaysbe desirable as the voice recorder can create a barrier between theresearcher and the respondent especially when it comes to dis-cussing politically sensitive issues Instead of recording detailed notescan be taken and transcribed electronically The transcribed inter-views can be coded Ideally interviews are conducted and codedby two researchers which will enhance the quality of theanalysis

323 SurveysBased on the workshops and the interviews some of the con-

straints may be eligible for broader study among specific groupsof stakeholders through the use of surveys Such surveys mayprovide more insights in for example the socio-economic impactsof climate change on smallholder agriculture in specific regionsthe quality of agricultural extension received by farmers in ad-dressing complex agricultural problems or access to agriculturalinputs for male or female headed households Surveys are not nec-essarily limited to farmers but can also be conducted with any ofthe other stakeholder groups involved For the data to be comple-mentary surveys should be completed in the same study sites aswhere the workshops were organised and among a representativesample of the targeted stakeholder group To achieve that a strati-fied random sampling strategy can be used to identify respondentsacross different study sites levels or stakeholder groups A (effi-cient) sampling method that allows for optimal allocation ofresources can be used to determine the sample size (eg Whitleyand Ball 2002)

324 Secondary data collectionSecondary data are written data with relevance for the analy-

sis of the complex agricultural problem innovation capacity of theagricultural system or the functioning of the agricultural innova-tion support system Examples are policy documents projectproposals and reports laws or legal procedures project evalua-tions curricula for agricultural education and training (agricultural)census and organisational records such as charts and budgets overa period of time The sampling of secondary data is not clear cutKey agricultural documents such as agricultural policies or agri-cultural research priorities should be included These documentscan refer to other relevant data Furthermore secondary data is oftenprovided during or following interviews Insights from secondarydata can be verified in interviews with stakeholders (eg the extentto which policy is implemented and enforced)

4 RAAISrsquo ability to provide specific and generic entry pointsfor innovation and lessons learnt from its application

We tested RAAIS through a case study aimed at analysing con-straints and opportunities for innovation to effectively addressparasitic weeds in rain-fed rice production systems in Tanzania(AprilndashOctober 2012) and Benin (JunendashAugust 2013) The results fromRAAIS in Tanzania are elaborated in Schut et al (2014c) Data weregathered across national zonal regional and district levels Multi-stakeholder workshops (with 68 participants in Tanzania and 66participants in Benin) were organised in three study sites (dis-tricts) in Tanzania and Benin where parasitic weeds are eminentIn-depth interviews were held with representatives of national-zonal- regional- and district-level representatives of farmer coop-eratives and associations NGO civil society private sectorgovernment and research and training institutes (42 in Tanzania65 in Benin) Across the three study sites in the countries a

Session 2

Session3

Session1

Categorisingconstraints

Exploring specific and generic entry points for innovation

Identifying constraints

Fig 2 The relation between the 13 Workshop Sessions and their sequence sub-divided over the three categories The dotted arrows indicate relations between thedifferent sessions in terms of triangulation and validation of data

6 M Schut et alAgricultural Systems 132 (2015) 1ndash11

socio-economic farmer survey (152 in Tanzania 182 in Benin) washeld to study the impact of parasitic weeds on rain-fed rice farming(see Nrsquocho et al 2014 for more information) In Tanzania a farmer-extensionist survey (120 farmers 30 agricultural extension officers)was held to explore the effectiveness of the national agricultural ex-tension policy across the three study sites (see Daniel 2013 for more

information) Additionally for both countries secondary data in-cluding crop protection extension and general agricultural policynational research priorities agricultural census and agricultural train-ing curricula were analysed Data gathering and initial analysis tookaround three months for each of the countries and involved tworesearchers We first conducted the in-depth interviews followed

Table 4The 13 Workshop Sessions subdivided over the three categories and their specific activities and objectives in the RAAIS

Categories Sessions Activities Objective(s)

Identifyingconstraints

1 Opening andparticipantintroduction

Participants (1) introduce themselves andreceive information about the workshopmethodology and (2) are subdivided overdifferent stakeholder groups (eg groupsidentified in Table 1)

bull To ensure an equal representation of participants over thedifferent stakeholder groups

2 Individualbrainstorming aboutconstraints

Participants individually identify fiveconstraints they face in their work

bull To make an inventory of general constraints in the agriculturalsystem faced by stakeholders

3 Developing a top-5 ofconstraints instakeholder groups

Participants (1) discuss constraints withinrespective stakeholder group (2) develop anstakeholder group top-5 of constraints (3)present the top-5 to other stakeholder groupsand (4) discuss within and betweenstakeholder group(s)

bull To gain insights in the key constraints in the agricultural system asfaced by different stakeholder groups

bull To create awareness and stimulate learning among stakeholders

Categorisingconstraints

4 Categorisingconstraints alongdifferent types ofinstitutions

Participants (1) categorise top-5 constraints aspolicy- research- education and training-extension- markets- and or politics-related(2) present results to the other groups and (3)discuss within and between the stakeholdergroup(s)

bull To gain insights in how key constraints relate to the different typesof institutions (institutional subsystem)

bull To create awareness and stimulate learning between stakeholders

5 Categorisingconstraints alongstructural conditionsthat can enable orconstrain innovation

Participants (1) categorise top-5 constraintsalong the structural conditions drivers ofinnovation (Table 2) and (2) discuss withinand between the stakeholder group(s)

bull To gain insights in how the stakeholder constraints relate tostructural conditions provided agricultural innovation supportsystem and whether these enable or constrain innovation capacity

bull To create awareness and stimulate learning between stakeholders

6 Categorisingconstraints acrossdifferent(administrative) levelswithin the institutionalsubsystems

Participants (1) categorise top-5 constraintsacross different administrative levels (egnational regional district) (2) discuss resultswith other stakeholder groups and (3) discusswithin and between the stakeholder group(s)

bull To gain insights in how key constraints relate to differentinstitutional (administrative) levels

bull To identify and analyse interactions between different levelsbull To create awareness and stimulate learning between stakeholders

7 Identifyingrelationships betweenconstraints andidentifying keyconstraints

Participants (1) jointly discuss and identifyrelations between the different constraints (2)identify constraints or challenges that arecentral in the analysis and (3) discuss withinand between the stakeholder group(s)

bull To analyse relationships between different constraintsbull To identify key constraintsbull To create awareness and stimulate learning between stakeholdersbull Identify generic entry points for enhancing the innovation

capacity in the agricultural system8 Categorising

constraints along thesectoral subsystem

Participants (1) categorise stakeholder grouptop-5 constraints along the segments of thevalue chain and (2) discuss within andbetween the stakeholder group(s)

bull To analyse constraints along the sectoral subsystembull To create awareness and stimulate learning between stakeholders

9 Categorisingconstraints alongdifferent technologicalsubsystems

Participants (1) categorise top-5 constraintsalong different technological or knowledgefields and (2) discuss within and between thestakeholder group(s)

bull To analyse constraints along different technological subsystemsbull To create awareness and stimulate learning between stakeholders

Exploringentrypoints forinnovation

10 Exploring constraintsstakeholder groups cansolve themselvesversus problems thatcan only be solvedwith or by others

Participants (1) categorise top-5 constraints aslsquocan be solved within the stakeholder grouprsquoor lsquocan only be solved in collaboration withother stakeholder groupsrsquo and (2) discussionwithin and between the stakeholder group(s)

bull To identify constraints that require collaboration betweenstakeholder groups

bull To create awareness and stimulate learning between stakeholdersbull Identify entry points for innovation in the agricultural system

11 Exploring constraintsthat are easy difficultto solve

Participants (1) categorise top-5 constraints asrelatively lsquoeasyrsquo or lsquodifficultrsquo to address and (2)discuss within and between the stakeholdergroup(s)

bull To explore which constraints require system optimisation (easy toaddress) and those that require system transformation (difficult toaddress)

bull To create awareness and stimulate learning between stakeholdersbull To triangulate data with Session 7 (are key constraints perceived

to be easy difficult to address)bull Identify entry points for enhancing the innovation capacity in the

agricultural system12 Exploring constraints

that are structuraloperational

Participants categorise top-5 constraints alonga four-step gradient ranging from lsquoverystructuralrsquo lsquostructuralrsquo lsquooperationalrsquo and lsquoveryoperationalrsquo challenges and constraints

bull To distinguish between structural constraints that require specificinnovation and more structural problems that require genericinnovation

bull To create awareness and stimulate learning between stakeholdersbull To triangulate data with Sessions 7 and 11 (relation between key

constraints how these are perceived by stakeholders)bull Identify generic entry points for enhancing the innovation

capacity in the agricultural system13 Identifying priorities

and solution strategiesParticipants (1) jointly discuss and develop anoverall top-5 of constraints and (2) jointlyidentify potential strategies to address theseconstraints

bull To explore opportunities for addressing systems constraintsthrough multi-stakeholder collaboration

bull To explore similarities and differences with the key systemsconstraints identified in Session 7

bull Identify key entry points for innovation

7M Schut et alAgricultural Systems 132 (2015) 1ndash11

by the multi-stakeholder workshops In Tanzania both the socio-economic farmer survey and the farmer-extensionist survey wereheld after the interviews and workshops In Benin the socio-economic farmer survey was held preceding the in-depth interviewsand workshops Secondary data collection occurred throughout thefieldwork Below we will further reflect on the main objectives ofRAAIS as well as provide recommendations for further improve-ments and use of RAAIS using our experiences from Tanzania andBenin

41 RAAISrsquo ability to provide specific entry points for innovation toaddress complex agricultural problems

RAAIS contributed to an integrated understanding of differentproblem dimensions multi-level interactions and multi-stakeholderdynamics related to parasitic weed problems With regard to thedifferent problem dimensions interviews demonstrated a poten-tial relation between for example the preference for growing localaromatic rice varieties (social-cultural dimension) the low capac-ity of farmer to purchase certified seeds (economic dimension) andthe spread of parasitic weed seeds through the local rice seed system(technological dimension) Additionally analysis of workshop datarevealed how the untimely and insufficient availability of agricul-tural inputs provided by the government (institutional dimension)and limited interaction and collaboration among networks of key

stakeholders (political dimensions) form additional bottlenecks foraddressing such problems It created awareness that describing andexplaining complex agricultural problems and exploring and de-signing solutions is unlikely to be successful if the different problemdimensions are analysed and treated separately (Hall and Clark 2010Spielman et al 2009)

Data gathering across different levels (national region and dis-trict level) enabled the analysis of the interactions and (mis)matchesbetween different levels (Cash et al 2006) An example that emergedduring the workshops and the interviews is Tanzaniarsquos nationalexport ban that prohibits export of agricultural produce (eg of rice)as long as the country has not been declared lsquofood securersquo This na-tional export ban influences local market prices and consequentlyalso farmersrsquo willingness and ability to invest in for example pur-chasing agricultural inputs such as fertilizers and seeds (eg Poultonet al 2010) This in turn provided an opportunity to identify entrypoints for innovation across different levels which has been iden-tified as a critical factor for addressing complex agricultural problems(eg Giller et al 2008 2011) As expected and confirming previ-ous reports (eg van Mierlo et al 2010) the participatory analysisof multi-level interactions showed that stakeholders (insiders) oftenidentify constraints at the level they represent (Schut et al 2014c)This was complemented by our analysis as researchers (outsiders)of the multi-level interactions regarding the parasitic weedproblems

Photo 1ndash4 Photo 1 (top left) Top-5 of constraints of NGO civil society representatives and their categorisation under the different components of the institutional sub-system (Session 4) Photo 2 (top right) The categorisation of the top-5 of the different stakeholder groups along different structural conditions that can enable or constraininnovation (Session 5) Photo 3 (bottom left) The identification of relationships between different constraints (arrows) and key problem (circled cards) (Session 7) Photo4 (bottom right) The categorisation of the top-5 of the different stakeholder groups along a four-step gradient ranging from structural to operational constraints (Session12) Photos were taken by M Schut during multi-stakeholder workshops in Tanzania held in October 2012

8 M Schut et alAgricultural Systems 132 (2015) 1ndash11

The involvement of different groups of stakeholders was essen-tial for enhancing the credibility validity and quality of RAAIS aswell as for delineating the boundaries of the agricultural system andthe agricultural innovation support system which is considered akey challenge when using AIS approaches to analyse complex ag-ricultural problems (Klerkx et al 2012b) Furthermore stakeholderparticipation provided a better understanding of the feasibility andacceptability of solutions for stakeholder groups Although we believethat the stakeholder groups included in the testing of RAAIS (Table 1)provide a good starting point other stakeholder groups (for in-stance the media) may be included in the sample (eg Ortiz et al2013) depending on the type of complex agricultural problem underreview The triangulation of data resulting from the different methodsenabled us to validate findings and to verify strategic communi-cation by stakeholders for instance to verify how the extensionsystem as described by policymakers in interviews functioned inreality according to surveyed farmers

42 RAAISrsquo ability to provide generic entry points for innovation

RAAIS demonstrates interactions between complex agricultur-al problems innovation capacity of the agricultural system ndashconsisting of institutional sectoral and technological subsystemsndash and the agricultural innovation support system For example ap-plying fertilizer (technological subsystem) in rain-fed rice productionis seen as a promising management strategy to reduce infection levelsof Rhamphicarpa one of the parasitic weeds involved in the studyand mitigate negative effects of the parasite on rice yields (Rodenburget al 2011) However as was highlighted during the RAAIS work-shops in both in Benin and in Tanzania fertilizers are difficult toaccess in rural areas In Benin there is no well-developed privateagro-dealer network and distribution infrastructure to support thesupply of agricultural inputs Furthermore interviews showed thatthe public extension and input supply systems in Benin focus onthe cotton sector rather than on cereal crops (sectoral subsys-tems) In Tanzania a private agro-dealer network and distributioninfrastructure exists but structures controlling the quality of fer-tilizers (institutional subsystem) are functioning sub-optimallyaccording to interviewed government officials In some areas fakeagro inputs are dominating the market resulting in a limited trustand willingness to invest in applying fertilizer according to farmerrepresentatives who participated in the workshops The exampleshows how the absence or poor performance of fertilizer distribu-tion infrastructure limited farmer-extensionist interaction and lackof functional institutions for quality control (being structural con-ditions for innovation) constrain the innovation capacity in theagricultural systems and its technological (in this case fertilizer) andsectoral (the rice value chain) subsystems Another example is basedon secondary data analyses that demonstrated the lack of an op-erational strategy to address parasitic weeds in Tanzania and BeninIn both the interviews and workshops stakeholders highlighted thegeneral lack of interaction and collaboration between stakehold-ers in the agricultural sector (being a structural condition forinnovation) as one of the main reasons for the absence or poor im-plementation of parasitic weed and other agricultural policies andstrategies

The aforementioned examples demonstrate how RAAIS cansupport the identification of generic entry points for innovation Suchinnovations can directly contribute to addressing the complex ag-ricultural problem under review but can also have a spill-over effectin terms of addressing broader constraints that hamper the inno-vation capacity in the agricultural system For example the lack ofstakeholder interactions and collaboration in the agricultural systemcan provide an entry point for the adaptation of the structuralconditions in the broader agricultural innovation support systemfor example through investments in innovation brokers or

multi-stakeholder platforms (Kilelu et al 2013 Klerkx et al 2010)Such structural adjustments can facilitate multi-stakeholder col-laboration in tackling parasitic weed as well as other complexagricultural problems

43 Lessons learnt from applying RAAIS and recommendations forfurther improvement

Based on our experiences in Tanzania and Benin we recom-mend conducting RAAIS in an interdisciplinary team of researcherswith expertise on different dimensions of complex agricultural prob-lems and on different data collection methods (Hulsebosch 2001)Other suggestions include the experimentation with other combi-nations of methods and on different types of complex agriculturalproblems The workshop methodology could be made more inter-active in the sense of directly feeding back results of the sessionsto participants to stimulate reflection and validate analyses duringthe workshops Post-workshop surveys could provide additionalinsight into whether stakeholders felt they could freely raise anddiscuss their ideas and needs

The multi-stakeholder workshops but also the surveys pre-sented a rather static picture of the complex agricultural problemunder review and the innovation capacity of the agricultural systemin which the problem is embedded However initial workshops andsurveys could function as a baseline to which future workshops andsurveys can be compared Other methods such as secondary dataanalysis or in-depth interviews present a more dynamic image ofhow for example collaborations between stakeholders evolve overthe years Our experiences in Tanzania and Benin show that ensur-ing social differentiation among workshop participants intervieweesand survey respondents (eg of different gender of age) was chal-lenging as for example the majority of workshop participants weremale Specific Workshop Sessions could have more attention for cat-egorisation and priority setting by different gender or age groupsThe facilitation of the multi-stakeholder workshops ensured thatdifferent stakeholder groups could raise and discuss their ideas(Hulsebosch 2001) Despite such efforts unequal power relationsand differences in the ability to debate and negotiate that inher-ently exist between groups may have played a role In line with ourexpectations politically sensitive issues were more freely dis-cussed in individual interviews as compared to multi-stakeholdersetting

The combination of different methods of data collection wasessential In terms of the sequence of data collection we recom-mend to first conduct and analyse the RAAIS multi-stakeholderworkshops to identify constraints and subsequently conduct thein-depth interviews and surveys that can provide more insight inthe distribution and underlying root causes of these constraintsThe workshops then provide a lsquofast-trackrsquo approach to identifyingentry points for innovation that can subsequently be validated andexplored in more detail using the in-depth interviews and stake-holder surveys This would furthermore increase the lsquorapidnessrsquoof RAAIS as a diagnostic tool

An updated version of the RAAIS multi-stakeholder workshopshas been used to identify constraints challenges and entry pointsfor innovations related to the lsquosustainable intensification of agri-cultural systemsrsquo in Burundi the Democratic Republic of CongoRwanda Nigeria and Cameroon under the CGIAR Research Pro-gramme for the Humid Tropics (Humidtropics) (Schut and Hinnou2014) Several of the recommendations made in this paper includ-ing the revised sequence of methods for data collection and the useof post-workshop participant questionnaires have been imple-mented and tested successfully Some of the bottlenecks identifiedsuch as social differentiation (eg gender and age groups) amongworkshop participants remained problematic and require furtherattention At the end of the Humidtropics RAAIS workshops

9M Schut et alAgricultural Systems 132 (2015) 1ndash11

participants developed action plans to address the prioritised con-straints (Workshop Session 13) This required an extension of theworkshops of half a day The development and implementation ofthe action plans forms an important element for continued stake-holder collaboration in multi-stakeholder platforms

5 Conclusions

This paper demonstrates the potential of RAAIS as a diagnostictool that can support and guide the integrated analysis of complexagricultural problems innovation capacity in the agricultural systemand the performance of the agricultural innovation support systemRAAIS combines multiple qualitative and quantitative methods andinsider (stakeholders) and outsider (researchers) analyses whichallow for critical triangulation and validation of the gathered dataSuch an analysis can provide specific entry points for innovationsto address the complex agricultural problem under study and genericentry points for innovation related to strengthening the innova-tion capacity of agricultural system and the functioning of theagricultural innovation support system

Recommendations for further improvement include using RAAISfor the analysis of other types of complex agricultural problems usingother combinations of methods of data collection and providing di-rectly feedback to workshop participants to stimulate reflection andvalidate workshop outcomes An adapted sequence of data collec-tion methods in which workshops provide a lsquofast-trackrsquo approachto identifying entry points for innovation followed up by more in-depth interviews and stakeholder surveys would increase the RAAISrsquodiagnostic capacity The participatory development of concrete actionplans based on RAAIS can provide a basis for continued multi-stakeholder collaboration to operationalise and implement specificand generic entry points for innovation

Acknowledgements

This research forms part of the PARASITE programme fundedby the Integrated Programmes Scheme of the NetherlandsOrganisation for Scientific Research ndash Science for Global Develop-ment (NWO-WOTRO) (wwwparasite-projectorg) Additional supportis provided by the CGIAR Research Program on Climate Change Ag-riculture and Food Security (CCAFS) The CGIAR Research Programmefor the Humid Tropics (httphumidtropicscgiarorg) provided theopportunity to further adapt apply and improve RAAIS across dif-ferent Humidtropics Action Areas Special thanks to Elifadhili Danielwho conducted the farmer-extensionist survey

References

Amankwah K Klerkx L Oosting SJ Sakyi-Dawson O van der Zijpp AJ MillarD 2012 Diagnosing constraints to market participation of small ruminantproducers in northern Ghana an innovation systems analysis NJAS-Wagen JLife Sc 60ndash63 37ndash47

Basu S Leeuwis C 2012 Understanding the rapid spread of System of RiceIntensification (SRI) in Andhra Pradesh exploring the building of supportnetworks and media representation Agr Syst 111 34ndash44

Beintema N Stads G-J Fuglie K Heisey P 2012 ASTI global assessment ofagricultural RampD spending International Food Policy Research InstituteWashington DC Agricultural Science and Technology Indicators Rome Italy andthe Global Forum on Agricultural Research Rome Italy

Blay-Palmer A 2005 Growing innovation policy the case of organic agriculturein Ontario Canada Environ Plann C Gov Policy 23 557ndash581

Carlsson B Stankiewicz R 1991 On the nature function and composition oftechnological systems J Evol Econ 1 93ndash118

Carlsson B Jacobsson S Holmeacuten M Rickne A 2002 Innovation systems analyticaland methodological issues Res Policy 31 233ndash245

Cash DW Adger WN Berkes F Garden P Lebel L Olsson P et al 2006 Scaleand cross-scale dynamics governance and information in a multilevel worldEcol Soc 11 8 Available from lthttpwwwecologyandsocietyorgvol11iss12art18gt [online]

Chambers R 1994 The origins and practice of participatory rural appraisal WorldDev 22 953ndash969

Chung CC 2012 National sectoral and technological innovation systems the caseof Taiwanese pharmaceutical biotechnology and agricultural biotechnologyinnovation systems (1945ndash2000) Sci Public Policy 39 271ndash281

Cooke P Uranga MG Etxebarria G 1997 Regional innovation systems Institutionaland organisational dimensions Res Policy 26 475ndash491

Daniel E 2013 Assessment of agricultural extension services in Tanzania A casestudy of Kyela Songea Rural and Morogoro Rural Districts Wageningen Universityand Research CentreAfrica Rice Center Wageningen the NetherlandsDar esSalaam Tanzania p 45

Douthwaite B Kuby T van de Fliert E Schulz S 2003 Impact pathway evaluationan approach for achieving and attributing impact in complex systems Agr Syst78 243ndash265

Engel PGH 1995 Facilitating Innovation An Action-Oriented Approach andParticipatory Methodology to Improve Innovative Social Practice in AgricultureWageningen University Wageningen the Netherlands p 300

Faysse N 2006 Troubles on the way an analysis of the challenges faced bymulti-stakeholder platforms Nat Resour Forum 30 219ndash229

Freeman C 1988 Japan a new national innovation system In Dosi G FreemanC Nelson R Soete T (Eds) Technology and Economic Theory London pp330ndash348

Freeman C 1995 The national system of innovation in historical perspective CambJ Econ 19 5ndash24

Funtowicz SO Ravetz JR 1993 Science for the post-normal age Futures 25739ndash755

Gildemacher PR Kaguongo W Ortiz O Tesfaye A Woldegiorgis G Wagoire WWet al 2009 Improving potato production in Kenya Uganda and Ethiopia a systemdiagnosis Potato Res 52 173ndash205

Giller KE Leeuwis C Andersson JA Andriesse W Brouwer A Frost P et al2008 Competing claims on natural resources what role for science Ecol Soc13 34 Available from lthttpwwwecologyandsocietyorgvol13iss32art34gt[online]

Giller KE Tittonell P Rufino MC van Wijk MT Zingore S Mapfumo P et al2011 Communicating complexity integrated assessment of trade-offs concerningsoil fertility management within African farming systems to support innovationand development Agr Syst 104 191ndash203

Guest G Bunce A Johnson L 2006 How many interviews are enough Anexperiment with data saturation and variability Field Methods 18 59ndash82

Hall A 2005 Capacity development for agricultural biotechnology in developingcountries an innovation systems view of what it is and how to develop it J IntDev 17 611ndash630

Hall A Clark N 2010 What do complex adaptive systems look like and what arethe implications for innovation policy J Int Dev 22 308ndash324

Hall A Rasheed Sulaiman V Clark N Yoganand B 2003 From measuring impactto learning institutional lessons an innovation systems perspective on improvingthe management of international agricultural research Agr Syst 78 213ndash241

Hall AJ Yoganand B Sulaiman RV Raina RS Prasad CS Naik GC et al 2004Innovations in innovation reflections on partnership institutions and learningICRISAT Andra Pradesh

Hekkert MP Suurs RAA Negro SO Kuhlmann S Smits REHM 2007 Functionsof innovation systems a new approach for analysing technological changeTechnol Forecast Soc Change 74 413ndash432

Hermans F Stuiver M Beers PJ Kok K 2013 The distribution of roles andfunctions for upscaling and outscaling innovations in agricultural innovationsystems Agr Syst 115 117ndash128

Hounkonnou D Kossou D Kuyper TW Leeuwis C Nederlof ES Roumlling N et al2012 An innovation systems approach to institutional change smallholderdevelopment in West Africa Agr Syst 108 74ndash83

Hulsebosch J 2001 The use of RAAKS for strengthening community-basedorganisations in Mali Dev Pract 11 622ndash632

Hung S-C Whittington R 2011 Agency in national innovation systems institutionalentrepreneurship and the professionalization of Taiwanese IT Res Policy 40526ndash538

International Institute for Sustainable Development 2014 Rapid Rural Appraisal(RRA)

Jiggins J 2012 Diagnosing the scope for innovation linking smallholder practicesand institutional context NJAS-Wagen J Life Sc 60ndash63 1ndash122

Kilelu CW Klerkx L Leeuwis C 2013 Unravelling the role of innovation platformsin supporting co-evolution of innovation contributions and tensions in asmallholder dairy development programme Agr Syst 118 65ndash77

Klein Woolthuis R Lankhuizen M Gilsing V 2005 A system failure frameworkfor innovation policy design Technovation 25 609ndash619

Klerkx L Aarts N Leeuwis C 2010 Adaptive management in agricultural innovationsystems the interactions between innovation networks and their environmentAgr Syst 103 390ndash400

Klerkx L Schut M Leeuwis C Kilelu C 2012a Advances in knowledge brokeringin the agricultural sector towards innovation system facilitation IDS Bull 4353ndash60

Klerkx L van Mierlo B Leeuwis C 2012b Evolution of systems approaches toagricultural innovation Concepts analysis and interventions In Darnhofer IGibbon D Dedieu B (Eds) Farming Systems Research into the 21st CenturyThe New Dynamic Springer Dordrecht pp 457ndash483

Leeuwis C 2000 Reconceptualizing participation for sustainable rural developmenttowards a negotiation approach Dev Change 31 931ndash959

Leeuwis C 2004 Communication for Rural Innovation Rethinking AgriculturalExtension (with Contributions of Anne van den Ban) Blackwell Science Oxford

10 M Schut et alAgricultural Systems 132 (2015) 1ndash11

questions are avoided or receive socially desirable responsesMethods that target stakeholders individually are more likely toprovide insights in such questions (International Institute forSustainable Development 2014)

5 Methods together should provide sufficient detail on the complexagricultural problem under review the innovation capacity inthe agricultural system and the functioning of the agriculturalinnovation support system (World Bank 2012)

Combining different types of methods and data collection tech-niques provides an opportunity to triangulate and validate dataDepending on the nature of the agricultural problem under reviewand the available resources and time different types of data col-lection methods can be used for RAAIS taking into account the abovecriteria for method selection

32 Methods of data collection

Based on the five criteria four complementary methods for datacollection were selected to be part of RAAIS (Table 3)

321 Multi-stakeholder workshopsMulti-stakeholder workshops mainly focused on the insider anal-

yses of innovation capacity in the agricultural system and thestructural conditions provided by the agricultural innovation supportsystem A participatory workshop methodology facilitates differ-ent groups of stakeholders to ndash individually and in homogeneousand heterogeneous groups ndash identify categorise and analyse con-straints for innovation in the agricultural system Depending on thetype of problem workshops can be organised with stakeholders rep-resenting national regional andor district levels or for instanceacross different study sites where a specific problem is eminent Tokeep the workshops manageable and to stimulate interaction anddebate the participation of a maximum of 25 participants per work-shop is proposed for instance consisting of five representatives ofthe five different stakeholder groups in Table 1 As much as possi-ble each group should be a representative sample with respect tofor instance gender age income or ethnic groups The work-shops should be held in the language that all participants speakand be facilitated by someone who is familiar with the culturalnorms has affinity with the problem and understands the reali-ties of the different stakeholder groups The proposed workshopmethodology consists of 13 Sessions subdivided into three catego-ries with each their own focus (1) identifying constraints (2)categorising constraints and (3) exploring specific and generic entrypoints for innovation Figure 2 and Table 4 provide an overview ofthe 13 Sessions their sequence and relations and their specific ob-jective in RAAIS

Workshops are designed to take approximately 1 day Besidesthe facilitator a note-taker documents the outcome of the differ-ent sessions and captures discussions among participants Workshopfacilitation and note-taking protocols ensure that the workshoporganisation facilitation and documentation is standardized whichis essential for comparing or aggregating the outcomes for in-stance across different study sites

A crucial element in the workshops is the use of coloured cardsAt the start of the workshop (Session 1) each of the stakeholdergroups is assigned a different colour During Session 2 each par-ticipant individually lists five constraints or challenges they face intheir work and writes them down on their coloured cards If fivestakeholder groups are equally represented this results in 125 cardsDuring Session 3 the participants discuss within their stakehold-er groups the listed constraints explore overlapping issues and jointlydevelop a stakeholder group top-5 If necessary constraints canbe reformulated based on discussions within the group Each of thestakeholder groups use their top-5 throughout the rest of the Ta

ble

3M

eth

odol

ogic

alfr

amew

ork

indi

cati

ng

how

diffe

ren

tm

eth

ods

for

data

coll

ecti

onco

rres

pon

dw

ith

the

sele

ctio

ncr

iter

iafo

rm

eth

ods

Sele

ctio

ncr

iter

iafo

rm

eth

ods

Met

hod

sfo

rda

taco

llect

ion

Typ

eof

data

Insi

der

outs

ider

Stak

ehol

der

grou

pSt

akeh

olde

rin

volv

emen

tA

ISan

alys

is

Qualitative

Quantitative

Insider

Outsider

Farmers

NGOcivilsociety

Privatesector

Government

Researchandtraining

Individual

Homogeneousgroups

Heterogeneousgroups

Complexagriculturalproblems

Innovationcapacityintheagriculturalsystem

Agriculturalinnovationsupportsystem

Mu

lti-

stak

ehol

der

wor

ksh

ops

Sem

i-st

ruct

ure

din

-dep

thin

terv

iew

s

Qu

esti

onn

aire

s

Seco

nda

ryda

taan

alys

is

N

aN

a

5M Schut et alAgricultural Systems 132 (2015) 1ndash11

sessions during the workshop hence 25 cards (five cards per stake-holder group) (Photo 1ndash4)

The use of the coloured cards facilitates the analysis of differ-ent sessions during and after the workshops As the cards are codedand recycled throughout the successive sessions photographs canbe taken to capture the results (for example Photo 1 and 4) Suchphotographs can be analysed after the workshop and can also beused to validate the note-takerrsquos data Furthermore the cards provideinsight into the relations between constraints identified by differ-ent stakeholder groups (Photo 2 and 3) Combining the results fromdifferent sessions can stimulate integrative analyses for instancecombining data resulting from Sessions 5 and 6 provides insight inthe structural conditions for innovation across different levels Sim-ilarly the outcome of Sessions 7 and 11 can be compared totriangulate the data as both seek to identify key constraints for in-novation in the agricultural system

322 Semi-structured in-depth interviewsTo guide the semi-structured interviews a topic list is pre-

pared and fine-tuned for each interview Using a topic list providesa degree of flexibility to identify and to anticipate interestingstorylines related to the problem under review and allows valida-tion of data that was gathered during previous interviews or duringthe workshops Interviews should take a maximum of 1 hour en-suring a high level of attentiveness of both the respondent as wellas the interviewer Sampling of interview respondents should followa stratified approach to ensure that stakeholders representing dif-ferent study sites different stakeholder groups and differentadministrative levels are included Within those strata respon-dents can be selected purposive or based on snowball samplingwhere interview respondents make suggestions for who else shouldbe included in the sample (Russell Bernard 2006) The sample sizecan be based on the concept of ldquosaturationrdquo or the point at which

no new information or themes are observed in the interview data(Guest et al 2006) Interviews can be recorded and transcribed elec-tronically From an ethical point of view interviewees should givepermission for interviews to be recorded and researchers shouldensure confidentiality of all interview data Recording may not alwaysbe desirable as the voice recorder can create a barrier between theresearcher and the respondent especially when it comes to dis-cussing politically sensitive issues Instead of recording detailed notescan be taken and transcribed electronically The transcribed inter-views can be coded Ideally interviews are conducted and codedby two researchers which will enhance the quality of theanalysis

323 SurveysBased on the workshops and the interviews some of the con-

straints may be eligible for broader study among specific groupsof stakeholders through the use of surveys Such surveys mayprovide more insights in for example the socio-economic impactsof climate change on smallholder agriculture in specific regionsthe quality of agricultural extension received by farmers in ad-dressing complex agricultural problems or access to agriculturalinputs for male or female headed households Surveys are not nec-essarily limited to farmers but can also be conducted with any ofthe other stakeholder groups involved For the data to be comple-mentary surveys should be completed in the same study sites aswhere the workshops were organised and among a representativesample of the targeted stakeholder group To achieve that a strati-fied random sampling strategy can be used to identify respondentsacross different study sites levels or stakeholder groups A (effi-cient) sampling method that allows for optimal allocation ofresources can be used to determine the sample size (eg Whitleyand Ball 2002)

324 Secondary data collectionSecondary data are written data with relevance for the analy-

sis of the complex agricultural problem innovation capacity of theagricultural system or the functioning of the agricultural innova-tion support system Examples are policy documents projectproposals and reports laws or legal procedures project evalua-tions curricula for agricultural education and training (agricultural)census and organisational records such as charts and budgets overa period of time The sampling of secondary data is not clear cutKey agricultural documents such as agricultural policies or agri-cultural research priorities should be included These documentscan refer to other relevant data Furthermore secondary data is oftenprovided during or following interviews Insights from secondarydata can be verified in interviews with stakeholders (eg the extentto which policy is implemented and enforced)

4 RAAISrsquo ability to provide specific and generic entry pointsfor innovation and lessons learnt from its application

We tested RAAIS through a case study aimed at analysing con-straints and opportunities for innovation to effectively addressparasitic weeds in rain-fed rice production systems in Tanzania(AprilndashOctober 2012) and Benin (JunendashAugust 2013) The results fromRAAIS in Tanzania are elaborated in Schut et al (2014c) Data weregathered across national zonal regional and district levels Multi-stakeholder workshops (with 68 participants in Tanzania and 66participants in Benin) were organised in three study sites (dis-tricts) in Tanzania and Benin where parasitic weeds are eminentIn-depth interviews were held with representatives of national-zonal- regional- and district-level representatives of farmer coop-eratives and associations NGO civil society private sectorgovernment and research and training institutes (42 in Tanzania65 in Benin) Across the three study sites in the countries a

Session 2

Session3

Session1

Categorisingconstraints

Exploring specific and generic entry points for innovation

Identifying constraints

Fig 2 The relation between the 13 Workshop Sessions and their sequence sub-divided over the three categories The dotted arrows indicate relations between thedifferent sessions in terms of triangulation and validation of data

6 M Schut et alAgricultural Systems 132 (2015) 1ndash11

socio-economic farmer survey (152 in Tanzania 182 in Benin) washeld to study the impact of parasitic weeds on rain-fed rice farming(see Nrsquocho et al 2014 for more information) In Tanzania a farmer-extensionist survey (120 farmers 30 agricultural extension officers)was held to explore the effectiveness of the national agricultural ex-tension policy across the three study sites (see Daniel 2013 for more

information) Additionally for both countries secondary data in-cluding crop protection extension and general agricultural policynational research priorities agricultural census and agricultural train-ing curricula were analysed Data gathering and initial analysis tookaround three months for each of the countries and involved tworesearchers We first conducted the in-depth interviews followed

Table 4The 13 Workshop Sessions subdivided over the three categories and their specific activities and objectives in the RAAIS

Categories Sessions Activities Objective(s)

Identifyingconstraints

1 Opening andparticipantintroduction

Participants (1) introduce themselves andreceive information about the workshopmethodology and (2) are subdivided overdifferent stakeholder groups (eg groupsidentified in Table 1)

bull To ensure an equal representation of participants over thedifferent stakeholder groups

2 Individualbrainstorming aboutconstraints

Participants individually identify fiveconstraints they face in their work

bull To make an inventory of general constraints in the agriculturalsystem faced by stakeholders

3 Developing a top-5 ofconstraints instakeholder groups

Participants (1) discuss constraints withinrespective stakeholder group (2) develop anstakeholder group top-5 of constraints (3)present the top-5 to other stakeholder groupsand (4) discuss within and betweenstakeholder group(s)

bull To gain insights in the key constraints in the agricultural system asfaced by different stakeholder groups

bull To create awareness and stimulate learning among stakeholders

Categorisingconstraints

4 Categorisingconstraints alongdifferent types ofinstitutions

Participants (1) categorise top-5 constraints aspolicy- research- education and training-extension- markets- and or politics-related(2) present results to the other groups and (3)discuss within and between the stakeholdergroup(s)

bull To gain insights in how key constraints relate to the different typesof institutions (institutional subsystem)

bull To create awareness and stimulate learning between stakeholders

5 Categorisingconstraints alongstructural conditionsthat can enable orconstrain innovation

Participants (1) categorise top-5 constraintsalong the structural conditions drivers ofinnovation (Table 2) and (2) discuss withinand between the stakeholder group(s)

bull To gain insights in how the stakeholder constraints relate tostructural conditions provided agricultural innovation supportsystem and whether these enable or constrain innovation capacity

bull To create awareness and stimulate learning between stakeholders

6 Categorisingconstraints acrossdifferent(administrative) levelswithin the institutionalsubsystems

Participants (1) categorise top-5 constraintsacross different administrative levels (egnational regional district) (2) discuss resultswith other stakeholder groups and (3) discusswithin and between the stakeholder group(s)

bull To gain insights in how key constraints relate to differentinstitutional (administrative) levels

bull To identify and analyse interactions between different levelsbull To create awareness and stimulate learning between stakeholders

7 Identifyingrelationships betweenconstraints andidentifying keyconstraints

Participants (1) jointly discuss and identifyrelations between the different constraints (2)identify constraints or challenges that arecentral in the analysis and (3) discuss withinand between the stakeholder group(s)

bull To analyse relationships between different constraintsbull To identify key constraintsbull To create awareness and stimulate learning between stakeholdersbull Identify generic entry points for enhancing the innovation

capacity in the agricultural system8 Categorising

constraints along thesectoral subsystem

Participants (1) categorise stakeholder grouptop-5 constraints along the segments of thevalue chain and (2) discuss within andbetween the stakeholder group(s)

bull To analyse constraints along the sectoral subsystembull To create awareness and stimulate learning between stakeholders

9 Categorisingconstraints alongdifferent technologicalsubsystems

Participants (1) categorise top-5 constraintsalong different technological or knowledgefields and (2) discuss within and between thestakeholder group(s)

bull To analyse constraints along different technological subsystemsbull To create awareness and stimulate learning between stakeholders

Exploringentrypoints forinnovation

10 Exploring constraintsstakeholder groups cansolve themselvesversus problems thatcan only be solvedwith or by others

Participants (1) categorise top-5 constraints aslsquocan be solved within the stakeholder grouprsquoor lsquocan only be solved in collaboration withother stakeholder groupsrsquo and (2) discussionwithin and between the stakeholder group(s)

bull To identify constraints that require collaboration betweenstakeholder groups

bull To create awareness and stimulate learning between stakeholdersbull Identify entry points for innovation in the agricultural system

11 Exploring constraintsthat are easy difficultto solve

Participants (1) categorise top-5 constraints asrelatively lsquoeasyrsquo or lsquodifficultrsquo to address and (2)discuss within and between the stakeholdergroup(s)

bull To explore which constraints require system optimisation (easy toaddress) and those that require system transformation (difficult toaddress)

bull To create awareness and stimulate learning between stakeholdersbull To triangulate data with Session 7 (are key constraints perceived

to be easy difficult to address)bull Identify entry points for enhancing the innovation capacity in the

agricultural system12 Exploring constraints

that are structuraloperational

Participants categorise top-5 constraints alonga four-step gradient ranging from lsquoverystructuralrsquo lsquostructuralrsquo lsquooperationalrsquo and lsquoveryoperationalrsquo challenges and constraints

bull To distinguish between structural constraints that require specificinnovation and more structural problems that require genericinnovation

bull To create awareness and stimulate learning between stakeholdersbull To triangulate data with Sessions 7 and 11 (relation between key

constraints how these are perceived by stakeholders)bull Identify generic entry points for enhancing the innovation

capacity in the agricultural system13 Identifying priorities

and solution strategiesParticipants (1) jointly discuss and develop anoverall top-5 of constraints and (2) jointlyidentify potential strategies to address theseconstraints

bull To explore opportunities for addressing systems constraintsthrough multi-stakeholder collaboration

bull To explore similarities and differences with the key systemsconstraints identified in Session 7

bull Identify key entry points for innovation

7M Schut et alAgricultural Systems 132 (2015) 1ndash11

by the multi-stakeholder workshops In Tanzania both the socio-economic farmer survey and the farmer-extensionist survey wereheld after the interviews and workshops In Benin the socio-economic farmer survey was held preceding the in-depth interviewsand workshops Secondary data collection occurred throughout thefieldwork Below we will further reflect on the main objectives ofRAAIS as well as provide recommendations for further improve-ments and use of RAAIS using our experiences from Tanzania andBenin

41 RAAISrsquo ability to provide specific entry points for innovation toaddress complex agricultural problems

RAAIS contributed to an integrated understanding of differentproblem dimensions multi-level interactions and multi-stakeholderdynamics related to parasitic weed problems With regard to thedifferent problem dimensions interviews demonstrated a poten-tial relation between for example the preference for growing localaromatic rice varieties (social-cultural dimension) the low capac-ity of farmer to purchase certified seeds (economic dimension) andthe spread of parasitic weed seeds through the local rice seed system(technological dimension) Additionally analysis of workshop datarevealed how the untimely and insufficient availability of agricul-tural inputs provided by the government (institutional dimension)and limited interaction and collaboration among networks of key

stakeholders (political dimensions) form additional bottlenecks foraddressing such problems It created awareness that describing andexplaining complex agricultural problems and exploring and de-signing solutions is unlikely to be successful if the different problemdimensions are analysed and treated separately (Hall and Clark 2010Spielman et al 2009)

Data gathering across different levels (national region and dis-trict level) enabled the analysis of the interactions and (mis)matchesbetween different levels (Cash et al 2006) An example that emergedduring the workshops and the interviews is Tanzaniarsquos nationalexport ban that prohibits export of agricultural produce (eg of rice)as long as the country has not been declared lsquofood securersquo This na-tional export ban influences local market prices and consequentlyalso farmersrsquo willingness and ability to invest in for example pur-chasing agricultural inputs such as fertilizers and seeds (eg Poultonet al 2010) This in turn provided an opportunity to identify entrypoints for innovation across different levels which has been iden-tified as a critical factor for addressing complex agricultural problems(eg Giller et al 2008 2011) As expected and confirming previ-ous reports (eg van Mierlo et al 2010) the participatory analysisof multi-level interactions showed that stakeholders (insiders) oftenidentify constraints at the level they represent (Schut et al 2014c)This was complemented by our analysis as researchers (outsiders)of the multi-level interactions regarding the parasitic weedproblems

Photo 1ndash4 Photo 1 (top left) Top-5 of constraints of NGO civil society representatives and their categorisation under the different components of the institutional sub-system (Session 4) Photo 2 (top right) The categorisation of the top-5 of the different stakeholder groups along different structural conditions that can enable or constraininnovation (Session 5) Photo 3 (bottom left) The identification of relationships between different constraints (arrows) and key problem (circled cards) (Session 7) Photo4 (bottom right) The categorisation of the top-5 of the different stakeholder groups along a four-step gradient ranging from structural to operational constraints (Session12) Photos were taken by M Schut during multi-stakeholder workshops in Tanzania held in October 2012

8 M Schut et alAgricultural Systems 132 (2015) 1ndash11

The involvement of different groups of stakeholders was essen-tial for enhancing the credibility validity and quality of RAAIS aswell as for delineating the boundaries of the agricultural system andthe agricultural innovation support system which is considered akey challenge when using AIS approaches to analyse complex ag-ricultural problems (Klerkx et al 2012b) Furthermore stakeholderparticipation provided a better understanding of the feasibility andacceptability of solutions for stakeholder groups Although we believethat the stakeholder groups included in the testing of RAAIS (Table 1)provide a good starting point other stakeholder groups (for in-stance the media) may be included in the sample (eg Ortiz et al2013) depending on the type of complex agricultural problem underreview The triangulation of data resulting from the different methodsenabled us to validate findings and to verify strategic communi-cation by stakeholders for instance to verify how the extensionsystem as described by policymakers in interviews functioned inreality according to surveyed farmers

42 RAAISrsquo ability to provide generic entry points for innovation

RAAIS demonstrates interactions between complex agricultur-al problems innovation capacity of the agricultural system ndashconsisting of institutional sectoral and technological subsystemsndash and the agricultural innovation support system For example ap-plying fertilizer (technological subsystem) in rain-fed rice productionis seen as a promising management strategy to reduce infection levelsof Rhamphicarpa one of the parasitic weeds involved in the studyand mitigate negative effects of the parasite on rice yields (Rodenburget al 2011) However as was highlighted during the RAAIS work-shops in both in Benin and in Tanzania fertilizers are difficult toaccess in rural areas In Benin there is no well-developed privateagro-dealer network and distribution infrastructure to support thesupply of agricultural inputs Furthermore interviews showed thatthe public extension and input supply systems in Benin focus onthe cotton sector rather than on cereal crops (sectoral subsys-tems) In Tanzania a private agro-dealer network and distributioninfrastructure exists but structures controlling the quality of fer-tilizers (institutional subsystem) are functioning sub-optimallyaccording to interviewed government officials In some areas fakeagro inputs are dominating the market resulting in a limited trustand willingness to invest in applying fertilizer according to farmerrepresentatives who participated in the workshops The exampleshows how the absence or poor performance of fertilizer distribu-tion infrastructure limited farmer-extensionist interaction and lackof functional institutions for quality control (being structural con-ditions for innovation) constrain the innovation capacity in theagricultural systems and its technological (in this case fertilizer) andsectoral (the rice value chain) subsystems Another example is basedon secondary data analyses that demonstrated the lack of an op-erational strategy to address parasitic weeds in Tanzania and BeninIn both the interviews and workshops stakeholders highlighted thegeneral lack of interaction and collaboration between stakehold-ers in the agricultural sector (being a structural condition forinnovation) as one of the main reasons for the absence or poor im-plementation of parasitic weed and other agricultural policies andstrategies

The aforementioned examples demonstrate how RAAIS cansupport the identification of generic entry points for innovation Suchinnovations can directly contribute to addressing the complex ag-ricultural problem under review but can also have a spill-over effectin terms of addressing broader constraints that hamper the inno-vation capacity in the agricultural system For example the lack ofstakeholder interactions and collaboration in the agricultural systemcan provide an entry point for the adaptation of the structuralconditions in the broader agricultural innovation support systemfor example through investments in innovation brokers or

multi-stakeholder platforms (Kilelu et al 2013 Klerkx et al 2010)Such structural adjustments can facilitate multi-stakeholder col-laboration in tackling parasitic weed as well as other complexagricultural problems

43 Lessons learnt from applying RAAIS and recommendations forfurther improvement

Based on our experiences in Tanzania and Benin we recom-mend conducting RAAIS in an interdisciplinary team of researcherswith expertise on different dimensions of complex agricultural prob-lems and on different data collection methods (Hulsebosch 2001)Other suggestions include the experimentation with other combi-nations of methods and on different types of complex agriculturalproblems The workshop methodology could be made more inter-active in the sense of directly feeding back results of the sessionsto participants to stimulate reflection and validate analyses duringthe workshops Post-workshop surveys could provide additionalinsight into whether stakeholders felt they could freely raise anddiscuss their ideas and needs

The multi-stakeholder workshops but also the surveys pre-sented a rather static picture of the complex agricultural problemunder review and the innovation capacity of the agricultural systemin which the problem is embedded However initial workshops andsurveys could function as a baseline to which future workshops andsurveys can be compared Other methods such as secondary dataanalysis or in-depth interviews present a more dynamic image ofhow for example collaborations between stakeholders evolve overthe years Our experiences in Tanzania and Benin show that ensur-ing social differentiation among workshop participants intervieweesand survey respondents (eg of different gender of age) was chal-lenging as for example the majority of workshop participants weremale Specific Workshop Sessions could have more attention for cat-egorisation and priority setting by different gender or age groupsThe facilitation of the multi-stakeholder workshops ensured thatdifferent stakeholder groups could raise and discuss their ideas(Hulsebosch 2001) Despite such efforts unequal power relationsand differences in the ability to debate and negotiate that inher-ently exist between groups may have played a role In line with ourexpectations politically sensitive issues were more freely dis-cussed in individual interviews as compared to multi-stakeholdersetting

The combination of different methods of data collection wasessential In terms of the sequence of data collection we recom-mend to first conduct and analyse the RAAIS multi-stakeholderworkshops to identify constraints and subsequently conduct thein-depth interviews and surveys that can provide more insight inthe distribution and underlying root causes of these constraintsThe workshops then provide a lsquofast-trackrsquo approach to identifyingentry points for innovation that can subsequently be validated andexplored in more detail using the in-depth interviews and stake-holder surveys This would furthermore increase the lsquorapidnessrsquoof RAAIS as a diagnostic tool

An updated version of the RAAIS multi-stakeholder workshopshas been used to identify constraints challenges and entry pointsfor innovations related to the lsquosustainable intensification of agri-cultural systemsrsquo in Burundi the Democratic Republic of CongoRwanda Nigeria and Cameroon under the CGIAR Research Pro-gramme for the Humid Tropics (Humidtropics) (Schut and Hinnou2014) Several of the recommendations made in this paper includ-ing the revised sequence of methods for data collection and the useof post-workshop participant questionnaires have been imple-mented and tested successfully Some of the bottlenecks identifiedsuch as social differentiation (eg gender and age groups) amongworkshop participants remained problematic and require furtherattention At the end of the Humidtropics RAAIS workshops

9M Schut et alAgricultural Systems 132 (2015) 1ndash11

participants developed action plans to address the prioritised con-straints (Workshop Session 13) This required an extension of theworkshops of half a day The development and implementation ofthe action plans forms an important element for continued stake-holder collaboration in multi-stakeholder platforms

5 Conclusions

This paper demonstrates the potential of RAAIS as a diagnostictool that can support and guide the integrated analysis of complexagricultural problems innovation capacity in the agricultural systemand the performance of the agricultural innovation support systemRAAIS combines multiple qualitative and quantitative methods andinsider (stakeholders) and outsider (researchers) analyses whichallow for critical triangulation and validation of the gathered dataSuch an analysis can provide specific entry points for innovationsto address the complex agricultural problem under study and genericentry points for innovation related to strengthening the innova-tion capacity of agricultural system and the functioning of theagricultural innovation support system

Recommendations for further improvement include using RAAISfor the analysis of other types of complex agricultural problems usingother combinations of methods of data collection and providing di-rectly feedback to workshop participants to stimulate reflection andvalidate workshop outcomes An adapted sequence of data collec-tion methods in which workshops provide a lsquofast-trackrsquo approachto identifying entry points for innovation followed up by more in-depth interviews and stakeholder surveys would increase the RAAISrsquodiagnostic capacity The participatory development of concrete actionplans based on RAAIS can provide a basis for continued multi-stakeholder collaboration to operationalise and implement specificand generic entry points for innovation

Acknowledgements

This research forms part of the PARASITE programme fundedby the Integrated Programmes Scheme of the NetherlandsOrganisation for Scientific Research ndash Science for Global Develop-ment (NWO-WOTRO) (wwwparasite-projectorg) Additional supportis provided by the CGIAR Research Program on Climate Change Ag-riculture and Food Security (CCAFS) The CGIAR Research Programmefor the Humid Tropics (httphumidtropicscgiarorg) provided theopportunity to further adapt apply and improve RAAIS across dif-ferent Humidtropics Action Areas Special thanks to Elifadhili Danielwho conducted the farmer-extensionist survey

References

Amankwah K Klerkx L Oosting SJ Sakyi-Dawson O van der Zijpp AJ MillarD 2012 Diagnosing constraints to market participation of small ruminantproducers in northern Ghana an innovation systems analysis NJAS-Wagen JLife Sc 60ndash63 37ndash47

Basu S Leeuwis C 2012 Understanding the rapid spread of System of RiceIntensification (SRI) in Andhra Pradesh exploring the building of supportnetworks and media representation Agr Syst 111 34ndash44

Beintema N Stads G-J Fuglie K Heisey P 2012 ASTI global assessment ofagricultural RampD spending International Food Policy Research InstituteWashington DC Agricultural Science and Technology Indicators Rome Italy andthe Global Forum on Agricultural Research Rome Italy

Blay-Palmer A 2005 Growing innovation policy the case of organic agriculturein Ontario Canada Environ Plann C Gov Policy 23 557ndash581

Carlsson B Stankiewicz R 1991 On the nature function and composition oftechnological systems J Evol Econ 1 93ndash118

Carlsson B Jacobsson S Holmeacuten M Rickne A 2002 Innovation systems analyticaland methodological issues Res Policy 31 233ndash245

Cash DW Adger WN Berkes F Garden P Lebel L Olsson P et al 2006 Scaleand cross-scale dynamics governance and information in a multilevel worldEcol Soc 11 8 Available from lthttpwwwecologyandsocietyorgvol11iss12art18gt [online]

Chambers R 1994 The origins and practice of participatory rural appraisal WorldDev 22 953ndash969

Chung CC 2012 National sectoral and technological innovation systems the caseof Taiwanese pharmaceutical biotechnology and agricultural biotechnologyinnovation systems (1945ndash2000) Sci Public Policy 39 271ndash281

Cooke P Uranga MG Etxebarria G 1997 Regional innovation systems Institutionaland organisational dimensions Res Policy 26 475ndash491

Daniel E 2013 Assessment of agricultural extension services in Tanzania A casestudy of Kyela Songea Rural and Morogoro Rural Districts Wageningen Universityand Research CentreAfrica Rice Center Wageningen the NetherlandsDar esSalaam Tanzania p 45

Douthwaite B Kuby T van de Fliert E Schulz S 2003 Impact pathway evaluationan approach for achieving and attributing impact in complex systems Agr Syst78 243ndash265

Engel PGH 1995 Facilitating Innovation An Action-Oriented Approach andParticipatory Methodology to Improve Innovative Social Practice in AgricultureWageningen University Wageningen the Netherlands p 300

Faysse N 2006 Troubles on the way an analysis of the challenges faced bymulti-stakeholder platforms Nat Resour Forum 30 219ndash229

Freeman C 1988 Japan a new national innovation system In Dosi G FreemanC Nelson R Soete T (Eds) Technology and Economic Theory London pp330ndash348

Freeman C 1995 The national system of innovation in historical perspective CambJ Econ 19 5ndash24

Funtowicz SO Ravetz JR 1993 Science for the post-normal age Futures 25739ndash755

Gildemacher PR Kaguongo W Ortiz O Tesfaye A Woldegiorgis G Wagoire WWet al 2009 Improving potato production in Kenya Uganda and Ethiopia a systemdiagnosis Potato Res 52 173ndash205

Giller KE Leeuwis C Andersson JA Andriesse W Brouwer A Frost P et al2008 Competing claims on natural resources what role for science Ecol Soc13 34 Available from lthttpwwwecologyandsocietyorgvol13iss32art34gt[online]

Giller KE Tittonell P Rufino MC van Wijk MT Zingore S Mapfumo P et al2011 Communicating complexity integrated assessment of trade-offs concerningsoil fertility management within African farming systems to support innovationand development Agr Syst 104 191ndash203

Guest G Bunce A Johnson L 2006 How many interviews are enough Anexperiment with data saturation and variability Field Methods 18 59ndash82

Hall A 2005 Capacity development for agricultural biotechnology in developingcountries an innovation systems view of what it is and how to develop it J IntDev 17 611ndash630

Hall A Clark N 2010 What do complex adaptive systems look like and what arethe implications for innovation policy J Int Dev 22 308ndash324

Hall A Rasheed Sulaiman V Clark N Yoganand B 2003 From measuring impactto learning institutional lessons an innovation systems perspective on improvingthe management of international agricultural research Agr Syst 78 213ndash241

Hall AJ Yoganand B Sulaiman RV Raina RS Prasad CS Naik GC et al 2004Innovations in innovation reflections on partnership institutions and learningICRISAT Andra Pradesh

Hekkert MP Suurs RAA Negro SO Kuhlmann S Smits REHM 2007 Functionsof innovation systems a new approach for analysing technological changeTechnol Forecast Soc Change 74 413ndash432

Hermans F Stuiver M Beers PJ Kok K 2013 The distribution of roles andfunctions for upscaling and outscaling innovations in agricultural innovationsystems Agr Syst 115 117ndash128

Hounkonnou D Kossou D Kuyper TW Leeuwis C Nederlof ES Roumlling N et al2012 An innovation systems approach to institutional change smallholderdevelopment in West Africa Agr Syst 108 74ndash83

Hulsebosch J 2001 The use of RAAKS for strengthening community-basedorganisations in Mali Dev Pract 11 622ndash632

Hung S-C Whittington R 2011 Agency in national innovation systems institutionalentrepreneurship and the professionalization of Taiwanese IT Res Policy 40526ndash538

International Institute for Sustainable Development 2014 Rapid Rural Appraisal(RRA)

Jiggins J 2012 Diagnosing the scope for innovation linking smallholder practicesand institutional context NJAS-Wagen J Life Sc 60ndash63 1ndash122

Kilelu CW Klerkx L Leeuwis C 2013 Unravelling the role of innovation platformsin supporting co-evolution of innovation contributions and tensions in asmallholder dairy development programme Agr Syst 118 65ndash77

Klein Woolthuis R Lankhuizen M Gilsing V 2005 A system failure frameworkfor innovation policy design Technovation 25 609ndash619

Klerkx L Aarts N Leeuwis C 2010 Adaptive management in agricultural innovationsystems the interactions between innovation networks and their environmentAgr Syst 103 390ndash400

Klerkx L Schut M Leeuwis C Kilelu C 2012a Advances in knowledge brokeringin the agricultural sector towards innovation system facilitation IDS Bull 4353ndash60

Klerkx L van Mierlo B Leeuwis C 2012b Evolution of systems approaches toagricultural innovation Concepts analysis and interventions In Darnhofer IGibbon D Dedieu B (Eds) Farming Systems Research into the 21st CenturyThe New Dynamic Springer Dordrecht pp 457ndash483

Leeuwis C 2000 Reconceptualizing participation for sustainable rural developmenttowards a negotiation approach Dev Change 31 931ndash959

Leeuwis C 2004 Communication for Rural Innovation Rethinking AgriculturalExtension (with Contributions of Anne van den Ban) Blackwell Science Oxford

10 M Schut et alAgricultural Systems 132 (2015) 1ndash11

sessions during the workshop hence 25 cards (five cards per stake-holder group) (Photo 1ndash4)

The use of the coloured cards facilitates the analysis of differ-ent sessions during and after the workshops As the cards are codedand recycled throughout the successive sessions photographs canbe taken to capture the results (for example Photo 1 and 4) Suchphotographs can be analysed after the workshop and can also beused to validate the note-takerrsquos data Furthermore the cards provideinsight into the relations between constraints identified by differ-ent stakeholder groups (Photo 2 and 3) Combining the results fromdifferent sessions can stimulate integrative analyses for instancecombining data resulting from Sessions 5 and 6 provides insight inthe structural conditions for innovation across different levels Sim-ilarly the outcome of Sessions 7 and 11 can be compared totriangulate the data as both seek to identify key constraints for in-novation in the agricultural system

322 Semi-structured in-depth interviewsTo guide the semi-structured interviews a topic list is pre-

pared and fine-tuned for each interview Using a topic list providesa degree of flexibility to identify and to anticipate interestingstorylines related to the problem under review and allows valida-tion of data that was gathered during previous interviews or duringthe workshops Interviews should take a maximum of 1 hour en-suring a high level of attentiveness of both the respondent as wellas the interviewer Sampling of interview respondents should followa stratified approach to ensure that stakeholders representing dif-ferent study sites different stakeholder groups and differentadministrative levels are included Within those strata respon-dents can be selected purposive or based on snowball samplingwhere interview respondents make suggestions for who else shouldbe included in the sample (Russell Bernard 2006) The sample sizecan be based on the concept of ldquosaturationrdquo or the point at which

no new information or themes are observed in the interview data(Guest et al 2006) Interviews can be recorded and transcribed elec-tronically From an ethical point of view interviewees should givepermission for interviews to be recorded and researchers shouldensure confidentiality of all interview data Recording may not alwaysbe desirable as the voice recorder can create a barrier between theresearcher and the respondent especially when it comes to dis-cussing politically sensitive issues Instead of recording detailed notescan be taken and transcribed electronically The transcribed inter-views can be coded Ideally interviews are conducted and codedby two researchers which will enhance the quality of theanalysis

323 SurveysBased on the workshops and the interviews some of the con-

straints may be eligible for broader study among specific groupsof stakeholders through the use of surveys Such surveys mayprovide more insights in for example the socio-economic impactsof climate change on smallholder agriculture in specific regionsthe quality of agricultural extension received by farmers in ad-dressing complex agricultural problems or access to agriculturalinputs for male or female headed households Surveys are not nec-essarily limited to farmers but can also be conducted with any ofthe other stakeholder groups involved For the data to be comple-mentary surveys should be completed in the same study sites aswhere the workshops were organised and among a representativesample of the targeted stakeholder group To achieve that a strati-fied random sampling strategy can be used to identify respondentsacross different study sites levels or stakeholder groups A (effi-cient) sampling method that allows for optimal allocation ofresources can be used to determine the sample size (eg Whitleyand Ball 2002)

324 Secondary data collectionSecondary data are written data with relevance for the analy-

sis of the complex agricultural problem innovation capacity of theagricultural system or the functioning of the agricultural innova-tion support system Examples are policy documents projectproposals and reports laws or legal procedures project evalua-tions curricula for agricultural education and training (agricultural)census and organisational records such as charts and budgets overa period of time The sampling of secondary data is not clear cutKey agricultural documents such as agricultural policies or agri-cultural research priorities should be included These documentscan refer to other relevant data Furthermore secondary data is oftenprovided during or following interviews Insights from secondarydata can be verified in interviews with stakeholders (eg the extentto which policy is implemented and enforced)

4 RAAISrsquo ability to provide specific and generic entry pointsfor innovation and lessons learnt from its application

We tested RAAIS through a case study aimed at analysing con-straints and opportunities for innovation to effectively addressparasitic weeds in rain-fed rice production systems in Tanzania(AprilndashOctober 2012) and Benin (JunendashAugust 2013) The results fromRAAIS in Tanzania are elaborated in Schut et al (2014c) Data weregathered across national zonal regional and district levels Multi-stakeholder workshops (with 68 participants in Tanzania and 66participants in Benin) were organised in three study sites (dis-tricts) in Tanzania and Benin where parasitic weeds are eminentIn-depth interviews were held with representatives of national-zonal- regional- and district-level representatives of farmer coop-eratives and associations NGO civil society private sectorgovernment and research and training institutes (42 in Tanzania65 in Benin) Across the three study sites in the countries a

Session 2

Session3

Session1

Categorisingconstraints

Exploring specific and generic entry points for innovation

Identifying constraints

Fig 2 The relation between the 13 Workshop Sessions and their sequence sub-divided over the three categories The dotted arrows indicate relations between thedifferent sessions in terms of triangulation and validation of data

6 M Schut et alAgricultural Systems 132 (2015) 1ndash11

socio-economic farmer survey (152 in Tanzania 182 in Benin) washeld to study the impact of parasitic weeds on rain-fed rice farming(see Nrsquocho et al 2014 for more information) In Tanzania a farmer-extensionist survey (120 farmers 30 agricultural extension officers)was held to explore the effectiveness of the national agricultural ex-tension policy across the three study sites (see Daniel 2013 for more

information) Additionally for both countries secondary data in-cluding crop protection extension and general agricultural policynational research priorities agricultural census and agricultural train-ing curricula were analysed Data gathering and initial analysis tookaround three months for each of the countries and involved tworesearchers We first conducted the in-depth interviews followed

Table 4The 13 Workshop Sessions subdivided over the three categories and their specific activities and objectives in the RAAIS

Categories Sessions Activities Objective(s)

Identifyingconstraints

1 Opening andparticipantintroduction

Participants (1) introduce themselves andreceive information about the workshopmethodology and (2) are subdivided overdifferent stakeholder groups (eg groupsidentified in Table 1)

bull To ensure an equal representation of participants over thedifferent stakeholder groups

2 Individualbrainstorming aboutconstraints

Participants individually identify fiveconstraints they face in their work

bull To make an inventory of general constraints in the agriculturalsystem faced by stakeholders

3 Developing a top-5 ofconstraints instakeholder groups

Participants (1) discuss constraints withinrespective stakeholder group (2) develop anstakeholder group top-5 of constraints (3)present the top-5 to other stakeholder groupsand (4) discuss within and betweenstakeholder group(s)

bull To gain insights in the key constraints in the agricultural system asfaced by different stakeholder groups

bull To create awareness and stimulate learning among stakeholders

Categorisingconstraints

4 Categorisingconstraints alongdifferent types ofinstitutions

Participants (1) categorise top-5 constraints aspolicy- research- education and training-extension- markets- and or politics-related(2) present results to the other groups and (3)discuss within and between the stakeholdergroup(s)

bull To gain insights in how key constraints relate to the different typesof institutions (institutional subsystem)

bull To create awareness and stimulate learning between stakeholders

5 Categorisingconstraints alongstructural conditionsthat can enable orconstrain innovation

Participants (1) categorise top-5 constraintsalong the structural conditions drivers ofinnovation (Table 2) and (2) discuss withinand between the stakeholder group(s)

bull To gain insights in how the stakeholder constraints relate tostructural conditions provided agricultural innovation supportsystem and whether these enable or constrain innovation capacity

bull To create awareness and stimulate learning between stakeholders

6 Categorisingconstraints acrossdifferent(administrative) levelswithin the institutionalsubsystems

Participants (1) categorise top-5 constraintsacross different administrative levels (egnational regional district) (2) discuss resultswith other stakeholder groups and (3) discusswithin and between the stakeholder group(s)

bull To gain insights in how key constraints relate to differentinstitutional (administrative) levels

bull To identify and analyse interactions between different levelsbull To create awareness and stimulate learning between stakeholders

7 Identifyingrelationships betweenconstraints andidentifying keyconstraints

Participants (1) jointly discuss and identifyrelations between the different constraints (2)identify constraints or challenges that arecentral in the analysis and (3) discuss withinand between the stakeholder group(s)

bull To analyse relationships between different constraintsbull To identify key constraintsbull To create awareness and stimulate learning between stakeholdersbull Identify generic entry points for enhancing the innovation

capacity in the agricultural system8 Categorising

constraints along thesectoral subsystem

Participants (1) categorise stakeholder grouptop-5 constraints along the segments of thevalue chain and (2) discuss within andbetween the stakeholder group(s)

bull To analyse constraints along the sectoral subsystembull To create awareness and stimulate learning between stakeholders

9 Categorisingconstraints alongdifferent technologicalsubsystems

Participants (1) categorise top-5 constraintsalong different technological or knowledgefields and (2) discuss within and between thestakeholder group(s)

bull To analyse constraints along different technological subsystemsbull To create awareness and stimulate learning between stakeholders

Exploringentrypoints forinnovation

10 Exploring constraintsstakeholder groups cansolve themselvesversus problems thatcan only be solvedwith or by others

Participants (1) categorise top-5 constraints aslsquocan be solved within the stakeholder grouprsquoor lsquocan only be solved in collaboration withother stakeholder groupsrsquo and (2) discussionwithin and between the stakeholder group(s)

bull To identify constraints that require collaboration betweenstakeholder groups

bull To create awareness and stimulate learning between stakeholdersbull Identify entry points for innovation in the agricultural system

11 Exploring constraintsthat are easy difficultto solve

Participants (1) categorise top-5 constraints asrelatively lsquoeasyrsquo or lsquodifficultrsquo to address and (2)discuss within and between the stakeholdergroup(s)

bull To explore which constraints require system optimisation (easy toaddress) and those that require system transformation (difficult toaddress)

bull To create awareness and stimulate learning between stakeholdersbull To triangulate data with Session 7 (are key constraints perceived

to be easy difficult to address)bull Identify entry points for enhancing the innovation capacity in the

agricultural system12 Exploring constraints

that are structuraloperational

Participants categorise top-5 constraints alonga four-step gradient ranging from lsquoverystructuralrsquo lsquostructuralrsquo lsquooperationalrsquo and lsquoveryoperationalrsquo challenges and constraints

bull To distinguish between structural constraints that require specificinnovation and more structural problems that require genericinnovation

bull To create awareness and stimulate learning between stakeholdersbull To triangulate data with Sessions 7 and 11 (relation between key

constraints how these are perceived by stakeholders)bull Identify generic entry points for enhancing the innovation

capacity in the agricultural system13 Identifying priorities

and solution strategiesParticipants (1) jointly discuss and develop anoverall top-5 of constraints and (2) jointlyidentify potential strategies to address theseconstraints

bull To explore opportunities for addressing systems constraintsthrough multi-stakeholder collaboration

bull To explore similarities and differences with the key systemsconstraints identified in Session 7

bull Identify key entry points for innovation

7M Schut et alAgricultural Systems 132 (2015) 1ndash11

by the multi-stakeholder workshops In Tanzania both the socio-economic farmer survey and the farmer-extensionist survey wereheld after the interviews and workshops In Benin the socio-economic farmer survey was held preceding the in-depth interviewsand workshops Secondary data collection occurred throughout thefieldwork Below we will further reflect on the main objectives ofRAAIS as well as provide recommendations for further improve-ments and use of RAAIS using our experiences from Tanzania andBenin

41 RAAISrsquo ability to provide specific entry points for innovation toaddress complex agricultural problems

RAAIS contributed to an integrated understanding of differentproblem dimensions multi-level interactions and multi-stakeholderdynamics related to parasitic weed problems With regard to thedifferent problem dimensions interviews demonstrated a poten-tial relation between for example the preference for growing localaromatic rice varieties (social-cultural dimension) the low capac-ity of farmer to purchase certified seeds (economic dimension) andthe spread of parasitic weed seeds through the local rice seed system(technological dimension) Additionally analysis of workshop datarevealed how the untimely and insufficient availability of agricul-tural inputs provided by the government (institutional dimension)and limited interaction and collaboration among networks of key

stakeholders (political dimensions) form additional bottlenecks foraddressing such problems It created awareness that describing andexplaining complex agricultural problems and exploring and de-signing solutions is unlikely to be successful if the different problemdimensions are analysed and treated separately (Hall and Clark 2010Spielman et al 2009)

Data gathering across different levels (national region and dis-trict level) enabled the analysis of the interactions and (mis)matchesbetween different levels (Cash et al 2006) An example that emergedduring the workshops and the interviews is Tanzaniarsquos nationalexport ban that prohibits export of agricultural produce (eg of rice)as long as the country has not been declared lsquofood securersquo This na-tional export ban influences local market prices and consequentlyalso farmersrsquo willingness and ability to invest in for example pur-chasing agricultural inputs such as fertilizers and seeds (eg Poultonet al 2010) This in turn provided an opportunity to identify entrypoints for innovation across different levels which has been iden-tified as a critical factor for addressing complex agricultural problems(eg Giller et al 2008 2011) As expected and confirming previ-ous reports (eg van Mierlo et al 2010) the participatory analysisof multi-level interactions showed that stakeholders (insiders) oftenidentify constraints at the level they represent (Schut et al 2014c)This was complemented by our analysis as researchers (outsiders)of the multi-level interactions regarding the parasitic weedproblems

Photo 1ndash4 Photo 1 (top left) Top-5 of constraints of NGO civil society representatives and their categorisation under the different components of the institutional sub-system (Session 4) Photo 2 (top right) The categorisation of the top-5 of the different stakeholder groups along different structural conditions that can enable or constraininnovation (Session 5) Photo 3 (bottom left) The identification of relationships between different constraints (arrows) and key problem (circled cards) (Session 7) Photo4 (bottom right) The categorisation of the top-5 of the different stakeholder groups along a four-step gradient ranging from structural to operational constraints (Session12) Photos were taken by M Schut during multi-stakeholder workshops in Tanzania held in October 2012

8 M Schut et alAgricultural Systems 132 (2015) 1ndash11

The involvement of different groups of stakeholders was essen-tial for enhancing the credibility validity and quality of RAAIS aswell as for delineating the boundaries of the agricultural system andthe agricultural innovation support system which is considered akey challenge when using AIS approaches to analyse complex ag-ricultural problems (Klerkx et al 2012b) Furthermore stakeholderparticipation provided a better understanding of the feasibility andacceptability of solutions for stakeholder groups Although we believethat the stakeholder groups included in the testing of RAAIS (Table 1)provide a good starting point other stakeholder groups (for in-stance the media) may be included in the sample (eg Ortiz et al2013) depending on the type of complex agricultural problem underreview The triangulation of data resulting from the different methodsenabled us to validate findings and to verify strategic communi-cation by stakeholders for instance to verify how the extensionsystem as described by policymakers in interviews functioned inreality according to surveyed farmers

42 RAAISrsquo ability to provide generic entry points for innovation

RAAIS demonstrates interactions between complex agricultur-al problems innovation capacity of the agricultural system ndashconsisting of institutional sectoral and technological subsystemsndash and the agricultural innovation support system For example ap-plying fertilizer (technological subsystem) in rain-fed rice productionis seen as a promising management strategy to reduce infection levelsof Rhamphicarpa one of the parasitic weeds involved in the studyand mitigate negative effects of the parasite on rice yields (Rodenburget al 2011) However as was highlighted during the RAAIS work-shops in both in Benin and in Tanzania fertilizers are difficult toaccess in rural areas In Benin there is no well-developed privateagro-dealer network and distribution infrastructure to support thesupply of agricultural inputs Furthermore interviews showed thatthe public extension and input supply systems in Benin focus onthe cotton sector rather than on cereal crops (sectoral subsys-tems) In Tanzania a private agro-dealer network and distributioninfrastructure exists but structures controlling the quality of fer-tilizers (institutional subsystem) are functioning sub-optimallyaccording to interviewed government officials In some areas fakeagro inputs are dominating the market resulting in a limited trustand willingness to invest in applying fertilizer according to farmerrepresentatives who participated in the workshops The exampleshows how the absence or poor performance of fertilizer distribu-tion infrastructure limited farmer-extensionist interaction and lackof functional institutions for quality control (being structural con-ditions for innovation) constrain the innovation capacity in theagricultural systems and its technological (in this case fertilizer) andsectoral (the rice value chain) subsystems Another example is basedon secondary data analyses that demonstrated the lack of an op-erational strategy to address parasitic weeds in Tanzania and BeninIn both the interviews and workshops stakeholders highlighted thegeneral lack of interaction and collaboration between stakehold-ers in the agricultural sector (being a structural condition forinnovation) as one of the main reasons for the absence or poor im-plementation of parasitic weed and other agricultural policies andstrategies

The aforementioned examples demonstrate how RAAIS cansupport the identification of generic entry points for innovation Suchinnovations can directly contribute to addressing the complex ag-ricultural problem under review but can also have a spill-over effectin terms of addressing broader constraints that hamper the inno-vation capacity in the agricultural system For example the lack ofstakeholder interactions and collaboration in the agricultural systemcan provide an entry point for the adaptation of the structuralconditions in the broader agricultural innovation support systemfor example through investments in innovation brokers or

multi-stakeholder platforms (Kilelu et al 2013 Klerkx et al 2010)Such structural adjustments can facilitate multi-stakeholder col-laboration in tackling parasitic weed as well as other complexagricultural problems

43 Lessons learnt from applying RAAIS and recommendations forfurther improvement

Based on our experiences in Tanzania and Benin we recom-mend conducting RAAIS in an interdisciplinary team of researcherswith expertise on different dimensions of complex agricultural prob-lems and on different data collection methods (Hulsebosch 2001)Other suggestions include the experimentation with other combi-nations of methods and on different types of complex agriculturalproblems The workshop methodology could be made more inter-active in the sense of directly feeding back results of the sessionsto participants to stimulate reflection and validate analyses duringthe workshops Post-workshop surveys could provide additionalinsight into whether stakeholders felt they could freely raise anddiscuss their ideas and needs

The multi-stakeholder workshops but also the surveys pre-sented a rather static picture of the complex agricultural problemunder review and the innovation capacity of the agricultural systemin which the problem is embedded However initial workshops andsurveys could function as a baseline to which future workshops andsurveys can be compared Other methods such as secondary dataanalysis or in-depth interviews present a more dynamic image ofhow for example collaborations between stakeholders evolve overthe years Our experiences in Tanzania and Benin show that ensur-ing social differentiation among workshop participants intervieweesand survey respondents (eg of different gender of age) was chal-lenging as for example the majority of workshop participants weremale Specific Workshop Sessions could have more attention for cat-egorisation and priority setting by different gender or age groupsThe facilitation of the multi-stakeholder workshops ensured thatdifferent stakeholder groups could raise and discuss their ideas(Hulsebosch 2001) Despite such efforts unequal power relationsand differences in the ability to debate and negotiate that inher-ently exist between groups may have played a role In line with ourexpectations politically sensitive issues were more freely dis-cussed in individual interviews as compared to multi-stakeholdersetting

The combination of different methods of data collection wasessential In terms of the sequence of data collection we recom-mend to first conduct and analyse the RAAIS multi-stakeholderworkshops to identify constraints and subsequently conduct thein-depth interviews and surveys that can provide more insight inthe distribution and underlying root causes of these constraintsThe workshops then provide a lsquofast-trackrsquo approach to identifyingentry points for innovation that can subsequently be validated andexplored in more detail using the in-depth interviews and stake-holder surveys This would furthermore increase the lsquorapidnessrsquoof RAAIS as a diagnostic tool

An updated version of the RAAIS multi-stakeholder workshopshas been used to identify constraints challenges and entry pointsfor innovations related to the lsquosustainable intensification of agri-cultural systemsrsquo in Burundi the Democratic Republic of CongoRwanda Nigeria and Cameroon under the CGIAR Research Pro-gramme for the Humid Tropics (Humidtropics) (Schut and Hinnou2014) Several of the recommendations made in this paper includ-ing the revised sequence of methods for data collection and the useof post-workshop participant questionnaires have been imple-mented and tested successfully Some of the bottlenecks identifiedsuch as social differentiation (eg gender and age groups) amongworkshop participants remained problematic and require furtherattention At the end of the Humidtropics RAAIS workshops

9M Schut et alAgricultural Systems 132 (2015) 1ndash11

participants developed action plans to address the prioritised con-straints (Workshop Session 13) This required an extension of theworkshops of half a day The development and implementation ofthe action plans forms an important element for continued stake-holder collaboration in multi-stakeholder platforms

5 Conclusions

This paper demonstrates the potential of RAAIS as a diagnostictool that can support and guide the integrated analysis of complexagricultural problems innovation capacity in the agricultural systemand the performance of the agricultural innovation support systemRAAIS combines multiple qualitative and quantitative methods andinsider (stakeholders) and outsider (researchers) analyses whichallow for critical triangulation and validation of the gathered dataSuch an analysis can provide specific entry points for innovationsto address the complex agricultural problem under study and genericentry points for innovation related to strengthening the innova-tion capacity of agricultural system and the functioning of theagricultural innovation support system

Recommendations for further improvement include using RAAISfor the analysis of other types of complex agricultural problems usingother combinations of methods of data collection and providing di-rectly feedback to workshop participants to stimulate reflection andvalidate workshop outcomes An adapted sequence of data collec-tion methods in which workshops provide a lsquofast-trackrsquo approachto identifying entry points for innovation followed up by more in-depth interviews and stakeholder surveys would increase the RAAISrsquodiagnostic capacity The participatory development of concrete actionplans based on RAAIS can provide a basis for continued multi-stakeholder collaboration to operationalise and implement specificand generic entry points for innovation

Acknowledgements

This research forms part of the PARASITE programme fundedby the Integrated Programmes Scheme of the NetherlandsOrganisation for Scientific Research ndash Science for Global Develop-ment (NWO-WOTRO) (wwwparasite-projectorg) Additional supportis provided by the CGIAR Research Program on Climate Change Ag-riculture and Food Security (CCAFS) The CGIAR Research Programmefor the Humid Tropics (httphumidtropicscgiarorg) provided theopportunity to further adapt apply and improve RAAIS across dif-ferent Humidtropics Action Areas Special thanks to Elifadhili Danielwho conducted the farmer-extensionist survey

References

Amankwah K Klerkx L Oosting SJ Sakyi-Dawson O van der Zijpp AJ MillarD 2012 Diagnosing constraints to market participation of small ruminantproducers in northern Ghana an innovation systems analysis NJAS-Wagen JLife Sc 60ndash63 37ndash47

Basu S Leeuwis C 2012 Understanding the rapid spread of System of RiceIntensification (SRI) in Andhra Pradesh exploring the building of supportnetworks and media representation Agr Syst 111 34ndash44

Beintema N Stads G-J Fuglie K Heisey P 2012 ASTI global assessment ofagricultural RampD spending International Food Policy Research InstituteWashington DC Agricultural Science and Technology Indicators Rome Italy andthe Global Forum on Agricultural Research Rome Italy

Blay-Palmer A 2005 Growing innovation policy the case of organic agriculturein Ontario Canada Environ Plann C Gov Policy 23 557ndash581

Carlsson B Stankiewicz R 1991 On the nature function and composition oftechnological systems J Evol Econ 1 93ndash118

Carlsson B Jacobsson S Holmeacuten M Rickne A 2002 Innovation systems analyticaland methodological issues Res Policy 31 233ndash245

Cash DW Adger WN Berkes F Garden P Lebel L Olsson P et al 2006 Scaleand cross-scale dynamics governance and information in a multilevel worldEcol Soc 11 8 Available from lthttpwwwecologyandsocietyorgvol11iss12art18gt [online]

Chambers R 1994 The origins and practice of participatory rural appraisal WorldDev 22 953ndash969

Chung CC 2012 National sectoral and technological innovation systems the caseof Taiwanese pharmaceutical biotechnology and agricultural biotechnologyinnovation systems (1945ndash2000) Sci Public Policy 39 271ndash281

Cooke P Uranga MG Etxebarria G 1997 Regional innovation systems Institutionaland organisational dimensions Res Policy 26 475ndash491

Daniel E 2013 Assessment of agricultural extension services in Tanzania A casestudy of Kyela Songea Rural and Morogoro Rural Districts Wageningen Universityand Research CentreAfrica Rice Center Wageningen the NetherlandsDar esSalaam Tanzania p 45

Douthwaite B Kuby T van de Fliert E Schulz S 2003 Impact pathway evaluationan approach for achieving and attributing impact in complex systems Agr Syst78 243ndash265

Engel PGH 1995 Facilitating Innovation An Action-Oriented Approach andParticipatory Methodology to Improve Innovative Social Practice in AgricultureWageningen University Wageningen the Netherlands p 300

Faysse N 2006 Troubles on the way an analysis of the challenges faced bymulti-stakeholder platforms Nat Resour Forum 30 219ndash229

Freeman C 1988 Japan a new national innovation system In Dosi G FreemanC Nelson R Soete T (Eds) Technology and Economic Theory London pp330ndash348

Freeman C 1995 The national system of innovation in historical perspective CambJ Econ 19 5ndash24

Funtowicz SO Ravetz JR 1993 Science for the post-normal age Futures 25739ndash755

Gildemacher PR Kaguongo W Ortiz O Tesfaye A Woldegiorgis G Wagoire WWet al 2009 Improving potato production in Kenya Uganda and Ethiopia a systemdiagnosis Potato Res 52 173ndash205

Giller KE Leeuwis C Andersson JA Andriesse W Brouwer A Frost P et al2008 Competing claims on natural resources what role for science Ecol Soc13 34 Available from lthttpwwwecologyandsocietyorgvol13iss32art34gt[online]

Giller KE Tittonell P Rufino MC van Wijk MT Zingore S Mapfumo P et al2011 Communicating complexity integrated assessment of trade-offs concerningsoil fertility management within African farming systems to support innovationand development Agr Syst 104 191ndash203

Guest G Bunce A Johnson L 2006 How many interviews are enough Anexperiment with data saturation and variability Field Methods 18 59ndash82

Hall A 2005 Capacity development for agricultural biotechnology in developingcountries an innovation systems view of what it is and how to develop it J IntDev 17 611ndash630

Hall A Clark N 2010 What do complex adaptive systems look like and what arethe implications for innovation policy J Int Dev 22 308ndash324

Hall A Rasheed Sulaiman V Clark N Yoganand B 2003 From measuring impactto learning institutional lessons an innovation systems perspective on improvingthe management of international agricultural research Agr Syst 78 213ndash241

Hall AJ Yoganand B Sulaiman RV Raina RS Prasad CS Naik GC et al 2004Innovations in innovation reflections on partnership institutions and learningICRISAT Andra Pradesh

Hekkert MP Suurs RAA Negro SO Kuhlmann S Smits REHM 2007 Functionsof innovation systems a new approach for analysing technological changeTechnol Forecast Soc Change 74 413ndash432

Hermans F Stuiver M Beers PJ Kok K 2013 The distribution of roles andfunctions for upscaling and outscaling innovations in agricultural innovationsystems Agr Syst 115 117ndash128

Hounkonnou D Kossou D Kuyper TW Leeuwis C Nederlof ES Roumlling N et al2012 An innovation systems approach to institutional change smallholderdevelopment in West Africa Agr Syst 108 74ndash83

Hulsebosch J 2001 The use of RAAKS for strengthening community-basedorganisations in Mali Dev Pract 11 622ndash632

Hung S-C Whittington R 2011 Agency in national innovation systems institutionalentrepreneurship and the professionalization of Taiwanese IT Res Policy 40526ndash538

International Institute for Sustainable Development 2014 Rapid Rural Appraisal(RRA)

Jiggins J 2012 Diagnosing the scope for innovation linking smallholder practicesand institutional context NJAS-Wagen J Life Sc 60ndash63 1ndash122

Kilelu CW Klerkx L Leeuwis C 2013 Unravelling the role of innovation platformsin supporting co-evolution of innovation contributions and tensions in asmallholder dairy development programme Agr Syst 118 65ndash77

Klein Woolthuis R Lankhuizen M Gilsing V 2005 A system failure frameworkfor innovation policy design Technovation 25 609ndash619

Klerkx L Aarts N Leeuwis C 2010 Adaptive management in agricultural innovationsystems the interactions between innovation networks and their environmentAgr Syst 103 390ndash400

Klerkx L Schut M Leeuwis C Kilelu C 2012a Advances in knowledge brokeringin the agricultural sector towards innovation system facilitation IDS Bull 4353ndash60

Klerkx L van Mierlo B Leeuwis C 2012b Evolution of systems approaches toagricultural innovation Concepts analysis and interventions In Darnhofer IGibbon D Dedieu B (Eds) Farming Systems Research into the 21st CenturyThe New Dynamic Springer Dordrecht pp 457ndash483

Leeuwis C 2000 Reconceptualizing participation for sustainable rural developmenttowards a negotiation approach Dev Change 31 931ndash959

Leeuwis C 2004 Communication for Rural Innovation Rethinking AgriculturalExtension (with Contributions of Anne van den Ban) Blackwell Science Oxford

10 M Schut et alAgricultural Systems 132 (2015) 1ndash11

socio-economic farmer survey (152 in Tanzania 182 in Benin) washeld to study the impact of parasitic weeds on rain-fed rice farming(see Nrsquocho et al 2014 for more information) In Tanzania a farmer-extensionist survey (120 farmers 30 agricultural extension officers)was held to explore the effectiveness of the national agricultural ex-tension policy across the three study sites (see Daniel 2013 for more

information) Additionally for both countries secondary data in-cluding crop protection extension and general agricultural policynational research priorities agricultural census and agricultural train-ing curricula were analysed Data gathering and initial analysis tookaround three months for each of the countries and involved tworesearchers We first conducted the in-depth interviews followed

Table 4The 13 Workshop Sessions subdivided over the three categories and their specific activities and objectives in the RAAIS

Categories Sessions Activities Objective(s)

Identifyingconstraints

1 Opening andparticipantintroduction

Participants (1) introduce themselves andreceive information about the workshopmethodology and (2) are subdivided overdifferent stakeholder groups (eg groupsidentified in Table 1)

bull To ensure an equal representation of participants over thedifferent stakeholder groups

2 Individualbrainstorming aboutconstraints

Participants individually identify fiveconstraints they face in their work

bull To make an inventory of general constraints in the agriculturalsystem faced by stakeholders

3 Developing a top-5 ofconstraints instakeholder groups

Participants (1) discuss constraints withinrespective stakeholder group (2) develop anstakeholder group top-5 of constraints (3)present the top-5 to other stakeholder groupsand (4) discuss within and betweenstakeholder group(s)

bull To gain insights in the key constraints in the agricultural system asfaced by different stakeholder groups

bull To create awareness and stimulate learning among stakeholders

Categorisingconstraints

4 Categorisingconstraints alongdifferent types ofinstitutions

Participants (1) categorise top-5 constraints aspolicy- research- education and training-extension- markets- and or politics-related(2) present results to the other groups and (3)discuss within and between the stakeholdergroup(s)

bull To gain insights in how key constraints relate to the different typesof institutions (institutional subsystem)

bull To create awareness and stimulate learning between stakeholders

5 Categorisingconstraints alongstructural conditionsthat can enable orconstrain innovation

Participants (1) categorise top-5 constraintsalong the structural conditions drivers ofinnovation (Table 2) and (2) discuss withinand between the stakeholder group(s)

bull To gain insights in how the stakeholder constraints relate tostructural conditions provided agricultural innovation supportsystem and whether these enable or constrain innovation capacity

bull To create awareness and stimulate learning between stakeholders

6 Categorisingconstraints acrossdifferent(administrative) levelswithin the institutionalsubsystems

Participants (1) categorise top-5 constraintsacross different administrative levels (egnational regional district) (2) discuss resultswith other stakeholder groups and (3) discusswithin and between the stakeholder group(s)

bull To gain insights in how key constraints relate to differentinstitutional (administrative) levels

bull To identify and analyse interactions between different levelsbull To create awareness and stimulate learning between stakeholders

7 Identifyingrelationships betweenconstraints andidentifying keyconstraints

Participants (1) jointly discuss and identifyrelations between the different constraints (2)identify constraints or challenges that arecentral in the analysis and (3) discuss withinand between the stakeholder group(s)

bull To analyse relationships between different constraintsbull To identify key constraintsbull To create awareness and stimulate learning between stakeholdersbull Identify generic entry points for enhancing the innovation

capacity in the agricultural system8 Categorising

constraints along thesectoral subsystem

Participants (1) categorise stakeholder grouptop-5 constraints along the segments of thevalue chain and (2) discuss within andbetween the stakeholder group(s)

bull To analyse constraints along the sectoral subsystembull To create awareness and stimulate learning between stakeholders

9 Categorisingconstraints alongdifferent technologicalsubsystems

Participants (1) categorise top-5 constraintsalong different technological or knowledgefields and (2) discuss within and between thestakeholder group(s)

bull To analyse constraints along different technological subsystemsbull To create awareness and stimulate learning between stakeholders

Exploringentrypoints forinnovation

10 Exploring constraintsstakeholder groups cansolve themselvesversus problems thatcan only be solvedwith or by others

Participants (1) categorise top-5 constraints aslsquocan be solved within the stakeholder grouprsquoor lsquocan only be solved in collaboration withother stakeholder groupsrsquo and (2) discussionwithin and between the stakeholder group(s)

bull To identify constraints that require collaboration betweenstakeholder groups

bull To create awareness and stimulate learning between stakeholdersbull Identify entry points for innovation in the agricultural system

11 Exploring constraintsthat are easy difficultto solve

Participants (1) categorise top-5 constraints asrelatively lsquoeasyrsquo or lsquodifficultrsquo to address and (2)discuss within and between the stakeholdergroup(s)

bull To explore which constraints require system optimisation (easy toaddress) and those that require system transformation (difficult toaddress)

bull To create awareness and stimulate learning between stakeholdersbull To triangulate data with Session 7 (are key constraints perceived

to be easy difficult to address)bull Identify entry points for enhancing the innovation capacity in the

agricultural system12 Exploring constraints

that are structuraloperational

Participants categorise top-5 constraints alonga four-step gradient ranging from lsquoverystructuralrsquo lsquostructuralrsquo lsquooperationalrsquo and lsquoveryoperationalrsquo challenges and constraints

bull To distinguish between structural constraints that require specificinnovation and more structural problems that require genericinnovation

bull To create awareness and stimulate learning between stakeholdersbull To triangulate data with Sessions 7 and 11 (relation between key

constraints how these are perceived by stakeholders)bull Identify generic entry points for enhancing the innovation

capacity in the agricultural system13 Identifying priorities

and solution strategiesParticipants (1) jointly discuss and develop anoverall top-5 of constraints and (2) jointlyidentify potential strategies to address theseconstraints

bull To explore opportunities for addressing systems constraintsthrough multi-stakeholder collaboration

bull To explore similarities and differences with the key systemsconstraints identified in Session 7

bull Identify key entry points for innovation

7M Schut et alAgricultural Systems 132 (2015) 1ndash11

by the multi-stakeholder workshops In Tanzania both the socio-economic farmer survey and the farmer-extensionist survey wereheld after the interviews and workshops In Benin the socio-economic farmer survey was held preceding the in-depth interviewsand workshops Secondary data collection occurred throughout thefieldwork Below we will further reflect on the main objectives ofRAAIS as well as provide recommendations for further improve-ments and use of RAAIS using our experiences from Tanzania andBenin

41 RAAISrsquo ability to provide specific entry points for innovation toaddress complex agricultural problems

RAAIS contributed to an integrated understanding of differentproblem dimensions multi-level interactions and multi-stakeholderdynamics related to parasitic weed problems With regard to thedifferent problem dimensions interviews demonstrated a poten-tial relation between for example the preference for growing localaromatic rice varieties (social-cultural dimension) the low capac-ity of farmer to purchase certified seeds (economic dimension) andthe spread of parasitic weed seeds through the local rice seed system(technological dimension) Additionally analysis of workshop datarevealed how the untimely and insufficient availability of agricul-tural inputs provided by the government (institutional dimension)and limited interaction and collaboration among networks of key

stakeholders (political dimensions) form additional bottlenecks foraddressing such problems It created awareness that describing andexplaining complex agricultural problems and exploring and de-signing solutions is unlikely to be successful if the different problemdimensions are analysed and treated separately (Hall and Clark 2010Spielman et al 2009)

Data gathering across different levels (national region and dis-trict level) enabled the analysis of the interactions and (mis)matchesbetween different levels (Cash et al 2006) An example that emergedduring the workshops and the interviews is Tanzaniarsquos nationalexport ban that prohibits export of agricultural produce (eg of rice)as long as the country has not been declared lsquofood securersquo This na-tional export ban influences local market prices and consequentlyalso farmersrsquo willingness and ability to invest in for example pur-chasing agricultural inputs such as fertilizers and seeds (eg Poultonet al 2010) This in turn provided an opportunity to identify entrypoints for innovation across different levels which has been iden-tified as a critical factor for addressing complex agricultural problems(eg Giller et al 2008 2011) As expected and confirming previ-ous reports (eg van Mierlo et al 2010) the participatory analysisof multi-level interactions showed that stakeholders (insiders) oftenidentify constraints at the level they represent (Schut et al 2014c)This was complemented by our analysis as researchers (outsiders)of the multi-level interactions regarding the parasitic weedproblems

Photo 1ndash4 Photo 1 (top left) Top-5 of constraints of NGO civil society representatives and their categorisation under the different components of the institutional sub-system (Session 4) Photo 2 (top right) The categorisation of the top-5 of the different stakeholder groups along different structural conditions that can enable or constraininnovation (Session 5) Photo 3 (bottom left) The identification of relationships between different constraints (arrows) and key problem (circled cards) (Session 7) Photo4 (bottom right) The categorisation of the top-5 of the different stakeholder groups along a four-step gradient ranging from structural to operational constraints (Session12) Photos were taken by M Schut during multi-stakeholder workshops in Tanzania held in October 2012

8 M Schut et alAgricultural Systems 132 (2015) 1ndash11

The involvement of different groups of stakeholders was essen-tial for enhancing the credibility validity and quality of RAAIS aswell as for delineating the boundaries of the agricultural system andthe agricultural innovation support system which is considered akey challenge when using AIS approaches to analyse complex ag-ricultural problems (Klerkx et al 2012b) Furthermore stakeholderparticipation provided a better understanding of the feasibility andacceptability of solutions for stakeholder groups Although we believethat the stakeholder groups included in the testing of RAAIS (Table 1)provide a good starting point other stakeholder groups (for in-stance the media) may be included in the sample (eg Ortiz et al2013) depending on the type of complex agricultural problem underreview The triangulation of data resulting from the different methodsenabled us to validate findings and to verify strategic communi-cation by stakeholders for instance to verify how the extensionsystem as described by policymakers in interviews functioned inreality according to surveyed farmers

42 RAAISrsquo ability to provide generic entry points for innovation

RAAIS demonstrates interactions between complex agricultur-al problems innovation capacity of the agricultural system ndashconsisting of institutional sectoral and technological subsystemsndash and the agricultural innovation support system For example ap-plying fertilizer (technological subsystem) in rain-fed rice productionis seen as a promising management strategy to reduce infection levelsof Rhamphicarpa one of the parasitic weeds involved in the studyand mitigate negative effects of the parasite on rice yields (Rodenburget al 2011) However as was highlighted during the RAAIS work-shops in both in Benin and in Tanzania fertilizers are difficult toaccess in rural areas In Benin there is no well-developed privateagro-dealer network and distribution infrastructure to support thesupply of agricultural inputs Furthermore interviews showed thatthe public extension and input supply systems in Benin focus onthe cotton sector rather than on cereal crops (sectoral subsys-tems) In Tanzania a private agro-dealer network and distributioninfrastructure exists but structures controlling the quality of fer-tilizers (institutional subsystem) are functioning sub-optimallyaccording to interviewed government officials In some areas fakeagro inputs are dominating the market resulting in a limited trustand willingness to invest in applying fertilizer according to farmerrepresentatives who participated in the workshops The exampleshows how the absence or poor performance of fertilizer distribu-tion infrastructure limited farmer-extensionist interaction and lackof functional institutions for quality control (being structural con-ditions for innovation) constrain the innovation capacity in theagricultural systems and its technological (in this case fertilizer) andsectoral (the rice value chain) subsystems Another example is basedon secondary data analyses that demonstrated the lack of an op-erational strategy to address parasitic weeds in Tanzania and BeninIn both the interviews and workshops stakeholders highlighted thegeneral lack of interaction and collaboration between stakehold-ers in the agricultural sector (being a structural condition forinnovation) as one of the main reasons for the absence or poor im-plementation of parasitic weed and other agricultural policies andstrategies

The aforementioned examples demonstrate how RAAIS cansupport the identification of generic entry points for innovation Suchinnovations can directly contribute to addressing the complex ag-ricultural problem under review but can also have a spill-over effectin terms of addressing broader constraints that hamper the inno-vation capacity in the agricultural system For example the lack ofstakeholder interactions and collaboration in the agricultural systemcan provide an entry point for the adaptation of the structuralconditions in the broader agricultural innovation support systemfor example through investments in innovation brokers or

multi-stakeholder platforms (Kilelu et al 2013 Klerkx et al 2010)Such structural adjustments can facilitate multi-stakeholder col-laboration in tackling parasitic weed as well as other complexagricultural problems

43 Lessons learnt from applying RAAIS and recommendations forfurther improvement

Based on our experiences in Tanzania and Benin we recom-mend conducting RAAIS in an interdisciplinary team of researcherswith expertise on different dimensions of complex agricultural prob-lems and on different data collection methods (Hulsebosch 2001)Other suggestions include the experimentation with other combi-nations of methods and on different types of complex agriculturalproblems The workshop methodology could be made more inter-active in the sense of directly feeding back results of the sessionsto participants to stimulate reflection and validate analyses duringthe workshops Post-workshop surveys could provide additionalinsight into whether stakeholders felt they could freely raise anddiscuss their ideas and needs

The multi-stakeholder workshops but also the surveys pre-sented a rather static picture of the complex agricultural problemunder review and the innovation capacity of the agricultural systemin which the problem is embedded However initial workshops andsurveys could function as a baseline to which future workshops andsurveys can be compared Other methods such as secondary dataanalysis or in-depth interviews present a more dynamic image ofhow for example collaborations between stakeholders evolve overthe years Our experiences in Tanzania and Benin show that ensur-ing social differentiation among workshop participants intervieweesand survey respondents (eg of different gender of age) was chal-lenging as for example the majority of workshop participants weremale Specific Workshop Sessions could have more attention for cat-egorisation and priority setting by different gender or age groupsThe facilitation of the multi-stakeholder workshops ensured thatdifferent stakeholder groups could raise and discuss their ideas(Hulsebosch 2001) Despite such efforts unequal power relationsand differences in the ability to debate and negotiate that inher-ently exist between groups may have played a role In line with ourexpectations politically sensitive issues were more freely dis-cussed in individual interviews as compared to multi-stakeholdersetting

The combination of different methods of data collection wasessential In terms of the sequence of data collection we recom-mend to first conduct and analyse the RAAIS multi-stakeholderworkshops to identify constraints and subsequently conduct thein-depth interviews and surveys that can provide more insight inthe distribution and underlying root causes of these constraintsThe workshops then provide a lsquofast-trackrsquo approach to identifyingentry points for innovation that can subsequently be validated andexplored in more detail using the in-depth interviews and stake-holder surveys This would furthermore increase the lsquorapidnessrsquoof RAAIS as a diagnostic tool

An updated version of the RAAIS multi-stakeholder workshopshas been used to identify constraints challenges and entry pointsfor innovations related to the lsquosustainable intensification of agri-cultural systemsrsquo in Burundi the Democratic Republic of CongoRwanda Nigeria and Cameroon under the CGIAR Research Pro-gramme for the Humid Tropics (Humidtropics) (Schut and Hinnou2014) Several of the recommendations made in this paper includ-ing the revised sequence of methods for data collection and the useof post-workshop participant questionnaires have been imple-mented and tested successfully Some of the bottlenecks identifiedsuch as social differentiation (eg gender and age groups) amongworkshop participants remained problematic and require furtherattention At the end of the Humidtropics RAAIS workshops

9M Schut et alAgricultural Systems 132 (2015) 1ndash11

participants developed action plans to address the prioritised con-straints (Workshop Session 13) This required an extension of theworkshops of half a day The development and implementation ofthe action plans forms an important element for continued stake-holder collaboration in multi-stakeholder platforms

5 Conclusions

This paper demonstrates the potential of RAAIS as a diagnostictool that can support and guide the integrated analysis of complexagricultural problems innovation capacity in the agricultural systemand the performance of the agricultural innovation support systemRAAIS combines multiple qualitative and quantitative methods andinsider (stakeholders) and outsider (researchers) analyses whichallow for critical triangulation and validation of the gathered dataSuch an analysis can provide specific entry points for innovationsto address the complex agricultural problem under study and genericentry points for innovation related to strengthening the innova-tion capacity of agricultural system and the functioning of theagricultural innovation support system

Recommendations for further improvement include using RAAISfor the analysis of other types of complex agricultural problems usingother combinations of methods of data collection and providing di-rectly feedback to workshop participants to stimulate reflection andvalidate workshop outcomes An adapted sequence of data collec-tion methods in which workshops provide a lsquofast-trackrsquo approachto identifying entry points for innovation followed up by more in-depth interviews and stakeholder surveys would increase the RAAISrsquodiagnostic capacity The participatory development of concrete actionplans based on RAAIS can provide a basis for continued multi-stakeholder collaboration to operationalise and implement specificand generic entry points for innovation

Acknowledgements

This research forms part of the PARASITE programme fundedby the Integrated Programmes Scheme of the NetherlandsOrganisation for Scientific Research ndash Science for Global Develop-ment (NWO-WOTRO) (wwwparasite-projectorg) Additional supportis provided by the CGIAR Research Program on Climate Change Ag-riculture and Food Security (CCAFS) The CGIAR Research Programmefor the Humid Tropics (httphumidtropicscgiarorg) provided theopportunity to further adapt apply and improve RAAIS across dif-ferent Humidtropics Action Areas Special thanks to Elifadhili Danielwho conducted the farmer-extensionist survey

References

Amankwah K Klerkx L Oosting SJ Sakyi-Dawson O van der Zijpp AJ MillarD 2012 Diagnosing constraints to market participation of small ruminantproducers in northern Ghana an innovation systems analysis NJAS-Wagen JLife Sc 60ndash63 37ndash47

Basu S Leeuwis C 2012 Understanding the rapid spread of System of RiceIntensification (SRI) in Andhra Pradesh exploring the building of supportnetworks and media representation Agr Syst 111 34ndash44

Beintema N Stads G-J Fuglie K Heisey P 2012 ASTI global assessment ofagricultural RampD spending International Food Policy Research InstituteWashington DC Agricultural Science and Technology Indicators Rome Italy andthe Global Forum on Agricultural Research Rome Italy

Blay-Palmer A 2005 Growing innovation policy the case of organic agriculturein Ontario Canada Environ Plann C Gov Policy 23 557ndash581

Carlsson B Stankiewicz R 1991 On the nature function and composition oftechnological systems J Evol Econ 1 93ndash118

Carlsson B Jacobsson S Holmeacuten M Rickne A 2002 Innovation systems analyticaland methodological issues Res Policy 31 233ndash245

Cash DW Adger WN Berkes F Garden P Lebel L Olsson P et al 2006 Scaleand cross-scale dynamics governance and information in a multilevel worldEcol Soc 11 8 Available from lthttpwwwecologyandsocietyorgvol11iss12art18gt [online]

Chambers R 1994 The origins and practice of participatory rural appraisal WorldDev 22 953ndash969

Chung CC 2012 National sectoral and technological innovation systems the caseof Taiwanese pharmaceutical biotechnology and agricultural biotechnologyinnovation systems (1945ndash2000) Sci Public Policy 39 271ndash281

Cooke P Uranga MG Etxebarria G 1997 Regional innovation systems Institutionaland organisational dimensions Res Policy 26 475ndash491

Daniel E 2013 Assessment of agricultural extension services in Tanzania A casestudy of Kyela Songea Rural and Morogoro Rural Districts Wageningen Universityand Research CentreAfrica Rice Center Wageningen the NetherlandsDar esSalaam Tanzania p 45

Douthwaite B Kuby T van de Fliert E Schulz S 2003 Impact pathway evaluationan approach for achieving and attributing impact in complex systems Agr Syst78 243ndash265

Engel PGH 1995 Facilitating Innovation An Action-Oriented Approach andParticipatory Methodology to Improve Innovative Social Practice in AgricultureWageningen University Wageningen the Netherlands p 300

Faysse N 2006 Troubles on the way an analysis of the challenges faced bymulti-stakeholder platforms Nat Resour Forum 30 219ndash229

Freeman C 1988 Japan a new national innovation system In Dosi G FreemanC Nelson R Soete T (Eds) Technology and Economic Theory London pp330ndash348

Freeman C 1995 The national system of innovation in historical perspective CambJ Econ 19 5ndash24

Funtowicz SO Ravetz JR 1993 Science for the post-normal age Futures 25739ndash755

Gildemacher PR Kaguongo W Ortiz O Tesfaye A Woldegiorgis G Wagoire WWet al 2009 Improving potato production in Kenya Uganda and Ethiopia a systemdiagnosis Potato Res 52 173ndash205

Giller KE Leeuwis C Andersson JA Andriesse W Brouwer A Frost P et al2008 Competing claims on natural resources what role for science Ecol Soc13 34 Available from lthttpwwwecologyandsocietyorgvol13iss32art34gt[online]

Giller KE Tittonell P Rufino MC van Wijk MT Zingore S Mapfumo P et al2011 Communicating complexity integrated assessment of trade-offs concerningsoil fertility management within African farming systems to support innovationand development Agr Syst 104 191ndash203

Guest G Bunce A Johnson L 2006 How many interviews are enough Anexperiment with data saturation and variability Field Methods 18 59ndash82

Hall A 2005 Capacity development for agricultural biotechnology in developingcountries an innovation systems view of what it is and how to develop it J IntDev 17 611ndash630

Hall A Clark N 2010 What do complex adaptive systems look like and what arethe implications for innovation policy J Int Dev 22 308ndash324

Hall A Rasheed Sulaiman V Clark N Yoganand B 2003 From measuring impactto learning institutional lessons an innovation systems perspective on improvingthe management of international agricultural research Agr Syst 78 213ndash241

Hall AJ Yoganand B Sulaiman RV Raina RS Prasad CS Naik GC et al 2004Innovations in innovation reflections on partnership institutions and learningICRISAT Andra Pradesh

Hekkert MP Suurs RAA Negro SO Kuhlmann S Smits REHM 2007 Functionsof innovation systems a new approach for analysing technological changeTechnol Forecast Soc Change 74 413ndash432

Hermans F Stuiver M Beers PJ Kok K 2013 The distribution of roles andfunctions for upscaling and outscaling innovations in agricultural innovationsystems Agr Syst 115 117ndash128

Hounkonnou D Kossou D Kuyper TW Leeuwis C Nederlof ES Roumlling N et al2012 An innovation systems approach to institutional change smallholderdevelopment in West Africa Agr Syst 108 74ndash83

Hulsebosch J 2001 The use of RAAKS for strengthening community-basedorganisations in Mali Dev Pract 11 622ndash632

Hung S-C Whittington R 2011 Agency in national innovation systems institutionalentrepreneurship and the professionalization of Taiwanese IT Res Policy 40526ndash538

International Institute for Sustainable Development 2014 Rapid Rural Appraisal(RRA)

Jiggins J 2012 Diagnosing the scope for innovation linking smallholder practicesand institutional context NJAS-Wagen J Life Sc 60ndash63 1ndash122

Kilelu CW Klerkx L Leeuwis C 2013 Unravelling the role of innovation platformsin supporting co-evolution of innovation contributions and tensions in asmallholder dairy development programme Agr Syst 118 65ndash77

Klein Woolthuis R Lankhuizen M Gilsing V 2005 A system failure frameworkfor innovation policy design Technovation 25 609ndash619

Klerkx L Aarts N Leeuwis C 2010 Adaptive management in agricultural innovationsystems the interactions between innovation networks and their environmentAgr Syst 103 390ndash400

Klerkx L Schut M Leeuwis C Kilelu C 2012a Advances in knowledge brokeringin the agricultural sector towards innovation system facilitation IDS Bull 4353ndash60

Klerkx L van Mierlo B Leeuwis C 2012b Evolution of systems approaches toagricultural innovation Concepts analysis and interventions In Darnhofer IGibbon D Dedieu B (Eds) Farming Systems Research into the 21st CenturyThe New Dynamic Springer Dordrecht pp 457ndash483

Leeuwis C 2000 Reconceptualizing participation for sustainable rural developmenttowards a negotiation approach Dev Change 31 931ndash959

Leeuwis C 2004 Communication for Rural Innovation Rethinking AgriculturalExtension (with Contributions of Anne van den Ban) Blackwell Science Oxford

10 M Schut et alAgricultural Systems 132 (2015) 1ndash11

by the multi-stakeholder workshops In Tanzania both the socio-economic farmer survey and the farmer-extensionist survey wereheld after the interviews and workshops In Benin the socio-economic farmer survey was held preceding the in-depth interviewsand workshops Secondary data collection occurred throughout thefieldwork Below we will further reflect on the main objectives ofRAAIS as well as provide recommendations for further improve-ments and use of RAAIS using our experiences from Tanzania andBenin

41 RAAISrsquo ability to provide specific entry points for innovation toaddress complex agricultural problems

RAAIS contributed to an integrated understanding of differentproblem dimensions multi-level interactions and multi-stakeholderdynamics related to parasitic weed problems With regard to thedifferent problem dimensions interviews demonstrated a poten-tial relation between for example the preference for growing localaromatic rice varieties (social-cultural dimension) the low capac-ity of farmer to purchase certified seeds (economic dimension) andthe spread of parasitic weed seeds through the local rice seed system(technological dimension) Additionally analysis of workshop datarevealed how the untimely and insufficient availability of agricul-tural inputs provided by the government (institutional dimension)and limited interaction and collaboration among networks of key

stakeholders (political dimensions) form additional bottlenecks foraddressing such problems It created awareness that describing andexplaining complex agricultural problems and exploring and de-signing solutions is unlikely to be successful if the different problemdimensions are analysed and treated separately (Hall and Clark 2010Spielman et al 2009)

Data gathering across different levels (national region and dis-trict level) enabled the analysis of the interactions and (mis)matchesbetween different levels (Cash et al 2006) An example that emergedduring the workshops and the interviews is Tanzaniarsquos nationalexport ban that prohibits export of agricultural produce (eg of rice)as long as the country has not been declared lsquofood securersquo This na-tional export ban influences local market prices and consequentlyalso farmersrsquo willingness and ability to invest in for example pur-chasing agricultural inputs such as fertilizers and seeds (eg Poultonet al 2010) This in turn provided an opportunity to identify entrypoints for innovation across different levels which has been iden-tified as a critical factor for addressing complex agricultural problems(eg Giller et al 2008 2011) As expected and confirming previ-ous reports (eg van Mierlo et al 2010) the participatory analysisof multi-level interactions showed that stakeholders (insiders) oftenidentify constraints at the level they represent (Schut et al 2014c)This was complemented by our analysis as researchers (outsiders)of the multi-level interactions regarding the parasitic weedproblems

Photo 1ndash4 Photo 1 (top left) Top-5 of constraints of NGO civil society representatives and their categorisation under the different components of the institutional sub-system (Session 4) Photo 2 (top right) The categorisation of the top-5 of the different stakeholder groups along different structural conditions that can enable or constraininnovation (Session 5) Photo 3 (bottom left) The identification of relationships between different constraints (arrows) and key problem (circled cards) (Session 7) Photo4 (bottom right) The categorisation of the top-5 of the different stakeholder groups along a four-step gradient ranging from structural to operational constraints (Session12) Photos were taken by M Schut during multi-stakeholder workshops in Tanzania held in October 2012

8 M Schut et alAgricultural Systems 132 (2015) 1ndash11

The involvement of different groups of stakeholders was essen-tial for enhancing the credibility validity and quality of RAAIS aswell as for delineating the boundaries of the agricultural system andthe agricultural innovation support system which is considered akey challenge when using AIS approaches to analyse complex ag-ricultural problems (Klerkx et al 2012b) Furthermore stakeholderparticipation provided a better understanding of the feasibility andacceptability of solutions for stakeholder groups Although we believethat the stakeholder groups included in the testing of RAAIS (Table 1)provide a good starting point other stakeholder groups (for in-stance the media) may be included in the sample (eg Ortiz et al2013) depending on the type of complex agricultural problem underreview The triangulation of data resulting from the different methodsenabled us to validate findings and to verify strategic communi-cation by stakeholders for instance to verify how the extensionsystem as described by policymakers in interviews functioned inreality according to surveyed farmers

42 RAAISrsquo ability to provide generic entry points for innovation

RAAIS demonstrates interactions between complex agricultur-al problems innovation capacity of the agricultural system ndashconsisting of institutional sectoral and technological subsystemsndash and the agricultural innovation support system For example ap-plying fertilizer (technological subsystem) in rain-fed rice productionis seen as a promising management strategy to reduce infection levelsof Rhamphicarpa one of the parasitic weeds involved in the studyand mitigate negative effects of the parasite on rice yields (Rodenburget al 2011) However as was highlighted during the RAAIS work-shops in both in Benin and in Tanzania fertilizers are difficult toaccess in rural areas In Benin there is no well-developed privateagro-dealer network and distribution infrastructure to support thesupply of agricultural inputs Furthermore interviews showed thatthe public extension and input supply systems in Benin focus onthe cotton sector rather than on cereal crops (sectoral subsys-tems) In Tanzania a private agro-dealer network and distributioninfrastructure exists but structures controlling the quality of fer-tilizers (institutional subsystem) are functioning sub-optimallyaccording to interviewed government officials In some areas fakeagro inputs are dominating the market resulting in a limited trustand willingness to invest in applying fertilizer according to farmerrepresentatives who participated in the workshops The exampleshows how the absence or poor performance of fertilizer distribu-tion infrastructure limited farmer-extensionist interaction and lackof functional institutions for quality control (being structural con-ditions for innovation) constrain the innovation capacity in theagricultural systems and its technological (in this case fertilizer) andsectoral (the rice value chain) subsystems Another example is basedon secondary data analyses that demonstrated the lack of an op-erational strategy to address parasitic weeds in Tanzania and BeninIn both the interviews and workshops stakeholders highlighted thegeneral lack of interaction and collaboration between stakehold-ers in the agricultural sector (being a structural condition forinnovation) as one of the main reasons for the absence or poor im-plementation of parasitic weed and other agricultural policies andstrategies

The aforementioned examples demonstrate how RAAIS cansupport the identification of generic entry points for innovation Suchinnovations can directly contribute to addressing the complex ag-ricultural problem under review but can also have a spill-over effectin terms of addressing broader constraints that hamper the inno-vation capacity in the agricultural system For example the lack ofstakeholder interactions and collaboration in the agricultural systemcan provide an entry point for the adaptation of the structuralconditions in the broader agricultural innovation support systemfor example through investments in innovation brokers or

multi-stakeholder platforms (Kilelu et al 2013 Klerkx et al 2010)Such structural adjustments can facilitate multi-stakeholder col-laboration in tackling parasitic weed as well as other complexagricultural problems

43 Lessons learnt from applying RAAIS and recommendations forfurther improvement

Based on our experiences in Tanzania and Benin we recom-mend conducting RAAIS in an interdisciplinary team of researcherswith expertise on different dimensions of complex agricultural prob-lems and on different data collection methods (Hulsebosch 2001)Other suggestions include the experimentation with other combi-nations of methods and on different types of complex agriculturalproblems The workshop methodology could be made more inter-active in the sense of directly feeding back results of the sessionsto participants to stimulate reflection and validate analyses duringthe workshops Post-workshop surveys could provide additionalinsight into whether stakeholders felt they could freely raise anddiscuss their ideas and needs

The multi-stakeholder workshops but also the surveys pre-sented a rather static picture of the complex agricultural problemunder review and the innovation capacity of the agricultural systemin which the problem is embedded However initial workshops andsurveys could function as a baseline to which future workshops andsurveys can be compared Other methods such as secondary dataanalysis or in-depth interviews present a more dynamic image ofhow for example collaborations between stakeholders evolve overthe years Our experiences in Tanzania and Benin show that ensur-ing social differentiation among workshop participants intervieweesand survey respondents (eg of different gender of age) was chal-lenging as for example the majority of workshop participants weremale Specific Workshop Sessions could have more attention for cat-egorisation and priority setting by different gender or age groupsThe facilitation of the multi-stakeholder workshops ensured thatdifferent stakeholder groups could raise and discuss their ideas(Hulsebosch 2001) Despite such efforts unequal power relationsand differences in the ability to debate and negotiate that inher-ently exist between groups may have played a role In line with ourexpectations politically sensitive issues were more freely dis-cussed in individual interviews as compared to multi-stakeholdersetting

The combination of different methods of data collection wasessential In terms of the sequence of data collection we recom-mend to first conduct and analyse the RAAIS multi-stakeholderworkshops to identify constraints and subsequently conduct thein-depth interviews and surveys that can provide more insight inthe distribution and underlying root causes of these constraintsThe workshops then provide a lsquofast-trackrsquo approach to identifyingentry points for innovation that can subsequently be validated andexplored in more detail using the in-depth interviews and stake-holder surveys This would furthermore increase the lsquorapidnessrsquoof RAAIS as a diagnostic tool

An updated version of the RAAIS multi-stakeholder workshopshas been used to identify constraints challenges and entry pointsfor innovations related to the lsquosustainable intensification of agri-cultural systemsrsquo in Burundi the Democratic Republic of CongoRwanda Nigeria and Cameroon under the CGIAR Research Pro-gramme for the Humid Tropics (Humidtropics) (Schut and Hinnou2014) Several of the recommendations made in this paper includ-ing the revised sequence of methods for data collection and the useof post-workshop participant questionnaires have been imple-mented and tested successfully Some of the bottlenecks identifiedsuch as social differentiation (eg gender and age groups) amongworkshop participants remained problematic and require furtherattention At the end of the Humidtropics RAAIS workshops

9M Schut et alAgricultural Systems 132 (2015) 1ndash11

participants developed action plans to address the prioritised con-straints (Workshop Session 13) This required an extension of theworkshops of half a day The development and implementation ofthe action plans forms an important element for continued stake-holder collaboration in multi-stakeholder platforms

5 Conclusions

This paper demonstrates the potential of RAAIS as a diagnostictool that can support and guide the integrated analysis of complexagricultural problems innovation capacity in the agricultural systemand the performance of the agricultural innovation support systemRAAIS combines multiple qualitative and quantitative methods andinsider (stakeholders) and outsider (researchers) analyses whichallow for critical triangulation and validation of the gathered dataSuch an analysis can provide specific entry points for innovationsto address the complex agricultural problem under study and genericentry points for innovation related to strengthening the innova-tion capacity of agricultural system and the functioning of theagricultural innovation support system

Recommendations for further improvement include using RAAISfor the analysis of other types of complex agricultural problems usingother combinations of methods of data collection and providing di-rectly feedback to workshop participants to stimulate reflection andvalidate workshop outcomes An adapted sequence of data collec-tion methods in which workshops provide a lsquofast-trackrsquo approachto identifying entry points for innovation followed up by more in-depth interviews and stakeholder surveys would increase the RAAISrsquodiagnostic capacity The participatory development of concrete actionplans based on RAAIS can provide a basis for continued multi-stakeholder collaboration to operationalise and implement specificand generic entry points for innovation

Acknowledgements

This research forms part of the PARASITE programme fundedby the Integrated Programmes Scheme of the NetherlandsOrganisation for Scientific Research ndash Science for Global Develop-ment (NWO-WOTRO) (wwwparasite-projectorg) Additional supportis provided by the CGIAR Research Program on Climate Change Ag-riculture and Food Security (CCAFS) The CGIAR Research Programmefor the Humid Tropics (httphumidtropicscgiarorg) provided theopportunity to further adapt apply and improve RAAIS across dif-ferent Humidtropics Action Areas Special thanks to Elifadhili Danielwho conducted the farmer-extensionist survey

References

Amankwah K Klerkx L Oosting SJ Sakyi-Dawson O van der Zijpp AJ MillarD 2012 Diagnosing constraints to market participation of small ruminantproducers in northern Ghana an innovation systems analysis NJAS-Wagen JLife Sc 60ndash63 37ndash47

Basu S Leeuwis C 2012 Understanding the rapid spread of System of RiceIntensification (SRI) in Andhra Pradesh exploring the building of supportnetworks and media representation Agr Syst 111 34ndash44

Beintema N Stads G-J Fuglie K Heisey P 2012 ASTI global assessment ofagricultural RampD spending International Food Policy Research InstituteWashington DC Agricultural Science and Technology Indicators Rome Italy andthe Global Forum on Agricultural Research Rome Italy

Blay-Palmer A 2005 Growing innovation policy the case of organic agriculturein Ontario Canada Environ Plann C Gov Policy 23 557ndash581

Carlsson B Stankiewicz R 1991 On the nature function and composition oftechnological systems J Evol Econ 1 93ndash118

Carlsson B Jacobsson S Holmeacuten M Rickne A 2002 Innovation systems analyticaland methodological issues Res Policy 31 233ndash245

Cash DW Adger WN Berkes F Garden P Lebel L Olsson P et al 2006 Scaleand cross-scale dynamics governance and information in a multilevel worldEcol Soc 11 8 Available from lthttpwwwecologyandsocietyorgvol11iss12art18gt [online]

Chambers R 1994 The origins and practice of participatory rural appraisal WorldDev 22 953ndash969

Chung CC 2012 National sectoral and technological innovation systems the caseof Taiwanese pharmaceutical biotechnology and agricultural biotechnologyinnovation systems (1945ndash2000) Sci Public Policy 39 271ndash281

Cooke P Uranga MG Etxebarria G 1997 Regional innovation systems Institutionaland organisational dimensions Res Policy 26 475ndash491

Daniel E 2013 Assessment of agricultural extension services in Tanzania A casestudy of Kyela Songea Rural and Morogoro Rural Districts Wageningen Universityand Research CentreAfrica Rice Center Wageningen the NetherlandsDar esSalaam Tanzania p 45

Douthwaite B Kuby T van de Fliert E Schulz S 2003 Impact pathway evaluationan approach for achieving and attributing impact in complex systems Agr Syst78 243ndash265

Engel PGH 1995 Facilitating Innovation An Action-Oriented Approach andParticipatory Methodology to Improve Innovative Social Practice in AgricultureWageningen University Wageningen the Netherlands p 300

Faysse N 2006 Troubles on the way an analysis of the challenges faced bymulti-stakeholder platforms Nat Resour Forum 30 219ndash229

Freeman C 1988 Japan a new national innovation system In Dosi G FreemanC Nelson R Soete T (Eds) Technology and Economic Theory London pp330ndash348

Freeman C 1995 The national system of innovation in historical perspective CambJ Econ 19 5ndash24

Funtowicz SO Ravetz JR 1993 Science for the post-normal age Futures 25739ndash755

Gildemacher PR Kaguongo W Ortiz O Tesfaye A Woldegiorgis G Wagoire WWet al 2009 Improving potato production in Kenya Uganda and Ethiopia a systemdiagnosis Potato Res 52 173ndash205

Giller KE Leeuwis C Andersson JA Andriesse W Brouwer A Frost P et al2008 Competing claims on natural resources what role for science Ecol Soc13 34 Available from lthttpwwwecologyandsocietyorgvol13iss32art34gt[online]

Giller KE Tittonell P Rufino MC van Wijk MT Zingore S Mapfumo P et al2011 Communicating complexity integrated assessment of trade-offs concerningsoil fertility management within African farming systems to support innovationand development Agr Syst 104 191ndash203

Guest G Bunce A Johnson L 2006 How many interviews are enough Anexperiment with data saturation and variability Field Methods 18 59ndash82

Hall A 2005 Capacity development for agricultural biotechnology in developingcountries an innovation systems view of what it is and how to develop it J IntDev 17 611ndash630

Hall A Clark N 2010 What do complex adaptive systems look like and what arethe implications for innovation policy J Int Dev 22 308ndash324

Hall A Rasheed Sulaiman V Clark N Yoganand B 2003 From measuring impactto learning institutional lessons an innovation systems perspective on improvingthe management of international agricultural research Agr Syst 78 213ndash241

Hall AJ Yoganand B Sulaiman RV Raina RS Prasad CS Naik GC et al 2004Innovations in innovation reflections on partnership institutions and learningICRISAT Andra Pradesh

Hekkert MP Suurs RAA Negro SO Kuhlmann S Smits REHM 2007 Functionsof innovation systems a new approach for analysing technological changeTechnol Forecast Soc Change 74 413ndash432

Hermans F Stuiver M Beers PJ Kok K 2013 The distribution of roles andfunctions for upscaling and outscaling innovations in agricultural innovationsystems Agr Syst 115 117ndash128

Hounkonnou D Kossou D Kuyper TW Leeuwis C Nederlof ES Roumlling N et al2012 An innovation systems approach to institutional change smallholderdevelopment in West Africa Agr Syst 108 74ndash83

Hulsebosch J 2001 The use of RAAKS for strengthening community-basedorganisations in Mali Dev Pract 11 622ndash632

Hung S-C Whittington R 2011 Agency in national innovation systems institutionalentrepreneurship and the professionalization of Taiwanese IT Res Policy 40526ndash538

International Institute for Sustainable Development 2014 Rapid Rural Appraisal(RRA)

Jiggins J 2012 Diagnosing the scope for innovation linking smallholder practicesand institutional context NJAS-Wagen J Life Sc 60ndash63 1ndash122

Kilelu CW Klerkx L Leeuwis C 2013 Unravelling the role of innovation platformsin supporting co-evolution of innovation contributions and tensions in asmallholder dairy development programme Agr Syst 118 65ndash77

Klein Woolthuis R Lankhuizen M Gilsing V 2005 A system failure frameworkfor innovation policy design Technovation 25 609ndash619

Klerkx L Aarts N Leeuwis C 2010 Adaptive management in agricultural innovationsystems the interactions between innovation networks and their environmentAgr Syst 103 390ndash400

Klerkx L Schut M Leeuwis C Kilelu C 2012a Advances in knowledge brokeringin the agricultural sector towards innovation system facilitation IDS Bull 4353ndash60

Klerkx L van Mierlo B Leeuwis C 2012b Evolution of systems approaches toagricultural innovation Concepts analysis and interventions In Darnhofer IGibbon D Dedieu B (Eds) Farming Systems Research into the 21st CenturyThe New Dynamic Springer Dordrecht pp 457ndash483

Leeuwis C 2000 Reconceptualizing participation for sustainable rural developmenttowards a negotiation approach Dev Change 31 931ndash959

Leeuwis C 2004 Communication for Rural Innovation Rethinking AgriculturalExtension (with Contributions of Anne van den Ban) Blackwell Science Oxford

10 M Schut et alAgricultural Systems 132 (2015) 1ndash11

The involvement of different groups of stakeholders was essen-tial for enhancing the credibility validity and quality of RAAIS aswell as for delineating the boundaries of the agricultural system andthe agricultural innovation support system which is considered akey challenge when using AIS approaches to analyse complex ag-ricultural problems (Klerkx et al 2012b) Furthermore stakeholderparticipation provided a better understanding of the feasibility andacceptability of solutions for stakeholder groups Although we believethat the stakeholder groups included in the testing of RAAIS (Table 1)provide a good starting point other stakeholder groups (for in-stance the media) may be included in the sample (eg Ortiz et al2013) depending on the type of complex agricultural problem underreview The triangulation of data resulting from the different methodsenabled us to validate findings and to verify strategic communi-cation by stakeholders for instance to verify how the extensionsystem as described by policymakers in interviews functioned inreality according to surveyed farmers

42 RAAISrsquo ability to provide generic entry points for innovation

RAAIS demonstrates interactions between complex agricultur-al problems innovation capacity of the agricultural system ndashconsisting of institutional sectoral and technological subsystemsndash and the agricultural innovation support system For example ap-plying fertilizer (technological subsystem) in rain-fed rice productionis seen as a promising management strategy to reduce infection levelsof Rhamphicarpa one of the parasitic weeds involved in the studyand mitigate negative effects of the parasite on rice yields (Rodenburget al 2011) However as was highlighted during the RAAIS work-shops in both in Benin and in Tanzania fertilizers are difficult toaccess in rural areas In Benin there is no well-developed privateagro-dealer network and distribution infrastructure to support thesupply of agricultural inputs Furthermore interviews showed thatthe public extension and input supply systems in Benin focus onthe cotton sector rather than on cereal crops (sectoral subsys-tems) In Tanzania a private agro-dealer network and distributioninfrastructure exists but structures controlling the quality of fer-tilizers (institutional subsystem) are functioning sub-optimallyaccording to interviewed government officials In some areas fakeagro inputs are dominating the market resulting in a limited trustand willingness to invest in applying fertilizer according to farmerrepresentatives who participated in the workshops The exampleshows how the absence or poor performance of fertilizer distribu-tion infrastructure limited farmer-extensionist interaction and lackof functional institutions for quality control (being structural con-ditions for innovation) constrain the innovation capacity in theagricultural systems and its technological (in this case fertilizer) andsectoral (the rice value chain) subsystems Another example is basedon secondary data analyses that demonstrated the lack of an op-erational strategy to address parasitic weeds in Tanzania and BeninIn both the interviews and workshops stakeholders highlighted thegeneral lack of interaction and collaboration between stakehold-ers in the agricultural sector (being a structural condition forinnovation) as one of the main reasons for the absence or poor im-plementation of parasitic weed and other agricultural policies andstrategies

The aforementioned examples demonstrate how RAAIS cansupport the identification of generic entry points for innovation Suchinnovations can directly contribute to addressing the complex ag-ricultural problem under review but can also have a spill-over effectin terms of addressing broader constraints that hamper the inno-vation capacity in the agricultural system For example the lack ofstakeholder interactions and collaboration in the agricultural systemcan provide an entry point for the adaptation of the structuralconditions in the broader agricultural innovation support systemfor example through investments in innovation brokers or

multi-stakeholder platforms (Kilelu et al 2013 Klerkx et al 2010)Such structural adjustments can facilitate multi-stakeholder col-laboration in tackling parasitic weed as well as other complexagricultural problems

43 Lessons learnt from applying RAAIS and recommendations forfurther improvement

Based on our experiences in Tanzania and Benin we recom-mend conducting RAAIS in an interdisciplinary team of researcherswith expertise on different dimensions of complex agricultural prob-lems and on different data collection methods (Hulsebosch 2001)Other suggestions include the experimentation with other combi-nations of methods and on different types of complex agriculturalproblems The workshop methodology could be made more inter-active in the sense of directly feeding back results of the sessionsto participants to stimulate reflection and validate analyses duringthe workshops Post-workshop surveys could provide additionalinsight into whether stakeholders felt they could freely raise anddiscuss their ideas and needs

The multi-stakeholder workshops but also the surveys pre-sented a rather static picture of the complex agricultural problemunder review and the innovation capacity of the agricultural systemin which the problem is embedded However initial workshops andsurveys could function as a baseline to which future workshops andsurveys can be compared Other methods such as secondary dataanalysis or in-depth interviews present a more dynamic image ofhow for example collaborations between stakeholders evolve overthe years Our experiences in Tanzania and Benin show that ensur-ing social differentiation among workshop participants intervieweesand survey respondents (eg of different gender of age) was chal-lenging as for example the majority of workshop participants weremale Specific Workshop Sessions could have more attention for cat-egorisation and priority setting by different gender or age groupsThe facilitation of the multi-stakeholder workshops ensured thatdifferent stakeholder groups could raise and discuss their ideas(Hulsebosch 2001) Despite such efforts unequal power relationsand differences in the ability to debate and negotiate that inher-ently exist between groups may have played a role In line with ourexpectations politically sensitive issues were more freely dis-cussed in individual interviews as compared to multi-stakeholdersetting

The combination of different methods of data collection wasessential In terms of the sequence of data collection we recom-mend to first conduct and analyse the RAAIS multi-stakeholderworkshops to identify constraints and subsequently conduct thein-depth interviews and surveys that can provide more insight inthe distribution and underlying root causes of these constraintsThe workshops then provide a lsquofast-trackrsquo approach to identifyingentry points for innovation that can subsequently be validated andexplored in more detail using the in-depth interviews and stake-holder surveys This would furthermore increase the lsquorapidnessrsquoof RAAIS as a diagnostic tool

An updated version of the RAAIS multi-stakeholder workshopshas been used to identify constraints challenges and entry pointsfor innovations related to the lsquosustainable intensification of agri-cultural systemsrsquo in Burundi the Democratic Republic of CongoRwanda Nigeria and Cameroon under the CGIAR Research Pro-gramme for the Humid Tropics (Humidtropics) (Schut and Hinnou2014) Several of the recommendations made in this paper includ-ing the revised sequence of methods for data collection and the useof post-workshop participant questionnaires have been imple-mented and tested successfully Some of the bottlenecks identifiedsuch as social differentiation (eg gender and age groups) amongworkshop participants remained problematic and require furtherattention At the end of the Humidtropics RAAIS workshops

9M Schut et alAgricultural Systems 132 (2015) 1ndash11

participants developed action plans to address the prioritised con-straints (Workshop Session 13) This required an extension of theworkshops of half a day The development and implementation ofthe action plans forms an important element for continued stake-holder collaboration in multi-stakeholder platforms

5 Conclusions

This paper demonstrates the potential of RAAIS as a diagnostictool that can support and guide the integrated analysis of complexagricultural problems innovation capacity in the agricultural systemand the performance of the agricultural innovation support systemRAAIS combines multiple qualitative and quantitative methods andinsider (stakeholders) and outsider (researchers) analyses whichallow for critical triangulation and validation of the gathered dataSuch an analysis can provide specific entry points for innovationsto address the complex agricultural problem under study and genericentry points for innovation related to strengthening the innova-tion capacity of agricultural system and the functioning of theagricultural innovation support system

Recommendations for further improvement include using RAAISfor the analysis of other types of complex agricultural problems usingother combinations of methods of data collection and providing di-rectly feedback to workshop participants to stimulate reflection andvalidate workshop outcomes An adapted sequence of data collec-tion methods in which workshops provide a lsquofast-trackrsquo approachto identifying entry points for innovation followed up by more in-depth interviews and stakeholder surveys would increase the RAAISrsquodiagnostic capacity The participatory development of concrete actionplans based on RAAIS can provide a basis for continued multi-stakeholder collaboration to operationalise and implement specificand generic entry points for innovation

Acknowledgements

This research forms part of the PARASITE programme fundedby the Integrated Programmes Scheme of the NetherlandsOrganisation for Scientific Research ndash Science for Global Develop-ment (NWO-WOTRO) (wwwparasite-projectorg) Additional supportis provided by the CGIAR Research Program on Climate Change Ag-riculture and Food Security (CCAFS) The CGIAR Research Programmefor the Humid Tropics (httphumidtropicscgiarorg) provided theopportunity to further adapt apply and improve RAAIS across dif-ferent Humidtropics Action Areas Special thanks to Elifadhili Danielwho conducted the farmer-extensionist survey

References

Amankwah K Klerkx L Oosting SJ Sakyi-Dawson O van der Zijpp AJ MillarD 2012 Diagnosing constraints to market participation of small ruminantproducers in northern Ghana an innovation systems analysis NJAS-Wagen JLife Sc 60ndash63 37ndash47

Basu S Leeuwis C 2012 Understanding the rapid spread of System of RiceIntensification (SRI) in Andhra Pradesh exploring the building of supportnetworks and media representation Agr Syst 111 34ndash44

Beintema N Stads G-J Fuglie K Heisey P 2012 ASTI global assessment ofagricultural RampD spending International Food Policy Research InstituteWashington DC Agricultural Science and Technology Indicators Rome Italy andthe Global Forum on Agricultural Research Rome Italy

Blay-Palmer A 2005 Growing innovation policy the case of organic agriculturein Ontario Canada Environ Plann C Gov Policy 23 557ndash581

Carlsson B Stankiewicz R 1991 On the nature function and composition oftechnological systems J Evol Econ 1 93ndash118

Carlsson B Jacobsson S Holmeacuten M Rickne A 2002 Innovation systems analyticaland methodological issues Res Policy 31 233ndash245

Cash DW Adger WN Berkes F Garden P Lebel L Olsson P et al 2006 Scaleand cross-scale dynamics governance and information in a multilevel worldEcol Soc 11 8 Available from lthttpwwwecologyandsocietyorgvol11iss12art18gt [online]

Chambers R 1994 The origins and practice of participatory rural appraisal WorldDev 22 953ndash969

Chung CC 2012 National sectoral and technological innovation systems the caseof Taiwanese pharmaceutical biotechnology and agricultural biotechnologyinnovation systems (1945ndash2000) Sci Public Policy 39 271ndash281

Cooke P Uranga MG Etxebarria G 1997 Regional innovation systems Institutionaland organisational dimensions Res Policy 26 475ndash491

Daniel E 2013 Assessment of agricultural extension services in Tanzania A casestudy of Kyela Songea Rural and Morogoro Rural Districts Wageningen Universityand Research CentreAfrica Rice Center Wageningen the NetherlandsDar esSalaam Tanzania p 45

Douthwaite B Kuby T van de Fliert E Schulz S 2003 Impact pathway evaluationan approach for achieving and attributing impact in complex systems Agr Syst78 243ndash265

Engel PGH 1995 Facilitating Innovation An Action-Oriented Approach andParticipatory Methodology to Improve Innovative Social Practice in AgricultureWageningen University Wageningen the Netherlands p 300

Faysse N 2006 Troubles on the way an analysis of the challenges faced bymulti-stakeholder platforms Nat Resour Forum 30 219ndash229

Freeman C 1988 Japan a new national innovation system In Dosi G FreemanC Nelson R Soete T (Eds) Technology and Economic Theory London pp330ndash348

Freeman C 1995 The national system of innovation in historical perspective CambJ Econ 19 5ndash24

Funtowicz SO Ravetz JR 1993 Science for the post-normal age Futures 25739ndash755

Gildemacher PR Kaguongo W Ortiz O Tesfaye A Woldegiorgis G Wagoire WWet al 2009 Improving potato production in Kenya Uganda and Ethiopia a systemdiagnosis Potato Res 52 173ndash205

Giller KE Leeuwis C Andersson JA Andriesse W Brouwer A Frost P et al2008 Competing claims on natural resources what role for science Ecol Soc13 34 Available from lthttpwwwecologyandsocietyorgvol13iss32art34gt[online]

Giller KE Tittonell P Rufino MC van Wijk MT Zingore S Mapfumo P et al2011 Communicating complexity integrated assessment of trade-offs concerningsoil fertility management within African farming systems to support innovationand development Agr Syst 104 191ndash203

Guest G Bunce A Johnson L 2006 How many interviews are enough Anexperiment with data saturation and variability Field Methods 18 59ndash82

Hall A 2005 Capacity development for agricultural biotechnology in developingcountries an innovation systems view of what it is and how to develop it J IntDev 17 611ndash630

Hall A Clark N 2010 What do complex adaptive systems look like and what arethe implications for innovation policy J Int Dev 22 308ndash324

Hall A Rasheed Sulaiman V Clark N Yoganand B 2003 From measuring impactto learning institutional lessons an innovation systems perspective on improvingthe management of international agricultural research Agr Syst 78 213ndash241

Hall AJ Yoganand B Sulaiman RV Raina RS Prasad CS Naik GC et al 2004Innovations in innovation reflections on partnership institutions and learningICRISAT Andra Pradesh

Hekkert MP Suurs RAA Negro SO Kuhlmann S Smits REHM 2007 Functionsof innovation systems a new approach for analysing technological changeTechnol Forecast Soc Change 74 413ndash432

Hermans F Stuiver M Beers PJ Kok K 2013 The distribution of roles andfunctions for upscaling and outscaling innovations in agricultural innovationsystems Agr Syst 115 117ndash128

Hounkonnou D Kossou D Kuyper TW Leeuwis C Nederlof ES Roumlling N et al2012 An innovation systems approach to institutional change smallholderdevelopment in West Africa Agr Syst 108 74ndash83

Hulsebosch J 2001 The use of RAAKS for strengthening community-basedorganisations in Mali Dev Pract 11 622ndash632

Hung S-C Whittington R 2011 Agency in national innovation systems institutionalentrepreneurship and the professionalization of Taiwanese IT Res Policy 40526ndash538

International Institute for Sustainable Development 2014 Rapid Rural Appraisal(RRA)

Jiggins J 2012 Diagnosing the scope for innovation linking smallholder practicesand institutional context NJAS-Wagen J Life Sc 60ndash63 1ndash122

Kilelu CW Klerkx L Leeuwis C 2013 Unravelling the role of innovation platformsin supporting co-evolution of innovation contributions and tensions in asmallholder dairy development programme Agr Syst 118 65ndash77

Klein Woolthuis R Lankhuizen M Gilsing V 2005 A system failure frameworkfor innovation policy design Technovation 25 609ndash619

Klerkx L Aarts N Leeuwis C 2010 Adaptive management in agricultural innovationsystems the interactions between innovation networks and their environmentAgr Syst 103 390ndash400

Klerkx L Schut M Leeuwis C Kilelu C 2012a Advances in knowledge brokeringin the agricultural sector towards innovation system facilitation IDS Bull 4353ndash60

Klerkx L van Mierlo B Leeuwis C 2012b Evolution of systems approaches toagricultural innovation Concepts analysis and interventions In Darnhofer IGibbon D Dedieu B (Eds) Farming Systems Research into the 21st CenturyThe New Dynamic Springer Dordrecht pp 457ndash483

Leeuwis C 2000 Reconceptualizing participation for sustainable rural developmenttowards a negotiation approach Dev Change 31 931ndash959

Leeuwis C 2004 Communication for Rural Innovation Rethinking AgriculturalExtension (with Contributions of Anne van den Ban) Blackwell Science Oxford

10 M Schut et alAgricultural Systems 132 (2015) 1ndash11

participants developed action plans to address the prioritised con-straints (Workshop Session 13) This required an extension of theworkshops of half a day The development and implementation ofthe action plans forms an important element for continued stake-holder collaboration in multi-stakeholder platforms

5 Conclusions

This paper demonstrates the potential of RAAIS as a diagnostictool that can support and guide the integrated analysis of complexagricultural problems innovation capacity in the agricultural systemand the performance of the agricultural innovation support systemRAAIS combines multiple qualitative and quantitative methods andinsider (stakeholders) and outsider (researchers) analyses whichallow for critical triangulation and validation of the gathered dataSuch an analysis can provide specific entry points for innovationsto address the complex agricultural problem under study and genericentry points for innovation related to strengthening the innova-tion capacity of agricultural system and the functioning of theagricultural innovation support system

Recommendations for further improvement include using RAAISfor the analysis of other types of complex agricultural problems usingother combinations of methods of data collection and providing di-rectly feedback to workshop participants to stimulate reflection andvalidate workshop outcomes An adapted sequence of data collec-tion methods in which workshops provide a lsquofast-trackrsquo approachto identifying entry points for innovation followed up by more in-depth interviews and stakeholder surveys would increase the RAAISrsquodiagnostic capacity The participatory development of concrete actionplans based on RAAIS can provide a basis for continued multi-stakeholder collaboration to operationalise and implement specificand generic entry points for innovation

Acknowledgements

This research forms part of the PARASITE programme fundedby the Integrated Programmes Scheme of the NetherlandsOrganisation for Scientific Research ndash Science for Global Develop-ment (NWO-WOTRO) (wwwparasite-projectorg) Additional supportis provided by the CGIAR Research Program on Climate Change Ag-riculture and Food Security (CCAFS) The CGIAR Research Programmefor the Humid Tropics (httphumidtropicscgiarorg) provided theopportunity to further adapt apply and improve RAAIS across dif-ferent Humidtropics Action Areas Special thanks to Elifadhili Danielwho conducted the farmer-extensionist survey

References

Amankwah K Klerkx L Oosting SJ Sakyi-Dawson O van der Zijpp AJ MillarD 2012 Diagnosing constraints to market participation of small ruminantproducers in northern Ghana an innovation systems analysis NJAS-Wagen JLife Sc 60ndash63 37ndash47

Basu S Leeuwis C 2012 Understanding the rapid spread of System of RiceIntensification (SRI) in Andhra Pradesh exploring the building of supportnetworks and media representation Agr Syst 111 34ndash44

Beintema N Stads G-J Fuglie K Heisey P 2012 ASTI global assessment ofagricultural RampD spending International Food Policy Research InstituteWashington DC Agricultural Science and Technology Indicators Rome Italy andthe Global Forum on Agricultural Research Rome Italy

Blay-Palmer A 2005 Growing innovation policy the case of organic agriculturein Ontario Canada Environ Plann C Gov Policy 23 557ndash581

Carlsson B Stankiewicz R 1991 On the nature function and composition oftechnological systems J Evol Econ 1 93ndash118

Carlsson B Jacobsson S Holmeacuten M Rickne A 2002 Innovation systems analyticaland methodological issues Res Policy 31 233ndash245

Cash DW Adger WN Berkes F Garden P Lebel L Olsson P et al 2006 Scaleand cross-scale dynamics governance and information in a multilevel worldEcol Soc 11 8 Available from lthttpwwwecologyandsocietyorgvol11iss12art18gt [online]

Chambers R 1994 The origins and practice of participatory rural appraisal WorldDev 22 953ndash969

Chung CC 2012 National sectoral and technological innovation systems the caseof Taiwanese pharmaceutical biotechnology and agricultural biotechnologyinnovation systems (1945ndash2000) Sci Public Policy 39 271ndash281

Cooke P Uranga MG Etxebarria G 1997 Regional innovation systems Institutionaland organisational dimensions Res Policy 26 475ndash491

Daniel E 2013 Assessment of agricultural extension services in Tanzania A casestudy of Kyela Songea Rural and Morogoro Rural Districts Wageningen Universityand Research CentreAfrica Rice Center Wageningen the NetherlandsDar esSalaam Tanzania p 45

Douthwaite B Kuby T van de Fliert E Schulz S 2003 Impact pathway evaluationan approach for achieving and attributing impact in complex systems Agr Syst78 243ndash265

Engel PGH 1995 Facilitating Innovation An Action-Oriented Approach andParticipatory Methodology to Improve Innovative Social Practice in AgricultureWageningen University Wageningen the Netherlands p 300

Faysse N 2006 Troubles on the way an analysis of the challenges faced bymulti-stakeholder platforms Nat Resour Forum 30 219ndash229

Freeman C 1988 Japan a new national innovation system In Dosi G FreemanC Nelson R Soete T (Eds) Technology and Economic Theory London pp330ndash348

Freeman C 1995 The national system of innovation in historical perspective CambJ Econ 19 5ndash24

Funtowicz SO Ravetz JR 1993 Science for the post-normal age Futures 25739ndash755

Gildemacher PR Kaguongo W Ortiz O Tesfaye A Woldegiorgis G Wagoire WWet al 2009 Improving potato production in Kenya Uganda and Ethiopia a systemdiagnosis Potato Res 52 173ndash205

Giller KE Leeuwis C Andersson JA Andriesse W Brouwer A Frost P et al2008 Competing claims on natural resources what role for science Ecol Soc13 34 Available from lthttpwwwecologyandsocietyorgvol13iss32art34gt[online]

Giller KE Tittonell P Rufino MC van Wijk MT Zingore S Mapfumo P et al2011 Communicating complexity integrated assessment of trade-offs concerningsoil fertility management within African farming systems to support innovationand development Agr Syst 104 191ndash203

Guest G Bunce A Johnson L 2006 How many interviews are enough Anexperiment with data saturation and variability Field Methods 18 59ndash82

Hall A 2005 Capacity development for agricultural biotechnology in developingcountries an innovation systems view of what it is and how to develop it J IntDev 17 611ndash630

Hall A Clark N 2010 What do complex adaptive systems look like and what arethe implications for innovation policy J Int Dev 22 308ndash324

Hall A Rasheed Sulaiman V Clark N Yoganand B 2003 From measuring impactto learning institutional lessons an innovation systems perspective on improvingthe management of international agricultural research Agr Syst 78 213ndash241

Hall AJ Yoganand B Sulaiman RV Raina RS Prasad CS Naik GC et al 2004Innovations in innovation reflections on partnership institutions and learningICRISAT Andra Pradesh

Hekkert MP Suurs RAA Negro SO Kuhlmann S Smits REHM 2007 Functionsof innovation systems a new approach for analysing technological changeTechnol Forecast Soc Change 74 413ndash432

Hermans F Stuiver M Beers PJ Kok K 2013 The distribution of roles andfunctions for upscaling and outscaling innovations in agricultural innovationsystems Agr Syst 115 117ndash128

Hounkonnou D Kossou D Kuyper TW Leeuwis C Nederlof ES Roumlling N et al2012 An innovation systems approach to institutional change smallholderdevelopment in West Africa Agr Syst 108 74ndash83

Hulsebosch J 2001 The use of RAAKS for strengthening community-basedorganisations in Mali Dev Pract 11 622ndash632

Hung S-C Whittington R 2011 Agency in national innovation systems institutionalentrepreneurship and the professionalization of Taiwanese IT Res Policy 40526ndash538

International Institute for Sustainable Development 2014 Rapid Rural Appraisal(RRA)

Jiggins J 2012 Diagnosing the scope for innovation linking smallholder practicesand institutional context NJAS-Wagen J Life Sc 60ndash63 1ndash122

Kilelu CW Klerkx L Leeuwis C 2013 Unravelling the role of innovation platformsin supporting co-evolution of innovation contributions and tensions in asmallholder dairy development programme Agr Syst 118 65ndash77

Klein Woolthuis R Lankhuizen M Gilsing V 2005 A system failure frameworkfor innovation policy design Technovation 25 609ndash619

Klerkx L Aarts N Leeuwis C 2010 Adaptive management in agricultural innovationsystems the interactions between innovation networks and their environmentAgr Syst 103 390ndash400

Klerkx L Schut M Leeuwis C Kilelu C 2012a Advances in knowledge brokeringin the agricultural sector towards innovation system facilitation IDS Bull 4353ndash60

Klerkx L van Mierlo B Leeuwis C 2012b Evolution of systems approaches toagricultural innovation Concepts analysis and interventions In Darnhofer IGibbon D Dedieu B (Eds) Farming Systems Research into the 21st CenturyThe New Dynamic Springer Dordrecht pp 457ndash483

Leeuwis C 2000 Reconceptualizing participation for sustainable rural developmenttowards a negotiation approach Dev Change 31 931ndash959

Leeuwis C 2004 Communication for Rural Innovation Rethinking AgriculturalExtension (with Contributions of Anne van den Ban) Blackwell Science Oxford

10 M Schut et alAgricultural Systems 132 (2015) 1ndash11

Leeuwis C Schut M Waters-Bayer A Mur R Atta-Krah K Douthwaite B 2014lsquoCapacity to innovatersquo from a system-CRP perspective System CGIAR ResearchPrograms (CRPs) p 5

Lundy M Gottret MV Ashby J 2005 Learning alliances an approach for buildingmulti-stakeholder innovation systems ILAC Brief 8 Bioversity Rome

Markard J Truffer B 2008 Technological innovation systems and the multi-levelperspective towards an integrated framework Res Policy 37 596ndash615

McNie EC 2007 Reconciling the supply of scientific information with user demandsan analysis of the problem and review of the literature Environ Sci Policy 1017ndash38

Nrsquocho SA Mourits M Rodenburg J Demont M Oude Lansink A 2014Determinants of parasitic weed infestation in rainfed lowland rice in Benin AgrSyst 130 105ndash115

Ortiz O Orrego R Pradel W Gildemacher P Castillo R Otiniano R et al 2013Insights into potato innovation systems in Bolivia Ethiopia Peru and UgandaAgr Syst 114 73ndash83

Papaioannou T Wield DV Chataway JC 2009 Knowledge ecologies andecosystems An empirically grounded reflection on recent developments ininnovation systems theory Environ Plann C Gov Policy 27 319ndash339

Poulton C Dorward A Kydd J 2010 The future of small farms new directionsfor services institutions and intermediation World Dev 38 1413ndash1428

Rodenburg J Zossou-Kouderin N Gbehounou G Ahanchede A Toure A KyaloG et al 2011 Rhamphicarpa fistulosa a parasitic weed threatening rain-fedlowland rice production in sub-Saharan Africa ndash a case study from Benin CropProt 30 1306ndash1314

Russell Bernard H 2006 Research Methods in Anthropology Qualitative andQuantitative Approaches fourth ed AltaMira Press Oxford UK

Schut M Florin MJ under review The policy and practice of biofuel sustainabilitybetween global frameworks and local heterogeneity The case of food securityin Mozambique

Schut M Hinnou LC 2014 Rapid Appraisal of Agricultural Innovation Systems(RAAIS) workshops Burundi DR Congo and Rwanda Wageningen University(WUR) and International Institute for Tropical Agriculture (IITA) WageningenThe Netherlands p 128

Schut M Rodenburg J Klerkx L van Ast A Bastiaans L 2014a Systems approachesto innovation in crop protection A systematic literature review Crop Prot 5698ndash108

Schut M van Paassen A Leeuwis C Klerkx L 2014b Towards dynamic researchconfigurations a framework for reflection on the contribution of research topolicy and innovation processes Sci Public Policy 41 207ndash218

Schut M Rodenburg J Klerkx L Kayeke J van Ast A Bastiaans L 2014c RAAISRapid Appraisal of Agricultural Innovation Systems (Part II) Integrated analysisof parasitic weed problems in rice in Tanzania Agr Syst bullbull bullbullndashbullbull

Sims BG Thierfelder C Kienzle J Friedrich T Kassam A 2012 Developmentof the conservation agriculture equipment industry in sub-Saharan Africa ApplEng Agric 28 813ndash823

Singh RK Murty HR Gupta SK Dikshit AK 2009 An overview of sustainabilityassessment methodologies Ecol Indic 9 189ndash212

Spedding CRW 1988 An Introduction to Agricultural Systems second ed ElsevierApplied Science Publishers New York

Spielman DJ 2005 Innovation Systems Perspectives on Developing-CountryAgriculture A Critical Review ISNAR Discussion Paper 2 IFPRI Washington p58

Spielman DJ Ekboir J Davis K 2009 The art and science of innovation systemsinquiry applications to sub-Saharan African agriculture Technol Soc 31399ndash405

Temel T Janssen W Karimov F 2003 Systems analysis by graph theoreticaltechniques assessment of the agricultural innovation system of Azerbaijan AgrSyst 77 91ndash116

Thitinunsomboon S Chairatana PA Keeratipibul S 2008 Sectoral innovationsystems in agriculture the case of rice in thailand Asian J Technol Innov 1683ndash100

Totin E van Mierlo B Saidou A Mongbo R Agbossou E Stroosnijder L et al2012 Barriers and opportunities for innovation in rice production in the inlandvalleys of Benin NJAS-Wagen J Life Sc 60ndash63 57ndash66

van Ittersum MK Ewert F Heckelei T Wery J Alkan Olsson J Andersen E et al2008 Integrated assessment of agricultural systems ndash A component-basedframework for the European Union (SEAMLESS) Agr Syst 96 150ndash165

van Mierlo B Leeuwis C Smits R Woolthuis RK 2010 Learning towards systeminnovation evaluating a systemic instrument Technol Forecast Soc Change 77318ndash334

Whitley E Ball J 2002 Statistics review 4 sample size calculations Crit Care 6335ndash341

Wieczorek AJ Hekkert MP 2012 Systemic instruments for systemic innovationproblems a framework for policy makers and innovation scholars Sci PublicPolicy 39 74ndash87

World Bank 2006 Enhancing agricultural Innovation How to Go Beyond theStrengthening of Research Systems World Bank Washington DC p 118

World Bank 2012 Agricultural Innovation Systems An Investment Sourcebook TheWorld Bank Washington DC

11M Schut et alAgricultural Systems 132 (2015) 1ndash11