Technological Forecasting & Social Change 80 (2013) 1815–1833

Contents lists available at ScienceDirect

Technological Forecasting & Social Change

A Delphi-based risk analysis — Identifying and assessing future challengesfor supply chain security in a multi-stakeholder environment

Christoph Markmann⁎, Inga-Lena Darkow, Heiko von der GrachtInstitute for Futures Studies and Knowledge Management (IFK), EBS Business School, Konrad-Adenauer-Ring 15, 65187 Wiesbaden, Germany

a r t i c l e i n f o

⁎ Corresponding author. Tel.: +49 611 7102 2107; f2107.

E-mail addresses: Christoph.Markmann@ebs.edu (Inga-Lena.Darkow@ebs.edu (I.-L. Darkow), Heiko.vond(H. von der Gracht).

0040-1625/$ – see front matter © 2012 Elsevier Inc. Ahttp://dx.doi.org/10.1016/j.techfore.2012.10.019

a b s t r a c t

Article history:Received 26 March 2012Received in revised form 6 September 2012Accepted 6 October 2012Available online 24 November 2012

Identifying and assessing the potential impact and likelihood of future events, whichmight evolveinto risks, are a prerequisite to identify future security challenges. In particular, risks associatedwith global supply chains are special since they involve a multitude of international stakeholderswith different perspectives on security needs andmeasures. Therefore, it is essential to determinewhich techniques and instruments are best suited for risk assessment in complex andmulti-organizational environments. The Delphi expert survey technique has proven to be avaluable instrument for long-term decision making support as well as foresight, and has apotential value for risk assessment. We contribute to this research strand and conduct aDelphi-based risk analysis. Our research concentrates on man-made risks in global supply chainswhich are particularly uncertain in terms of type, location, and affected supply chain partners andcan therefore be classified as inherently “wicked” issues, i.e. issues that aremultidimensionalwithoften unpleasant outcomes. We illustrate that Delphi research makes a fivefold contribution torisk analysis by: (1) identifying and quantifying risks; (2) analyzing stakeholder perceptions andworldviews; (3) stimulating a global communication process; (4) identifyingweak signals, outlieropinions, and wildcards; (5) and facilitating risk scenario development.

© 2012 Elsevier Inc. All rights reserved.

Keywords:RiskSupply chain securityTerroristic attacksDelphiMulti-stakeholderFuture

1. Introduction

Much has been discussed about the various sources andimpacts of risks likely to affect global supply chains in the 21stcentury. Increasing complexity, due to globalization, andleanness of structures and processes are major driving forcesof supply chain risks and, therefore, supply chain vulnerability[1–3]. Furthermore, the variety of stakeholders involved inmanaging supply chains, such as suppliers, manufacturers,retailers, logistics service providers, infrastructure providerslike port authorities, as well as national and internationalgovernmental institutions contributes to the complexity andsusceptibility of supply chains [4]. Consequently, disruptions to

ax: +49 611 7102 10

C. Markmann),erGracht@ebs.edu

ll rights reserved.

global flows of goods and related states of affairs around theworld have drawn companies' and governments' attention tosuch situations [5]. The causes of such disruptions includenatural catastrophes (e.g. flood or earthquake), man-madeaccidents (e.g. technological breakdowns), or intentionalman-made attacks (e.g. theft or terrorism). Moreover, the UNMillennium Project recently identified organized crime andterrorism as two of the fourmost impactful global problems forthe next 10–20 years. Furthermore, the risk of informationwarfare and cyber attacks is an emerging problem in ourInternet-dependent global economy [6]. The risks associatedwith the operation of global supply chains, whether theseconcern the management of maritime shipping lines, supplychain information systems, or logistics infrastructural hubs, arequite high. Especially the substantial and far reaching lossesdue to man-made attacks – terrorism, crime, cyber attacks orpiracy – have drawn attention. For example, maritime piracywas estimated to cost the international economy between $7and $12 billion in 2010 [7]. Furthermore, re-routing ships due

1816 C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

to piracy attacks costs Egypt $642 million a year due to lostrevenues from Suez Canal fees, Kenya and Yemen together$564 million due to reduced trading activities and theSeychelles around $6 million due to fewer tourists. However,recent natural disasters (e.g. the flood in Thailand) andaccidents (e.g. the nuclear accident in Fukushima) have forcedindustries and governments to re-evaluate their assessments,preparations for and handling of emergency situations. As aconsequence, redesigning global supply chains, in order tomake themmore resilient and less susceptible to various kindsof disruptions by proactive planning, has become a majormanagement issue [5,8,9]. Furthermore, researchers andpractitioners seek appropriate strategies tomitigate the impactof harmful supply chain disruptions [5,10,11].

Usually, a common understanding or consensus does notexist about the problems as well as the sources and impactsthat cause risks in international supply chains on a global level[12–14]. Furthermore, man-made risks in global supply chainsare uncertain in terms of type, location, and affected supplychain partners and are therefore inherently “wicked” issues[13,15]. A wicked problem is defined as an issue that ismultidimensional with often unpalatable trade-offs. As de-scribed by Camillus [13], “a wicked problem has innumerablecauses, is tough to describe, and doesn't have a right answer”. Awicked issue often involves multiple stakeholders withdifferent perceptions of the problem, different perceptions ofthe appropriate procedure to solve the problem, and differentperceptions of how the results and success of the solutionshould be evaluated [13,16]. In such uncertain wickedenvironments, it is difficult for relevant stakeholders to processinformation and make effective decisions [17].

Gonzalez [18] emphasized the importance of empiricalstudies to identify and assess relevant information in uncertainenvironments in order to derive appropriate strategies. Duncan[19] elaborated on the characteristics of the organizationalenvironment and perceived environmental uncertainty in aseminal paper. Both authors argued that possible images of thefuture significantly affect strategies to cope with futureuncertainties. Stakeholders from different regions and culturesoften have different perceptions of situations and risks whichcould affect security in supply chains, depending on national orcultural backgrounds, their own position within the valuechain, their experience, and so on [20,21]. In such situations, itis common to accidentally neglect relevant factors andimportant information, or draw misleading conclusions [22].

In order to be better prepared for the future, we need tosystematically consider different stakeholder conditions, con-texts, and limitations in order to gain a complete perspective ofthe wicked problem: supply chain security (SCS) [15]. Anappropriate procedure which collects and evaluates all stake-holder aspects, including stakeholders' images of the futureand opinions of the greatest challenges in SCS needs to beapplied. Therefore, the first step is to identify which risksstakeholders perceive to be relevant in long-term SCS. Theserisks are inherently uncertain and have varying impacts andseverity on supply chain partners [23]. The risk conditions areshrouded by ambiguity, information asymmetry, and organi-zational fragmentation [15]. In a second step, the identifiedrisks need to be evaluated in a risk analysis process, whichincludes an estimation of risk probability and risk impact [24].Due to similar challenges and characteristics of risk analysis

and foresight, Koivisto et al. [25] recently proposed toinvestigate foresight methods for risk analysis due to theirproactive nature. Several authors [18,26,27] have alreadyproclaimed the potential value of the Delphimethod to analyzerisks in a future setting. The authors argued that in uncertainenvironments it is necessary to analyze how uncertainties canimpact business and supply chains. The conventional Delphitechnique follows a structured, anonymous, multi-roundsurvey process in written form, where expert opinions onfuture events and developments are summarized [28]. There-by, this method overcomes drawbacks of traditional groupdiscussions, such as the halo and bandwagon effects [29], andproduces answers quicker and more accurately than individ-uals on the average [30,31]. Furthermore, the Delphi techniquehas been found to be helpful in examining uncertain worldevents, such as geopolitical changes, terrorist activities orvolatile military actions [32].

Following this logic, we contribute to the field of supplychain security by examining the potentials of Delphi researchfor risk analysis. Our overall aim is to gauge to what degree theDelphi technique can support and improve risk analysis. Weconduct a Delphi survey on the future of supply chain securityin 2030 and aggregate the perceptions and views of the worldof 80 international top decision makers from a multitude ofstakeholder groups, such as industry, academia, politics andother associations. Thereby, this multi-stakeholder approachimproves the general validity of our findings concerning thefuture of SCS.

The remainder of the paper is organized as follows: InSection 2, we provide an overview of literature that points tothe challenges of risk analysis or presents experiences withthe Delphi method in the field of risk analysis. In Section 3,we present the methodology of the Delphi survey that wasapplied to gather data and the perceptions of stakeholders forthe future of SCS. In Section 4, we propose how the Delphidata can be applied for risk analysis, before we discuss andevaluate the benefits of the collected Delphi data for riskanalysis in Section 5: we illustrate how we (1) identify andquantify security risks for global supply chains, (2) determinestakeholder perspectives and world views, (3) stimulate aglobal communication process, (4) identify weak signals,outlier opinions, and wildcards, and finally (5) facilitate a riskscenario development. Section 6 concludes with overallreflection, limitations and suggestions for future research.

2. Literature review

Having uncertainties in supply chains is – as the namesuggests – unfavorable for running stable processes. Therefore,uncertainties in supply chains are commonly described as risksand require more specific information regarding probability ofoccurrence and potential consequences [33]. Furthermore,risks usually refer to a certain event or development, whichdisrupts the ordinary course of action by inducing exceptionalconditions. The scope of such disruptive events can range fromhigh-probability, low-impact events (e.g. a screw missing inthe production process) to low-probability, high-impact events(e.g. nuclear contamination of an entire region). Risk may alsopertain to natural disasters andman-made accidents or attacks.

Sheffi [34] firstly addressed the topic of SCSmanagement inthe direct aftermath of the 9/11 attack in the US. He noted that

1817C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

companies need to change their way of thinking and theirpractices in order to successfully operate under increasinguncertainty and to be prepared for similar future events. Inyears following the attack, numerous scientific articleshighlighted how SCS and supply chain risks are interlinked.Supply chain risk management refers to reducing supply chainvulnerability by ensuring business continuity via planning. Thediscipline analyses to what extent the supply chain issusceptible to disruptions and thereby could be potentiallyharmful to a company's overall profitability and performance[35,36].

SCS management, in turn, is an element of supply chainrisk management and seeks to prevent man-made attackssuch as theft and damage to or destruction of products andassets [34,37,38]. This new field of study identifies thedeficiencies of existing logistics networks [39], developssecurity concepts within companies [10,40] and analyzesmaritime, rail, road and air transportation security, as well asthe roles of law enforcement, security technologies andcorporate security training [39]. Lee and Wolfe [41] linkedthe topic to established quality management concepts andproposed an approach for prevention, process control anddesign improvements. Williams et al. [38] provided anextensive literature review, in which they demonstrate themain challenges in the field and concluded that SCS researchlacks in portraying intra-organizational activities, the inter-action among supply chain partners, and the role of thegovernment, as well as quantitative assessments to betterunderstand the rationale, targets, sources and causes of SCSinitiatives. Furthermore, the authors revealed a researchgap: the link between the consequences of and resultingmeasures against supply chain disruptions and organiza-tional performance. In their extensive report on investing inSCS, Rice and Spayd [42] concluded that SCS research is, ingeneral, normative, seldom based on empirical studies andoffers examples of reaction to past events rather than givinginsights into efforts of companies to prevent and beprepared for these kinds of events. However, the moreinformation is available about disruptive events and themore accurate the evaluations of probability and impact are,the more appropriate reactive and proactive measures canbe [9].

However, risk analysis methods have several shortcom-ings when applied to the context of global, far-reaching, andrelatively unknown situations [43]. The quality of riskanalysis is strongly dependent on the availability andactuality of information, as well as on the experience of therisk assessor [20,21]. In general, there is a lack of up-to-dateinformation and experienced advisors. Risks are usuallyanalyzed and evaluated by a small group of people, whohave just a fraction of the globally available information andexperience in the topic. Therefore, it is difficult to evaluatesuch circumstances and appropriately prepare for emergen-cy settings. Furthermore, risk analysis is subjective. Riskassessment varies according to the risk perception of theassessor [20,21]. Since it is not possible to determine the“right” perception of a risk or to accurately weight differentrisks, multiple perspectives should be surveyed and inte-grated [44] on a continual basis [43]. Traditional scientificapproaches of setting goals and deducing actions often failhere and are too inert to be applicable in suddenly occurring

emergency situations. Instead, more incremental ap-proaches, based on “the science of muddling through” [45]or the newer High Reliability Organizations (HRO) [46], arecurrently being revitalized in emergency managementoperations [43]. These concepts are suitable to emergencysituations because they integrate multiple stakeholders inthe decision process. Since they rely on many smalladaptions instead of a few large changes, they are moreflexible. However, these approaches require ongoing mon-itoring, continual exchange of information and perspectives,and the ability to act short-term, which is quite difficult forlarge companies or governmental institutions.

The risks which accompany disruptive events also varyaccording to individual situations [47]. Due to the mainly localimpact of disruptive events in the past, risk analysis subjectiv-ity might have been of minor importance. The global characterof today's supply chains and the dependency of supply chainpartners on each other may cause one company to exposeother companies to their risks [48]. Especially superior risks,such as natural disasters or terrorist attacks, disrupt partnercompanies in a similar way. Proactive exchange of informationamong people with similar or complementary duties wouldimprove the respective preparation for emergency situations[43,49]. Therefore, the risk identification and assessment offar-reaching events should become as global as their disruptiveeffect, however without additional time and effort detrimen-tally affecting process agility.

These current concerns about the traditional risk analysis inthe context of global, far-reaching, disruptive events indicatethat the process needs to be reviewed and adapted regardinginformation gathering, validity and actuality of probability andimpact assessment, as well as multifaceted stakeholderinvolvement [43]. Koivisto et al. [25] proposed investigatingmethods from foresight and future-oriented technology anal-ysis (FTA) regarding their applicability in the risk analysisprocess since these research areas have similar challenges.Similar to risk analysis, futures research deals with uncer-tainties about the occurrence and the impact of future eventsand developments, which influence different stakeholders.There are various methods to approach future developmentsand thereby to reduce uncertainty for even longer timehorizons. By using scenarios, wildcard analyses and expertworkshops for instance, futures research aims to enhance theknowledge base for medium- and long-term decision making,which also prepares stakeholders for times of emergency.

Due to the possible risk of being attacked by the SovietUnion during the Cold War, the US RAND Corporationdeveloped a survey method called Delphi, which facilitatesin generating different images of the future, based on theshared knowledge of the involved experts [50]. The provenadvantage of the Delphi method in this context included thestructured, systematic identification and the collectiveassessment of rare or even unique, hardly imaginable andunexperienced events [18,32,43]. In addition to a systematicgroup communication process, early applications particu-larly focused – although not the primary aim – on consensusbuilding among an expert panel about which strategies toadopt in an emergency situation [28]. Risk analysis, based onexpert opinion, was mainly used within a non-businessenvironment and was often related to national securitytopics in the years following the Delphi inception.

1818 C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

The Delphi method evolved over the years and was oftenapplied in project management to analyze risks in large-scale orinternational projects (see Table 1). For example, Tummala andBurchett [51] applied the Delphi method to risk assessment inorder to determine and address possible public concerns aboutbudgeting in a public urban planning project as early as possible.Similarly, Renn [52] used a Delphi approach in order toincorporate the public view into risk management in severalurban planning projects. He illustrated a communication andconsensus-oriented process which fosters managerial decisionsthat are usually dominated by discussions among technical andscientific experts. Furthermore, he revealed cultural differencesin risk perception among experts from different countries.

With the rise of information technology (IT) and theperceived vulnerability of highly complex IT systems, Delphiwas applied in risk analysis for information systems research.Thereby the Delphi method became more prevalent in civil andbusiness related environments. Rainer, Snyder and Carr pub-lished apaper in 1991, inwhich they applied amulti-method riskanalysis, based on the Delphi technique, to identify and assessthe vulnerability of information systems. Similarly, Schmidt,Lyytinen, Keil and Cule [55] used the Delphi method foridentifying risks in software projects by conducting an empiricalstudy in Finland, the U.S. and Hong Kong and elaborated on thecultural impact on the rank order of identified risks. In hisempirical paper, Addison [56] applied the Delphi method toidentify and assess risks in e-commerce projects and demon-strated what determines practitioners' opinion. While Nakatsuand Iacovou [57] only considered the perspective of a singlestakeholder group (i.e. clients) in their comparative study aboutrisks in the offshore and domestic outsourcing of softwareprojects, Chapman [54] considered the Delphi technique as oneof several risk identification methods that have the potential todeal with the challenges of a multiple stakeholder environment.Turoff et al. [43] proposed the Delphi method to deal moreefficiently with the increasing number of emergency situationsand their inherent risks, and even recommends further devel-opment towards a more dynamic and continuous Delphi votingsystem.

The literature review shows that a Delphi-based riskanalysis is used in public decision making in order to involvemultiple stakeholders in identifying and assessing risks[50–52]. In the corporate world, the rising need to managecomplex projects makes risk identification and, to a lesserextent, assessment exercises, which incorporate variousfacets of the Delphi method, essential [53,55–57]. Further-more, the increasing number of far-reaching emergencysituations requires rethinking current risk analysis approaches.Methods, such as the dynamic Delphi evaluation system,facilitate extensive and effective multi-stakeholder communica-tion and collaboration processes [43]. In addition, themain use ofthe Delphi method in rather complex and multi-stakeholder-driven environments [58] indicates that the method has thecapability to assess risks in “deeply uncertain” situations. Underdeep uncertainty, “parties to a decision do not know or do notagree on the system model relating actions to consequences orthe prior probability distributions for input parameters to thesesystem models” [59,60]. This definition means that neither anappropriatemodel, which describes the relationships among thekey driving forces can be developed, nor the probabilitydistributions used to represent uncertainty or the desirability

value of alternative outcomes [59,60]. According to Lempert andcolleagues [59,60], especially decisionswith a long-termhorizon,novel or poorly understood systems, or surprises have to dealwith deep uncertainty. Therefore, deeply uncertain environ-ments primarily cause wicked problems [13], which involvemultidimensional issues and various stakeholders. Ambiguityprevails as to what is the most appropriate procedure to solvethe problem and how the results of the decision should beevaluated.

Although Delphi research was initiated to imagine possibleevents in the long-term future and to address man-madethreats, the method was rarely used by the public orcommercial sector for this purpose over the last decade [58].As previously stated, the global economy could be threatenedby natural and man-made disruptions. Therefore, analysismethods are required, which are suitable in analyzing complexissues and include different risk perceptions [43,52,55].However, Delphi-based risk analysis has seldom been pub-lished in the non-military environment of man-made securityrisks. Moreover, the presented papers use the Delphi methodfor risk analysis only to a limited extent. Nevertheless, Ogden etal. [61] found that the Delphi method is particularly valuable inconsensus-building about future supply chain strategies.According to the authors, themethod involvesmultiple expertswhile avoiding the common pitfalls of other forecastingmethods, such as reliance on single expert assessments orpremature averaging of experts' forecasts without allowing foran unfiltered exchange of information or biases. Turoff et al.[43] even recommended using the Delphi method as an online,dynamic voting system that encourages continual communi-cation and collaboration in risk assessment of potentialemergency situations. Squire and Chu [23] applied the Delphimethod in the complex, multi-stakeholder environment ofglobal supply chains to identify typical risk factors and rankedthem according to their severity, which the authors define asthe product of probability and impact assessment. Due to thevery broad scope of identifying and ranking supply chain risksin general, the authors do not have the opportunity to exhaustthe potential of the Delphi method nor to elaborate on andspecify the risks of man-made attacks. They simply summa-rized all potential disruptions due to sabotage, terrorism, crimeand war under the term “terrorist attacks”.

Therefore, we aim to address this research gap andinvestigate the potential of the Delphi method for risk analysisof complex and far-reaching events. We apply the method tothe currently challenging, complex and comparablyunexplored topic of SCS in order to determine a solidevaluation of the Delphi technique's applicability for emergen-cy preparedness as well as to contribute to current researchabout SCS. In this paper, we perform a Delphi-based riskanalysis to identify and assess global, man-made risks for thelong-term future of SCS. The analysis includes differentworldviews and risk perceptions in order to consolidate imagesof the future in a multi-stakeholder approach.

3. Survey methodology

3.1. Delphi survey method

The Delphi method has proven to be an efficient surveymethod when only a limited amount of data on a topic is

Table 1Most relevant literature on Delphi for risk analysis.

# Author, year Journal Title Key results

1 Dalkey and Helmer,1963 [50]

ManagementScience

An experimental application of the Delphimethod for experts

▪ First academic description of the Delphi method▪ Presentation of results of Delphi experiment withmethodological shortcomings

▪ Focus on the Delphi method itself2 Rainer, Snyder and

Carr, 1991 [53]Journal ofManagementInformationSystems

Risk analysis for information technology ▪ Presentation of a multi-method risk analysisincluding qualitative and quantitative aspects ininformation technology project

▪ Focus on risk analysis using Delphi method for ITresource and assets

▪ Procedural development of 8 steps for risk identificationand risk assessment in IT

3 Chapman, 1998 [54] InternationalJournal of ProjectManagement

The effectiveness of working group riskidentification and assessment techniques

▪ Conceptual approach to compare three risk identificationand risk assessment methods: brainstorming, nominalgroup technique and Delphi

▪ Application at construction industry▪ Conclusion that nominal group technique is mostbeneficial for the needs in the construction industry andrespective stakeholders involved

4 Tummala andBurchett, 1999 [51]

InternationalJournal of ProjectManagement

Applying a risk management process (RMP)to manage cost risk for an EHV transmissionline project

▪ Delphi-based risk analysis in the project definition phaseof extra high voltage (EHV) transmission line in order tosupport the budgeting process

▪ Risks identified via brainstorming and Pareto analyses tomeet the expectations of the public

▪ Risk assessed by Delphi survey to estimate probabilitydistributions of risk occurrence

▪ Inclusion of Delphi results in a simulation model todescribe risk structure for the project budget

5 Renn, 1999 [52] Environ. Sci.Technol.

A model for an analytic–deliberative processin risk management

▪ Involvement of the public in risk analysis and decisionmaking for various urban planning projects in Germany,Switzerland and U.S.

▪ Application of a method similar to Delphi to identify risksand evaluate impacts related to policy options

▪ Focus on organizing and structuring the process tointegrate various stakeholders and their concerns intourban planning projects

6 Schmidt, Lyytinen,Keiland Cule, 2001 [55]

Journal ofManagementInformationSystems

Identifying software project risks: aninternational Delphi study

▪ Identification of software project risks in three cultures(US, Finland and Hong Kong)

▪ Application of Delphi method to rank software projectrisks

▪ Identification of cultural differences in risk perception7 Addison, 2003 [56] International

Journal ofInformationManagement

E-commerce project developmentrisks: evidence from a Delphi survey

▪ Identification of e-commerce related risks▪ Application of Delphi method to evaluate e-commerceproject risks including various stakeholders(project managers, developers, users and academics)

▪ Result: ranking of top 10 risks for e-commerce projects8 Nakatsu and Iacovou,

2009 [57]Information andManagement

A comparative study of important riskfactors involved in offshore and domesticoutsourcing of software developmentprojects: a two-panel Delphi study

▪ Creation of an empirically generated list of risk factors forsoftware development projects

▪ Explorative two-panel Delphi study for domestically- andoffshore-outsourced projects

▪ Application of Delphi method to identify softwaredevelopment risks

▪ Focus on single stakeholder approach, here clientperspective

▪ Results: two lists of top project risks9 Turoff, Hiltz, White,

Plotnick, Hendela, andYao, 2009 [43]

EmergencyPreparedness andManagement

The past as the future of emergencypreparedness and management

▪ Current risk analysis approaches need to be rethought inorder to make them more long-term and less monetaryoriented

▪ Better ways to involve and inform the public inemergency situations are required

▪ Communication and collaboration between differentstakeholders in and before an emergency situation needto be encouraged

▪ Delphi method should be further developed towards adynamic and continuous voting system in order to meetthe requirements for emergency management

1819C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

1. Development of projections

2. Selection of experts

3. Collection of data

4. Data analyses

Fig. 1. Structure of the Delphi survey process.

1820 C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

available [29,62,63]. Themajor goal of the technique is to assistand structure the group communication process [28]. Based onthe modifications to the initial Delphi method, we chose avariant in which selected experts assessed a number ofprovoking future projections regarding expected probability,industry impact and desirability. Furthermore, we asked theexperts to supplement their quantitative assessments withqualitative arguments. This Delphi approach appeared mostsuitable to fulfill the previously mentioned challenges of riskanalysis in a deeply uncertain environment.While themajorityof Delphi studies concentrate on consensus among the expertpanel on a certain topic [64,65], dissent is equally valuable fordata interpretation [28,62,66,67], especially in the context ofterrorism [68] or policy making [69]. Therefore, the measure-ment of consensus and dissent should be considered valuableinterpretation components in Delphi research [70].

Since we strived to obtain an in-depth exchange ofarguments with different perspectives on the topic from aglobal panel of experts, we conducted our research in the formof a real-time Delphi on the Internet [71], based upon theearlier methodological rationales of Gordon and Pease [72].Recent research has proven that both conventionalround-based as well as modern real-time Delphi methodslead to comparable results [71,73]. Gnatzy et al. [71] and Geist[73] demonstrated that an important advancement of thereal-time variant is that experts receive immediate feedback,which streamlines the process, thereby reducing researchfatigue, eliminating panel mortality over rounds, and increas-ing the validity of the data.

Our Delphi process followed four central phases (Fig. 1).First, we conducted intensive literature, database and deskresearch on potential issues for the future of supply chainsecurity 2030, which resulted in a long list of issues. Duringworkshops with a mixed delegation of researchers andindustry experts, the list was consolidated into a short list ofstrategic issues, which are considered to be particularlyrelevant for SCS. Strategic issues included both scientific aswell as practical relevance. Second, we identified and invitedexperts with an appropriate level of expertise in the field toparticipate. In a third step, we facilitated the Delphi surveyprocess by sending participants regular reminders and in-vitations to participate in the survey. In the last step, weanalyzed the quantitative and qualitative data in various waysto identify the contribution of the survey to the field of SCS. Thesteps of the Delphi survey process are explained in detail in theremaining sections of Section 3 as well as Section 4.

1 STEP (social, technological, economic, and political analysis) describes aframework of macro-environmental factors used in environmental scanning.

3.2. Development of projections

The Delphi projections were developed in a series of fourparticipatory expert workshops. In the first workshop, the coreresearch team, consisting of four academics and four subjectmatter experts from industry, compiled a list of issues that theydeemed relevant for the research topic. This list was enriched

by intensive desk research on the Internet and on variousdatabases (e.g. Global Terrorism Database or Statista). It wasalso enhanced with current governmental security initiatives(e.g. C-TPAT, ISO 28 000, Authorized Economic Operator(AEO)) and scientific publications [10,34,39].

Identifying key issues for projection development wasparticularly challenging due to the multitude of involvedregions, governments, transport modes and routes, differentorganizational andmarket structures, involved technologies, aswell as the variety of man-made threats for global supplychains. Since the major goal of the study was to collect globalinsights of concerns, prospects and strategies of supply chainpartners, macro-level key issues were surveyed (e.g. securitycosts or transport time). The projections had to be generallyunderstandable and assessable, regardless if the participantwas from an American OEM, a European logistics serviceprovider, or an Asian shipping company. Therefore, we eitherfocused on publications and security initiatives with a globalfocus or searched for common factors in more specificdocuments (e.g. national security regulations).

The initial list included 132 issues which were furthercondensed, refined, clustered and prioritized in a subsequentsecondworkshop, resulting in a short list of 27 strategic issues.We used a STEP-analysis1 [74] to structure the strategic issuesof the contextual environment to ensure that the set ofprojections was balanced. In addition, we clustered thestrategic issues according to the four major supply chain riskconstructs of Jüttner et al. [75]: targets (projections 1–2),sources and causes (projections 4–5), consequences (pro-jections 3, 6, 13, 14) and measures (projections 7–12).

In a third workshop, projections were formulated based onthe strategic issues identified previously. To ensure methodo-logical rigor, we employed proven methodological rules, suchas concerning formulation, non-ambiguity and optimal num-ber of words, in order to ensure high validity and reliability[76,77].Moreover,we aimed to ensure that the processing timeto fill the questionnaire was kept as short as possible sinceproper consideration of a few questions was more valuable forus than the cursory evaluation of a large number of questions[78]. The preliminary set included 16 projections on the futureof SCS 2030.

Before the final workshop of the core research team, amonitoring team, consisting of two independent researchers,separately checked for content and face validity of theprojections through comprehensive literature review andpersonal assessment [69]. Due to potential overlap of discus-sions and after final refinement of projection formulations, twoprojections were eliminated resulting in a final set of 14projections. Table 2 presents the final set of 14 projections andtheir origin in literature.

Table 2Strategic issues and literature for future projections.

No. Group Strategic issue Underlying literature (selection) Projection

1 T Number of attacks Sheffi [34]; NCTC [79] 2030: The number of attacks on supply chainshas increased.

2 T Hubs and nodes Hendricks and Singhal [80];Ratick, Meacham et al. [81]

2030: Logistics hubs (e.g. ports, airports) andinfrastructural nodes (e.g. bridges, channels)are preferred targets for attacks.

3 C Regional impact Bram, Orr et al. [82]; Banks [83] 2030: Targeted attacks on supply chains orhubs have destabilized the economies of someregions.

4 S Cyber attacks Warren and Hutchinson [84];Goodman, Kirk et al. [85]

2030: Cyber attacks cause more damage tosupply chains than physical attacks.

5 S Competitors Banks [83]; Speier, Whippleet al. [86]

2030: The number of attacks on supply chainsby competitors (e.g. sabotage, espionage) hasincreased significantly.

6 C Cost Lee and Wolfe [41]; Sarathy [87] 2030: Security has become one of the mostimportant cost drivers for logistics.

7 M Government Sheffi [34]; Closs, Speier et al. [40] 2030: Government institutions play the leadingrole in ensuring secure supply chains.

8 M Technology Mitnick [88]; Closs, Speier et al. [40] 2030: Using advanced technology is the best wayto guarantee security.

9 M Transport routes Wilson [89]; Kendall and Kendall [90] 2030: Regional threats to security have causedshifts in transport routes.

10 M Complexity Modarress, Ansari et al. [39];Vecchiato [91]

2030: Supply chain complexity has been reduceddue to unresolved security problems.

11 M Audits Closs and McGarrell [37]; Sarathy [87] 2030: Security audits are compulsory along theentire supply chain, from raw material deliveryto point of sale.

12 M Prevention/reaction Sheffi [34]; Lee and Wolfe [41] 2030: Strategies to cope with emergencies aremore effective in dealing with disruptions thanpreventive measures.

13 C Data privacy Lee and Rao [92]; Aquilina [93] 2030: Concerns about data privacy are increasinglyignored in favor of greater security.

14 C Time Lee and Whang [94]; Modarress,Ansari et al. [39]

2030: Additional security measures have resultedin increased transport times.

T — targets, S — sources and causes, M — measures, and C — consequences.

1821C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

3.3. Selection of Delphi panel

In order to select the most suitable experts for our Delphipanel, we followed a rigorous selection procedure thatincluded expertise dimensions such as current managementlevel, academic background, job specialization, education,functions inside and outside of the organization, publications,and age [95–97]. In total, we were able to identify 745potential SCS experts for our survey, who were invited toparticipate. Additional support in expert recruitment wasgiven by the Transported Asset Protection Association(TAPA)2 which is a forum with approximately 600 membersthat unites global manufacturers, logistics providers, freightcarriers, law enforcement agencies, and other stakeholderswith the common aim of reducing losses in internationalsupply chains.

3.4. Collection of data

For participation, experts received an invitation hyperlinkvia email by which they could access the Delphi survey portalat any time over a three-month time period. In the onlinequestionnaire, experts were asked to assess a number of

2 www.tapaemea.com.

projections regarding their estimated probability (EP) ofoccurrence (scale ranging from 0 to 100%), impact (I) on thetransportation and logistics industry (5-point Likert scale),and desirability (D) of occurrence (5-point Likert scale) forthe year 2030. The far horizon of 2030 was purposefullychosen in order to stimulate “out-of-the-box” thinking [98].In addition, experts were motivated to provide qualitativearguments to support their quantitative evaluations in all thethree dimensions. Afterwards, experts' initial assessmentswere saved, group feedback was provided, and participantshad the opportunity to revise prior evaluations.

Overall, 80 of the invited 745 experts (10.7%) participatedin the study, of which 55 (69%) were from industry, 16 (20%)were from science, and 9 (11%) came from politics or otherassociations. The industry share included representativesfrom all modes of transport in the supply chain as well asinfrastructure operators, such as logistics service providers,ports, hubs and airports. Participants were based in 25different countries, ensuring a global view and differentperceptions on security. The final panel included, amongothers, CEOs and managing directors, strategic planning andbusiness development managers, risk and crisis managers,compliance managers, as well as corporate security man-agers. By including such a diverse mix of SCS experts in ourDelphi panel, we were able to achieve a multi-stakeholderview on the topic and to motivate a controversial discussion

1822 C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

among experts. Moreover, our panel size exceeded theminimum recommended number of 30 Delphi participants[99,100], which further increased the quality and robustnessof the survey results due to the higher number of incorpo-rated perspectives.

4. Data analyses

The Delphi research method generates an extensiveamount of qualitative and quantitative data. The investiga-tion of the applicability of the Delphi foresight method forrisk analysis by identifying and evaluating methodologicalbenefits requires a thorough analysis of this Delphi data.

4.1. Quantification of risks

We start our analysis by examining the quantitative dataof experts' estimated probability, impact and desirabilityassessments of the surveyed projections. This serves as aquantitative evaluation basis of the expected future devel-opment of the identified emerging security risk topics. Wealso calculated the interquartile range, which is an acceptedindicator for consensus measurement [101,102]. Further-more, we measured the convergence rate to reveal whetherthe standard deviations of initial assessments decreased incomparison to final assessments. By doing so, we could studywhether the exchange of arguments in the Delphi commu-nication process contributed to the exploration of the SCStopic.

4.2. Analysis of stakeholder perspectives

Due to the large number and the heterogeneity of ourexpert panel and consequent differing security perspectivesand interests, experts' evaluations of SCS topics can and shouldbe further analyzed. We applied the approach by Ecken et al.[103] to test and correct the desirability bias of our Delphi data.Desirability bias describes an expert's high desirability assess-ment and accompanied high probability assessment of aprojection to become true due to individual preferences.Afterwards, we conducted a stakeholder analysis in order toexploit the quantitative data in more detail. To do so, weapplied statistical tests (e.g. t-test) in order to identify patternsin experts' assessment behavior. Additionally, we analyzedcertain characteristics from experts (i.e. gender, nationality andworking sector) when we split up the expert panel.

3 Also known as black swans [106].4 Outliers are observations that are numerically distant from the rest of

the data. They lie between one and a half times and three times theinterquartile range [109].

5 Extreme points are observations that are numerically distant from therest of the data. They lie beyond three times the interquartile range [109].

4.3. Analysis of communication process

In order to conduct a sound analysis of the qualitative data,we followed two approaches. First, we coded the writtenarguments and aggregated them for every projection andevaluation dimension (EP, I, D). We followed the general andestablished coding procedure by Corbin and Strauss [104] andworked against coding bias by involving two different re-searchers. The result of the coding process enabled us toconduct a frequency analysis and to identify the prevalentreasoning.

4.4. Identification of rare events and extreme positions

In the second step of our qualitative analysis, we focusedon unique arguments and extreme assessments of pro-jections' impact and desirability. As Schwarz [105] remarked,the Delphi method is in particular useful in identifying andexploring new issues. Therefore, we aimed to identify newrisks in the topic by the Delphi method. We were able toreveal outlier opinions, weak signals and wildcards,3 such ashigh-impact, low-probability events [107,108], which maylead to changes in future developments and therefore need tobe considered in future risk analysis. Outlier opinions can bedetected during qualitative data analysis of arguments butalso during statistical analysis of quantitative data. In thelatter case, researchers look for outliers4 or even extremepoints5 in the data as well as their argumentations. Weaksignals are insinuations that are too incomplete for anaccurate impact assessment and corresponding measures[110] but are considered to be initial indicators for potentiallyimpactful developments, such as threats or opportunities[111,112]. Therefore, qualitative data needs to be carefullyanalyzed for indications that point to emerging opportunitiesor threats. Pattern analyses (e.g. by data mining) can help tofind these hints at an early stage. For the identification ofwildcards, an inductive as well as a deductive approach exists[113,114]. While the inductive approach is based on commonknowledge about certain possible developments, the deduc-tive approach aims to generate new (i.e. unconsidered)disruptive events. This requires the use of future projectionsand scenarios which describe an unexpected picture of thefuture. The picture of the future becomes even more realisticbut also complex, by combining Delphi with a cross-impactanalysis, which facilitates in creating scenarios in which theeffect of different risk factors on each other is also considered[115]. Reasoning the circumstances that are required to get insuch a situation allows new wildcards to be determined[113,114]. Therefore, the Delphi method offers the requiredcircumstances to derive new wildcards with limited effort.Within our analysis, we pay particular attention to theprojections with distinct probability estimations (eitherhigh or low) and look for arguments that point in the otherdirection.

4.5. Development of risk scenarios

In the last step of our analysis, we combined quantitativeand qualitative data to generate scenarios. We understandscenarios as internally consistent, plausible, and challengingnarrative descriptions of possible situations in the future, basedon a complex network of influencing factors [116,117]. Theyare considered to be an inevitable approach to expand peoples'cognition and to describe possible futures [118], since there areno methods that can certainly predict relevant driving factors,probability values or consequences of developments [59].

1823C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

Within this paper, we present the development of threedifferent kinds of scenarios: (im)probable, (un)desirable and“extreme” scenarios. For the development of reasonablescenarios, we clustered the projections according to theirestimated probability and impact [61,119,120] and received(im)probable scenarios. In a second step, we took thedesirability and probability assessment into account and weretherefore able to derive (un)desirable scenarios for the topic ofSCS. For development of extreme scenarios, we applied thescenario axis approach [121,122]. The scenario axes are awidely recognized and presented tool in foresight research toconstruct scenarios in a coherent and systematic way [121]. Inthis method, two driving forces or key factors that are highlyuncertain and have a high impact are selected. While somecriticize this approach as a subjective and unjustifiednarrowing of the future along merely two factors, othersappreciate the method as a supportive framework to approachcomplex future topics [121,122]. The selected most importantfactors are then to be plotted on two axes, resulting in fourdifferent scenario quadrants. In a succeeding step, the fourscenarios are filled in by creating four different stories of howthe future may unfold. Such scenarios entail the entireparameter range of two selected projections by describing thefour possible combinations of extreme values for these pro-jections. These scenarios stimulate “out-of-the-box” thinkingby confronting multiple stakeholders with extreme situations.

Table 3Quantification of projections.

No. Projections on SCS EP final rou

1 2030: The number of attacks on supply chainshas increased.

56.0%

2 2030: Logistics hubs (e.g. ports, airports) andinfrastructural nodes (e.g. bridges, channels)are preferred targets for attacks.

57.7%

3 2030: Targeted attacks on supply chains or hubshave destabilized the economies of some regions.

49.3%

4 2030: Cyber attacks cause more damage to supplychains than physical attacks.

49.4%

5 2030: The number of attacks on supply chainsby competitors (e.g. sabotage, espionage) hasincreased significantly.

29.9%

6 2030: Security has become one of the most importantcost drivers for logistics.

56.5%

7 2030: Government institutions play the leadingrole in ensuring secure supply chains.

44.1%

8 2030: Using advanced technology is the best way toguarantee security.

58.6%

9 2030: Regional threats to security have caused shiftsin transport routes.

60.6%

10 2030: Supply chain complexity has been reduced dueto unresolved security problems.

30.4%

11 2030: Security audits are compulsory along the entiresupply chain, from raw material delivery to point of sale.

70.2%

12 2030: Strategies to cope with emergencies are moreeffective in dealing with disruptions than preventivemeasures.

44.3%

13 2030: Concerns about data privacy are increasinglyignored in favor of greater security.

55.7%

14 2030: Additional security measures have resulted inincreased transport times.

64.4%

EP — estimated probability of occurrence (0–100%); D — desirability (5-point-Likeimpact (5-point-Likert scale); IQR — interquartile range (≤25 equals consensus).

Thereby, fictitious environments are offered to decisionmakers, which allow the vulnerability of their business inextreme situations to be tested by conducting impact analyses[123] as well as measures that increase emergency prepared-ness to be derived. In an advanced approach, Delphi could becombined with cross impact analysis to develop even morecomplex scenarios in which events are not isolated butinterrelated [115].

5. Results and discussion

The overall aim of this paper is to illustrate and assess thepotentials of the Delphi method for risk analysis, includingidentification and estimation thereof. The application of theDelphi method generally results in a large amount ofquantitative and qualitative data. In the following discussion,we investigate to what extent the data can be used for riskanalysis. We structure our argumentation and analyses alongthe previously derived quality criteria for risk analysis.

5.1. Identification and quantification of risks

In a first step of our analysis, we study the quantitativeresults of ourDelphi survey in-depth. As previouslymentioned,an important step in the risk analysis is the quantification of theidentified risks in order to support decisionmakerswith a solid

nd EP initial round EP adjusted IQR CV I D

54.8% – 31 −11.6% 3.7 1.4

57.7% 71.4% 30 −4.0% 3.9 1.2

49.8% – 36 −2.1% 3.5 1.3

51.5% – 40 −5.6% 3.7 1.4

34.6% – 30 −16.4% 2.9 1.3

57.5% – 30 −2.3% 3.5 2.2

45.9% – 30 −7.7% 3.4 2.8

58.7% 58.1% 25 −4.9% 3.5 3.0

60.0% – 25 −10.2% 3.6 1.9

31.3% – 20 −10.3% 3.2 2.2

67.0% 65.1% 25 −12.6% 3.8 3.4

42.8% 50.7% 20 −12.4% 3.0 2.5

56.2% – 20 −6.1% 3.1 2.1

59.7% – 30 −8.7% 3.8 1.9

rt scale); CV — convergence rate (≤10% equals moderate convergence); I —

1824 C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

data base. Therefore, we listed in Table 3 the 14 projections onSCS together with their accumulated quantitative evaluations.We calculated mean values of the assessments for estimatedprobability (EP) and desirability of occurrence (D) as well asimpact (I) on the industry.

These values are valuable for multiple stakeholders anddecision makers for two reasons. First, the values express theestimations of the surveyed, relevant stakeholders and,therefore, provide a clear picture of their perceptions of therelevant SCS factors. Second, the assessment of the factorsprovides a quantitative data basis on which multiple stake-holders and decision makers can develop strategies andmeasurements — a previous research gap which has beenhighlighted by several authors (e. g. Rice and Spayd [42]).However, the reported values may not be considered sepa-rately. Their conjoint reflection is necessary in order to conductan appropriate risk assessment, whichmay also support in laterderivation of adequate strategies. A high probability value of aprojection, for instance, together with a quite low impact valuewould make the respective projection rather irrelevant forconsideration. On the other hand, a high impact value of aprobable projection would convey the relevance of theunderlying factor and consequently cause an in-detailexamination.

Table 3 reveals that the means of nearly all impact ratingsare 3, that is medium, or above, which approves theimportance of the examined SCS topics and can be consid-ered as a quality criterion for the relevance of the developedprojections. The assessment of desirability in the context ofsecurity concerns is consistently low, as anticipated forprojections which treat the targets or sources of man-madeattacks. The desirability values vary when it comes tomeasures that could increase the security of supply chains(projections 6–14). Projections which contribute to en-hanced security (projections 7, 8, 11) have higher valuescompared to projections that imply a decrease of efficiencyfactors at the same time (e.g. transport time, distance or cost)(projections 6, 9, 10, 12–14). The mean desirability values arerelatively low, which is not astonishing in the context ofsecurity. However, querying the desirability estimation of aprojection to become true facilitates in calculating thedesirability bias of the expert, which might have biased theEP assessment. In the most cases, the Delphi methodcompensates for biases and subjectivity in the assessment.According to the approach of Ecken et al. [103], a significantbias in the EP assessment according to experts' wishes,determined by the desirability value, could still be identifiedfor four projections (projections 2, 8, 11, 12). The strongestimpact of this bias was determined for projection 2.Consequently, it can be assumed that the expected probabil-ity of projection 2 is significantly higher and that the topic ofsecuring logistics hubs and infrastructural nodes will requiremore attention in the future. This is supported by literatureabout current needs in supply chain security management aswell as governmental and international security initiatives,where hot spots and infrastructure bottlenecks are majortopics [37,38,41,124].

The probability of occurrence (EP) value determines anessential indicator: whether a certain topic might arise in thefuture. For the specific risk analysis of firms, the use of thesevalues is limited, due to the broad and generic formulation of

the projections. However, the evaluation gives an objectiveindication as to which of the topics are expected to gainimportance in the future. An initial assessment of a relativelyunexplored topic, such as SCS, is especially useful for decisionmakers from politics and associations who are working in aglobal environment. However, it is also presented tocompanies so that they know where they should focus theirefforts and more specific risk analyses on. In fact, previousauthors suggested making quantitative assessments of futurerisks of supply chain disruptions and identified the lack ofquantitative assessment as being one of the major researchgaps in supply chain security literature [38].

In addition, the Delphi data can be studied in order to detectwhether convergence in experts' assessments could be ob-served. This aspect is especially relevant in the context of riskanalysis since it demonstrateswhether the extensive exchangeof information led to a homogenization of risk assessments. Thenegative convergence values for each projection reveal that thegroup communication process facilitated a consensus buildingprocess, and led to convergence in the projection assessment,consequently to a harmonization of the different securityperceptions. It additionally indicates that the Delphi methodstrongly facilitated the consensus building process. For 6 out of14 projections, consensus among experts, which we define asIQR≤25 according to former successful Delphi studies[101,125,126], could be achieved. It is a valuable indicator ofthe former degree of uncertainty in the probability estimates.Strong consensus on an estimate indicates more certainty andreliance for risk analysis than high dissent. High-dissentprojections are, however, of particular interest for more in-depth analysis since outliers or opposing opinions mightindicate further weak signals, which have not yet beenidentified.

5.2. Elicitation of stakeholder perspectives and worldviews

Altogether, the Delphi process showed dissent for 8 out of14 projections. This corresponds with the aim of the Delphiprocess to integrate multiple stakeholders and their variousperspectives [62].

Considering different worldviews in a cross-national field,such as SCS, is highly relevant for risk analysts and decisionmakers for two reasons. First, the review of various globaloptions, concerns and supporting evidence improves thequality of decisions. Second, the discussion of disputabletopics can enhance the acceptance of decisions and therebytheir implementation due to the recognized critical reflectionof the topic. The results show that attention is paid to thegeneration of the strongest possible opposing views on thepotential resolutions of a major policy issue [69]. To enhancethe explanatory power of our quantitative data, we analyzedthe responses of the expert panel in more detail in order findpatterns among panelists' assessments.

Therefore, we used the probability of occurrence evalua-tion from projection 1 ‘number of attacks’ for two reasons:the projection generally asks whether an expert expects theSCS topic to increase in the future. Consequently, the resultsfrom projection 1 offer valuable insights about the generalmindset of an expert regarding the future threat for globalsupply chains. Furthermore, this projection demands afundamental reflection of the topic by the expert. Therefore,

Table 4Differences in experts' assessments.

EP assessment of expertsassessing P1…

P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14

Relaxed 46.82 37.88 42.12 27.55 54.55 42.18 52.67 57.12 26.67 60.85 42.42 51.21 56.06Concerned 65.26 57.34 54.57 31.49 57.81 45.49 62.77 63.00 32.96 76.81 45.68 58.79 70.21Average 57.61 49.27 49.38 29.85 56.46 44.12 58.58 60.56 30.35 70.19 44.33 55.65 64.34Asymp. sig. (2-tailed) 0.001⁎ .000⁎ 0.04⁎ 0.196 0.491 0.965 0.076 0.196 0.102 0.004⁎ 0.559 0.178 0.007⁎

P=Projection.⁎ Significant at the 0.1 level.

1825C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

experts' probability assessment of the following projectionsmight be influenced by the probability assessment of the firstprojection, which is described in the literature as “anchoringeffect” [74]. Depending on whether an expert assessedprojection number 1 to be probable or improbable, wedefined his/her group membership. Those experts, whoassessed the occurrence of projection 1 to be greater than50%, were defined as belonging to the group of concernedexperts, while the remaining participants were defined asrelaxed experts. Following this allocation, we conducted anindependent sample t-test and studied whether concernedand relaxed experts' assessment for the remaining 13projections differed significantly.

In total, the assessment of 6 projections significantlydiffered depending on whether a participant belonged to thegroup of concerned or relaxed experts (cf. Table 4). The groupof concerned experts, who strongly believe that the number ofattacks on supply chains will increase by 2030, includes 59% ofthe participants. On the other hand, the remaining 41% of theparticipants are more relaxed and do not believe an increase insupply chain attacks to be likely to happen. Depending on theiraffiliation to the group of concerned or relaxed experts,probability assessments vary.

According to this classification, experts who expect thenumber of attacks on supply chains to increase also believe, forinstance, that these attackswill mainly happen at logistics hubsor via Internet. Therefore, by splitting the expert panel in‘concerned’ and ‘relaxed’ experts, we illustrate the existence oftwo worldviews among experts regarding SCS. Sinceman-made attacks usually strike weak links, relying on onejoint worldview could be misleading: security topics in whichneither location nor involved parties are known may causedissent at an international stage — and creating a discordantrisk assessment may create such weak links. Therefore, theidentification of ambiguous topics and the analysis of thecorresponding qualitative arguments can provide indicationsthat might be helpful for timely adaption or development ofappropriate policies and strategies.

Table 5Stakeholder analysis.

No. of experts Countries

Europe Asia America Australia Afri

Total 52 10 14 2 2Concerned 31 4 9 1 2Relaxed 21 6 5 1 0

The stakeholder analysis of Delphi panelists can offerfurther valuable findings for risk assessment. Multiple stake-holders can distinguish if an issue is expected to be critical inthe future in general or only for a certain group of stakeholders,which for instance can be characterized by nationality, industryor gender. These characteristics can be used in order to splitgroups even further or to identify completely other groupswithin a Delphi panel. We demonstrated this procedureexemplarily by investigating the characteristics of the expertsof our initial panel into ‘concerned’ and ‘relaxed’ experts. Westrive to highlight the relevance for multiple stakeholders toanalyze the groupswithin a Delphi panel, since the analysis canbe helpful to better adjust risk strategies (e.g. according toregional relevance, organizational structure or businessmodel). Although the sample size does not allow for furtherstatistical tests due to the limited number of cases, there issome indication that no difference exists between women andmen in terms of risk perception, but that Europeans andAmericans are more concerned than relaxed. Almost allacademics in our sample are concerned stakeholders (cf.Table 5).

5.3. Stimulation of a global communication process

In addition to the quantitative data, the Delphi method hasthe potential to generate an extensive amount of qualitativedata. This data offers valuable insights for risk assessmentwhich help to complement the quantitative data and to draw aclearer picture on projections. The Delphi method stimulatesmore than other risk analysis methods, such as simulation,brainstorming, nominal group technique, and the communica-tion process among experts [54,127]. Overall, approximately850 arguments were provided by the Delphi panelists in orderto justify their quantitative assessments. This demonstratesexperts' high interest in the topic as well as the strong desirefor a comprehensive exchange of information. Furthermore,the online platform allowed for surveying SCS experts from 25countries. The internationality of expert panels is especially

Gender Sector

ca Male Female Academia Industry Politics

67 13 15 56 939 8 13 30 428 5 2 26 5

1826 C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

relevant for global challenges, such as the security of globalsupply chains. Perceptions of different stakeholders are takeninto account and discussed in a global context.We recognize anincreased interest of experts for extensive discussion due to theglobal character of this survey. The multi-faceted discussion ofprojections, in combination with different national andregional perspectives, offers a unique handling of the wickedproblem of SCS and is expected to produce valuable findings.

Table 6Comparison of coded arguments from ‘relaxed’ and ‘concerned’ experts.

Relaxed experts (top 3 in terms of frequency) Ent

1.) 2030: The number of attacks on supply chains has increased.

Total number of comments: 75 (25 arguments for low probability; 50 argumentsGovernmental security initiatives and prevention awareness will preventthis (cf. Authorized Economic Operator (AEO), Customs–TradePartnership Against Terrorism (CT-PAT))

6

Security processes will improve 3

The state of vigilance will improve considerably worldwide 3

2.) 2030: Logistics hubs (e.g. ports, airports) and infrastructural nodes (e.g. bridges,

Total number of comments: 57 (24 arguments for low probability; 33 argumentsAdequate security systems and technology safeguard attacks withininfrastructural nodes

5

Logistics hubs are easily defendable compared to transport modes (truck,train, etc.) along the entire transport routes

4

Terrorists are more interested in injuring people than in damaginginfrastructure

3

3.) 2030: Targeted attacks on supply chains or hubs have destabilized the economies

Total number of comments: 33 (19 arguments for low probability; 14 argumentsGlobal supply chains will provide alternates and resilience to the degreethat regional destabilization will be short-lived

3

This might only be the case for some regions as it can already be seen today 3

Targeted attacks will only happen occasionally and cause temporarylimitations. Therefore, they will not seriously destabilize an entire region

3

4.) 2030: Cyber attacks cause more damage to supply chains than physical attacks.

Total number of comments: 41 (17 arguments for low probability; 24 argumentsCyber disruptions could clearly occur, but firms have potent incentives toincrease their IT security level in the near future

5

Physical attacks are very effective and lead to predictable results 2

If the scarcity of resources is the cause of attacks on supply chains, then itfollows that physical attacks are more likely than cyber ones

2

11.) 2030: Security audits are compulsory along the entire supply chain, from raw mTotal number of comments: 33 (15 arguments for low probability; 18 argumentsOnly necessary for certain goods and industries 6

Impossible without a huge re-structuring of security systems andorganizations, globally

3

Depends on technological capabilities and advancements 2

14.) 2030: Additional security measures have resulted in increased transport times.

Total number of comments: 36 (16 arguments for low probability; 20 argumentsTechnology will drive efficiency and therefore, transportation lead timeswill stay the same

7

It depends on qualification of the people and the security measure 2Adequate security measures will be integrated into logistics processes 2

Furthermore, the many considered perceptions of securityin supply chains increase the quality of the final resultssince it allows for cross-validation and data enrichment tomanage the situation of deep uncertainty. Consequently, aDelphi-based risk analysis can be expected to incorporateworldviews of multiple stakeholders and therefore to yieldrobust decision-making [14] regarding mitigation strate-gies. This addresses one of the major concerns in risk

ries Concerned experts (top 3 in terms of frequency) Entries

for high probability)Terrorists and criminals have discovered supply chains(SCs) as a means to their end due to SCs' highvulnerability and severe impact on the world economy

11

Number of attacks on supply chains has increased overthe last years

10

Expected to increase due to economic downturn and theconsequent rising unemployment and increasing gapbetween rich and poor

7

channels) are preferred targets for attacks.

for high probability)Hubs are an easy target with high severe economicconsequences and high media visibility

10

They are the weakest link in transport infrastructure(nodes) or the most crowded and important places(hubs) for any supply chain

7

Increase in attacks can already be observed today.Due to the growing cargo and passenger traffic, thenumber of attacks is likely to increase as well

4

of some regions.

for high probability)Exposed vulnerability in the media will undermineconfidence in that region as well as its stability

2

The effect depends on the size and location of the regionas well as its dependency on this hub

2

High risk areas influence future strategies of companieswhich can cause economic weakening of the region

2

for high probability)Cyber attacks can be a neater way of disrupting SCs(low resources required; high impact)

5

The increasing IT dependency makes it possible thatcyber attacks can shut down supply chains

4

With technology playing a major role in SCs, cyber attackswill increase

4

aterial delivery to point of sale.for high probability)

Current security initiatives will require an auditingprocess

6

That will be mandatory for any quality of servicestatements from a company

2

Adequate security systems will deteriorate without control 2

for high probability)This is probable to happen, even if technology andcounter-movements will mitigate such effects

4

It is currently moving in that direction 4Probable for some origins/destinations 2

1827C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

analysis, as discussed by Slovic [20] as well as Acedo andFlorin [21].

Another major advantage of the Delphi method – especiallyif applied in a real-time version – is its capability to facilitate aninternational exchange of information in a minimum of time.Over a period of only three months, experts participated in ourmulti-round survey. According to their specific time restrictionsand level of interest, they could access the survey platformmore than once and participate in the international dialogue.The immediate feedback and the efficient survey design did notlead to panel mortality – a classical phenomenon over multiplerounds in paper and pencil Delphi surveys – since the minimumof one initial round and one revision round are completed in onesession. Especially when involving experts from many timezones, cultures and disciplines, a survey design that providesdirect benefits, such as direct and time-independent feedback[43], is required.

We recommend stimulating the experts to share theirthoughts on a topic in order to add valuable information totheir quantitative evaluation. The entered arguments helpmultiple stakeholders to underpin the numerical data withinsightful content. This is especially valuable when it comesto stakeholder analysis. Thereby, concerns of the differentstakeholders can be identified and accordingly consolidated.Furthermore, the explicitly requested provision of argumentsprovokes extensive discussion, which is highly relevant forespecially controversial risk topics [68,69]. Our previousgroup analysis revealed that even security related topics,which might be expected to lead to a homogeneousassessment, are evaluated differently. We apply our groupassignment with ‘relaxed’ and ‘concerned’ experts from theprevious analysis and present an abstract of the soundreasoning of both parties for the controversially discussedprojections (cf. Table 6).

The analysis of these arguments reveals that there arecomprehensible explanations for both directions of futuredevelopment in the presented topics. In general, concernedexperts worry most about current trends in terroristic attacksand are dominated by their perceived vulnerability of oureconomic system, represented by global supply chains. Theirargumentation points to a certain bias due to recentterroristic events [32]. Relaxed experts evaluate technologyimprovements and governmental measures along all diverse-ly discussed projections as the two major driving forces togain control over man-made attacks. Both topics are alsostrongly discussed in literature, but without relating it todifferent worldviews (e. g. [41]). Therefore, these two topicswill be used in Section 5.5 for the development of extremescenarios.

Table 7Exemplarily findings in supply chain related wildcard identificat

No. Wildcard

1. Security-relevant company in2. Introduction of a “war insuran3. New transport routes due to d4. Certain means of transport (e

by hackers5. Invention of mind reading sca6. Privatization of important tran

5.4. Identification of weak signals, outlier opinions and wildcards

The request for qualitative arguments as part of a Delphisurvey has many advantages. The major purpose of theexchange of arguments in a Delphi survey is to reduce theinformation asymmetry and to generate convergence ordivergence in experts' assessments. However, the identifica-tion of extreme positions within a survey group can also offervaluable findings for risk analysis. The specific evaluation ofexperts with an outlier opinion often disappears or isneglected in the statistical analysis of surveys with numerousparticipants although the consideration of unique argumentscan be productively used for early risk identification. We canconclude a twofold advantage from our qualitative Delphidata for risk identification: First, data can be analyzed toidentify weak signals. Raising early awareness for thesepotential changes and their possible consequences can helpto reduce uncertainty by taking them into account when itcomes to risk assessment. This can reduce the vulnerabilitiesof supply chains as well as support the implementation ofpreventative or the development of reactive measures at anearly stage. In fact, this supports the conceptual approach ofLee and Wolfe [41], who suggest applying prevention,process control and design improvement according toestablished quality management procedures.

Second, the qualitative data can be examined to identifywildcards through qualitative data analysis of expert com-ments. However, the identification of weak signals andwildcards is time consuming for large expert panels andstrongly dependent on the experience and subjective assess-ment of the data analyst. Based on the discussion and theresults of our Delphi survey, we were able to identify a list ofwildcards which are relevant to the topic of SCS (cf. Table 7).

Wildcards are of special interest for topics which areconsensually assessed to be very probable or improbable. Forsuch topics, the future development is often valuated to beknown. Consequently, the occurrence of an unexpected eventwould have a devastating effect. Following this logic, we payparticular attention to the projection with the lowestestimated probability of occurrence (i.e. projection 5 “com-petitors”). The arguments from experts regarding this topicreveal that it is highly improbable that companies would risktheir public reputation by sabotaging competitors. Consider-ing the recent case of WikiLeaks, in which highly confidentialdata was published on the Internet, a comparable situationcan be imagined. As demonstrated with the first wildcard inTable 7, the provision of security relevant information abouta company by displeased or fired employees would facilitatethe planning of harmful actions and lower the timidity of

ion.

formation are traded online (cf. WikiLeaks)ce” for dangerous transport routeseglaciation of Arctic icebergs.g. container ships or trains) become remote-controlled

nners and installation at major hubssport routes

uncertain, medium impact

probable, medium impact

improbable, medium impact

Fig. 2. Portfolio-based scenario development: probability vs. impact.

1828 C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

competitors to act in secrecy. Many ongoing examples (e.g.illegal file sharing or violence glorifying videos) explicitlyprove how difficult it can be to prohibit content on theInternet or to remove data permanently from it. Onceidentified, risk assessment should include a description ofthe wildcard events and further facts and figures from deskresearch for plausibility purposes and impact analysis.

5.5. Facilitation of risk scenario development

As another part in the risk analysis process, we combinedthe quantitative and qualitative data of the Delphi survey inorder to develop risk scenarios that expand the future horizonand include further perspectives and worldviews for the riskanalysis. We identified a particular value of scenarios for riskanalysis in the field of stakeholder engagement, communicationfor awareness, but also as simulation and discussion platformsof impactful disruptive events [123]. Within the scope of ourresearch, we found two valuable analyses that facilitate riskscenario development: portfolio-based scenario development

uncertain, undesirable

improbable, undesirable

pr

Fig. 3. Portfolio-based scenario develop

(strategymaps) and the use of Delphi data for the developmentof scenario axes.

The Delphi data can be illustrated on scatterplots usingquantitative assessment of the different projections regardingestimated probability of occurrence, industry impact anddesirability. Depending on the values, scenarios such assurprising (i.e. low-probability, high-impact) or expected (i.e.high-probability, high-impact) can be analyzed (see Figs. 2 and3). Clustering of projections can be either subjectivelyperformed or by cluster analysis. For exemplary purposes, wefollowed subjective clustering. The creation of the (im)probable scenarios can be conducted by a graphical illustrationof the probability and impact evaluations (Fig. 2). For thispurpose, we only take projections with a high or lowprobability assessment into account. Furthermore, the selectedprojections should have a certain impact on the industry inorder to develop relevant scenarios. Probable scenarios can beused to understand how a certain topic is expected to evolve,which decreases the inherent uncertainty. Instead, improbablescenarios should be considered in order to be aware of possiblechanges and wildcards which might seriously jeopardize

obable, desirable

ment probability vs. desirability.

1829C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

business. Consequently, improbable scenarios assist in reduc-ing an organization's vulnerability for worst case develop-ments. In Fig. 3, we apply a similar approach for thedevelopment of (un)desirable scenarios. This kind ofportfolio-based scenario is helpful in specifying differentperceptions and attitudes. Therefore, conclusions can bedrawn about what behavioral patterns can be expected fromcertain stakeholders. Combined with the results of a formergroup analysis, both (im)probable and (un)desirable scenarioscan be further assessed regarding their relevance for a specificcountry, company or business model. Furthermore, thegraphical approach facilitates the development of scenariosand enhances their acceptance due to a comprehensible andtraceable development procedure.

Second, applying our Delphi data, we exemplarilyconstructed four extreme scenarios for the future of SCS. Forthis purpose, we selected two projections which showed anambiguous probability assessment (i.e. EP value close to 50%)andwere estimated by the experts to have a serious impact onthe future development of SCS: the ‘role of governments’(projection 7) and the ‘role of technology’ (projection 8) forproviding future security. In addition, the results of the initialdesk research for the projection development (e.g. [34,40]), aswell as the analysis of the arguments provided by ‘concerned’and ‘relaxed’ experts, identified these topics as two drivingforces for SCS. We used the provided qualitative arguments ofthe projections to characterize the different scenarios in moredetail. If one considers each axis as a metric scale from 0 to100 percent probability, i.e. an operationalization of the

Scenario 1: Open personal privacy at work

Scenario 3: Conjoint fight against crime

Technology can provide se

No involvement of

goverments

• Every employee represents a security issue

• Companies ignore personal data privacy in favor of security

• Surveillance technology monitors every area in a company

• Crime and deception are outlawed • Every employee perceives himself

and is perceived as a security officer by his company

• Strong moral courage and solidarity instead of expensive security equipment

Technologycannot provide

Probable

Fig. 4. Extreme scenarios for SCS considering th

projection itself, using Delphi data for scenario axes evenallows sketching in the probable Delphi data based scenario.Fig. 4 illustrates the scenario axes as support for further riskanalysis.

6. Conclusion

The overall aim of this research was to illustrate andassess the applicability of the Delphi technique for riskanalysis, including risk identification and estimation thereof.In order to be able to draw future-oriented conclusions, weselected the topic SCS, which has been determined toinclude wicked problems such as terrorism [13]. We wereable to illustrate and assess fivemajor benefits of Delphi datafor risk analysis: (1) identification and quantification ofrisks, (2) analysis of stakeholder perceptions and world-views, (3) stimulation of global communication process, (4)identification of weak signals, outlier opinions, andwildcards, (5) and facilitation of risk scenario development.In essence, Delphi can assist in reducing uncertainty andthereby the companies' susceptibility and vulnerability tovarious kinds of disruptions.

The method has already been recommended to be espe-cially helpful in the field of security and terrorism [32,68]without enlisting specific reasons in more detail or providingempirical data. Consequently, there was a research gap ofempirical support in how to apply the method in this field orwhat specific benefits it has for analyzing risks in a globalenvironment, which we attempted to close.

Scenario 2: „Big Brother“

Scenario 4: Open! Control

itself curity

Strong involvement

of goverments

• State control by highly sophisticated surveillance technology

• Mind reading technology helps to identify offenders

• Security is governmental duty

alone security

• Authorities randomly control factories, offices and other facilitiescomparable to today‘s food controls

• Undercover SCS marshals in meansof transport and logistics hubs

• Disclosure and control of companies’ top secret topics

Scenario

e role of governments and technologies.

1830 C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

The Delphi method – especially if applied in a real-timeversion – has several benefits which are particularly helpfulfor risk analysis. First, it allows quantifying uncertainty andrisks by conducting a multi-stakeholder survey without beingrestricted by the variety of experts' time and place. Includingthe different worldviews from an international panel ofexperts is crucial for topics such as supply chain managementor terrorism, which have global sources and consequences.We demonstrated an effective evaluation process of relevantfactors of SCS regarding probability of occurrence, desirabilityof occurrence and impact on industry using an online Delphiplatform. Thereby, we were able to determine which topicsare expected to be relevant in the future and which are not inorder to address crucial uncertainty in SCS.

Second, the stakeholder analysis of our panel revealedthat there are topics on which experts are currentlydiscordant. Consequently, we split our sample into ‘relaxed’and ‘concerned’ experts in order to gain more insights fromour data. We noted that the individual positions of the twogroups could be logically substantiated by the collectedarguments. This differentiation would not be possible byonly considering quantitative data. A further analysis of thecharacteristics of ‘relaxed’ and ‘concerned’ experts can beconducted unboundedly according to the interest of the riskanalyst. Applying the Delphi technique for risk analysisshould therefore include written justifications for estimatesin order to allow for a more in-depth stakeholder analysis.

Third, we demonstrated how the Delphi method stimu-lates and facilitates an extensive communication process bygiving direct and reasonable feedback. Thereby, the methodassists the convergence or respectively divergence amongpanelists and therefore unveils various worldviews in riskperception. Both processes offer valuable findings for multi-ple stakeholders. Especially extreme arguments and outlieropinions can be productively used to generate wildcards andto identify weak signals in a fourth step. These weak signalshave the potential to initiate a serious change in technolog-ical, social, economical or political developments and aretherefore valuable to be aware of as early as possible.Wildcards, on the other hand, can be used to evaluate therobustness of a company in case of an unexpected andimprobable event.

Finally, we conducted scenario development in order toexpand the future horizon and scope of risk analysis. Weillustrate that both portfolio-based scenarios and scenarioaxes can add a complementary value for risk analysis.

Furthermore, we contribute to supply chain risk and securityresearch by providing a methodology for quantitative assess-ment of targets, sources and causes as well as measures andconsequences of supply chain disruptions in a security context.We provide empirical data for assessing risks and possiblemitigation strategies. Additionally, we consider the topic in aninter-organizational set-up including perspectives of the corpo-rate world as well as governmental institutions. Therefore, weaddress some of the main research gaps identified in the supplychain domain. Although our focuswas onman-made attacks, theapplication of the Delphi method is not limited to this kind ofdisruption. It can also be used to analyze risks related toaccidents or natural catastrophes.

However, as with any research endeavor, this paper hassome limitations. First, the conventional Delphi technique

may be more time consuming than other techniques, such asgroup discussions. We tried to compensate this by using areal-time version of the technique. However, selecting andinvolving appropriate experts still require a great deal ofeffort. Therefore, implementing an (expert) network orDelphi modifications which allow a more continuousanalysis process [28,115] would counteract this aspect.Moreover, the expert panel assessed only three dimensions:probability, impact, and desirability. Future research mightexamine the potentials of other dimensions for risk analysis,such as feasibility of measures, urgency for actions, orawareness of issues.

Second, certain subjectivity in risk analysis by Delphi stillexists. Even if we were able to reduce it, the projectiondevelopment, as well as the selection of the surveyed topics,remains arbitrary to the facilitator. Moreover, the identifica-tion of weak signals and wildcards, as well as the develop-ment of the extreme scenarios, is still dependent on theexperience of the analyst.

Third, we assume that the assumption of Parenté [32],that a close terrorist attack can bias the Delphi results, mightalso be applicable to our data. If one considers that piracyalong the coast of Somalia and Yemen has strongly increasedsince 2008, and this has been communicated in the media,the dissent in the projections about the role of infrastructuralnodes (2), regional consequences (3) and increased transporttimes (14) could be caused by experts, who are more or lessaffected by the consequences of piracy. However, this doesnot weaken our findings, but confirms the usability of thereal-time Delphi method in the field of SCS since we wereable to identify the current areas of increased interest amongexperts.

Fourth, we focused our research on the contribution ofDelphi for risk analysis, its identification and assessment.Future research might expand this perspective and examinepotential linkages of Delphi in the entire risk managementprocess and its combination with other supportive risk orforesight methods to a multi-method approach (e.g. incombination with cross-impact analysis [115]).It mighteven be possible to develop a framework for Delphi-basedrisk management.

Acknowledgments

The contents of this publication are partly based on workof the joint research project “Competitiveness Monitor”,funded by the German Federal Ministry of Education andResearch (project reference number: 01IC10L18 A) in thecourse of its leading-edge cluster initiative. Joint researchpartners include Bayer MaterialScience, BrainNet, Dilotec,and EBS Business School. We would like to thank all partiesinvolved in this leading-edge cluster initiative as well as itspartners and sponsors. Furthermore, we would like to thankPricewaterhouseCoopers Germany for their supportive spon-sorship and valuable consulting. We highly appreciate theDelphi panelists' participation in the survey and the supportof our colleague Janice Magel. We would also like to thankThorsten Neumann, chairman of TAPA EMEA, for his supportand the allowance of access to his network of security expertsfor this research.

1831C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

References

[1] J.-H. Thun, D. Hoenig, An empirical analysis of supply chain riskmanagement in the German automotive industry, Int. J. Prod. Econ.131 (2011) 242–249.

[2] S.M. Wagner, C. Bode, An empirical investigation into supply chainvulnerability, J. Purch. Supply Manag. 12 (2006) 301–312.

[3] A. Norrman, U. Jansson, Ericsson's proactive supply chain riskmanagement approach after a serious sub-supplier accident, Int. J.Phys. Distrib. Logist. Manag. 34 (2004) 434–456.

[4] L. Ritter, J.M. Barrett, R.A. Wilson, Securing Global TransportationNetworks: a Total Security Management Approach, McGraw-HillProfessional, New York, 2007.

[5] P.R. Kleindorfer, G.H. Saad, Managing disruption risks in supplychains, Prod. Oper. Manag. 14 (2005) 53–68.

[6] J.C. Glenn, Updating the global scorecard: the 2011 state of the future,in: The Futurist, Nov.–Dec. 2011, pp. 24–28.

[7] One Earth Future Foundation, The economic costs of maritime piracy,in: O.E.F. Foundation (Ed.), , 2010.

[8] Y. Sheffi, Building a resilient supply chain, Harv. Bus. Rev. 1 (2005)1–4.

[9] A.M. Knemeyer, W. Zinn, C. Eroglu, Proactive planning for catastroph-ic events in supply chains, J. Oper. Manag. 27 (2009) 141–153.

[10] C.W. Autry, L.M. Bobbitt, Supply chain security orientation: concep-tual development and a proposed framework, Int. J. Logist. Manag. 19(2008) 42–64.

[11] J. Blackhurst, C.W. Craighead, D. Elkins, R.B. Handfield, An empiricallyderived agenda of critical research issues for managing supply-chaindisruptions, Int. J. Prod. Res. 43 (2005) 4067–4081.

[12] V.A.W.J. Marchau, W.E. Walker, G.P. van Wee, Dynamic adaptivetransport policies for handling deep uncertainty, Technol. Forecast.Soc. Chang. 77 (2010) 940–950.

[13] J.C. Camillus, Strategy as a wicked problem, Harv. Bus. Rev. 86 (2008)98–106.

[14] L. Dixon, R.J. Lempert, T. LaTourrette, R.T. Reville, The Federal Role inTerrorism Insurance: Evaluating Alternatives in an Uncertain World,RAND Corporation, 2007.

[15] M. Lodge, The public management of risk: the case for deliberatingamong worldviews, Rev. Policy Res. 26 (2009) 395–408.

[16] H.W.J. Rittel, M.M. Webber, Dilemmas in a general theory of planning,Policy. Sci. 4 (1973) 155–169.

[17] T. Gnatzy, R. Moser, Scenario development for an evolving healthinsurance industry in rural India: INPUT for business model innovation,Technol. Forecast. Soc. Chang. 79 (2012) 688–699.

[18] M.V. Gonzalez, Environmental uncertainty, futures studies, andstrategic planning, Technol. Forecast. Soc. Chang. 42 (1992)335–349.

[19] R.B. Duncan, Characteristics of organizational environments andperceived environmental uncertainty, Adm. Sci. Q. 17 (1972) 313–327.

[20] P. Slovic, Perception of risk, Science 236 (1987) 280–285.[21] F.J. Acedo, J. Florin, Understanding the risk perception of strategic

opportunities: a tripartite model, Strateg. Chang. 16 (2007) 97–116.[22] L. Tihanyi, W.B. Thomas, Information-processing demands and the

multinational enterprise: a comparison of foreign and domesticearnings estimates, J. Bus. Res. 58 (2005) 285–292.

[23] B. Squire, Y. Chu, Supply Chains at Risk: a Delphi Study Report,University of Bath, School of Management, Bath, 2011.

[24] N.P. Høj, W. Kröger, Risk analyses of transportation on road andrailway from a European perspective, Saf. Sci. 40 (2002) 337–357.

[25] R. Koivisto, N. Wessberg, A. Eerola, T. Ahlqvist, S. Kivisaari, J. Myllyoja,M. Halonen, Integrating future-oriented technology analysis and riskassessment methodologies, Technol. Forecast. Soc. Chang. 76 (2009)1163–1176.

[26] R.W. Blanning, B.A. Reinig, Political event analysis using group supportsystems, Technol. Forecast. Soc. Chang. 69 (2002) 153–172.

[27] H.A. von der Gracht, I.-L. Darkow, Scenarios for the logistics servicesindustry: a Delphi-based analysis for 2025, Int. J. Prod. Econ. 127 (2010)46–59.

[28] H.A. Linstone, M. Turoff, Delphi: a brief look backward and forward,Technol. Forecast. Soc. Chang. 78 (2011) 1712–1719.

[29] G. Rowe, G. Wright, F. Bolger, Delphi: a reevaluation of research andtheory, Technol. Forecast. Soc. Chang. 39 (1991) 235–251.

[30] J.H. Davis, Individual-group problem solving, subject preference, andproblem type, J. Personal. Soc. Psychol. 13 (1969) 362–374.

[31] F.J. Parente, J.K. Anderson, P. Myers, T. O'Brien, An examination offactors contributing to Delphi accuracy, J. Forecast. 3 (1984)173–182.

[32] R.J. Parenté, T.N. Hiöb, R.A. Silver, C. Jenkins, M.P. Poe, R.J. Mullins, TheDelphi method, impeachment and terrorism: accuracies of short-range

forecasts for volatile world events, Technol. Forecast. Soc. Chang. 72(2005) 401–411.

[33] J. Hallikas, V.M. Virolainen, Risk Management in Supplier Relation-ships and Networks, in: C. Brindley (Ed.), Supply Chain Risk AshgatePublishing, Hampshire, Burlington, 2004, pp. 43–65.

[34] Y. Sheffi, Supply chain management under the threat of internationalterrorism, Int. J. Logist. Manag. 12 (2001) 1–11.

[35] L.C. Giunipero, R.A. Eltantawy, Securing the upstream supply chain: arisk management approach, Int. J. Phys. Distrib. Logist. Manag. 34(2004) 698–713.

[36] G.A. Zsidisin, S.A. Melnyk, G.L. Ragatz, An institutional theory perspectiveof business continuity planning for purchasing and supply management,Int. J. Prod. Res. 43 (2005) 3401–3420.

[37] D.J. Closs, E.F. McGarrell, Enhancing security throughout the supplychain, Special Report to the IBM Center for the Business of Government,2004.

[38] Z. Williams, J.E. Lueg, S.A. LeMay, Supply chain security: an overviewand research agenda, Int. J. Logist. Manag. 19 (2008) 254–281.

[39] B. Modarress, A. Ansari, D. Lockwood, Raising the alarm for logisticsnetworks security, Int. J. Bus. Res. 7 (2007) 221–225.

[40] D. Closs, C. Speier, J. Whipple, M.D. Voss, A framework for protectingyour supply chain, Supply Chain Manag. Rev. 12 (2008) 38–45.

[41] H.L. Lee, M. Wolfe, Supply chain security without tears, Supply ChainManag. Rev. 7 (2003) 12.

[42] J.B. Rice Jr., P.W. Spayd, Investing in supply chain security: collateralbenefits, in: Special Report Series: IBM Center for the Business ofGovernment, 2005, pp. 1–34.

[43] M. Turoff, S.R. Hiltz, C. White, L. Plotnick, A. Hendela, X. Yoa, The pastas the future of emergency preparedness and management, Int. J. Inf.Syst. Crisis Res. Manag. (IJISCRAM) 1 (2009) 12–28.

[44] B. Bowonder, H.A. Linstone, Notes on the Bhopal accident: riskanalysis and multiple perspectives, Technol. Forecast. Soc. Chang. 32(1987) 183–202.

[45] C.E. Lindblom, The science of “muddling” through, Public Adm. Rev.19 (1959) 79–88.

[46] K. Weick, K. Sutcliffe, Managing the Unexpected: Assuring HighPerformance in an Age of Complexity, Jossey-Bass Publishers, 2001.

[47] M.S. Carolan, The precautionary principle and traditional riskassessment: rethinking how we assess and mitigate environmentalthreats, Organ. Environ. 20 (2007) 5–24.

[48] J. Hallikas, I. Karvonen, U. Pulkkinen, V.-M. Virolainen, M. Tuominen,Risk management processes in supplier networks, Int. J. Prod. Econ. 90(2004) 47–58.

[49] M. Turoff, M. Chumer, R. Hiltz, R. Klashner, M. Alles, M. Vasarhelyi, A.Kogan, Assuring homeland security: continuous monitoring, control &assurance of emergency preparedness, J. Inf. Technol. Theory Appl.(JITTA) 6 (2004) 1–24.

[50] N. Dalkey, O. Helmer, An experimental application of the Delphimethod to the use of experts, Manag. Sci. 9 (1963) 458–467.

[51] V.M.R. Tummala, J.F. Burchett, Applying a risk management process(RMP) to manage cost risk for an EHV transmission line project, Int. J.Proj. Manag. 17 (1999) 223–235.

[52] O. Renn, A model for an analytic–deliberative process in riskmanagement, Environ. Sci. Technol. 33 (1999) 3049–3055.

[53] R.K. Rainer Jr., C.A. Snyder, H.H. Carr, Risk analysis for informationtechnology, J. Manag. Inf. Syst. 8 (1991) 129–147.

[54] R.J. Chapman, The effectiveness of working group risk identificationand assessment techniques, Int. J. Proj. Manag. 16 (1998) 333–343.

[55] R. Schmidt, K. Lyytinen, M. Keil, P. Cule, Identifying software projectrisks: an international Delphi study, J. Manag. Inf. Syst. 17 (2001) 5–36.

[56] T. Addison, E-commerce project development risks: evidence from aDelphi survey1, Int. J. Inf. Manag. 23 (2003) 25–40.

[57] R.T. Nakatsu, C.L. Iacovou, A comparative study of important riskfactors involved in offshore and domestic outsourcing of softwaredevelopment projects: a two-panel Delphi study, Inf. Manag. 46(2009) 57–68.

[58] U.G. Gupta, R.E. Clarke, Theory and applications of the Delphitechnique: a bibliography (1975–1994), Technol. Forecast. Soc.Chang. 53 (1996) 185–211.

[59] R.J. Lempert, S.W. Popper, S.C. Bankes, Shaping the Next One HundredYears: New Methods for Quantitative, Long-term Policy Analysis,Rand Corp, Santa Monica, CA, 2003.

[60] R.J. Lempert, M.T. Collins, Managing the risk of uncertain thresholdresponses: comparison of robust, optimum, and precautionaryapproaches, Risk Anal. 27 (2007) 1009–1026.

[61] J.A. Ogden, K.J. Petersen, J.R. Carter, R.M. Monczka, Supply manage-ment strategies for the future: a Delphi study, J. Supply Chain Manag.41 (2005) 29–48.

[62] H.A. Linstone, M. Turoff, The Delphi Method: Techniques andApplications, Addison-Wesley, Reading, 1975.

1832 C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

[63] W.E. Riggs, The Delphi technique: an experimental evaluation,Technol. Forecast. Soc. Chang. 23 (1983) 89–94.

[64] M.S. Raskin, The Delphi study in field instruction revisited: expertconsensus on issues and research priorities, J. Soc. Work. Educ. 30(1994) 75–89.

[65] M.K. Rayens, E.J. Hahn, Building consensus using the policy Delphimethod, Policy, Polit. Nurs. Pract. 1 (2000) 308–315.

[66] P. Tapio, Disaggregative policy Delphi: using cluster analysis as a toolfor systematic scenario formation, Technol. Forecast. Soc. Chang. 70(2003) 83–101.

[67] M. Steinert, A dissensus based online Delphi approach: an explorativeresearch tool, Technol. Forecast. Soc. Chang. 76 (2009) 291–300.

[68] E. van de Linde, P. van der Duin, The Delphi method as earlywarning: linking global societal trends to future radicalization andterrorism in the Netherlands, Technol. Forecast. Soc. Chang. 78(2011) 1557–1564.

[69] M. Turoff, The policy Delphi, in: H.A. Linstone, M. Turoff (Eds.), TheDelphi Method: Techniques and Applications, Addison-Wesley,Reading, 1975, pp. 84–101.

[70] H.A. von der Gracht, Consensus measurement in Delphi studies:review and implications for future quality assurance, TechnologicalForecasting and Social Change 79 (2012) 1525–1536.

[71] T. Gnatzy, J. Warth, H. von der Gracht, I.-L. Darkow, Validating aninnovative real-time Delphi approach — a methodological compari-son between real-time and conventional Delphi studies, Technol.Forecast. Soc. Chang. 78 (2011) 1681–1694.

[72] T. Gordon, A. Pease, RT Delphi: an efficient, “round-less” almost realtime Delphi method, Technol. Forecast. Soc. Chang. 73 (2006) 321–333.

[73] M.R. Geist, Using the Delphi method to engage stakeholders: acomparison of two studies, Eval. Program Plan. 33 (2010)147–154.

[74] P. Goodwin, G. Wright, The limits of forecasting methods in anticipatingrare events, Technol. Forecast. Soc. Chang. 77 (2010) 355–368.

[75] U. Jüttner, H. Peck, M. Christopher, Supply chain risk management:outlining an agenda for future research, Int. J. Logist. 6 (2003)197–210.

[76] J.R. Salancik, W. Wenger, E. Helfer, The construction of Delphi eventstatements, Technol. Forecast. Soc. Chang. 3 (1971) 65–73.

[77] D. Loveridge, On Delphi questions, Ideas in Progress Paper, 2002.[78] V. Mitchell, Assessing the reliability and validity of questionnaires: an

empirical example, J. Appl. Manag. Stud. 5 (1996) 199–208.[79] National Counter Terrorism Center (NCTC), Worldwide Incidents

Tracking System (WITS), in 2011.[80] K.B. Hendricks, V.R. Singhal, An empirical analysis of the effect of

supply chain disruptions on long run stock price performance andequity risk of the firm, Prod. Oper. Manag. 14 (2005) 35–52.

[81] S. Ratick, B. Meacham, Y. Aoyama, Locating backup facilities toenhance supply chain disaster resilience, Growth Chang. 39 (2008)642–666.

[82] J. Bram, J. Orr, C. Rapaport, Measuring the effects of the September 11attack on New York City, Fed. Reserv. Bank N. Y. Econ. Policy Rev.8 (2002) 5–20.

[83] E. Banks, Catastrophic Risk: Analysis and Management, John Wiley &Sons, Chichester, 2005.

[84] M. Warren, W. Hutchinson, Cyber attacks against supply chainmanagement systems: a short note, Int. J. Phys. Distrib. Logist. Manag.30 (2000) 710–716.

[85] S.E. Goodman, J.C. Kirk, M.H. Kirk, Cyberspace as a medium forterrorists, Technol. Forecast. Soc. Chang. 74 (2007) 193–210.

[86] C. Speier, J.M. Whipple, D.J. Closs, M.D. Voss, Global supply chaindesign considerations: mitigating product safety and security risks, J.Oper. Manag. 29 (2011) 721–736.

[87] R. Sarathy, Security and the global supply chain, Transp. J. 45 (2006)28–51.

[88] K.D. Mitnick, Are you the weak link? Harv. Bus. Rev. 81 (2003)18–20.

[89] M.C. Wilson, The impact of transportation disruptions on supply chainperformance, Transp. Res. E 43 (2007) 295–320.

[90] W.R. Kendall, L.K. Kendall, The consequences to shipping of aresurgence of violence at sea: piracy, terrorism, war or self defense?Eur. J. Econ. Finance Adm. Sci. (2010) 138–146.

[91] R. Vecchiato, Environmental uncertainty, foresight and strategicdecision making: an integrated study, Technol. Forecast. Soc. Change79 (3) (March 2012) 436–447.

[92] J. Lee, H.R. Rao, Perceived risks, counter-beliefs, and intentions to useanti-/counter-terrorismwebsites: an exploratory study of government–citizens online interactions in a turbulent environment, Decis. Support.Syst. 43 (2007) 1431–1449.

[93] K. Aquilina, Public security versus privacy in technology law: abalancing act? Comput. Law Secur. Rev. 26 (2010) 130–143.

[94] H.L. Lee, S. Whang, Higher supply chain security with lower cost:lessons from total quality management, Int. J. Prod. Econ. 96(2005) 289–300.

[95] A.L. Delbecq, A.H. Van de Ven, D.H. Gustafson, Group Techniques forProgram Planning: a Guide to Nominal Group and Delphi Processes,Green Briar Pr, Middleton, 1986.

[96] A. Lipinski, D. Loveridge, Institute for the future's study of the UK,1978–1995, Futures 14 (1982) 205–239.

[97] R.I. Mehr, S. Neumann, Delphi forecasting project, J. Risk Insur. 37(1970) 241–246.

[98] S.A.W. Drew, Building technology foresight: using scenarios toembrace innovation, Eur. J. Innov. Manag. 9 (2006) 241–257.

[99] F.J. Parentè, J.K. Anderson-Parentè, Delphi inquiry systems, in: G.Wright, P. Ayton (Eds.), Judgemental Forecasting, John Wiley & Sons,Chichester, 1987, pp. 129–156.

[100] G.J. Skulmoski, F.T. Hartman, J. Krahn, The Delphi method for graduateresearch, J. Inf. Technol. Educ. 6 (2007) 1–21.

[101] E. De Vet, J. Brug, J. De Nooijer, A. Dijkstra, N.K. De Vries, Determinants offorward stage transitions: a Delphi study, Heal. Educ. Res. 20 (2005)195–205.

[102] M. Scheibe, M. Skutsch, J. Schofer, Experiments in Delphi methodology,in: H.A. Linstone, M. Turoff (Eds.), The DelphiMethod— Techniques andApplications, Addison-Wesley, Reading, 1975, pp. 262–287.

[103] P. Ecken, T. Gnatzy, H.A. von der Gracht, Desirability bias in foresight:consequences for decision quality based on Delphi results, Technol.Forecast. Soc. Chang. 78 (2011) 1654–1670.

[104] J.M. Corbin, A. Strauss, Grounded theory research: procedures,canons, and evaluative criteria, Qual. Sociol. 13 (1990) 3–21.

[105] J.O. Schwarz, Linking strategic issue management to futures studies,Futur. Res. Q. 21 (2005) 39–56.

[106] N. Taleb, The Black Swan: the Impact of the Highly Improbable,Random House, New York, 2007.

[107] E. Cornish, The WILD cards in our future, Futurist 37 (2003) 18.[108] I. Grossmann, Critical and strategic factors for scenario development

and discontinuity tracing, Futures 39 (2007) 878–894.[109] J. Pallant, SPSS Survival Manual: a Step by Step Guide to Data Analysis

Using SPSS for Windows, Open University Press, Maidenhead, 2001.[110] H.I. Ansoff, Strategic issue management, Strateg. Manag. J. 1 (1980)

131–148.[111] S. Mendonça, M. Pina e Cunha, J. Kaivo-oja, F. Ruff, Wild cards, weak

signals and organisational improvisation, Futures 36 (2004)201–218.

[112] B.S. Coffman, Weak signal research, Part III: Sampling, Uncertainty andPhase Shifts in Weak Signal Evolution, 1997. Retrieved January 15, 2012,from http://www.mgtaylor.com/mgtaylor/jotm/winter97/wsrsampl.htm.

[113] H. von der Gracht, M. Linz, I.-L. Darkow, Delphi-based deductivewildcard analysis, in: European Futurists Conference Lucerne, TippingPoints Workshop, Lucerne, Switzerland, 2011.

[114] S. Schuckmann, M. Linz, H.A. von der Gracht, I.-L. Darkow, Delphibased disruptive and surprising transformation scenarios on thefuture of aviation, in: European Foresight Platform (EFP) (Ed.),Publication Series in European Foresight, EFP Brief No. 192, 2011.

[115] V.A. Bañuls, M. Turoff, Scenario construction via Delphi and cross-impactanalysis, Technol. Forecast. Soc. Chang. 78 (2011) 1579–1602.

[116] J. Gausemeier, A. Fink, O. Schlake, Scenario management: an approachto develop future potentials, Technol. Forecast. Soc. Chang. 59 (1998)111–130.

[117] K. Van der Heijden, Scenarios: the Art of Strategic Conversation (2nded.), 2nd ed. John Wiley & Sons, Chichester, 2005.

[118] P.J.H. Schoemaker, Scenario planning: a tool for strategic thinking,Sloan Manag. Rev. 36 (1995) 25–40.

[119] H.A. Akkermans, P. Bogerd, E. Yücesan, L.N. van Wassenhove, Theimpact of ERP on supply chain management: exploratory findingsfrom a European Delphi study, Eur. J. Oper. Res. 146 (2003) 284–301.

[120] P. Rikkonen, J. Kaivo-oja, J. Aakkula, Delphi expert panels in thescenario-based strategic planning of agriculture, foresight, J. Futur.Stud. Strateg. Think. Policy 8 (2006) 66–81.

[121] S.A. van 't Klooster, M.B.A. van Asselt, Practising the scenario-axestechnique, Futures 38 (2006) 15–30.

[122] R. De Mooij, P. Tang, Four Futures of Europe, CPB, 2003.[123] X. Qu, Q. Meng, The economic importance of the Straits of Malacca

and Singapore: an extreme-scenario analysis, Transp. Res. E 48 (2012)258–265.

[124] A.M. Goulielmos, A.A. Anastasakos, Worldwide security measures forshipping, seafarers and ports: an impact assessment of ISPS code,Disaster Prev. Manag. 14 (2005) 462–478.

[125] E.J. Hahn, C.P. Toumey, M.K. Rayens, C.A. McCoy, Kentucky legislators'views on tobacco policy, Am. J. Prev. Med. 16 (1999) 81–88.

[126] T. Spinelli, The Delphi decision-making process, J. Psychol. 113 (1983)73–80.

1833C. Markmann et al. / Technological Forecasting & Social Change 80 (2013) 1815–1833

[127] K. Mulder, C. van de Weijer, V. Marchau, Prospects for externalsources of vehicle propulsion: results of a Delphi study, Futures 28(1996) 919–945.

Christoph Markmann is a doctoral student at the Institute for Futures Studiesand Knowledge Management (IFK) of the EBS Business School in Wiesbaden,Germany. He obtained his engineering diploma in management of technologyfrom the University of Stuttgart. His main research interests include risks insupply chains, development ofmethodologies for foresight analyses (especiallythe advancement of Delphi studies) and technological forecasting.

Dr. Inga-Lena Darkow is a research fellow at the Institute for Futures Studiesand Knowledge Management (IFK) at EBS Business School, Wiesbaden,Germany, and Senior Expert Supply Chain Strategy at BASF, Ludwigshafen,Germany. During her career, she has held various academic positions inresearch and university education. Her research interests include corporateforesight and futures studies, especially in logistics and supply chainmanagement. Her works have been published in several books and inpeer-reviewed journals, among them Technological Forecasting & Social Change,Futures, International Journal of Production Economics, and Journal of BusinessLogistics.

Dr. Heiko A. von der Gracht is the founder and director of the Institute forFutures Studies and Knowledge Management (IFK) and a post-doctoralresearcher at EBS Business School in Wiesbaden, Germany. His researchinterests are corporate foresight, Delphi and scenario techniques, decisionsupport, and quality in futures research. His works have been published inseveral books and in peer-reviewed journals, among them TechnologicalForecasting & Social Change, Futures, and International Journal of ProductionEconomics.