© Blackwell Publishers Ltd 2001. Published by Blackwell Publishers, 108 Cowley Road, Oxford OX4 1JF, UKand 350 Main Street, Malden, MA 02148, USA.

ENHANCING STRATEGY EVALUATION IN SCENARIO PLANNING:A ROLE FOR DECISION ANALYSIS

P G

University of Bath

G W

University of Strathclyde

Scenario planning can be a useful and attractive tool in strategic management.In a rapidly changing environment it can avoid the pitfalls of more traditionalmethods. Moreover, it provides a means of addressing uncertainty withoutrecourse to the use of subjective probabilities, which can suffer from serious cog-nitive biases. However, one underdeveloped element of scenario planning is theevaluation of alternative strategies across the range of scenarios. If this is carriedout informally then inferior strategies may be selected, while those formal evalua-tion procedures that have been suggested in relation to scenario planning areunlikely to be practical in most contexts. This paper demonstrates how decisionanalysis can be used to structure the strategy evaluation process in a way whichavoids the problems associated with earlier proposals. The method is flexible,versatile and transparent and leads to a clear and documented rationale for theselection of a particular strategy.

Scenario planning is a tool that has been receiving increasing attention over the last few years. A recent book on scenario planning by van der Heijden (1996) appeared in the business book best-seller lists, and the employment ofthe technique at Royal Dutch Shell has received widespread publicity. Interest inthe use of scenarios as a framework for strategic planning is not surprising. In a volatile and rapidly changing environment more traditional aids to planning,such as extrapolations of past trends, are unlikely to produce reliable forecasts in the medium or long term (Makridakis and Gaba, 1998). Moreover, such extrapolations are often presented as single-figure forecasts and it is usually diffi-cult to make accurate estimates about the level of uncertainty that is associated

Address for reprints: Paul Goodwin, School of Management, University of Bath, Bath BA2 7AY, UK.

Journal of Management Studies 38:1 January 20010022-2380

with these forecasts. In contrast, scenario planning involves the structured use of management judgment to construct multiple ‘script-like characterization(s) of possible futures’ (Schoemaker, 1991). These characterizations focus on thedynamics of how a particular future might unfold by paying attention to causalrelationships, prevailing trends, the behaviour of key players and internal consistency. The resulting multiple scenarios attempt to bound the un-certainties that are seen to be inherent in the future. As such they can be seen to be analogous to a wind tunnel (Wack, 1985), enabling the performance of a given strategy to be tested under the range of possible futures that mightdevelop.

Scenario planning has been developed largely by practitioners and, as such,lacks the theoretical and axiomatic underpinning of other decision-aiding tools,such as decision analysis and statistical forecasting. Nevertheless, a number ofadvantages are claimed for the approach. Most obviously, it provides a way ofaddressing uncertainty which avoids the need to estimate subjective probabilities.The psychological biases associated with subjective probability estimation, such asoverconfidence in prediction, have been widely researched and documented(Hogarth and Makridakis, 1981; Tversky and Kahneman, 1974). In addition, thepresentation of scenarios as stories of how the future might unfold, with the focuson causality, is likely to be attractive to managers (van der Heijden, 1994). Indeed,in a study of how judges and juries make decisions in court, Wagenaar (1994) hasargued that ‘good’ stories provide a context that gives an easy and natural expla-nation of why ‘actors’ behaved in the way they did. So story telling via scenarioplanning may be a natural way of making sense of the world. The constructionof these ‘stories’ also enables a variety of viewpoints about the future to bereflected and can serve to challenge the prior assumptions of participants (Schoemaker, 1991).

Despite these advantages, one element of scenario planning that has beenunderdeveloped is the evaluation of the performance of strategies across the rangeof scenarios. In this paper we argue that, where an organization has a plurality ofobjectives, there is a danger that the selection of strategies, without recourse to aformal structured method, will lead to poor decisions. Moreover, those formalstrategy evaluation methods that have been suggested in relation to scenario planning may not be useful in many contexts because they require the estimationof subjective probabilities, thereby negating one of the most widely advertisedadvantages of the technique. We then demonstrate how decision analysis can beused to structure the strategy evaluation process in a way which avoids the prob-lems associated with earlier proposals. We argue that, by breaking the evaluationtask into a series of smaller and easier tasks, the judgments of planners can beemployed to greater effect. The flexibility, simplicity and transparency of our sug-gested approach should lead to insights into the nature of the strategy selectionproblem, while also improving communication within the planning team. More-over, the outcome of the process will be a clear and documented rationale for theselection of a particular strategy.

The paper is structured as follows. We first give a brief outline of scenario planning as currently practised. We then specify the potential weaknesses ofcurrent practice, and show why previous suggestions for combining decision analy-sis with scenario planning are likely to have limited applicability. Then we use acase-orientated example to demonstrate our alternative proposal.

2 . .

© Blackwell Publishers Ltd 2001

In this section we give a very brief overview of scenario planning. More detailedaccounts can be found in van der Heijden (1996) and Schoemaker (1995). Differ-ent practitioners have tended to develop different approaches, but essentially all ofthe methods initially involve assessments of how the future will evolve from thepresent day to the horizon year. This is done by identifying predetermined trends– that will impact on the issue of concern (e.g. demographic trends) and will, bydefinition, be present in all scenarios – and critical uncertainties (e.g. the successor otherwise of a new technology). The degree to which these trends and resolveduncertainties will have a negative or positive effect on the issues of concern is nextassessed. In the extreme world method of scenario planning two extreme worldsare then created by putting all negatively and positively resolved uncertainties inseparate scenarios and then checking for internal consistency. For example, a sce-nario that includes both a high pound-dollar exchange rate and an improved com-petitive advantage for UK suppliers of products to the USA may be judged to beimplausible. Next the actions of key players (e.g. rivals), who will be impacted bythe future described in a scenario, are assessed. Finally, each scenario is written upas a ‘story’ describing how that particular future will develop. The result shouldbe a set of vivid and detailed pictures of plausible futures.

Some practitioners advocate the development of more than two scenarios wherethe variety of possible outcomes suggests that this is appropriate or because thevery extremeness of just two scenarios may cause doubts amongst managers abouttheir plausibility. However, as Schoemaker (1991) points out, the scenarios are nottrue states of nature: they do not represent a mutually exclusive and exhaustiveset of future states. Nor are they forecasts: each scenario has an infinitesimal probability of actually occurring. Instead, the objective of scenario planning is togenerate a set of scenarios that collectively bound the perceived range of possiblefutures. This is achieved by permutating those uncertainties whose resolution willhave the greatest degree of perceived impact, positive or negative, on the issue ofconcern.

Once the scenarios have been generated they can be used for at least two relatedpurposes. The first is strategy design. For example, the objective may be to designa strategy that is robust, in the sense that it will ensure that the organization sur-vives, or even thrives, under all conditions that are assumed to be plausible. In thiscase the range of scenarios can be used to determine the features that need to bebuilt into a strategy so that it will be equipped to meet the demands and challengesthat may occur in the future. A more formalized approach is offered by Schoe-maker’s key-success-factor-matrix (Schoemaker, 1992). This can be used to identify ‘competitive assets or core capabilities’ that the organization will need ineach of its strategic segments in order to prosper under each scenario.

Secondly, the scenarios can be used in the evaluation of proposed strategies andin the selection of the most appropriate strategy. Note that design and evaluationshould not be regarded as successive stages in a non-recursive process. The processof evaluation may incubate new ideas in the planners’ minds and hence lead tothe formulation of new strategies (Ng and McConnell, 1993). This is demonstratedin Figure 1. Here, the ticks in the matrix represent positive performance of a strategy under a given scenario (the more ticks the better the performance), whilecrosses represent negative performance. If only strategies 1, 2 and 3 are being

3

© Blackwell Publishers Ltd 2001

considered, it can be seen that strategy 1 potentially offers the least worst nega-tive performance (it is the so-called maximin option). This strategy may thereforebe judged to be the most robust option – though adopting it would imply anextreme aversion to risk on the part of the decision makers. However, the insightsgenerated by the process of evaluation have led to the design of a new strategywhich clearly dominates the alternatives in that it will perform at least as well asthem under the entire range of scenarios.

Clearly, the process of strategy evaluation is not only important in its own right,but it can also act as a potential stimulus to improved strategy design. However, itis likely, in many organizations, that this evaluation will need to be carried out byassessing strategies against multiple objectives ( Johnson and Scholes, 1993, p. 304).This obviously increases the complexity of the strategic decision – several strate-gies have to be compared against multiple objectives under each of the sets of con-ditions suggested by a multiplicity of scenarios. The limited information processingcapacity of the human mind (Simon, 1955) means that unaided planners will beunlikely to deal optimally with these complexities. To cope with them they arelikely to employ over simplified heuristics (Kahneman and Tversky, 1982) and relatively poor choices of strategy may result.

Psychologists have identified a number of commonly used heuristics whichunaided decision makers employ when faced with the complexity of decisionsinvolving multiple objectives. The simplest is the lexicographic heuristic (Payne et al., 1993, p. 26). The decision maker using this approach focuses on what is

4 . .

© Blackwell Publishers Ltd 2001

Scenario 1 Scenario 2 Scenario 3

Strategy 1

Strategy 2

Strategy 3

New strategy

Figure 1. Testing the performance of strategies against scenarios

considered to be the most important objective (e.g. maximize share of domesticmarket) and then selects the alternative which performs best against that objec-tive. In the event of a tie between options in relation to the most important objec-tive, the option is chosen which performs best on the second most importantobjective, and so on. Clearly, the resulting decision will be based on only a smallpart of the information which is available to the decision maker. Moreover, theapproach is non-compensatory. Poor performance of an option in relation to oneobjective is not compensated by good performance on another. For example, asecond strategy may lead to a slightly smaller domestic market share, but muchhigher revenue generated from exports. With more careful thought the decisionmaker may have considered that this higher export revenue would more than compensate for the small reductions in domestic market share, but the use of thelexicographic strategy would preclude such considerations.

A similar approach, known as the semi-lexicographic heuristic, has been iden-tified by Tversky (1969). A decision maker employing this heuristic will considertwo options to be tied if they perform similarly in relation to an objective. Todemonstrate this, suppose that two strategies are considered to be tied if theirresulting estimated domestic market shares are within 5% of each other, in whichcase the strategy which has the least damaging impact on the natural environmentwill be chosen. Consider the strategies in Table I. The heuristic suggests that Awill be preferred to B and B to C. However, it also suggests that C will be pre-ferred to A. Thus there is a violation of a fundamental axiom of most normativedecision procedures, namely transitivity. This can lead to a number of dangers.For example, suppose that the selection of a strategy is made by first comparingthem in pairs, with the ‘worst’ of each pair being eliminated from further consid-eration. In this case, the final choice of strategy might merely reflect the order inwhich the strategies were compared.

Researchers have also found evidence that unaided decision makers sometimesemploy a more sophisticated heuristic known as elimination by aspects (EBA)(Tversky, 1972). Here the decision maker identifies the most important objective(say: maximize short-term profit). Next a cut-off value is set for that objective (say$8 million) and any option that fails to meet this value is eliminated. The processis repeated for the second most important objective (say annual growth in turnover)and continues until only one option is left. Despite the likelihood that EBA willlead to the use of more of the available information than the other two heuristicsit is still non-compensatory and does not ensure that options retained are superiorto those eliminated. The decision maker’s focus is on one objective at a time, ratherthan possible trade-offs between performance on different objectives.

Of course, these heuristics do not constitute an exhaustive list of the methodsused by decision makers in multi-objective choice. Indeed, psychologists have

5

© Blackwell Publishers Ltd 2001

Table I. Choice of strategy

Strategy Domestic market share Damage to natural environment

A 33% MediumB 36% HighC 30% Low

identified other heuristics such as ‘majority of confirming dimensions’, ‘frequencyof good and bad features’ and ‘the equal weight heuristic’, which ignores infor-mation on the relative importance of each attribute (see Payne et al. (1993) for asummary). Also, in some circumstances people appear to use mixed approacheswith, for example, EBA being used to reduce a long list of options and then otherapproaches being used to examine the options remaining (Payne, 1976). However,like the three heuristics outlined above, all of these approaches can also lead to achoice of options which can depart markedly from that suggested by a normativedecision process (Dahlstrand and Montgomery, 1984; Russo and Rosen, 1975;Svenson and Edland, 1987). In particular, as the number of objectives and alternatives in a decision problem increases then usually a smaller percentage ofobjectives will be considered and simpler heuristics will be used (Dahlstrand andMontgomery, 1984).

How can decision makers be helped to cope with the complexity of strategy evaluation so that they can avoid the problems associated with the use of over-simple heuristics? The central idea of decision analysis is that this can be achievedthrough decomposition, that is breaking the evaluation task into a series of smallerand easier tasks. After the decision maker has focused separately on each of thesetasks, his or her judgments in relation to these tasks are recomposed, and theoverall choice of option that is consistent with these decomposed judgments isidentified. The recomposition process is carried out in accordance with a formalset of axioms. For example, the transitivity axiom suggests that if A was preferredto B in one judgment and B to C in another, then it can be inferred that, out ofthe three options, A is the preferred choice.

Of course, the decomposition–recomposition process is at the heart of scenarioconstruction – judgmental decomposition into uncertainties, trends and actorbehaviours precedes the recomposition of these elements into the scenarios,though, unlike decision analysis, there are no axioms to structure the recomposi-tion. There is thus arguably a degree of incongruity in scenario planning whenthe strategy evaluation stage alone is carried out without the potential benefits ofdecomposition.

Surprisingly, there are few references in the literature which address this incon-gruity and suggest practical ways of aiding strategy evaluation within scenarioplanning. Schoemaker (1991) suggests that scenario planning should be used as apreliminary phase in the decision making process, enabling the decision makers’ideas to be clarified, before moving on to a formal decision analysis method thatis designed to support decision making under uncertainty: multiattribute utilitytheory (MAU) (see e.g. Keeney and Raiffa, 1976). While multiattribute utilitytheory is a decomposition procedure that is founded on sound theoretical prin-ciples, its use here raises a number of problems.

First, as we stated earlier, the scenarios do not represent states of nature. Yetthe application of multiattribute utility requires a set of mutually exclusive andexhaustive states of nature and is unclear how the transition should be madebetween writing the scenarios and the identification of the states of nature. Bell(1982) discusses this problem from a theoretical point of view, but concedes the

6 . .

© Blackwell Publishers Ltd 2001

‘practicalities are not that cooperative’. Perhaps more significantly, even if thisproblem can be overcome, then MAU requires probabilities to be assessed for thestates of nature. As we pointed out earlier, one of the most widely argued advan-tages of scenario planning is that it obviates the need to estimate probabilities andhence avoids the extensively documented biases that are associated with subjec-tive probabilities. Indeed, after advocating the use of MAU from a theoreticalpoint of view, Bell points out that ‘no one would dream of assessing p(x) for theworld energy outlook’. Second, it is important to note that estimating probabil-ities after the scenarios have been formulated may exacerbate particular biases.The scenarios are designed to be highly plausible and this may exaggerate the per-ceived probability of occurrence of the events associated with them. Logically, themore specific and detailed a scenario is then the less likely it is to occur, yet thereis evidence that people’s assessment of the probability of occurrence increases asthe level of detail, and hence the plausibility of the scenario, increases – a phenom-enon known as the conjunction fallacy (Tversky and Kahneman, 1983).

The third concern about the application of MAU is that it is can requireextremely difficult judgments from decision makers so that elicited responses maycontain errors (Edwards and Baron, 1994). Indeed, the process by which multiat-tribute utilities are obtained is unlikely to be transparent to most decision makersinvolved with scenario planning. The outcomes of this process may therefore lackcredibility or they might fail to yield insights to individual decision makers –insights that might be crucial if any further strategy design is contemplated. Sim-ilarly, when groups of decision makers are attempting to reach a consensus on thechoice of strategy, this lack of transparency may hinder informed debate. AsPhillips (1989) points out, ‘even within the same company there is usually no sharedunderstanding of terms like mission, vision, goal, objective, strategy, option, sce-nario and risk’ and the use of a complex evaluation procedure is unlikely to mitigate this problem. In the light of these three major concerns about MAU it is perhaps significant that Schoemaker’s (1991) proposed use of the method asa technique for scenario/strategy evaluation is a theoretical presentation.

All of this suggests that a formal strategy evaluation procedure is needed withinthe scenario planning process that meets the following conditions:

(1) Transparency – so that the derivation of the results can be understood andtherefore trusted. Indeed the modelling process must be capable of beingunderstood by managerial specialists from different parts of the organiza-tion and by general managers. In this way it will provide a common lan-guage that decision makers can share, thereby enhancing communicationamongst the decision making team (Wooler, 1987).

(2) Ease of judgment – so that there is less chance of errors in the elicitationprocess. To achieve this, decomposed judgments will need to be cognitivelyless demanding than holistic judgments (Slovic et al., 1977). Decompositionis also unlikely to be beneficial where the decomposed judgments are un-familiar. Nor will it be useful where boredom and fatigue occur because theduration of the evaluation task is protracted by the requirement to make alarge number of decomposed judgments (Goodwin and Wright, 1993).Moreover, ease of judgment should not be bought at the cost of superfi-ciality, which will preclude the potential benefits of thinking hard about theproblem (Edwards and Baron, 1994).

7

© Blackwell Publishers Ltd 2001

(3) Versatility – in that the evaluation should be able to incorporate both financialand non-financial objectives. Without such a method, comparing monetaryoutcomes, like return on capital, with ‘qualitative’ outcomes like ‘effect on theenvironment’ may be like comparing apples with oranges (Phillips, 1989) andthere may be a temptation to focus on the more-easily-measured financialoutcomes. In particular, the method will need to allow the incorporation ofattributes such as risk, without resorting to the complexities of MAU.

(4) Flexibility – so that changes in perspective can be accommodated as insightsand understanding increase and also so that the alternative perspectives ofdifferent participants in the decision making process can be modelled andcompared.

(5) Theoretical correctness – so that the ordering of the strategic options suggestedby the method is consistent with the judgments elicited from the decisionmakers.

In the next sections we demonstrate that these conditions can be met by applyinga decision analysis method known as multiattribute value modelling. We first usea simplified case example to explain how multiattribute value modelling can beadapted to allow it to be applied within the scenario planning process.

A number of assumptions underpin the approach we outline here. We believe thatthe formal assessment of probabilities of given scenarios prevailing is not advis-able, given problems, such as overconfidence, which we referred to earlier. Instead,one of the key objectives of the methods is the identification of strategies whichperform well, or at least acceptably, over a wide range of plausible scenarios, irre-spective of the apparent likelihood of these scenarios at the time of the strategydecision. While we will define our objectives as ‘maximize A’ or ‘minimize B’ thiswill serve to indicate our preferred direction of movement (e.g. higher profits arepreferred). Given the uncertainty inherent in strategic planning, we assume thatrealistically we can only seek or design strategies which are ‘good or acceptable’rather than optimal. Indeed, the objective of our approach is to inform and struc-ture debate and increase the salience of key issues, rather than to prescribe an‘optimal’ course of action.

The case we will use to illustrate the suggested approach is hypothetical. It con-cerns a newly privatized national mail company which needs to formulate strate-gies with a ten year planning horizon. To date, the company has been protectedby legislation which allows it to operate as a monopoly on letter deliveries. Thisprotection has engendered a culture of muddling through (i.e. minor adjustmentsto policies in reaction to events, with no clear sense of overall direction). However,the environment within which the company may operate in the future is likely to fundamentally change. For example, there is a possibility that it will lose itsmonopoly position, while technological developments pose long term threats tothe volume of letter mail.

Two ‘extreme-world’ scenarios have been written by the company’s planningteam who have also formulated three alternative strategies. These scenarios andstrategies are set out below.

8 . .

© Blackwell Publishers Ltd 2001

Scenario 1: (DOG FIGHT). The company loses its monopoly. Rival companies takeseveral years to develop their own delivery systems, but within five years there iskeen competition on price, delivery times and reliability. Growth in the usage ofelectronic communications, particularly by direct marketing organizations leads toa large reduction in the total volume of paper mail which needs to be delivered.This reduction is exacerbated by poor economic conditions.

Scenario 2: (MAIL MOUNTAIN). The company retains its monopoly on letter delivery. Despite increases in the use of electronic communications, taxes leviedon e-mail messages mean that paper mail remains popular. Buoyant economicconditions lead to increases in the volume of mail generated by direct marketingorganizations. Increased ‘home working’ also leads to increases in the number ofpaper documents which need to be delivered by mail.

Strategy A: (STATUS QUO). Continue with the current strategy of specializing inletter delivery, continuing to take advantage of increased mechanization whereappropriate, by buying the technology from foreign suppliers.

Strategy B: (R&D). Continue specializing in letter delivery, but allocate very largeamounts of investment to R&D with the objective of becoming a world leader inletter sorting technology.

Strategy C: (DIVERSIFY). As A, but also diversify into electronic communicationsby becoming an Internet service provider and by seeking to merge with a telecom-munications company.

Clearly, in order to evaluate the strategies, a clear and unambiguous set of objec-tives needs to be identified. This can be achieved in multiattribute value analysisby formulating an objectives hierarchy or value tree (Figure 2) (Keeney and Raiffa,1976). The hierarchy enables general objectives to be decomposed to a level ofspecificity which is sufficient for the decision maker to be able to assess the per-formance of the strategies. For example, the objective ‘maximize size of business’may be too broadly defined, or vague, to allow decision makers to compare strate-gies against it. By decomposing this objective into ‘maximize market share’ and‘maximize growth’, the decision makers may find that the strategy comparison ismade easier and more meaningful. In this case, the five ‘lowest level’ objectivesidentified are: maximize short and long term profit, market share, growth and theflexibility of any strategy. Flexibility refers here to the extent to which a strategycan be adapted to the different conditions which might prevail within a given scenario (e.g. to counter the changing tactics of rival companies).

It is important to establish that the objectives hierarchy meets a number of con-ditions before proceeding (Keeney and Raiffa, 1976). First, it is necessary to elimi-nate objectives from the hierarchy which appear more than once under differentguises or which overlap considerably (e.g. maximize reliability of service and maximize customer satisfaction) otherwise these objectives will receive undueweighting when strategies are evaluated. The objectives on the tree should also be judgmentally independent. This means that it should be possible to judge theperformance of a strategy against a given objective without having to take intoaccount its performance on other objectives. For example, it may be difficult toevaluate a strategy’s performance on the objective ‘maximize company growth’

9

© Blackwell Publishers Ltd 2001

without also taking into account the objective ‘minimize risk’. A high growth strategy may be less desirable than a medium growth strategy if it is associated with a high level of risk. If this is the case, judgmental independence would nothave been achieved. Usually, a restructuring of the tree can remove this problem(von Winterfeldt and Edwards, 1986).

A final problem to avoid is the inclusion of objectives whose importance variesbetween different scenarios. This is likely to occur where an objective is really ameans towards the achievement of an ultimate objective. For example, the mailcompany may initially identify ‘maximize the chances of providing of a low-priceservice’ as an objective. However, this is judged to be more important under a sce-nario involving keen competition, than one where a monopoly is maintained. Thisis because the low-price objective is a means towards achieving other objectivessuch as maximizing market share and profit.

Once the objectives hierarchy has been agreed by the decision making team,the next task is to assess the performance of the strategies against these objectivesunder the conditions of the different scenarios (though this is not usually a linearprocess and the objectives hierarchy might be modified at a later stage as the modelling process yields further insights). As a first step, for each ‘lowest level’objective all the strategy–scenario combinations are ranked from (1) best to (6)worst to reflect their performance against that objective. Examples of ranks fortwo of the objectives are given in Table II.

Thus the R&D strategy, under the MAIL MOUNTAIN scenario would, it isthought, lead to the best long term profit, while the STATUS QUO strategy, underthe DOG FIGHT scenario, would yield the worst.

These ranks can next be converted into scores. Table III shows the scores forthe two objectives we have just considered. A score of 100 is assigned to the mostpreferred strategy–scenario combination and 0 the least preferred. Intermediatescores should be designed to reflect the relative performance of the strategiesbetween these two extremes.[1] Note that these scores do not need to be exact.Indeed, the process of determining the values and the focused thinking that thisengenders is likely to be at least as valuable as the quantitative result which isobtained at the end of the analysis.

At this stage any strategy whose performance on any objective, in any scenario,is unacceptable should be removed from the analysis. In this case the analysis

10 . .

© Blackwell Publishers Ltd 2001

Figure 2. An objectives hierarchy for the national mail company

will have served to alert decision makers to the dangers of pursuing particularstrategies. It may also, of course, enable strategies to be modified to avoid suchdangers. Should strategies be removed, the remaining scores will need to re-scaledwith the worst of the remaining strategies on each objective now being assigneda score of zero. In the mail company case it is assumed that all strategies are acceptable.

Clearly, some objectives will be more important than others and it is tempting, atthis stage, to attach weights to the objectives to reflect their importance. However,while an objective may have a high level of importance per se, its importance in thechoice between strategies may be relatively minor. To see this, consider the objec-tive of maximizing long term profit, which, in itself, may be considered to be ofmajor importance. Suppose that all strategies lead to exactly the same long termprofits in all scenarios. Since the same profit would be achieved whichever scenariois selected, long term profit would have no importance in discriminating betweenthe options. An extension of this argument suggests that where the improvementbetween the worst performance on an objective and the best performance is relatively small then the objective should have a relatively low weight.

To take this into account, the importance of the improvement (or ‘swing’) betweenthe worst and best performances on the different objectives should be compared

11

© Blackwell Publishers Ltd 2001

Table II. Ranks for two objectives

Strategy Scenario

DOG FIGHT MAIL MOUNTAIN

Objective: maximize long term profitSTATUS QUO 6 2R&D 5 1DIVERSIFY 4 3

Objective: maximize share of letter marketSTATUS QUO 5= 1=R&D 4 1=DIVERSIFY 5= 1=

Table III. Scores for the two objectives

Strategy Scenario

DOG FIGHT MAIL MOUNTAIN

Objective: maximize long term profitSTATUS QUO 0 80R&D 30 100DIVERSIFY 50 60

Objective: maximize share of letter marketSTATUS QUO 0 100R&D 80 100DIVERSIFY 0 100

and ranked. Comparing these ‘0 to 100’ swings in scores for the objectives of thenational mail company leads to the ranks shown in Table IV. Once the ranks havebeen agreed by the decision makers a weight of 100 can be attached to the mostimportant swing. The importance of the other swings can then be compared withthis using a 0 to 100 scale.[2]

The weights assessed for the mail company are given in Table V. Note that forease of calculation it is conventional to normalize the weights so that they sum to1. This is achieved by simply dividing each weight by the sum of the weights. Foreach scenario, an aggregate score can now be obtained to measure the perfor-mance of a given strategy over all the objectives. This is calculated by multiplyingthe score for each objective by the normalized weight for that objective andsumming the resulting products. For example, the performance of the STATUSQUO strategy in the MAIL MOUNTAIN scenario is calculated as shown in Table VI.

By repeating this process for all of the other strategy–scenario combinations,the matrix shown in Table VII is obtained. This matrix can now be used tocompare the strategies’ performance. The scores show that the STATUS QUO isdominated by both the R&D and DIVERSIFY strategies. In other words, it per-forms the worst under both scenarios and therefore does not appear to be a strat-egy which is worth considering. While there is little to choose between the R&Dand DIVERSIFY strategies in the DOG FIGHT scenario, the R&D strategy isclearly superior in the MAIL MOUNTAIN scenario. Provisionally, the R&D strat-egy appears to be the most attractive. Of course, it is possible that by fosteringnew insights into the problem the decision analysis process will enable new andmore robust strategies to be designed.

12 . .

© Blackwell Publishers Ltd 2001

Table IV. Ranking of swing between worst and best performance

Swing Ranka

Worst long term profit to best 1Least market share to highest 2Least flexibility to most 3Least growth to highest 4Worst short term profit to best 5

Note:a 1 = most important swing.

Table V. Assessed weights for mail company

Swing Weight Normalizedweights

Worst long term profit to best 100 0.50Least market share to highest 40 0.20Least flexibility to most 30 0.15Least growth to highest 20 0.10Worst short term profit to best 10 0.05

Sum 200 1.00

A final, but important, stage of multiattribute value modelling is sensitivityanalysis. The scores and weights used in the analysis were based in rough andready judgments. Moreover, in a group of decision makers there are likely to bedifferent opinions, or minority views, on which scores and weights are appro-priate. For these reasons it can be useful to investigate the effect of changes in thesevalues on the aggregate scores of the strategy–scenario combinations. Often therelative performance of strategies is robust to changes in these judgmental inputs(von Winterfeldt and Edwards, 1986). This can sometimes lead to the resolutionof disputes between members of a planning team who, for example, may see thatthe same strategy is always superior whichever of a pair of competing weights isattached to an objective.

The use of multiattribute value modelling that we have just outlined meets the needsthat we identified for a formal strategy evaluation process within scenario planning.Its recommendations are not obtained from complex mathematics or from a ‘black-box’ algorithmic procedure. Their derivation is therefore transparent and can easilybe traced from the judgments put forward by decision makers. Secondly, the objec-tives hierarchy enables the evaluation problem to be decomposed to the level whererelatively easy judgments can be made, while the difficult task of estimating probabil-ities for states of nature that might prevail in the long term is avoided.

The method is also versatile in that it allows the decision makers to address prob-lems where trade-offs need to made between financial and non-financial objec-tives. As such, it avoids the risk that strategy evaluation will focus solely on easilyquantifiable objectives. Experience of using multiattribute value analysis with

13

© Blackwell Publishers Ltd 2001

Table VI. Performance of STATUS QUO strategy in MAILMOUNTAIN scenario

Objective Weight Score Weight ¥ score

Short term profit 0.05 100 5.0Long term profit 0.50 80 40.0Market share 0.20 100 20.0Growth 0.10 70 7.0Flexibility 0.15 10 1.5

Aggregate score 73.5

Table VII. Aggregate scores for all strategy–scenario combinations

Strategy Scenario

DOG FIGHT MAIL MOUNTAIN

STATUS QUO 4.5 73.5R&D 41.5 87.5DIVERSIFY 42.3 76.0

managers in a decision conferencing environment (e.g. Phillips, 1989) has shownthat it is considerably flexible in that the model can be quickly and easily updatedor modified to accommodate changing perceptions of the decision problem, orthe alternative perspectives of different managers. This means that multiattributevalue modelling can be used as a tool to explore the decision problem, rather thanacting as a static summary of decision makers’ initial views. Indeed, Phillips hasargued that the insights that result from exploration of conflicts between man-agers’ intuitive views and the recommendations of the models is one of the mostvaluable outcomes of decision analysis. This interaction between the model andthe managers’ perceptions is likely to lead to changes in both the model and man-agers’ views until, what Phillips calls, ‘a requisite decision model’ has been obtained(Phillips, 1989). Finally, in multiattribute value analysis this flexibility and sim-plicity is not bought at the cost of theoretical unsoundness. The approach is firmlyunderpinned by theory and the recomposition of judgments is founded on aformal set of axioms that represent testable descriptions of rational behaviour (seefor example, Dyer, 1990; von Winterfeldt and Edwards, 1986).

Why should the use of multiattribute value modelling improve on existing sce-nario planning practice? First, research into how unaided decision makers makedecisions suggests that they make inefficient use of available information and makenon-compensatory choices. Multiattribute value modelling is a normative decisionmethod that encourages and facilitates the use of all available information andwhich is explicitly designed to guide the decision maker through a process of com-pensatory choice. As a result, the biases emanating from the heuristics employedby unaided decision makers are likely to be avoided.

Secondly, research into the processes employed by groups of decision makershas revealed serious failings that can occur when the decision process is unstruc-tured. In particular, Janis and Mann (1977) have identified a phenomenon theycalled groupthink where high group cohesiveness can lead to a suppression ofdebate. As a result, there is an insufficient search for alternative courses of actionand a collective rationalization of the option perceived to be favoured by the group– even though this option may be extremely risky. The absence of a structuredapproach for making the decision is a known precursor of groupthink. By requir-ing the expression of objectives, scores and weights in explicit and unambiguousterms and by using a formal and logical process to arrive at its results, multi-attribute value modelling can serve to challenge the group’s rationalizations. More-over, its transparency and flexibility can maximize the participation of decisionmakers from diverse backgrounds and hence encourage new perspectives anddebate.

Finally, there are a number of self evident advantages to be gained by employ-ing multiattribute value modelling in scenario planning. Because the choice ofoption is based on a formal decision process, it is possible to retain a record of thedecision model. This can yield a documented and defensible rationale for why aparticular strategy was chosen. Clearly, this can be important when the decisionhas to be justified to senior colleagues, outside agencies or the general public.Indeed, should unforeseeable events mean that the chosen strategy fails, theemployment of a formal decision model may itself be seen as evidence that carewas taken to make the decision in good faith and in a ‘professional way’ (forexample Glantz (1982) describes a case where the perceived lack of professionalpractice in applying judgmental adjustments to drought forecasts led farmers to

14 . .

© Blackwell Publishers Ltd 2001

take legal action against the US government). In future years this documentationmay also prove valuable as a learning tool, enabling future decision makers toassess, with the benefit of hindsight, the validity of judgments that underpinnedpast decisions (Russo and Schoemaker, 1989).

[1] Technically, the scores are measured on an interval scale. It is not therefore possibleto say that strategy–scenario combination scoring 100 is twice as attractive as onescoring 50. However, the improvement in switching from a strategy scenario combina-tion scoring 0 to one scoring 100 can be said to be twice as attractive as the improve-ment obtained by switching from a combination scoring 0 to one scoring 50.

[2] Edwards and Baron (1994) have suggested that approximate weights, referred to asrank order centroid (ROC) weights can be inferred from the ranking of the swings.This clearly simplifies the task of the decision maker by removing the need to estimateweights directly. While ROC weights provided good approximations to directly elicitedweights in simulated decisions, we are, as yet, unaware of any testing of the approachin the field.

B, D. (1982). ‘Potential contributions to decision analysis’. Decision Sciences, 13, 534–40.D, U. and M, H. (1984). ‘Information search and evaluative

processes in decision making: a computer based process tracing study’. Acta Psychologica,56, 113–23.

D, J. S. (1990). ‘Remarks on the analytic hierarchy process’. Management Science, 36,249–58.

E, W. and B, F. H. (1994). ‘SMARTS and SMARTER: improved simplemethods for multiattribute utility measurement’. Organizational Behavior and Human Decision Processes, 60, 306–25.

G, M. H. (1982). ‘Consequences and responsibilities in drought forecasting: the caseof Yakima, 1977’. Water Resources Research, 18, 3–13.

G, P. and W, G. (1993). ‘Improving judgmental time series forecasting: areview of the guidance provided by research’. International Journal of Forecasting, 9,147–61.

H, R. H. and M, S. (1981). ‘Forecasting and planning: an evaluation’.Management Science, 27, 115–38.

J, I. L. and M, L. (1977). Decision Making, a Psychological Analysis of Conflict, Choice andCommitment. New York: Free Press.

J, G. and S, K. (1993). Exploring Corporate Strategy. New York: Prentice Hall,304.

K, D. and T, A. (1982). ‘Variants of uncertainty’. In Kahneman,D., Slovic, P. and Tversky, A. (Eds), Judgment Under Uncertainty: Heuristics and Biases.Cambridge: Cambridge University Press.

K, R. L. and R, H. (1976). Decisions with Multiple Objectives: Preferences and ValueTradeoffs. New York: Wiley.

M, S. and G, A. (1998). ‘Judgment: its role and value for strategy’. In Wright,G. and Goodwin, P. (Eds), Forecasting With Judgment. Chichester: Wiley.

N, S. M. W. and MC, D. (1993). ‘Development of a coherent European strat-egy as part of a global strategy for a multinational corporation in the liner industry’.Unpublished MBA dissertation, Strathclyde Graduate Business School, UK.

15

© Blackwell Publishers Ltd 2001

P, J. W. (1976). ‘Task complexity and contingent processing in decision making: aninformation search and protocol analysis’. Organizational Behavior and Human Performance,16, 366–87.

P, J. W., B, J. R. and J, E. J. (1993). The Adaptive Decision Maker. Cam-bridge: Cambridge University Press.

P, L. D. (1989). ‘People centred group decision support’. In Doukidis, G., Land, F.and Miller, G. (Eds), Knowledge Based Management Support Systems. Chichester: EllisHorwood.

R, J. E. and R, L. D. (1975). ‘An eye fixation analysis of multi-alternative choice’.Memory and Cognition, 3, 267–76.

R, J. E. and S, P. J. H. (1989). Confident Decision Making. London: Piatkus.S, P. J. H. (1991). ‘When and how to use scenario planning’. Journal of Fore-

casting, 10, 549–64.S, P. J. H. (1992). ‘How to link strategic vision to core capabilities’. Sloan

Management Review, Fall, 34, 67–81.S, P. J. H. (1995). ‘Scenario planning: a tool for strategic thinking’. Sloan

Management Review, Winter, 25–40.S, H. A. (1955). ‘A behavioral model of rational choice’. Quarterly Journal of Eco-

nomics, 69, 99–118.S, P., F, F. and L, S. (1977). ‘Behavioral decision theory’. Annual

Review of Psychology, 28, 1–39.S, O. and E, A. (1987). ‘Changes of preference under time pressure: choices

and judgments’. Scandinavian Journal of Psychology, 28, 322–30.T, A. (1969). ‘Intransitivity of preferences’. Psychological Review, 76, 31–48.T, A. (1972). ‘Elimination by aspects: a theory of choice’. Psychological Review, 79,

281–99.T, A. and K, D. (1974). ‘Judgment under uncertainty: heuristics and

biases’. Science, 185, 1124–31.T, A. and K, D. (1983). ‘Extensional vs. intuitive reasoning: the conjunc-

tion fallacy in probability judgment’. Psychological Review, 90, 293–315. H, K. (1994). ‘Probabilistic planning and scenario planning’. In Wright, G.

and Ayton, P. (Eds), Subjective Probability. Chichester: Wiley. H, K. (1996). Scenarios: The Art of Strategic Conversation. Chichester: Wiley. W, D. and E, W. (1986). Decision Analysis and Behavioral Research.

Cambridge: Cambridge University Press.W, P. (1985). ‘Scenarios: uncharted waters ahead’. Harvard Business Review,

63, 6, November–December, 131–42.W (1994). ‘The subjective probability of guilt’. In Wright, G. and Ayton, P. (Eds),

Subjective Probability. Chichester: Wiley.W, S. (1987). ‘Analysis of decision conferences: interpretation of decision makers’

activities in problem identification, problem expressing and problem structuring’. Tech-nical Report, 87–2. Decision Analysis Unit, London School of Economics and PoliticalScience, London.

16 . .

© Blackwell Publishers Ltd 2001