This article was downloaded by: [University of Sussex Library]On: 01 July 2014, At: 05:08Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Local Environment: The InternationalJournal of Justice and SustainabilityPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/cloe20

The appraisal of sustainability: Someproblems and possible responsesAndrew Stirling aa Science Policy Research Unit , University of Sussex ,Brighton, BN1 9RF, UK Phone: 01273 686758 Fax: 01273 686758E-mail:Published online: 02 May 2007.

To cite this article: Andrew Stirling (1999) The appraisal of sustainability: Some problems andpossible responses, Local Environment: The International Journal of Justice and Sustainability,4:2, 111-135, DOI: 10.1080/13549839908725588

To link to this article: http://dx.doi.org/10.1080/13549839908725588

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoeveras to the accuracy, completeness, or suitability for any purpose of the Content. Anyopinions and views expressed in this publication are the opinions and views of theauthors, and are not the views of or endorsed by Taylor & Francis. The accuracyof the Content should not be relied upon and should be independently verifiedwith primary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages, and otherliabilities whatsoever or howsoever caused arising directly or indirectly in connectionwith, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms& Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Local Environment, Vol. 4, No. 2, 1999

ARTICLE

The Appraisal of Sustainability: someproblems and possible responsesANDREW STIRLING

ABSTRACT The development of 'indicators' and associated techniques for theappraisal of 'sustainability' requires efforts to systematically define, quantifyand aggregate many disparate dimensions of social, environmental and econ-omic performance. This necessarily raises a number of serious theoretical andmethodological difficulties, including those relating to the selection and framingof 'problems' and 'options', the treatment of deep uncertainties and the'impossibility' of aggregating in analysis the divergent social interests and valuejudgements which govern the prioritisation of the different dimensions of'sustainability'. After exploring the depth and scope of some of these difficulties,this paper argues that they render futile any attempt to develop an 'analyticalfix' for the problems of appraisal. In this light, systematic public participationis recognised not just as an issue of political efficacy and legitimacy, but alsoas a fundamental matter of analytical rigour. However, it is also concluded thatonce aspirations to the 'analytical fix' are renounced, there is much that mightbe contributed by transparent, straightforward quantitative analytical tools andthe paper ends with some recommendations and an example in this regard.

Indicators and 'Analytical Fixes'

Aspirations to 'sustainable development' are a pervasive and growing theme inlocal, national and global politics alike. Notions of 'sustainability' are coming torival the much-theorised concept of 'risk' as a key organising principle in 'latemodern' industrial societies (e.g. Giddens, 1990; Luhmann, 1991; Beck, 1992;Lash et al., 1996). Yet, for all the rhetorical tumult, formidable questions remainover how to make operational the central ideas of 'sustainability'. Here, as withrisk, the high profile has not always served to clarify the issues. Indeed, thepractical business of appraising the relative 'sustainability' of real decisionoptions seems in many cases actually to be impaired by the ambiguities,polemics and expediencies associated with much mainstream commercial andpolitical interest in 'sustainable development'.

Despite the difficulties, strenuous efforts continue to be directed at meeting thedemand for robust characterisations of what it means to be 'sustainable' invarious different contexts. The development of clearly defined quantitative'indicators of sustainability' must be an important element in any overall

Andrew Stirling, Science Policy Research Unit, University of Sussex, Brighton BN1 9RF, UK. Tel:01273 686758. Fax: 01273 685865. Email: a.c.stirling@sussex.ac.uk

1354-9839/99/020111-25 © 1999 Taylor & Francis Ltd.

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

A. Stirling

strategy for transparent and effective decision making for sustainable develop-ment. Yet, amidst the institutional enthusiasm and academic complexity, theremay sometimes be a tendency to forget that the formulation of general indicatorsof sustainability is only a means to an end, rather than an end in itself. A set ofindicators is only as good as the decisions which it enables. The purpose, afterall, is not monitoring for its own sake, but the informing of difficult andcontentious social choices. Here, it is not enough simply to have developed ageneral suite of indicators, no matter how robust. The crucial—and moreintractable—challenges lie in the case by case use of such indicators to developand prioritise criteria for the evaluation of real investment, technology and policyoptions.

Of course, the appraisal of the relative degree of 'sustainability' displayed bydifferent decision options is an enormous field of activity in its own right. Awide range of disciplines and techniques are available for use in a variety ofcircumstances. However, as is the case in the analysis of related concepts of'risk', 'environmental costs' and 'environmental impacts' (addressed in similarterms elsewhere by the present author, e.g: Stirling, 1998a), the choice ofapproach is often as much a reflection of disciplinary affiliations and rivalries asit is of the intrinsic merits or shortcomings of different approaches in differentcontexts. The result is a confusing array of methodological options. Decision andpolicy analysis, life cycle and environmental impact assessment, multi-criteriaand comparative risk analysis, orthodox and 'constructive' technology assess-ment, as well as the various forms of environmental cost-benefit and cost-effec-tiveness analysis all compete for a niche in the market place of methods. On abroader canvas, economic (e.g. Pearce, 1989), co-evolutionary (e.g. Norgaard,1994) and social theory (e.g. Redclift, 1987) all make creditable bids to establishhegemony over the notion of 'sustainable development'. At a still deeper levelof generalisation, periodic attempts are made effectively to annex the theoreticalfoundations of the concept of 'sustainability' as a proprietary sub-discipline ofnatural sciences such as physics and chemistry (e.g. Azar et ah, 1996) or ecology(see Lele & Norgaard, 1995). If the matter were judged simply on the basis ofambition, then the business of characterising sustainability might be thought wellin hand!

Unfortunately, of course, a proliferation of candidate understandings is notnecessarily a sign of imminent enlightenment! In fact, some of the mostproblematic features of the quest to characterise sustainability are held incommon by virtually all the available approaches to appraisal. At root, theimplication of many such approaches is that 'sustainability', at least in its mostimportant aspects, may be regarded as if it were an objectively determinatequantity. Under such a view, the task of appraisal is simply to identify the 'best'among an array of policy, technology or investment options. To this extent, inthe terms of the philosopher of science Thomas Kuhn, such approaches share theobjective of converting the fuzzy and controversial socio-political problems ofsustainability into precisely defined and relatively tractable analytical puzzles(Kuhn, 1970, Funtowicz & Ravetz, 1990). In short, many of the availableapproaches aspire to develop a nice, clean 'analytical fix' for the messy (andintrinsically political) business of decision making on sustainability.

112

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

The Appraisal of Sustainability

Variability and Ambiguity in the Appraisal of Sustainability

Aspirations to the 'analytical fix' in appraisal may, at least in part, be due tothe strong convergence of interest between those seeking to promote particularacademic disciplines and those seeking convenient ways to justify difficult orpotentially unpopular decisions. Whether or not this is the case, the intoxicat-ing mixture of institutional interests and high decision stakes does little toenhance the humility of advocates of the 'analytical fix' in the appraisal ofsustainability. Unfortunately, the assertive self-confidence and heroic pre-cision with which the results of so much environmental and economicappraisal are typically expressed do not seem to be borne out in their practicalaccuracy or in the manifest resilience which they display in the face ofcriticism. One particularly clear (but not unique) example, might be drawnfrom the use of environmental cost-benefit analysis in the appraisal of 'energysustainability'.

The first problem is that the application of different (but equally 'reasonable')value judgements and framing assumptions in analysis may lead to a situationwhere different studies yield radically different results. Figure 1 displays on alogarithmic monetary scale the values obtained for the external environmentalcosts of new coal power published in some 32 different government- andindustry-sponsored studies over the past two decades in industrialised countries.The studies are ordered vertically by year of publication. Where an individualstudy acknowledges variability or uncertainty by stating a range of values, thisis represented by a horizontal bar. Nevertheless, it is clear that the range ofresults obtained over the literature as a whole far exceeds the range expressedin any individual study (Stirling, 1998b). There appears to be no pronouncedtrend towards convergence of results over time. In fact, the difference betweenthe highest and lowest values is more than four orders of magnitude: a factor ofmore than 50 000!

On the face of it, it seems obvious that this picture does not bode well foraspirations to determine by analysis the degree of 'sustainability' of anyindividual technology, policy or investment option. Unfortunately, however,the implications for comparative appraisal are even more severe. Figure 2displays on the same scale the ranges of values attributed in the same studiesto the external environmental costs of eight different electricity generatingtechnologies. Again, it is clear that there is significant variability in the resultsobtained for all the different options. The general trend from top left to bottomright might be taken as a (somewhat fuzzy) endorsement of the ratheruncontroversial overall impression that renewable energy options displaygenerally better environmental performance than do the fossil fuels or nuclearpower. However, even this slightly woolly (and rather anti-climactic!) con-clusion must be qualified. For such is the degree of overlap between the rangesof external environmental costs obtained for different options that—byjudicious choice of methodological conventions and framing assumptions—the results obtained across the literature as a whole might be employed todefend virtually any conceivable ranking order for these eight options (Stirling,1998b).

113

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

A. Stirling

stated external environmental cost of new coal power

logarithmic scale

0.01 100

1995 c/kWh

1000

0.01 0.1 1

| | excludes global warming

10 100 1000

addresses global warming

FIGURE 1. Variability in the analysis of environmental performance (an example from theenvironmental cost-benefit analysis of energy options). Note: The results are expressed in US dollarsat 1995 prices. They are, in the order displayed in Figure 1, those of: Ramsay, 1979; Shuman andCavanagh, 1982; ECO Northwest, 1987;EPRI, 1987;Hohmeyer, 1988, Chernick and Caverhill, 1989;Shilberg, 1989; CEC, 1989; Friedrich et al., 1990; Kcomey, 1990; Hohmeyer, 1990; Bernow &Matron, 1990; Ottinger, 1990; Bernow, 1990; Hagen, 1991; Koomey, 1991; Stacker, 1991; DTI,1992; Hohmeyer, 1992; Cline, 1992; Ferguson, 1992; Hohmeyer & Walz, 1993; ExternE, 1993;Friedrich, 1993; Eyre & Holland, 1993; Fankhauser, 1993; Pearce, 1993; Lazarus etal, 1993; Meyer,

1994; Eyre, 1995; ExternE, 1995; Tol, 1995.

114

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

The Appraisal of Sustainability

stated external environmental cost (bars represent range over a variety of studies)

logarithmic scale

0.001 0.01 0.1 1 10

"NUCLEAR-nssioNlil'BS

OILS

COALS

FIGURE 2. Ambiguities in the environmental ranking of contending options (an example from theenvironmental cost-benefit analysis of energy options).

Completeness, System Boundaries and Indicator Choice

So what, if anything, is going wrong? It must be emphasised that the environ-mental valuation of energy options is taken as an example here not so muchbecause it is uniquely deficient in this respect, but rather because it is a field withwhich the author happens to be relatively familiar. Similar problems evidentlyarise under other appraisal techniques such as risk (Stirling, 1997b) or multi-criteria analysis (Stirling, 1996a) and in different empirical fields (such astransport or agricultural policy). At the present level of generalisation, thefundamental issues are essentially the same. In the environmental valuation ofenergy options, as in these other areas, there is the same uncomfortable question.How is it that the analysis of such a serious topic by such competent specialistsfrom such authoritative institutions should be expressed with such fine numericalprecision, yet yield such an ambiguous overall picture?

This question is addressed in some detail elsewhere by the present author(Stirling, 1997a). For all the potential complexity, the basic features of theproblem are really just a matter of common sense and quite readily conveyed.For instance, the apparent simplicity and precision of numerical results tends toobscure the fact that different studies include and exclude different types ofimpact. This is illustrated schematically in Figure 3 for some of the studieswhose results are displayed in Figures 1 and 2. The particular effects which itis felt appropriate to address in analysis will depend on the wider context of

115

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

A. Stirling

globalwarminq

nuclearproliferation

ecosystemdamage

aestheticimpacts

Shuman & Cavanagh

1982

Hohmeyer

1988

* * * * \

Ottinger1990

ExternE1995

,, £ JKjt it*.-

effectneglected

effectaddressed

FIGURE 3. Completeness of scope in a sample of different environmental appraisal studies (anexample from the environmental cost-benefit analysis of energy options).

appraisal and will vary from case to case. However, even were there to beestablished a rigid universal convention concerning which effects to include andwhich to exclude from analysis, the problem would remain that the adoption ofany candidate convention would itself have important implications for theordering of different options in terms of their relative 'sustainability'. Alterna-tive, but equally 'reasonable' conventions would yield different ranking ordersfor the 'sustainability' of the different technology policy or investment optionsunder scrutiny.

Another difficulty lies in the fact (illustrated in Figure 4) that even within aparticular appraisal discipline, individual studies often employ entirely differentaccounting methods in the assessment of different effects. For instance, someenvironmental valuation studies express costs in terms of the alleviation ofenvironmental damage once committed (i.e. 'mitigation costs' such as afforesta-tion or sea defences). Others assess costs by examining prevailing property orwage markets or responses to questionnaires ('hedonic market' or 'contingentvaluation' respectively). Still others take the costs of controlling pollution atsource as a proxy indicator for the social costs of the environmental impactsthereby avoided ('abatement costs' such as flue gas desulphurisation). A finalgroup of studies conduct 'bottom-up' assessment of the costs associated witheach physical dose-response relationship ('damage costs'). It cannot be assumedthat such different methods will necessarily provide directly comparable picturesof the effects which they are intended to characterise—indeed, the differencesare often a point at issue. The point is, that the results obtained for specificeffects are often found to very between different methods (Peterson & Driver,1988; DTI, 1992; Stirling, 1997b).

In environmental economics (as in many of the other appraisal disciplinesmentioned above) there is a considerable technical literature (and little consen-sus) concerning the relative merits and deficiencies of different measurement

116

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

The Appraisal of Sustainability

abatementcost

mitigationcost

hypotheticalmarkets

damagecost

Hohmeyer1988

Ottinger1990

•

Tellus1991

-

ExternE1995

principal environmental valuation technique

FIGURE 4. The use of different methodologies in different environmental appraisal studies (anexample from the environmental cost-benefit analysis of energy options).

techniques and accounting methods. The conditions for the applicability of eachindividual approach are often strongly contested and are acknowledged even byadvocates to be circumscribed and circumstantial. A particular case in point withregard to 'sustainability', concerns the crucial role of assumptions over thediscounting of future impacts (Norgaard & Howarth, 1991; Stirling, 1998b). Theresulting mosaic of complex implications for the interpretation of appraisal tendsto remain neglected. Under all analytical appraisal approaches alike, then,serious questions are raised concerning the extent to which results reflect theperformance of the options under appraisal and the degree to which they areeffectively determined by particular methodological choices concerning indica-tors, framing assumptions or accounting methods.

A full account of the impacts on results of framing assumptions and choice ofindicators and methodologies would be prohibitively long. As a final illustrationof the importance of such factors, there is the question of the boundaries of thetechnological, policy or project 'systems' under appraisal. These are often farfrom obvious and yet can have a critical influence on the results (OECD, 1983;UNEP, 1985; IAEA, 1991). For instance, how far should analysis include thevarious different stages in the 'resource chains' associated with different options:from the extraction of raw materials and other inputs, through the various stagesof processing to the management of any emissions and residues which arisealong the way? Likewise, there is the question of the 'life cycles' of the variousfacilities involved at each stage in these chains, from construction, throughoperation to decommissioning. Should analysis include the materials and energyconsumed in the construction of all associated facilities? What of the resourcechains and life cycles associated with the provision of these material and energyinputs? What of more intangible inputs and contingencies, such as 'information'and infrastructural interdependencies? There is clearly enormous latitude forchoice of framing assumptions, with each step regressing into successive depthsof the economy as a whole. Ultimately, the choice of the conventions to

117

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

A. Stirling

Hohmeyer,1988

Ottinger1990

DTI, 1992

stage in the 'resource chain'

extraction

processing

transport

storage

conversion

residues

phase in the 'life cycle'

materialinputs

energyinputs

construction

operation

capacity

decommissioninq

addressed partlyaddressed

not addressed

FIGURE 5. The treatment of system boundaries in different environmental appraisal studies (an

example from the environmental cost-benefit analysis of energy options).

adopt in the setting of 'system boundaries' for analysis are a matter of essentiallysubjective judgement.

As shown in Figure 5, different appraisal studies typically include and excludedifferent stages in the 'resource chains' and phases in the 'life cycles' ofdifferent options. Again, the important point here is that different stages andphases play dominant roles in the environmental and social impacts of differentoptions. Different (but equally 'reasonable') conventions over system boundariesmay therefore exert a significant influence on any final picture of the relative'sustainability' of different options.

'Impossibility' and the Multiple Dimensions of 'Sustainability'

One of the most basic tasks in appraisal is to achieve, for any given set ofassumptions, at least some notion of the relative ordering of the various options

118

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

The Appraisal of Sustainability

under consideration. Without this, there must be serious questions over whethera particular appraisal methodology is of any practical policy use at all. It hasalready been commented here that the sustainability indicator literature has insome ways tended hitherto to be preoccupied with the essentially passivebusiness of monitoring rather than with the active appraisal of real technology,policy or investment options. However, the picture arising from the field ofenvironmental valuation (along with that equally evident in other broad quanti-tative approaches to environmental appraisal, such as risk and multi-criteriaanalysis) offer a salutary lesson for any who aspire to use 'sustainabilityindicators' in the development of a definitive broad analytical characterisation of'sustainable development'. The ambition to achieve some sort of uniquely robustand transcendent authority is undermined by the intrinsic subjectivity of keyframing assumptions and conventions such as those discussed here concerningthe scope of appraisal, the choice of indicators and accounting procedures andthe setting of system boundaries.

The problem is that 'sustainability' (like related notions of 'risk' and 'environ-mental cost') is an inherently multi-dimensional concept. Many of the disparateenvironmental, social and economic factors involved are mutually incommen-surable. Issues such as occupational safety, child mortality, future cancer risks,biodiversity loss, employment, regional development and social, gender andglobal equity simply cannot adequately be captured under the same yardstick.The aggregation of performance measured under any one criterion with thatunder any other criterion will require (either implicitly or explicitly) theimposition of subjective value judgements concerning the relative importance ofthe different aspects of sustainability.

Where certain technology, policy or investment options are manifestly supe-rior to others under virtually all criteria, there will—of course—exist a clearpotential for 'no regrets' choices and strategies. Indeed, the identification of suchopportunities may often be an important and highly positive result of formalappraisal. However, it is all too often the case that difficult trade-offs must bemade between different classes of environmental effect and between theseeffects and other factors such as employment, welfare or equity implications. Tothe scope for confusion over analytical conventions such as those alreadydiscussed, must be added a bewildering constellation of cross-cutting consider-ations concerning the nature and priority of the different aspects of sustainability,their implications for the autonomy of those affected and the way in which theyare distributed.

Some of the implications of incommensurability in social appraisal werefamously explored in another context by the Nobel Prize-winning economistKenneth Arrow more than 30 years ago. Despite a voluminous subsequentliterature, the resulting 'Arrow Impossibility Theorem' is acknowledged todemonstrate in quite general terms that it is impossible both democratically andconsistently to aggregate individual preferences in a plural society (Arrow,1963). In the terms of economics, the derivation of any single social preferenceordering (or aggregate social welfare function) will violate at least one of aminimal set of conditions held to be axiomatic in the characterisation ofindividual choice. No matter how much information is available, and no matter

119

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

A. Stirling

TABLE 1. The formal conditions applied in deriving the 'Arrow Impossibility'

• 'free triple condition': the ordering of social preferences for each of a set of options shouldbe the same irrespective of the way sub-sets of these options aregrouped together.

• 'non-negative association' any option that is increasingly favoured by all individuals, should beincreasingly favoured in the expression of social preference.

• 'independence of the introduction of new options, or the omission of old ones, shouldirrelevant alternatives' not alter the ordering of preferences for the other options.

• 'non-imposition' if individuals are able to choose between any two options, then it shouldbe possible to derive a social preference for one of these two options.

• 'non-dictatorship' under no conditions should social preference be determined by thepreferences of any single individual.

Note: It is interesting that a condition imposing equity of weighting to the preferences of all individualsis absent from Arrow's list. Despite this, however, Arrow showed that it is logically impossiblesimultaneously to satisfy even this more permissive set of five conditions. These issues are discussedin more detail by Kelly (1978), MacKay (1980) and Bonner (1986) with a convenient summary ofthe discussion provided in Pearce & Nash (1981)

how much consultation and consideration are involved, no purely analyticalprocedure can fulfil the role of a democratic political process. In other words,even in terms of the theoretical framework underlying the assessment method-ologies themselves, there can be no uniquely 'rational' way to resolve contradic-tory perspectives or conflicts of interest over incommensurable issues in a pluralsociety.

It is well known that the different types of environmental and social impactassociated with notions of 'sustainability' may be distinguished under dimen-sions such as: their severity (e.g. the balance of morbidity to mortality); theirimmediacy (e.g. injury versus disease), their duration, reversibility, familiarity orcontrollability, their geographic or demographic distribution and their 'gravity'(whether they are concentrated in single serious episodes or spread over anumber of relatively minor events) (cf. Fischhoff et ah, 1981). Not only theindividual effects themselves, then, but also these cross-cutting dimensions areincommensurable, in the sense that they cannot readily be aggregated under anysingle yardstick. It is possible to take different but equally reasonable views onthe relative importance of the different dimensions. The problem is not that it isdifficult in practice to assign overall priorities to the different classes anddimensions of a disparate set of sustainability impacts (and benefits). Rather, itis impossible in principle even meaningfully to conceive of a single 'objective'ordering of social priorities under these dimensions. Here, as in policy analysismore generally, there can be no straightforward 'analytical fix' for the problemsfaced in the social appraisal of 'sustainability' (cf. O'Neill, 1993).

Drawing primarily on the comparative risk assessment literature, a range ofmore than 30 different 'dimensions of sustainability' are presented in Table 2.In effect, these take the form of simple but potentially highly challengingquestions which might be asked of any ostensibly precise quantitative appraisalof overall sustainability. In short, judgements over the relative importance of

120

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

The Appraisal of Sustainability

factors such as severity, reversibility, familiarity, gravity and uncertainty areobviously and inescapably context-specific as well being subjective, political andethical in character. Seen in this light, aspirations to construct a robust andunambiguous general analytical characterisation of sustainability solely on thebasis of formal quantitative indicators look not only futile, but potentially quiteseriously misleading and wasteful of time and resources. Despite the frequentplaintive cries from beleaguered policy makers, the problem is not one ofbecoming more 'rational' about issues such as 'risk' and 'sustainability'. Thequestion is rather how decision making on sustainability can at the same time bebased on the best available technical and scientific information, whilst remainingsensitive to (and honest about) divergent interests and conflicting value judge-ments.

'Ignorance' and the Dilemmas of Uncertainty

The 'Arrow Impossibility' applies under plural perspectives even where there isperfect information concerning performance under each individual perspective.Unfortunately, in appraising performance in terms which are as extensive, asnebulous and as long-term in their implications as 'sustainability', such infor-mation is seldom anywhere near 'perfect'. Indeed, in cases like nuclear safety,global warming or genetic modification, uncertainties concerning key aspects ofperformance are often contentious dimensions of appraisal in their own right.Here, we encounter a second problem for the 'analytical fix' in the appraisal ofsustainability, one which is equally fundamental to that addressed in Arrow'sImpossibility. Yet, unlike 'impossibility', this is an issue which may emergeunder even the most tightly coherent of perspectives. It may arise even where alldimensions of performance are held to be entirely commensurable. This is theproblem of ignorance.

Despite the unfortunate pejorative connotations of the word itself, the conceptwhich is conventionally labelled as 'ignorance' has long been recognised in aneutral formal sense which implies no criticism of those to whom it applies.Although only relatively rarely acknowledged in the huge literature on 'decisionmaking under uncertainty', the condition of 'ignorance' is symmetrical with (andimplied by) the most systematic and widely accepted definitions of 'risk' and'uncertainty' embodied in probability theory (see Figure 6). Put briefly, 'igno-rance' is, in these terms, a condition under which it is possible neither to resolvea discrete set of probabilities (or a density function) along a scale of outcomes(as is possible under risk proper—in the top left of the grid in Figure 6), nor evento define a comprehensive set of outcomes (as under uncertainty—in the bottomleft of Figure 6). Ignorance, in this formal sense, then, lies in the bottom righthand corner of the grid in Figure 6.

The appraisal of 'sustainability' implies the consideration of enormouslycomplex causal chains (such as those associated with the interactions ofatmospheric emissions and climate) operating over very long time periods (suchas those governing the management of irradiated nuclear materials). It requiresthe adoption of a perspective which is global in extent (as when considering theeconomic context for 'sustainable development') but with the resolution of fine

121

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

A. Stirling

TABLE 2. Some different dimensions of 'sustainability' (i.e., some questions from risk assessmentwhich apply also to any aggregated representation of 'sustainability')

Severity:

Immediacy:

Gravity:

Reversibility:

Spatial distribution:

Balance of benefitsand burdens:

Fairness:

Public or workerexposure:Intergenerationalequity:Human ornon-human:

Voluntariness:

Controllability:

Familiarity:

Trust:

The Form of Different Aspects of SustainabilityDo the options differ in the ratios of risks of death to risks of injury or diseasewhich they pose? How much illness or how many serious injuries equate inseverity with one death?Are the effects associated with different options equally immediate in theirmanifestation or do they differ in the degree of latency between the initialcommitment of a burden and the eventual realisation of an effect? For instance,are some risks manifest as injuries and others as disease?Are the effects associated with some options dominated by low probabilitiesof large impacts, while those of other options are characterised predominantlyas high probabilities of relatively low impacts? To what extent are impacts theresult of single or repeated events?Are the effects associated with different options all equally reversible after theyhave been committed?

The Distribution of Different Aspects of SustainabilityAre the effects associated with different options identical in their spatialextents? Is it better that impacts of a given magnitude be geographicallyconcentrated or dispersed?To what extent is the social distribution of the environmental burdens causedby each option balanced by the distribution of any associated social or economicbenefits?To what extent do the distributions of burdens imposed by the different optionsact to alleviate or compound pre-existing patterns of privilege or socialdisadvantage? To what extent should exposure to other (unrelated) factors betaken into account in the assessment sustainability?To what extent do different options impose different distributions of effectsacross workers and the general public?Do the effects associated with certain options present risks to future generationsto a degree not associated with others? How should these effects be balanced?Do the options differ in the degree to which their impacts affect the well-beingof humans and non-human organisms?

The Autonomy of those Affected by Different Aspects of SustainabilityDo the environmental effects of different options vary in the degree to whichexposure may be considered to be 'voluntary' prior to the commitment of animpact?Once committed, are the impacts associated with different options all equallycontrollable from the point of view of the affected individuals or communities?Do certain effects require efforts at control which are perceived to pose a threatto democratic institutions or processes?Do the effects associated with different options differ in terms of the degreeto they are familiar to individuals, communities and established socialinstitutions? Do responses to the different effects involve equally disruptivechanges to normal routines and attitudes?Do options differ in terms of the degree of trust enjoyed in the wider societyby the institutions and communities charged with evaluating and managingtheir associated impacts? Does the appraisal of certain options tend to be morea specialised undertaking than that of others?

122

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

The Appraisal of Sustainability

TABLE 2. Continued.

The Choice of Indicators of SustainabilityAre the effects associated with different options all equally quantifiable? Howhas sustainability appraisal avoided a disproportionate emphasis on the morequantifiable aspects—and thus an overemphasis of the impacts of the associatedoptions?What is the fidelity of the models employed to track the relationship betweenthe magnitudes of particular burdens and the scale of their associated effects?Are there any non-linearities, or even 'non-monotonicities' in the dose-re-sponse functions?How well does each sustainability indicator resolve the full character and scopeof the individual effect which it is intended to represent?How coherent is the classificatory scheme adopted in any particular study withrespect to the full range of the different aspects of sustainability? Are there gapsor overlaps between the different classes of effect which are recognised for thepurposes of analysis?

Quantifiability:

Fidelity:

Resolution:

Coherence:

Ignorance:

Data quality:

Aetiology.

Specificity:

Trajectories:

System boundaries:

Completeness:

Articulation:

The Treatment of UncertaintyHow important is the element of surprise? Do some effects involve complex,novel or highly contingent mechanisms more than others? Are there are largediscrepancies in the degree of established experience with particular optionsor effects?Are the performance data for the different options all of comparable qualityand pertinence? Do some derive from ex post and other from ex ante studies?Are the effects of all options equally 'direct' in their manifestation, or do someinvolve complex, contingent or synergistic interactions with other agents oractivities to a greater extent than others? What possible interactions might existbetween the effects associated with different options in an overall portfolio?

The Framing and Presentation of Sustainability AppraisalsHow site-specific is the performance data for the different options? Howsensitive are results to assumptions about the operational characteristics of theindividual options and of the system in which they are embedded?How long a historic data series is appropriate as a basis for the appraisal ofcurrent options? How robust are assumptions concerning the likely futurebehaviour of those affected? Are different options on different 'learning curves'in terms of the potential for future improvements in performance?How systematically does the sustainability appraisal address the resourcechains and facility life cycles associated with the different options? How farback into the wider economy should analysis regress in assessing energy andmaterial inputs?How complete is the scope of appraisal with respect to the full range of relevantsocial, environmental and economic effects? How might changes in the scopeof appraisal alter the apparent degree of sustainability of the different options?How are the results of sustainability appraisal to be articulated with widerconsiderations and the subsequent decision making process. At what point doesthe domain of analysis end and that of politics begin?

123

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

A. Stirling

Km

tm

3WLBDGS

6LIH0OOS i

firm basisfor

probabilities

shaky basisfor

probabilities

no basisfor

probabilities

KSOWLEOGiE JS808T OUTCOMES ;

continuum ofoutcomes

*ei of discreteoutcomes

outcomespoorly defined

RISK

apply:

frequentistdistribution

functions

Bayesiandistribution

functions

discretefrequentist

probabilities

discreteBayesian

probabilities

UNCERTAINTY

apply: scenarioanalysis

'FUZZINESS1

apply:

fuzzy

logic

IGNORANCE

apply:flexibilitydiversity

FIGURE 6. 'Risk', 'Uncertainty' and 'Ignorance'. Note: The concepts set out here are discussed more

fully in Knight, 1922; Loasby, 1976; Collingridge, 1932; Smithson, 1989; Wynne, 1992; Faber &

Proops, 1994.

local scale details (for instance, in understanding the determinants of fuel woodscarcity). Account must be taken of human, biological and physical systemswhich are highly dynamic over time (as with technological change, pestpopulations and weather) and which may be essentially indeterminate in theirinternal mechanics (as in social and ecosystem change and many other forms ofnon-linear system). When these predicaments are compounded by the enormousscope for interactions between disparate political, cultural, economic, ecological,biological, chemical and physical aspects of 'sustainability', simple numericalexpressions of likelihood begin to look inadequate. Indeed, it is all too often thecase that long before consideration turns to 'probabilities', comprehension isconfounded even in the characterisation of the 'possibilities' themselves.

Although it may be seen variously as epistemological or ontological incharacter, the concept of ignorance is thus of considerable practical importancein the appraisal of 'sustainability' (cf., Winkler, 1986, and other essays inWinterfeldt & Edwards, 1986). It arises from many familiar sources, includingincomplete knowledge, contradictory information, conceptual imprecision, diver-gent frames of reference and the intrinsic complexity or indeterminacy of manynatural and social processes (cf., Granger Morgan et al., 1990; Funtowicz &

124

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

The Appraisal of Sustainability

Ravetz, 1990). Indeed, the frequent association of a need for 'precaution' withthe implementation of policies on 'sustainability' underscore widespread recog-nition of the pervasive importance of the condition of ignorance. Even where aprinciple of precaution has not been accepted or made operational, ignorance isoften acknowledged to be the dominant condition in many areas of long-termtechnology choice, investment appraisal and risk assessment (e.g. Wynne, 1992;O'Riordan & Cameron, 1994; Dovers & Handmer, 1995). Under such circum-stances, despite their sophistication and elegance, there exists no credible basisat all for the application of the techniques of probability theory. To the extentthat it requires the (implicitly) definitive characterisation of an (ostensibly)comprehensive set of (nominally) discrete outcomes, even simple scenarioanalysis may be seriously unrealistic.

Any characterisation of complex, dynamic, multidimensional performance interms of a restricted set of scalar indices requires a similar collapse in conceptsof uncertainty and ignorance. Indeed, even the more sophisticated approaches(based on an 'expected utility' calculus) involve the treatment of uncertainty andignorance as if they were mere 'risk'. In view of the full scope and depth of theissues involved, all such approaches display what Hayek once called the"pretence at knowledge" (Hayek, 1978, p.23). The condition of ignorance is, bydefinition, uncharacterisable in analysis. Yet ignorance is quite simply thedominant condition in the appraisal of many aspects of sustainability. No matterhow well informed they may be, judgements concerning the extent to which 'wedon't know what we don't know' remain intrinsically subjective and value laden.The quest for an 'analytical fix' in the appraisal of 'sustainability' looks evenmore futile.

Some Practical Implications

None of this is new. Much of what has been discussed here is well known tomany of those involved in the development of sustainability indicators and in thepractice of the various disciplines of environmental appraisal. The problem is notthat the difficulties are unknown. The issue is rather that the implications remainon the whole relatively under-explored by theoreticians and under-addressed bypractitioners. More importantly, they tend to be under-communicated to thesponsors of analysis, to the wider discourse on sustainability and—most seri-ously—to the stakeholders and other interested parties who stand to be affectedby any individual decisions which are informed by appraisal under sustainabilityindicators.

Although too rarely given the attention they deserve in the published litera-ture, these issues are sometimes more readily acknowledged informally. Here, avariety of rationales are employed for avoiding fuller or more formal discussionof the difficulties with the 'analytical fix' in environmental appraisal and thecharacterisation of sustainability. Interestingly, some of the arguments employedin this vein are mutually contradictory. For instance, to some, difficulties suchas those discussed here may seem self-evident and even trivial. To others,however, they may appear unduly complicated and philosophical. On the onehand, the highlighting of limitations to a positivistic approach to appraisal may

125

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

A. Stirling

be perceived as 'anti-scientific', 'postmodern' or even 'politically correct' in itsimplications. On the other hand, a focus on such issues may be seen as animpractical 'fundamentalist' preoccupation with 'pristine' analytical rigour. Ineither event, the implication is that the profound general difficulties with the'analytical fix' approach to environmental appraisal and the characterisation ofsustainability might reasonably be set aside as epi-phenomena. While of interestto philosophers, the argument goes, such issues might be held to have littlepractical relevance for the hard-nosed business of appraisal in a real world ofcomplex and dynamic problems and constrained time and resources.

Unfortunately, neither of these caricatures is true. Far from being 'anti-sci-entific' or 'politically correct' in flavour, acknowledgement of the difficulties inaspiring to the 'analytical fix' in appraisal is (as has been argued) founded on thesome of the most robust and scientifically 'respectable' of theoretical founda-tions. Being implicit in any application of social choice or probability theory, theArrow Impossibility and the condition of ignorance are both intrinsic to thedeepest foundations of appraisal. A glance at Figures 1 and 2 should be sufficientto indicate the enormous practical repercussions. Perhaps more surprisingly,though, the serious difficulties raised in this paper need not be taken asprohibitive to the practical business of appraisal. There is no need to descendinto some sort of fatalistic 'post-modern' crisis of confidence and abandon allnotions of good practice. Indeed, it is not even the case that we ought necessarilyto renounce the discipline and clarity of quantitative techniques. Some of theimplications of this critique are quite readily made operational in the practicalcontext of the appraisal of real investment, technology or policy options. It is onthis final (more positive) theme that this paper will close.

The first implication is rather straightforward and already more widelyaccepted in the field of sustainability indicators than in areas such as riskassessment or environmental economics. Since we cannot meaningfully express'sustainability' in terms of a single discrete scalar number, then we might see itrather as a vector with as many elements as there are relevant dimensions in anygiven context (Funtowicz & Ravetz, 1990; Gregory et ah, 1993; Vatn &Bromley, 1994). It is, of course, for this very reason that sustainability tends tobe discussed in terms of 'indicators' (plural) more often than in terms of a single'index' (although this latter ambition has apparently still not been entirelyrelinquished in certain quarters, e.g. Daly & Cobb, 1989).

The second practical implication seems less well recognised, although it is noless readily addressed. If 'sustainability' cannot be expressed as a singledeterminate quantity, then we might usefully think of it as a pattern ofsensitivities. Rather than seeking to express 'sustainability' in terms of discretenumerical results obtained under a particular set of indicators, appraisal mightinstead aim systematically to 'map' the sensitivities of results under divergentassumptions. Here, it is curious that quantitative techniques involving matrix andsensitivity analysis should be well established in relatively deterministic fieldssuch as engineering and yet so poorly represented in the areas of environmentalassessment and social appraisal (which are, if anything, more complex andindeterminate). Indeed, even in the field of multi-criteria evaluation, it isoften the case that analysis attempts to evade the implications of ignorance

126

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

The Appraisal of Sustainability

and impossibility by aspiring in some sense to express performance in termsof aggregated numerical values and a unitary set of rankings. The outputs ofanalysis for the social appraisal of sustainability should not be a discrete setof prescriptive 'results' (numerical or otherwise). Rather, they should take theform of 'political sensitivity analysis' explicitly displaying the way in which theapparent rankings of options change under different social perspectives.

It is here that we encounter arguably the most important single implication ofthis theoretical and methodological critique of the 'analytical fix' in appraisal.The need for active public participation in the characterisation of 'sustainabledevelopment' and in the appraisal of 'sustainability' are well established themesin the wider debate. Much useful work has been conducted in developingprocedures and institutions, such as citizen's juries, consensus conferences,planning cells and focus groups (cf., Webler et ah, 1995; Durant & Joss, 1995;MacNaghten et ah, 1995; Renn et ah, 1996). Indeed, with the emphasis givento these issues under Agenda 21, the field of 'sustainable development' might inthis respect be seen to be quite well ahead of other areas (such as risk analysisand environmental economics) where the same necessities are relatively less wellrecognised. However, the rationable for active public participation is still oftenexpressed in terms of considerations such as equity, political legitimacy or theease of decision implementation (Fiorino, 1989). These are undoubtedly ofcritical importance, but they are, in themselves, incomplete. The implications ofthe present paper are that a further rationale should be added for the solicitingof active and inclusive participation of all interested parties in appraisal. This isrationale of analytical rigour.

With the insight that even in principle we cannot expect to find definitiveanalytical justifications for one sustainability strategy over another, the role ofappraisal becomes the systematic and transparent exploring of the implicationsof different perspectives. Rather than counterpoising technical analysis andpublic deliberation in appraisal, the two are thus better seen as intrinsicallyinterdependent. For, it is only through public deliberation that analysis mayobtain the essential empirical inputs concerning the selection, definition andprioritisation of the appraisal criteria which are essential to analysis. Based onthe best available scientific and technical data, combined with empirically-grounded weighting schemes for the different incommensurable dimensionsinvolved, straightforward multi-criteria techniques may be used to generate'political sensitivity maps'. Where they are based on systematic inclusion of allinterested and affected parties, these would offer a robust and transparentdisplay, both to decision makers and other stakeholders, of the political andethical implications of different decisions, without seeking to order these interms of their 'rationality' (Stirling, 1997b).

Beyond this, there is a final possible practical analytical tool in the appraisalof 'sustainability'. If we cannot confidently employ 'first past the post' ap-proaches to identify a single objectively 'most sustainable' choice from the pointof view of society as a whole, then might we not instead begin to think in termsof portfolios of choices? Here, there is a remarkable and rare piece of good luckon the side of those interested in the analytical aspects of appraisal. Incontemplating the intractabilities of ignorance and Arrow's Impossibility, there

127

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

A. Stirling

is one common sense strategic response which stands out beyond all others.Don't put all your eggs in one basket! The pursuit in parallel of a diverseportfolio of policy or investment options may at the same time accommodatedivergent value judgements and help to hedge against ignorance in appraisal.Although itself a challenging task, the characterisation of diversity amongst aseries of contending decision options presents a potentially far more tractableproblem than the definitive characterisation of all possible future states of theworld and the objective definition of an aggregate social welfare function acrossall available options (cf., Stirling, 1994a, 1996b, 1997a). Indeed, where diversityis precisely characterised, it is even possible to derive quite robust numericalindices (cf: Stirling 1994a, 1995). In this way, there is no reason in principlewhy the diversification of options in the face of ignorance and pluralism mightnot be handled as readily as any other dimension of the appraisal exercise.All-important trade-offs between diversity, cost and the various dimensions of'sustainability' may all equally be treated by 'political sensitivity analysis'.

An Illustrative Example

So what would a real exercise in the 'political sensitivity analysis' of strategiesfor 'sustainability' actually look like? This is a highly demanding question whichthere is space here only to address in the most cursory of illustrative terms.Based on work published in greater detail elsewhere (Stirling, 1997), Figure 7displays a schematic picture (drawn from a hypothetical exercise), which showsin very general terms what the results of such an appraisal might look like.

For present purposes, it is sufficient to note simply that this highly stylisedpicture relates to a set of 63 permutations of possible perspectives concerning therelative importance of five different performance criteria in the appraisal of therelative 'sustainability' of three broad electricity generating options for the UK.Each of the perspectives represented as a pie chart in Figure 7 is characterisedby a set of numerical weightings, reflecting systematic permutations of therelative priorities that might be attached to the criteria of financial cost, airpollution, land use issues, radiation issues and technological diversity (as ameans to address ignorance and social pluralism—as well as other factors suchas energy security of supply). These weighting schemes are then articulated(using a 'linear additive weighting' approach—the simplest of multi-criteriaprocedures), with the best available technical data (acknowledging uncertainties)concerning the performance of the various options under each criterion. Usingthe simple quantitative characterisation of diversity mentioned above, thisprocedure identifies for each perspective a particular mix of options which wouldconstitute (in the terms of this paper) the 'most sustainable' objective of currentpolicy or investment strategies—taking account both of subjective values andbest available technical data.

Perhaps the best way to describe the practical policy or decision makingimplications of such a procedure would be to imagine that the results displayedin Figure 7 were arrived at through an inclusive and authoritative participatoryappraisal exercise, making use of deliberative procedures such as those men-tioned earlier in this paper. Here, the most striking point is the contrast which

128

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

The Appraisal of Sustainability

PRIORITYASSIGNED TODIFFERENTISSUES

VIEW CONCERNING MARKET FAILURE

externalities > costs externalities = costs externalities < costs

View concerning need for diversity

none some lots none some lots none some lots

Land use dominates

Sceptical aboutpollution

Sceptical aboutnuclear risks

Nuclear risk dominates

Sceptical aboutland use

Sceptical aboutpollution

o

Pollution dominates

Sceptical aboutland use

Sceptical aboutnuclear risks O O (I

© © (3

(3

All issues ofequal importance ©

KEY: fossilfuels

nuclearpower

renewableenergy

this picture presents with the conventional presentation of appraisal results.Instead of the heavily prescriptive overtones associated with a discrete set ofindividual numerical values, ranges of values, or ranking orders, a 'politicalsensitivity map' such as that in Figure 7 conveys a divergent array of signals.

129

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

A. Stirling

However, far from being a perverse feature of this approach, this is simply anindication of the fidelity with which such a technique might communicate thedegree of discord over social values which is almost entirely neglected by otheranalytical appraisal techniques.

Although the map as a whole encompasses a wide range of possible technol-ogy, policy and investment strategies, it is not the case that 'anything goes'.Where there exist regularities in the 'political sensitivity map' these will be ofsome potential importance in the business of arriving at a final decision. In thepresent illustrative example, for instance, it is clear that renewable energyis—under the majority of those perspectives which attach some value todiversity—consistently included as an objective for a sustainable UK electricitysupply mix at levels far in excess of the current 3%. Even in those cases whereno value at all is attached to diversity, renewables end up being assigned theentirety of the supply mix under some two-fifths of perspectives (twice thenumber in which the mix is nuclear-dominated). Were this to be the outcome ofa real appraisal exercise, the ensuing implications for the relative merits ofnuclear- and renewable-based sustainable energy strategies would be all themore robust because they are not dependent on any particular set of assumptionsand value judgements, but on a synthesis of a wide and contentious field ofdebate. In the face of evidence such as this, it would be difficult indeed to justifya strategy which did not involve a substantial programme of encouragement toinvestments in renewable electricity supply capacity.

Further, similarly robust, conclusions might also be drawn from such apicture. Not least of these is that the implications of divergent social values areevidently so profound for the development of a sustainable electricity generatingstrategy for the UK, that some degree of diversity seems essential simply on thegrounds of pluralism. However, the point here is that this kind of 'politicalsensitivity analysis' technique does not provide a means to construct some sortof artificial consensus. Indeed, the value of this approach lies more in thetransparency with which it reproduces genuine conflicts of values and interests.To the extent that the chronic problem of a lack of trust in appraisal arises fromthe way in which individual (especially minority) perspectives have a tendencyto become lost, such an approach might be expected to earn a greater degree oftrust than more opaque approaches. Again, the central point is that, where areasof consensus do emerge in such an exercise, these are of genuine significance—offering guidance to subsequent decision making which is far more robust thanthat yielded by orthodox prescriptive appraisal.

Ultimately, however, the final decision over which permutation of valuesshould be taken to define the objective of a real 'sustainable electricity' strategyis fundamentally political in character. The purpose of the 'political sensitivitymap' is simply to take the business of analysis as far as it can reasonably go.Further, more prescriptive, conclusions would necessarily involve the adoptionof a particular set of subjective values. If these are dressed up as 'analysis' ratherthan 'polities', then we are dealing with a case of technocracy which might bejudged not only corrosive to a democratic political discourse, but also sloppy andinvalid in analytical terms as well. Rather than being concealed by technocraticfig leaves, then, the final justification for investment decisions or policy choices

130

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

The Appraisal of Sustainability

TABLE 3. Some key emerging themes in the appraisal of 'sustainability'

• Treat the appraisal of 'sustainability' as a social process, not an analytical act.• Recognise the intrinsic subjectivity in prioritising the dimensions of 'sustainability'.• Disaggregate and treat separately the different dimensions of 'sustainability'.• Fully acknowledge the importance and scope of uncertainty and ignorance.• Include participation as an integral part of appraisal rather than as a 'bolt-on'.• Ensure that the assessment process is as transparent as possible to third parties.• Concentrate on exploring sensitivities rather than on prescribing definitive 'results'.• Focus on diverse portfolios rather than on single 'most sustainable' options.

for sustainability, must be through the institutions and procedures of democraticpolitical accountability. Although no panacea, a 'political sensitivity map'approach provides a tool which directly requires, and facilitates, such anunderstanding.

Conclusions

The conclusions of this argument are simply put. It is not possible to determineby analysis which of an array of investment, technology or policy options offersthe most sustainable strategy from the point of view of society as a whole. Theobstacles to the development of an 'analytical fix' for the appraisal of 'sustain-ability', are not just problems of political efficacy and legitimacy. Prohibitivedifficulties arise also in some of the deepest foundations of analysis itself—bothin social choice and in probability theory. The imperative for the systematic andtransparent inclusion in appraisal of divergent interests and perspectives is thusnot a question of expediency or 'political correctness'. Participatory deliberationis not a separate activity to technical analysis for the appraisal of sustainability.It is a fundamental matter of analytical rigour.

Once this is acknowledged, a series of quite positive implications unfold.There is no need for the practical business of appraisal to descend into some sortof post-modern paralysis. With the notion of unitary 'objective' prescriptiveanalytical results set to one side, straightforward quantitative techniques never-theless offer an indispensable set of tools. They provide a means to ensure theessential qualities of transparency and specificity in the participatory, delibera-tive appraisal of 'sustainability'. Some of the practical implications for decisionmaking and appraisal methodologies concerning 'sustainability' are summarisedin Table 3. It is curious that problems as daunting and as fundamental asimpossibility and ignorance should, even in principle, be tractable to suchstraightforward responses. The challenge now is to take the matter out of therealm of academic debate and into the practical business of policy making.

Acknowledgements

The author is grateful for helpful comments from anonymous referees and fromparticipants in the Sustainable Cities Network Meeting on 'Indicators of Sustain-ability and Local Environments' held at Imperial College, London on 18

131

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

A. Stirling

February 1998 to which a version of this paper was submitted. This research isfunded by the Global Environmental Change Programme of the ESRC.

References

Arrow, K. J. (1963) Social Choice and Individual Values, 2nd edn (New York, Wiley).Azar, C., Holmberg, J. & Lindgren, K. (1996) Socio-ecological indicators for sustainability, Ecological

Economics, 18, pp. 89-112.Beck, U. (1992) Risk Society: towards a new modernity (London, Sage).Bernow, S. (1990) Environmental externalities measurement: quantification, valuation and monetization, in:

O. Hohmeyer & R. Ottinger (Eds) External Environmental Costs of Electric Power (Berlin, Springer-Verlag).

Bernow, S. & Marron, D. (1990) Valuation of Environmental Externalities for Energy Planning andOperations, May 1990 Update (Boston, Tellus Institute).

Bonner, J. (1986) Politics, Economics and Welfare: an elementary introduction to social choice (Brighton,Harvester Press).

CEC (1989) Energy Technology Status Report (California Energy Commission).Chernick, P. & Caverhill E. (1989) The Valuation of Externalities from Energy Production, Delivery and Use,

Fall 1989 Update (Boston, Chernick and Caverhill Inc).Cline, W. (1992) The Economics of Global Warming (Washington DC, Institute for International Economics).Collingridge, D. (1982) Critical Decision Theory: a new theory of social choice (London, Frances Pinter).Daly, H. & Cobb, J. (1989) For the Common Good: redirecting the economy towards community, the

environment and a sustainable future (London, GreenPrint).Dovers, S. & Handmer, J. (1995) Ignorance, the precautionary principle and sustainability, Ambio, 24(2),

March.DTI (1992) D. Pearce, C. Bann, S. Georgiou, The Social Cost of the Fuel Cycles, report to the UK Department

of Trade and Industry by the Centre for Social and Economic Research on the Global Environment(London, HMSO).

Durant, J. & Joss, S. (Eds) (1995) Public Participation in Science: the role of consensus conferences inEurope, (London, Science Museum).

ECO Northwest (1987) Generic Coal Study: quantification and valuation of environmental impacts, reportcommissioned by Bonneville Power Administration.

EPRI (1987) Results cited but not referenced in Hohmeyer, 1990ExternE (1993) UK Energy Technology Support Unit, CEC, DG XII et al. Assessment of the external costs

of the coal fuel cycle, draft position paper prepared for the CEC/US Joint Study on Fuel Cycle Costs,Harwell.

ExternE (1995) European Commission Externe: externalities of energy. Volumes 1-6, EUR 16520-16525 EN(Brussels, European Commission).

Eyre, N. (1995) The external costs of wind energy and what they mean for energy policy, paper to the ThirdInternational Conference on External Costs, Ladenburg, May.

Eyre, N. & Holland, M. (1993) External costs of electricity generation using coal, in OECD (Ed.) ExpertWorkshop on Life Cycle Analysis of Energy Systems (Paris, OECD).

Faber, M. & Proops, J. (1994) Evolution, Time, Production and the Environment (Berlin, Springer).Fankhauser, S. (1993) Global Warming Damage Costs: some monetary estimates, CSERGE GEC Working

Paper 92-29 (Norwich, University of East Anglia).Ferguson, R. (1992) Results cited in Hohmeyer, 1990 and HoC, 1992 and confirmed by personal communi-

cation, November.Fiorino, D. (1989) Environmental risk and democratic process: a critical review, Columbia Journal of

Environmental Law, 14, p. 501.Fischhoff, B., Derby, S. L. & Lichtenstein, S. (1981) Acceptable Risk (Cambridge, Cambridge University

Press).Friedrich, R. (1993) External costs of electricity generation, Energy Policy, 21.Friedrich, R. & Voss, A. (1990) External costs of electricity generation, in: O. Hohmeyer & R. Ottinger (Eds)

External Environmental Costs of Electric Power (Berlin, Springer-Verlag).

132

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

The Appraisal of Sustainability

Funtowicz, S. & Ravetz, J. (1990) Uncertainty and Quality in Science for Policy (Dordrecht, Kluwer AcademicPublishers).

Giddens, A. (1990) The Consequences of Modernity (Cambridge, Polity).Granger Morgan, M., Henrion, M. & Small, M. et al. (1990) Uncertainty: a guide to dealing with uncertainty

in quantitative risk and policy analysis (Cambridge, Cambridge University Press).Gregory, R., Lichtenstein, S. & Slovic, P. (1993) Valuing environmental resources--a constructive approach,

Journal of Risk and Uncertainty, 7(2), pp. 177-197.Hagen, D. (1991) Application of Solar Thermal Technology in Reducing Greenhouse Gas Emissions--

opportunities and benefits for Australian industry (Canberra, ANUTECH Pty).Haszeldine, R. & Smythe, D. (1996) Radioactive Waste Disposal at Sellafield, UK: site selection, geological

and engineering problems (Glasgow, Glasgow University Press).Hayek, F. (1978) New Studies in Philosophy, Politics, Economics and the History of Ideas (Chicago, Chicago

University Press).Hohmeyer, O. (1988) Social Costs of Energy Consumption: external effects of electricity generation in the

Federal Republic of Germany, prepared for DG XII of the European Commission by the FraunhoferInstitut fuer Systemtechnik und Innovationforschung, (Berlin, Springer-Verlag).

Hohmeyer, O. (1990) Latest results of the international discussion on the social costs of energy--how doeswind compare today?, paper presented at the 1990 European Wind Energy Conference, Madrid, October.

Hohmeyer, O. (1992) Renewables and the full costs of energy, Energy Policy, 20(4), pp. 365-375.Hohmeyer, O. & Ottinger, R. (1990) External Environmental Costs of Electric Power (Berlin, Springer-

Verlag).Hohmeyer, O. & Walz, R. (1993) The analysis of indirect emissions due to intermediate production, in OECD

(Ed.) Expert Workshop on Life Cycle Analysis of Energy Systems (Paris, OECD).IAEA (1991) Senior Expert Symposium on Electricity and the Environment: key issues papers (Vienna IAEA).IIASA (1979) Management of Energy/Environment Systems: methods and case studies (New York, Wiley

Interscience for IIASA).Kelly, J. (1978) Arrow Impossibility Theorems (New York, Academic Press).Klir, G. & Folger, T. (1988) Fuzzy Sets, Uncertainty and Information (New Jersey, Prentice-Hall).Koomey, J. (1990) Comparative analysis of monetary estimates associated with the combustion of fossil fuels,

in: New England conference of Public Utilities Commissioners (Eds) Environmental ExternalitiesWorkshop, Portsmouth, NH, June.

Koomey, J. (1991) Comparative Analysis of Monetary Estimates Associated with the Combustion of FossilFuels paper presented to conference of News England Public Utility Commissions EnvironmentalExternalities Workshop.

Knight, F. (1922) Risk, Uncertainty and Profit (London, Houghton Mifflin).Kuhn, T. (1970) The Structure of Scientific Revolutions (Chicago, Chicago University Press).Lash, S., Szerszynski, B. & Wynne, B. (1996) Risk Environment and Modernity: towards a new ecology

(London, Sage).Lazarus, M., Greber, L., Bernow, S. et al. (1993) Towards Global Energy Security: the next energy

transition-an energy scenario for a fossil free energy future (Stockholm Environment Institute,Greenpeace International, Amsterdam).

Lele S. & Norgaard, R. (1995) Sustainability and the scientists burden, Conservation Biology, 12(3),pp. 213-237.

Loasby, B. (1976) Choice, Complexity and Ignorance: an inquiry into economic theory and the practice ofdecision making (Cambridge, Cambridge University Press).

Loveridge, D. (Ed.) (1996) Special issue on technology assessment, International Journal of TechnologyManagement, 11(5/6).

Luhmann, N. (1991) Risk: a sociological theory (Berlin, de Gruyter).MacKay, A. (1980) Arrow's Theorem: the paradox of social choice--a case study in the philosophy of

economics (New Haven, Yale University Press).MacNaghten, P., Grove-White, R. & Jacobs, M. (1995) Public Perceptions and Sustainability in Lancashire

(Lancaster, Lancaster University).Meyer, H., Morthorst, P. & Schleisner, L. (1994) Assessment of environmental costs: external effect of energy

production, submitted to 18th IAEE International Conference, revised version, 5-8 July, Washington.Norgaard, R. (1994) Development Betrayed: the end of progress and a co-evolutionary a revisioning of the

future (London, Routledge).

133

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

A. Stirling

Norgaard, R. & Howarth, R. (1991) Sustainability and discounting the future, in: R. Costanza (Ed.) EcologicalEconomics (New York, Columbia University Press).

OECD (1983) Environmental Effects of Energy Systems: the OECD Compass Project (Paris, OECD).OECD (1993) Expert Workshop on Life Cycle Analysis of Energy Systems (Paris, OECD).O'Riordan, T. & Cameron, J. (Eds) (1994) Interpreting the Precautionary Principle (London, Earthscan).Ottinger, R. (1990) Environmental Costs of Electricity, prepared for NYSERDA and US DOE by Pace

University Center for Environmental Legal Studies (New York, Oceana Publications).Pearce, D. (1989) Blueprint for a Green Economy (London, Earthscan).Pearce, D. (1993) The economic value of externalities from electricity sources, paper presented at Green

College Seminar, Environment and British Energy, University of Oxford, April.Pearce, D. & Nash, C. (1981) The Social Appraisal of Projects: a text in cost-benefit analysis (London,

Methuen).Peterson, G. & Driver, B. (Eds) (1988) Amenity Resource Valuation: integrating economics with other

disciplines (Philadelphia, Venture).Ramsay, W. (1979) Unpaid Costs of Electrical Energy: health and environmental impacts from coal and

nuclear power (Washington, DC, John Hopkins University Press).Redclift, M. (1987) Sustainable Development: exploring the contradictions (London, Routledge).Renn, O., Webler, T. & Wiedeman, P. (Eds) (1996) Fairness and Competence in Citizen Participation

(Dordrecht, Kluwer).Shilberg, G. (1989) Valuing Reductions in Air Emissions and Incorporation into Electric Resource Planning:

theoretical and quantitative aspects (Chicago, JBS Energy Inc).Shuman, M. & Cavanagh, R. (1982) A Model Conservation and Electric Power Plan for the Pacific Northwest.

Appendix 2: environmental costs (Seattle, Northwest Conservation Act Coalition).Smithson, M. (1989) Ignorance and Uncertainly: emerging paradigms (New York, Springer).Stern, P. & Fineberg, H. (Eds) (1996) Understanding Risk: informing decisions in a democratic society, Report

of the US National Research Council Committee on Risk Characterisation (Washington, DC, NationalAcademy Press).

Stirling, A. (1992) Regulating the electricity supply industry by valuing environmental effects, Futures, 24(10),December.

Stirling, A. (1994a) Diversity and ignorance in electricity supply investment: addressing the solution ratherthan the problem, Energy Policy, 22(3), pp. 195-216.

Stirling, A. (1994b) Technology choice for electricity supply: putting the money where the mouth is?, D.Philthesis, Science Policy Research Unit University of Sussex.

Stirling, A. (1995) Diversity in electricity supply: a response to the criticism of Lucas et al., Energy Policy,22(12), p. 94.

Stirling, A. (1996a) Optimising UK electricity portfolio diversity, In: G. MacKerron & P. Pearson, (Eds) TheUK Energy Experience: a model or a warning? (London, Imperial College Press).

Stirling, A. (1996b) On the NIREX MADA, in: R. Haszeldine & D. Smythe (Eds) Radioactive Disposal atSellafield, UK (Glasgow, Glasgow University Press).

Stirling, A. (1997a) Multicriteria mapping: mitigating the problems of environmental valuation?, in: J. Foster,(Ed.), Valuing Nature: economics, ethics and environment (London, Routledge).

Stirling, A. (1997b) Limits to the value of external costs, Energy Policy, 25(5), pp. 517-540.Stirling, A. (1998a) Risk at a turning point?. Journal of Risk Research, 1(2), pp. 97-109.Stirling, A. (1998b) Valuing the environmental impacts of electricity production: a critical review of some 'first

generation' studies, Energy Sources, 20(4/5).Stocker, L. (1991) Comprehensive Costs of Electricity in Western Australia (Canberra, Australian National

University).Suter, G. (1991) Ecological Risk Assessment (Lewis, Boca Raton).Tol, R. (1995) results cited in ExternE 1995.UNEP (1985) Comparative Data on the Emissions, Residuals and Health Hazards of Energy Sources,

Environmental Impacts of the Production and Use of Energy, Part IV, Phase I, UNEP.Vatn, A. & Bromley, D. (1994) Choices without prices without apologies, Journal of Environment Economics

and Management, 26.Webler, T., Kastenholz, H. & Renn, O. (1995) Public participation in impact assessment: a social learning

perspective, Environmental Impact Assessment Review, 15, pp. 443-463.Winkler, G. (1986) Necessity, chance and freedom, in: D. Winterfeldt & W. Edwards, Decision Analysis and

Behavioural Research (Cambridge, Cambridge University Press).

134

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014

The Appraisal of Sustainability

Winterfeldt, D. & Edwards W. (1986) Decision Analysis and Behavioral Research (Cambridge, CambridgeUniversity Press).

Wynne, B. (1992) Uncertainty and environmental learning: reconceiving science and policy in the preventiveparadigm, Global Environmental Change, 2(2), pp. 111-127.

135

Dow

nloa

ded

by [

Uni

vers

ity o

f Su

ssex

Lib

rary

] at

05:

08 0

1 Ju

ly 2

014