An early version of this paper was presented at the Second Conference of the European Section of the1
Society for Risk Analysis, IIASA, Laxenburg, Austria, April 2-3, 1990.
APPLICATIONS OF PROBABILISTIC RISK ASSESSMENTSThe Selection of Appropriate Tools 1
Joanne Linnerooth-Bayer and Björn Wahlström International Institute for Applied Systems Analysis, A-2361 Laxenburg, Austria
ABSTRACT
Probabilistic risk assessment (PRA) is an important methodology for assessing the risks of complex
technologies. This paper discusses the strengths and weaknesses of PRA. Its application is explored in three
different settings: adversarial policy processes, regulatory/licensing procedures, and plant safety audits. It
is concluded that PRA is a valuable tool for auditing safety precautions of existing or planned technologies,
especially when it is carried out as an interactive process involving designers and plant personnel who are
familiar with actual, everyday operations. PRA has not proven to be as well-suited in providing absolute
risk estimates in public policy debates concerning the acceptability of a technology, or for the licensing and
regulatory procedures. The reasons for this are discussed.
Key words: Probabilistic risk assessment, risk and policy making, regulatory processes, safety
improvements, public acceptance.
3
1. Introduction
In the United States, and to a more limited extent in European countries, risk analysis emerged during the
1980s as a major methodology for regulatory policy making. As a part of risk analysis, quantitative risk assessment(1)
includes both the identification of risks to public health and safety and to the environment, and an estimation of the
probability and severity of harm associated with them. Quantitative risk assessment has been applied extensively in
such areas as work place hazards, the transport of hazardous substances, public exposure to toxic chemicals, and the
development of technologies with a low-probability of a potentially very high-consequence accident.
Risk estimates for low-probability events are problematic since historical data is usually inadequate to
calculate meaningful accident frequencies. Probabilistic risk assessment (PRA) circumvents this problem by
providing estimates of overall accident probabilities synthesized from the design and performance characteristics
of individual components of the technology, such as pipes, pumps, valves, pressure vessels, control equipment and
human operators. In effect, PRA is a simplified model of the system and its operation.
The task of risk assessment has been viewed to be advantageous when separated from risk management,
that is, separating the scientific findings and policy judgments embodied in risk assessments from the political,
economic, and technical considerations that influence the choice of policy. This separation is steadily becoming(2)
more difficult as risk issues and technologies become more complex and consequently require increasingly difficult
judgments regarding the definition of issues, the selection of analytical structure, the designation of assumptions,
the interpretation of data, and the use of results. Large uncertainties, and even ignorance, dominate many areas of
risk to the extent that the very lack of knowledge is unsuspected.(3)
Ravetz suggests that information provided by such applied methodologies as PRA (as distinct from the
methodology itself) be viewed as a `tool' which must be shaped so as to be robust in the performance of its intended
purpose. This tool metaphor implies that probabilistic risk assessment may be more appropriate for some(4)
applications than for others. Since their inception in the late 1960s, three distinct uses of PRAs have emerged:
(1) to provide overall risk estimates of a technology for policy making purposes, public inquiries, and
environmental impact statements;
(2) to aid regulatory decisions and licensing procedures by providing quantified estimates of risk and
risk reductions; and
4
(3) to identify potential safety improvements in plant design and operation, quality control, backfitting
measures, and maintenance.
The first, and to some extent the second, of these applications are meant to aid political policy processes
by providing comparative data on the risks of the contested or regulated technology, whereas the third application
mainly provides information to the industry for its internal decisions on safety improvements. Crucial as this
distinction between public policy making and decision making within the industry is, it is often not fully appreciated
when considering the applicability and value of PRAs. Similarly, whether a PRA is employed to assess the risk of
a technology before its deployment, or to improve an existing plant, is a related and also important consideration
of its usefulness.
In this paper we review the PRA methodology and examine its use in three different contexts: adversarial
policy settings, regulatory procedures, and industry design and safety audits. We conclude that one of its more
valuable uses may be as an interactive tool that brings together safety analysts, plant designers and operators for the
purpose of improving a plant design or for continuously monitoring and improving the safety of plant equipment and
procedures. The use of PRA for establishing an absolute level of risk that can both inform and guide pluralistic
policy processes is more limited. This is due to the nature of political procedures which make tacit demands for
objective risk numbers, as well as to the nature of the PRA methodology which can only provide estimates from
existing data and expert judgment. Institutional reforms are needed to better integrate the judgment of experts with
the `common wisdom' of those affected by the policies they influence.
2. Probabilistic Risk Assessment (PRA)
Probabilistic risk assessment is a methodology for assessing and improving the safety of a technology. The
methodology entails the construction of possible chains of events called `event trees' which lead to unwanted
consequences or, working backwards, constructing chains of faults called `fault trees' in the search for accident
precursors. The risks are quantified by calculating an estimate of the probability of these event or fault sequences
5
and combining this with an estimate of the damages. The damages are usually expressed in terms of lives lost,
injuries, plant and property loss, and sometimes ecological harm.
In the nuclear industry, probabilistic methods for assessing plant safety were introduced as an alternative
to deterministic methods which have been the basis of most safety criteria in the past, e.g., the single-failure criterion
and the fail-safe principle. Designing redundant safety systems and applying `defence-in-depth' principles is
facilitated by the use of probabilistic methods for assessing the risks. A deterministic approach adopts conservative
assumptions, and consequently focuses on worst-case accident scenarios which provide an unrealistic picture of the
safety systems and give little guidance on the relative rankings of safety improvements. The PRA methodology(5)
consists of three general steps: (1) identification of the events that initiate and lead to unwanted consequences; (2)
modeling of the identified event sequences with respect to probabilities and consequences; and (3) quantification
of the risks involved.
A PRA can be carried out at different levels of detail. In the nuclear field there are three levels of PRA.
Level 1 considers only the probability of a reactor core melt; Level 2 considers releases of radioactivity to the
environment; and Level 3 considers the resulting individual and societal risks. The latter is often referred to as a
comprehensive or large-scale risk assessment.
Following the first comprehensive application of PRA, the U.S. Reactor Safety Study, in 1975 , more than(6)
15 large-scale (Level 3) PRAs have been carried out for nuclear power plants in the U.S. In Sweden, as of 1986,(7)
seven PRAs had been completed. A major nuclear risk study in Germany assessed the risks of 19 sites with a(8) (9)
total of 25 reactors. PRAs have found applications in many other areas, including production of chemicals, LNG
transport and storage, the transport of dangerous materials, oil drilling rigs, aircraft safety, and the space industry.
The methodology has also been used for comparing the risks of different technologies.(10)
There is little doubt that probabilistic risk assessments have contributed significantly to our understanding
of technological risks. In the nuclear field, for instance, PRAs have led to retrofitting actions that have decreased
the estimated frequency of core melts. The studies have also demonstrated that accidents beyond the `design base'
are major contributors to risk and that human interactions are extremely important. Moreover, comparative risk(11)
studies have provided important insights as to how the risks of different technologies are related. However, despite
these and other contributions there are acknowledged limitations of the methodology which affect its applicability.(12)
2.1. Completeness and failure data
6
The main limitations of the PRA methodology include problems of ensuring the completeness of the
analysis and the adequacy of the data. No analysis can be guaranteed complete, which means in the PRA case
considering everything of significance that might go wrong. This issue is the most crucial problem of the
methodology, since omissions can always remain with serious consequences. The problems of completeness typically
include human interactions and modeling of common-cause failures.
A cut-off level for small risks has to be assigned for practical reasons. This means that the analyst must
determine which event sequences are of such low probability that they can be ignored. In cases where these
sequences are very unlikely, no significant error will be made. However, unexpected common-cause failures can
introduce estimates which understate the probability by several orders of magnitude, thereby rendering the
simplification unjustified. As new information emerges, earlier estimates can appear to be too optimistic as was
demonstrated by intergranular stress corrosion cracking in boiling water reactor piping.(13)
External events such as earthquakes, aircraft crashes, floods and extreme weather conditions can also be
major contributors to accident risks. Since the external environment involves more complexity and lack of structure
than do internal plant operations, these estimates are often very uncertain. There is also a tacit assumption in many
risk assessments that the plant is built as designed and is adequately maintained. Violations of safety technical
specifications and sabotage are rarely included in the studies.
The treatment of common-cause failures, i.e., multiple failures resulting from the same cause, can be
problematic in a PRA. The difficulty of modeling these failures has been evidenced by large differences in their
inclusion in recent PRAs. These differences have been attributed not only to the dissimilarity of plants but also to
the analysis team, the purpose of the study, and the analytical methods used.(14)
The significance of human interactions with technological systems, another limitation to the methodology, was
demonstrated by the accidents at Three Mile Island, Bhopal and Chernobyl. However, in spite of two decades of
research in the human factors area there are still no practical methods, other than expert judgement, available for
identifying human malfunction and for assigning error probabilities.(15)
The accuracy of the data base may also be a limiting factor. The empirical experience on component failure
is often good enough for reasonable statistical extrapolations in terms of failure rates but the question always remains
as to whether these failure rates can be transplanted from one context to another.
2.2. Assumptions and the use of expert judgment
7
The results of the analyses depend on a series of analytical assumptions and judgements which must be
made when modeling the systems and combining the models with data obtained from historical experience. A
considerable amount of expert judgement will be used in this process. This judgement can be valid and useful when
recognized as such. In fact, engineering and analytical judgements enter a PRA analysis in many diverse ways:(16)
characterizing the risks; choosing how to fill gaps in the data and indicating what contingencies should be simply
left out; modeling complex physical phenomena, portraying the degree of confidence in the results, and choosing
which formats to use for presentation. Throughout the analysis assumptions must be made and all assumptions(17)
require judgments as to their appropriateness. In addition, experts themselves are immersed in their own analytical
frameworks and work under scientific norms that encourage a precision that can lead to difficulties in appreciating
variances within the socio-technological systems.(18)
Considering the inherent subjectivity of the studies, it is not surprising that the results often vary
significantly, even when different analyses are carried out for the same plant. Experience from reliability benchmark
calculations, for instance, has shown that probability estimates can range even over two orders of magnitude. This(19)
variation was also demonstrated in an early review of the use of PRAs to estimate the risks of liquid-natural-gas
storage facilities. The results of the PRAs for one planned site, when expressed in terms of the individual risk to(20)
people living in the vicinity of the facility, differed over orders of magnitude. This was not due to deficiencies in the
analyses but rather to subtle differences in the specification of the problem, assumptions, and the models employed.
This variability in the results of PRAs has also been documented in the nuclear field. Current core damage
frequency estimates for US reactors range from about 10 to 10 per year. This variance is not due solely to-5 -3 (21)
different plant designs and site characteristics as Fullwood and Hall have noted, the differences also stem from the
scope of the studies, the PRA methods employed, and the analytical assumptions. Large variations in PRAs due(22)
to differences in modeling approaches have also been documented in Sweden. In a study on the state-of-the-art(23)
in PRA, the U.S. Government Accounting Office concluded that inconsistencies in PRA results limit their
comparability and represent a fundamental problem of the methodology.(24)
2.3. Expressing uncertainties
The variability of different PRAs underscores the importance of estimating and reporting the underlying
assumptions and uncertainties. End results should be documented not as single numbers reflecting whatever measures
for risk are chosen, but rather as probability distributions reflecting the certainty of these numbers. Indeed, the
8
expression of uncertainties is one of the basic ideas of the PRA methodology. As the IAEA points out, a main
advantages of PRA over more deterministic estimates is that a substantial part of the uncertainties can be identified
and quantified.(25)
For the most part, PRA uncertainties stem from the completeness of the analysis, the modeling accuracy,
and the adequacy of the parameter estimates. As to the latter, the uncertainties can be calculated by propagating the
probability distributions of the data through the analysis, assuming, however, that the data is appropriate for the plant
in question. The uncertainties arising from the inherent incompleteness of the analysis and the model accuracy are
more elusive. These are generally dealt with by the use of sensitivity analysis, although Bayesian methods have also
been proposed.(26)
The practice of reflecting uncertainties in the analyses is somewhat different from the theory. Hirschberg,
for instance, points to examples of careless treatment of uncertainties in nuclear PRAs. "No clear distinctions are
usually made between the parametric, modeling and incompleteness uncertainties and their relative importance. The
analysts are not always aware of which type of framework (frequentist or subjectivist, i.e., Bayesian) they actually
apply...".(27)
Similar problems were reported much earlier with respect to the PRAs carried out for LNG storage
facilities. The analysts invariably overstated the confidence of the results by failing to acknowledge the existence
of experts and models with conflicting results; by eliminating some accidents from consideration based on the
judgement of the analysts but failing to acknowledge the possibility of this judgement being erroneous, and by
choosing parameters that sometimes differed by several orders of magnitude without qualifying the results with a
discussion of the uncertainties. A recent OECD study in which a number of non-nuclear risk assessments were(28)
examined lamented that the problem of treating uncertainties is often raised but rarely tackled in any depth.(29)
The failure to be explicit about uncertainties is also an issue with respect to risk assessments of toxic
chemicals. Finkel , who advocates that uncertainty analysis be incorporated in risk assessments and risk(30)
management decisions, warns that artificially precise but very fragile numbers may mask the fact that analysts are
ignorant of or ignoring uncertainties. Even the expression of uncertainty, however, requires "judgment calls" since,
as Finkel recognizes, "uncertainty assessment is both a science and an art, and it does not always aspire to scientific
`truth' or rigorous scientific standards". Judgments must be made in documenting the confidence of the results(31)
without being so inclusive as to engender paralysis.
9
2.4 Complexity
The increasing complexity and interdependence of technological systems raises a number of issues. Large-
scale nuclear risk assessments, for example, encompass a myriad of different systems requiring the involvement of
expertise from widely different areas. As Wynne has noted, the description of possible chains of events is so(32)
complicated and open to interpretation that there is often room for dispute after an accident as to whether this was
described in the PRA or not. The sheer magnitude of the calculations can be daunting. A PRA for a nuclear power
plant requires thousands of parameter estimates and the report may contain several thousand pages. This hinders
meaningful communication of the results to the final users of the study.
Yet nuclear power generation is a relatively simple and well-understood technology; many types of
chemical plants are far more complex and less well understood. While PRA with its `divide-and-conquer' approach(33)
is arguably suited for assessing the risks of complex systems, it may be appropriate only for those complex systems
that are, at the same time, structured and well defined. This may lead the assessment process to frame the risk issues
in exactly these terms. The danger is that the risk issue is narrowed or bounded to match the limitations of the
assessment methodology.
There is a natural tendency to define a problem in such a way that the analytical results are valid and
credible. This tendency has been apparent in the nuclear field. As an official at the U.K. Atomic Energy Agency
observed, hazards which can be evaluated with confidence have been given comparatively more attention than other
hazards, even those which were considered salient in the Reactor Safety Study but for which the analytical tools have
not greatly advanced since that time. Early PRAs focused on a single plant and mainly on hardware malfunctions(34)
leading to accidents. It was natural to focus on hardware functions, according to Hirschberg, as estimates could be
supported by reliable data and were therefore more credible and less controversial. Moreover, accident(35)
consequences have usually been defined in relatively narrow terms, that is by plant and property damage and health
effects to the immediately surrounding population. Far less attention has been given to the possibility of more diffuse
consequences such as psychological effects, social disruption, and threats to the ecological sustainability of a region.
As assessment methodologies have become more mature, technical risk issues have expanded in scope to
include factors more difficult to quantify. This broadening of the analytical scope has been positive in the sense that
it has simultaneously broadened the whole concept of technology as it relates to social and ecological systems. Quite
naturally, however, this has led to increasingly `soft' and less precise analytical results, but results which give a more
sensitive picture of the social and environmental ramifications of complex, technological systems.
10
With global risk problems emerging on political agendas, the question of analytical scope has become more
urgent. In many risk areas, analysts can no longer justifiably consider only the consequences of an accident at a
single or several similar plants and exclude consideration of perhaps far-reaching consequences in the event that the
technology proliferates in many different forms over the globe. Completeness in the PRA methodology might thus
be accompanied by an equally important concept of completeness with respect to the scope of the risk issue: the risk
to whom and over what time scale? Although this issue does not challenge the appropriateness of PRA, per se, with
its focus at the plant level, it is important to keep in mind the broader issues when considering the uses, and
limitations, of the PRA methodology.
3. PRA in Public Policy Debates
PRA is used in public policy procedures in three main ways: The first, which has been most visible in the
nuclear field, is to provide a basis for debate on the social acceptability of a technology. One motive for these studies
has been to clarify the risks and, in so doing, to narrow the debate to the areas of real disagreement. In the energy
area, this has often entailed carrying out comparative risk studies of different ways of generating electricity. A(36)
second use concerns the licensing of hazardous technologies, where the public authorities are called upon to give
approval for a site. PRAs are produced as quantitative evidence of the public risks involved. The third major use is
to satisfy regulatory requirements on existing or planned facilities. This application has intensified in Europe
following the implementation of European Community's Seveso Directive.
An early comparative study on the use of PRA in siting procedures for liquid natural gas terminals in four
countries concluded that technical risk analyses contributed little to building a consensus concerning the safety of
competing sites. The expert disagreements even appeared to be reinforced by the quantitative studies. In a more(37)
recent comparative study of nuclear risk analysis carried out in five countries, this finding was repeated: The results
showed that the risk studies "enlarged, not narrowed, the debate. Instead of shaping an accepted discourse for the
discussion of risk, they intensified the conflict ... Instead of greater acceptance of the bases upon which numerical
calculations rest, they heightened suspicions over the motivations for the studies and those who conduct them".(38)
11
There appears to be a mismatch between the acclaimed purposes of PRA in promoting consensus through
greater understanding of the risks and the actual uses made of PRAs in pluralistic policy debates. One reason is
undoubtedly the complexity of the problems and the PRA methodology itself, which makes it difficult even for
specialists to acquire a balanced view of the evidence provided in a study. Another reason may stem from the rather
fundamental differences between individual decision making and public policy making. Policies, unlike individual
decisions, need to gain a consensus in order to be viable. A consensus within and/or beyond an organization can only
be reached with convincing and institutionally appropriate arguments. In the Swedish nuclear debate, for example,(39)
Hansson observed that the energy risk studies "were predominantly used in argumentative contexts, in most cases
as support for particular views, but also as objects of criticism".(40)
The role of argumentation in the policy arena is crucial in appreciating the use of PRA studies and their
often misfit in policy debates. In a policy context, the interested parties often compete for credibility and influence
by appealing to scientific evidence which can be interpreted in such a way as to support and justify their arguments.
The best justification is viewed as `solid' factual evidence, a demand which cannot be fulfilled by PRAs where the
estimates are informed to an important extent by expert judgment. This has led stakeholders to underplay the
uncertainties of the analyses, or ignore them altogether. Rather than focusing the debate on the range of judgments(41)
and the uncertainties of the analyses, this has pitted experts against experts. As Fiorini has observed, "Science and
expert knowledge have not taken the politics out of technically-based policy issues, as many observers expected only
decades ago. Instead, the increasing involvement of technical experts in policy disputes has politicized expertise".(42)
In addition to the difficulties of bringing subjective estimates as evidence in pluralistic policy processes,
there are other important reasons why PRAs have not led to more `rational' debate. The risk perception research
demonstrates that lay judgements of risk are influenced by a number of factors not acknowledged in the technical
risk concept embedded in PRAs. Voluntary and familiar risks are of less public concern than those that are assumed
involuntarily, dreaded, or viewed as potentially catastrophic. These differential risk perceptions cannot, however,(43)
fully explain the intense conflicts witnessed in relation to many risk issues. In fact, the notion that differential risk
perceptions are fundamental to technological risk conflicts, or even that probability of harm plays an important role
in public concern, has been challenged. If technically defined risk, or risk as probability times harm, is not fully(44)
relevant to public concern, then the rationality implicit in risk calculations may not be shared by many individuals
of the public. The idea of `plural rationalities' which has been put forward by cultural anthropologists , suggests(45)
that individuals have different views of nature and that these views are rooted more in social contexts than in
12
individual perceptions of threat. The acceptability of risky technologies is colored, if not determined, by these social
contexts. For example, according to Schwarz and Thompson, the controversy over nuclear waste is viewed
differently by "those who are convinced that the wastes we already have are already pointing the finger of death at
thousands of children not yet born" and "those who would, as the saying goes, happily sprinkle that same waste on
their cornflakes".(46)
The fact that individuals have different ways of viewing the issues and different logics for their resolution,
or different concepts of rationality and reasonableness, points to the difficulty of arriving at a social consensus only
through a technical definition of risk. This consensus requires democratic procedures for extended debate and
deliberation that allow the full participation of all stakeholders in the process of policy making. But this participation,
as good as it is in theory, is problematic in practice, especially in view of the increasing complexity of technological
systems. The gap may be widening between the expectations of people regarding their capacity to influence decisions
and the realities of policy making.
Few groups outside the technical community, for example, have the requisite expertise and resources to
review a large-scale PRA, which can easily exceed ten volumes with thousands of pages of technical appendices.(47)
Institutional reforms that assure financial and other resources to intervenor groups is one (much discussed)
improvement. However, a more serious impediment to participation may be presented by the difficulties in framing
debates so that diverse public concerns each receive a hearing. Yet if, as Fiorini argues, risk analysis is viewed as
a political process informed by expert judgment rather than an expert process with token citizen representation ,(48)
refining procedures for communication and participation may then be as crucial to risk management as refining the
assessment methodologies.
4. PRA in regulatory procedures
The purposes of probabilistic risk assessment in the regulatory process are varied. Some are required, as
discussed above, for addressing issues of social acceptability, others for checking if the plant meets safety
requirements, or for checking if the licensee has a good understanding of the hazards, and some are used to develop
emergency plans. Risk is almost always defined as the probability of harm to the public, although environmental
13
damage is also entering the assessments. Risk assessments, while not always quantitative, have become practically
a routine requirement for many licensing procedures. Most new industrial facilities in Western Europe are subject
to planning or licensing procedures which require some sort of risk assessment. The so-called Seveso Directive of
the European Community requires that probabilistic risk assessments be carried out for some 1800 hazardous(49)
facilities now in operation. While there is no federal legislation in the U.S. and Canada requiring operating licenses
(except for the nuclear industry), federal projects in the U.S. are subject to Environmental Impact Statements which
are increasingly concerned with accidental risks.
The Netherlands is a forerunner in requiring quantitative, probabilistic risk assessments. Safety targets in
terms of acceptable and unacceptable risks to individuals (i.e., annual probability of death to a person) and to society
(i.e., the cumulative accident frequency) have also been proposed. The acceptable risk criteria inherent in these(50)
safety targets also underlie the `bottom line' approach proposed in the Clean Air Act amendments being considered
by the U.S. Congress. According to this approach, a point estimate separates reasonable from unreasonable risk. The
Netherlands and the U.S. are the only countries proposing such explicit, quantitative regulatory criteria, but the
OECD reports that quantitative safety targets are in use, if mostly on an informal level, in practically all its member
countries. Nuclear legislation, for example, makes frequent reference to them, and there is a great deal of interest(51)
in safety targets at the industrial level.(52)
While government officials in France and Germany are familiar with these criteria and even employ them
on an informal basis, they consider them to be too rigid and inappropriately removed from the social context to be
incorporated into formal criteria. This reservation was also voiced in a recent report by Bengtsson who concluded
that PRAs are not mature enough to permit stringent comparisons of quantitative `bottom-line' estimates with
prescribed safety goals. The same conclusion could be drawn with respect to comparing PRA results with other(53)
types of risks, for example, with risks in nature, with risks that people knowingly accept in their everyday lives, or
with risks from alternative technologies. Such comparisons require an absolute measure of risk, e.g., the number of
expected deaths per year in a certain population or an expected frequency of deaths for the exposed population.
Where uncertainty prevails, as is typical in PRAs, ranking different risks simply by comparing their point estimates
is problematic. If the uncertainty of the estimates is taken into account, this might well change the rank order. As(54)
Finkel has pointed out, a risk assessment is not always suited to prioritizing within a spectrum of risks if it has
difficulties assigning absolute values to them. Moreover assigning conservative estimates to the parameters as a(55)
way of circumventing the uncertainty can only aggravate the problem of comparison and prioritization.
14
This does not mean that PRAs cannot be useful for regulatory decision making, but it does suggest extreme
caution in making risk comparisons. It also suggests that a good understanding of the uncertainties, assumptions and
subjective judgements underlying the estimates may be as informative as the estimates themselves. If these judgments
are exposed and made scrutable, then PRA can be a far more valuable tool for policy makers than more deterministic
methods that do not lend themselves to such scrutiny.
This potential, however, tends to be limited by the political and institutional settings in which PRAs are
carried out. Especially in the U.S., where the system of judicial oversight requires supportive and preferably
quantitative evidence to defend agency decisions, the need for ex post justification may reduce the potential for
candid assessment. The same tendencies can be seen in European government policies which are increasingly
challenged by environmental groups. (56)
As expertise becomes more politicized, institutional reforms are needed to accommodate the results of
PRAs and other types of analyses that rely on expert judgment. But just as analytical expertise formed by judgment
is valuable to political decisions, so is the wisdom of those who are affected by or merely concerned about these
decisions. Functowitz and Ravetz argue for a new type of science to accommodate problems that do not have neat(57)
solutions, when the phenomena themselves are ambiguous and the mathematical techniques open to methodological
criticism. This science should include local wisdom, since those whose lives depend on the solution of these
problems will have a keen, `back yard' awareness the general principles involved.
5. PRAs for Safety Audits
Given the justifiable skepticism regarding the use of PRAs for providing absolute estimates of overall safety
levels , a seemingly more modest, but perhaps more valuable use of the tool is to improve the understanding of(58)
safety systems within a plant and to create an improved awareness of safety in its design and operation. This
application is sometimes referred to as reliability analysis, which can be carried out as a compulsory part of licensing
or regulatory procedures, or as an internal safety audit initiated by the plant management. Its main purpose is: to
identify weak points in overall safety precautions; to improve safety systems; to optimize safety technical
specifications; to select schedules for tests and maintenance; and to determine the need for operational
15
improvements. These analyses usually rely on fault trees which, even without quantification, can yield significant
insights into the structure of the system.
A recent development is the so-called `living PRA', which provides a continuously updated representation
of the plant that allows the integration of new experience and plant modifications. Such a continuously evolving
assessment can improve plant safety management by promoting the effective use of new information. The evaluation
of operational experience is also depending on a context to be beneficial. These approaches can help to overcome(59)
the usual split between safety assurance and the reliability segments in the organizational structures of large
utilities. The `living PRA' can, in principle, begin at the design stage before the plant is actually constructed. At(60)
this stage, the PRA is naturally crude since it requires information about how the plant will eventually be built, but
it is valuable in that it draws attention to the safety of the overall system. The PRA can then be refined as the design
and construction process progresses.
The value of the PRA methodology for improving the understanding of plant safety may be greatly
enhanced if the tool is developed by those persons, system experts and plant operators, intimately involved in plant
operations. Carried out as an internal and interactive exercise, the value of the PRA may lie more in the process than
in the final risk quantification. Since the methodology builds on a knowledge of the system behavior under normal
and adverse conditions, it can obviously benefit from informed knowledge of the everyday, routine conditions of
the system and, most importantly, how the plant and equipment are actually being operated. This knowledge can be
enhanced with the involvement of the operators and other staff in the PRA effort. This involvement not only has the
potential of improving and enlightening the analysis, but it can be useful as a training exercise as well.
As promising as a PRA may be in a process of internal and interactive assessment, this application appears
more the exception than the rule. PRAs are rarely carried out for internal safety purposes except in the anticipation
of public acceptance problems. Moreover, plant personnel are not generally involved in the assessment process.(61)
While big chemical and petroleum companies usually have specialized risk assessment departments, most PRAs are
carried out by consulting firms and their subcontractors. The risk assessment activities of these departments and(62)
firms aim mainly at demonstrating compliance with regulations, which means that the feedback to actual day-to-day
operation can be superfluous.
In some cases, however, there are positive `spin-offs' in the form of genuine safety improvements during
the PRA process. In fact, the regulatory relationship is sometimes such that improvements in the plant are negotiated
continuously during the regulatory assessment procedures. This is not always the case, however, and there is a
16
concern that many analyses carried out for regulatory purposes by outside consultants have little relationship to or
feedback for the actual operation of the plant. Increasing standardization of the methodology with the use of
computerized packages, which has considerably decreased the costs of analysis and made tailored analyses possible,
has, at the same time, led to concern that there is too little focus on the nuances of different plants.
6. Concluding Remarks
Probabilistic risk assessment has contributed to our understanding of many complex technological systems.
This contribution has relied on the strengths of the PRA tool, namely, decomposing complex systems and
extrapolating failure rates derived from historical operating data on the component parts. It is also matched by
experience, which shows that the PRA methodology is reasonably well suited for identifying safety improvements
in plant design and operation. In this application even relative estimates of the effects of proposed safety measures
can be informative, although it should be kept in mind that a rank ordering can be misleading when the uncertainties
are large. This application can also be based on relatively consistent assumptions since the analysis is not comparing
risks of entirely different technologies. Most importantly, the methodology can reveal large oversights in the safety
design, although smaller omissions become increasingly difficult to detect.
Although experience is more limited, the methodology can also provide a basis for safety management by
continuously integrating operational experience. The use of PRAs for improving plant safety might be enhanced if
carried out as an internal exercise, either at the initiative of the plant management for improving safety or for
fulfilling regulatory requirements. An interactive study that brings together personnel involved in plant operations
on a daily basis to identify event and fault sequences, and even to estimate their probabilities, can enhance safety
awareness and give important insights on the safety of actual operations without requiring precise, absolute risk
figures. In this use, the process of carrying out a PRA may be as important as the results.
A more problematic use of the tool is in public policy procedures, especially the generation of point
estimates of overall risk to compare with established safety targets or with other risk sources. This assumes a
precision in the results that cannot be produced in practice. This does not mean that an application of the
methodology cannot provide useful and important insights for comparative analyses. Even without precise risk
17
estimates, such studies can identify important risk contributors and their sometimes major uncertainties in the life
cycle of a technology. However, it cannot be assumed that the technical definition of risk embodied in PRAs is
central to public concern regarding hazardous technologies.
The PRA methodology relies to an important extent on expert judgment in characterizing the risks, deciding
on the appropriateness of the data, incorporating human interactions, modeling complex physical phenomena, and
even in interpreting the results. To the extent that analytical judgments and assumptions are made transparent and
scrutable, the tool can be valuable in exposing the source of the probability estimates and the uncertainty surrounding
these estimates. In theory, a PRA can provide public policy debates then with clearer and more informative evidence
on low-probabilistic risks than more deterministic assessments. That this potential has not been fully realized in
actual policy procedures can be partly attributed to the mismatch between multi-stakeholder processes, where tacit
demands are often made on analysts to provide `objective' risk numbers, and the PRA methodology, where the
estimates are informed by analytical judgment. This mismatch suggests that institutional reforms that can better
accommodate subjective scientific evidence (as well as `local wisdom') are as important for effective and credible
public policy as refinements of the PRA methodology. Any strategy for change will call for greater knowledge as
to how political and institutional forces affect the conduct and use of PRAs.(63)
18
1. R. Zimmerman, "The Management of Risk," Risk Evaluation and Management (V. Covello, J. Menkes,and J. Mumpower, eds.) 435-460 (Plenum, 1986).
2. U.S. National Academy of Sciences Risk Assessment in the Federal Government: Managing the Process(Washington, D.C.:NAS, 1983).
3. J. Ravetz, "Uncertainty, Ignorance and Policy," Science for Public Policy, (H. Brooks and C. Cogsei, eds.)77-95 (Pergamon 1987).
4. J. Ravetz, ibid.
5. S. Hirschberg, Dependencies, Human Interactions and Uncertainties in Probabilistic Safety Assessment(Nordic Liaison Committee for Atomic Energy, 1990).
6. U.S. Nuclear Regulatory Commission, Reactor Safety Study, (WASH-1400, NUREG-75/014. Washington,DC. 1975).
7. U.S. Nuclear Regulatory Commission, Probabilistic Risk Assessment (PRA) Reference Document,(Washington D.C. The Commission, 1984).
8. B. Hansson, "Major Energy Risk Assessments in Sweden: Information Flow and Impacts," Nuclear RiskAnalysis in Comparative Perspective, (R. Kasperson and J. Kasperson, eds.). 50-84 (Allen and Unwin,London, 1987).
9. Gesellschaft für Reaktorsicherheit, "Deutsche Risikostudie Kernkraftwerke: Eine Untersuchung zu demdurch Störfalle in Kernkraftwerken verursachten Risiko," (Cologne: Verlag TÜV Rheinland, 1980).
10. Herbert Inhaber, Energy Risk Assessment (Gordon and Breach Science Publishers, 1982).
11. U.S. Nuclear Regulatory Commission, op.cit. 6.
12. R. Fullwood and R. Hall, Probabilistic Risk Assessment in the Nuclear Power Industry: Fundamentals andApplications (Pergamon, New York, 1988).
13. G. S. Holman, "Application of Reliability Technicques to Prioritize BWR Recirculation Loop Welds forIn-Service Inspection", p.4, (U.S.Nuclear Regulatory Commission, NUREG/CR-5486, 1989).
14. S. Hirschberg, op.cit. 5.
15. E.M. Dougherty, "Human reliability analysis - Where shouldst thou turn?" Reliability Engineering &System Safety, 29:3 (1990).
16. R. Keeney and D. von Winterfeldt, "On The Uses of Expert Judgement on Complex Technical Problems,"IEEE Transactions on Engineering Management, 36:83-86 (1989).
17. J. Lathrop and J. Linnerooth, "The Use of Risk Assessments in a Political Decision Process," (IIASAWorking Paper WP-81-119, International Institute of Applied Systems Analysis, Laxenburg, Austria,1981).
18. B. Wynne, "Risk Assessment of Technological Systems: Dimensions of Uncertainty," Risk Managementand Hazardous Waste, 269-310 (Springer, Berlin, 1987).
19. Hirschberg, op.cit. 5, pp. 4.1-4.59
Notes
19
20. C. Mandl and J. Lathrop, "LEG Risk Assessments: Experts Disagree," Risk Analysis and DecisionProcesses (H. Kunreuther and J. Linnerooth, eds.) 148-177 (Springer, Berlin, 1983).
21. R. Fullwood and R. Hall, op.cit. 12, p. 279.
22. Ibid. p. 279.
23. S. Carlsson, S. Hirschberg, and G. Johanson, "Qualitative Review of Probabilistic Safety AssessmentCharacteristics," PSA '87 - International SNS/ENS/ANS Topical Meeting on Probabilistic SafetyAssessment and Risk Management (Zurich, Switzerland, August 30-Sept. 4, 1987).
24. U.S. Government Accounting Office (GAO), "Probabilistic Risk Assessment: An Emerging Aid to NuclearPower Plant Safety Regulation", GAO/RCED-85-11 (Washington, D.C. 1985).
25. IAEA. Draft Guidelines for Conducting Probabilistic Safety Assessment of Nuclear Power Plants, SafetySeries Report, p. 141 (1989).
26. C. Clarotti, "PSA, Subjective Probability and Decision Making," PSA '89- International Topical Meetingon Probability, Reliability and Safety Assessment (Pittsburgh, Pennsylvania, April 2-7, 1989) and U.Pulkkinen, "Bayesian Uncertainty Analysis of Probabilistic Risk Models," PSA '89 - International TopicalMeeting on Probability, Reliability and Safety Assessment (Pittsburgh, Pennsylvania, April 2-7, 1989).
27. S. Hirschberg, op.cit. 5, p.4-61.
28. J. Lathrop and J. Linnerooth, op.cit. 17.
29. OECD, "Risk Assessment and Risk Management for Accidents Connected with Industrial Activities",Environment Monographs, 19:57 (Paris, 1989).
30. A. Finkel, Confronting Uncertainty in Risk Management: A Guide for Decision Makers, (Center for RiskManagement, Resources for the Future, Washington D.C., 1990).
31. A. Finkel, Ibid., p. xiv.
32. B. Wynne, op.cit. 18, p. 278.
33. OECD, op.cit. 29.
34. T. O'Riordan, "Nuclear Risk in the United Kingdom," Nuclear Risk Analysis in Comparative Perspective(Kasperson, R. and J. Kasperson, eds.) p. 207 (1987).
35. S. Hirschberg, op.cit. 5, p.3-43.
36. A. Fritzsche, "The health risks of energy production," Risk Analysis, 9:565-577 (1989).
37. H. Kunreuther and J. Linnerooth, Risk Analysis and Decision Processes: The Siting of LEG Facilities inFour Countries (Springer Verlag, Berlin, 1983).
38. R.E. Kasperson, J. Dooley, B. Hansson, J. Kasperson, T. O'Riordan, and H. Paschen, "Large-scale NuclearRisk Analysis: Its Impacts and Future," Nuclear Risk Analysis in Comparative Perspective (R. Kaspersonand J. Kasperson, eds.) p. 226 (Allen and Unwin, 1987).
39. G. Majone, The Uses of Policy Analysis (Yale University Press, New Haven, Conn., 1984).
20
40. B. Hansson, "Major Energy Risk Assessments in Sweden: Information Flow and Impacts" Nuclear RiskAnalysis in Comparative Perspective (R. Kasperson and J. Kasperson, eds.) p. 82 (Allen and Unwin,Winchester, Mass., 1987).
41. J. Linnerooth, "The Political Processing of Uncertainty," Acta Psychologica 56:219-231 (1984).
42. D. Fiorini. "Environmental Risk and Democratic Process: A Critical Review," Columbia Journal ofEnvironmental Law, 14:501-547 (1989).
43. P. Slovic, "Perception of Risk," Science 236, 280-285 (1987).
44. H. Otway and D. von Winterfeld, "Beyond Acceptable Risk: On the Social Acceptability of Technologies,"Policy Sciences 14:247-256 (1982), and S. Rayner and R. Cantor, "How Fair is Safe Enough? The CulturalApproach to Societal Technology Choice," Risk Analysis 7, 3-9 (1987).
45. M. Douglas, Cultural Bias, Occasional Paper no. 35 (London: Royal Anthropological Institute of GreatBritain and Ireland, 1978), M. Douglas, Essays in the Sociology of Perception (Routeledge and KeganPaul, London, 1982), M. Thompson, "Postscript: A Cultural Basis for Comparison, Risk Analysis andDecision Processes (Kunreuther and J. Linnerooth, eds.) (Springer, Berlin, 1983).
46. M. Schwarz and M. Thompson, Divided We Stand: Redefining Politics, Technology and Social Choice,p. 57, (University of Pennsylvania Press, 1990).
47. L. Lederman and F. Niehaus, "Probabilistic Safety Assessment (PSA) as a Tool for Risk Assessment andManagement," Industrial Risk Management and Clean Technology (S. Maltrezow, A. McTry, and W.Irwin, eds.) 82-89 (1990).
48. D. Fiorini, "Technical and Democratic Values in Risk Analysis," Risk Analysis 9:293-299 (1989).
49. Council Directive of 24 June 1982 (82/501/EEC).
50. For a discussion, see OECD, op.cit. 29.
51. OECD, Ibid.
52. L. Lederman and F. Niehaus, op.cit. 47, p. 87, and IAEA, "Status, experience and future prospects for thedevelopment of probabilistic safety criteria," IAEA-TECDOC-524 (1989).
53. G. Bengtsson, Risk Analysis and Safety Rationale: Final Report of a Joint Nordic Research Program inNuclear Safety, p. ix, (Nordic Liaison Committee for Atomic Energy, 1990).
54. Finkel, op.cit. 30, p. xvi.
55. Finkel, Ibid., p. xvi.
56. R. Brickman, "Science and the Politics of Toxic Chemicals Regulations: US and European Contrasts,"Science, Technology and Human Values 9:107-11 (1984).
57. S. Funtowicz and J. Ravetz, "Post-normal science: A New Science for New Times," Scientific European,20-22 (1990) .
58. US Government Accounting Office (GAO), op.cit. 24.
59. J. W. Minarick, "The USNRC Accident Sequence Precursor Program: Present Methods and Findings",Reliability Engineering and System Safety, 27:1 (1990).
21
60. R. Kasperson et al., op.cit. 38, p. 232.
61. OECD, op.cit., 29, p. 31.
62. OECD, op.cit. 29, p. 31.
63. M. Kraft, "The Political and Institutional Setting for Risk Analysis," Risk Evaluation and Management(V. Covello, J. Menkes and J. Mumpower, eds.), 413-434 (Plenum, 1986).
Top Related