Eliciting and classifying concerns: A methodological critique

Risk Analysis, Vol. IS, No. 3, I995

Eliciting and Classifying Concerns: A Methodological Critique

Thomas Webler,**5 Horst Rakel; Ortwin Renn: and Branden Johnson:

Received October 13, 1993

Two methodological steps in the study of peoples’ concerns are elicitation and classification. Elicitation of concerns through analytical methods such as surveys can be supplemented with techniques that perform more diversively. We present two examples of how this can be accomplished one in the expert community and one in the lay community. A classification taxonomy is a subjective choice of the researcher and it can only be evaluated against the stated objectives of the research. We present a classification schema that is explicitly oriented toward diagnosing the substantive needs of public discourses about risk decision making. To illustrate how concerns can be elicited in a social setting and how this classification tool can be applied, we report on a public participation exercise in New Jersey where citizens discussed the impacts of land application of sewage sludge at an experimental farm.

KEY WORDS: Public participation; risk communication; democracy; concerns; causal model; policy making.

1. INTRODUCTION

b s k decision making is inherently difficult because of the need to meet two normative conditions. First, the decisions should be based on the best available scientific knowledge. Second, in a democratic society those citizens with a relevant interest in the situation should have some say in over the decision. The “best” decision is the one that is both scientifically competent and demo- cratically accepted.

When the concerns or demands of members of the lay community contradict the advice of the expert community, risk decision makers are left holding the stick.

I 132 Farley Road, Wendell, Massachusetts 01379. * Center for Environmental and Risk Management, School of Environ-

mental Sciences, University of East Anglia, Norwich NR4 7TJ, United Kingdom. Center for Technology Assessment, Industriestrasse 5, D-70565 Stuttgart, Germany. New Jersey Department of Environmental Protection, Division of Science and Research, Risk Communication Unit, CN 409,401 East State Street, Trenton, New Jersey 08625.

* To whom correspondence should be addressed.

In the absence of political leadership, they may try to get to the root of the controversy by ordering more scientific analyses of expert and lay perceptions and concerns, or they may organize opportunities for discussion among risk professionals and members of the lay community in hopes of producing a consensus or broad agreement for action.

In this paper we build on an ongoing tradition of social science research into risk perception:’“) risk com- rn~nication:~.~) expert judgment,c9) and public participa- t i ~ n ( ~ & ’ ~ ) and take a close and critical look at the prev- alent methodologies for eliciting and analyzing concerns. In line with this body of research we share the belief that objective risk characterization and risk communication are impossible; and we emphasize the need of science to provide sound methodological tools for analyzing concerns and organizing public discussions about risks in fair and competent ways. We suggest that the present method of eliciting concerns can benefit from the addition of discursive techniques and that these can be applied in both the lay and expert communities. Fur- thermore, we suggest that, since the classification of data

42 1 0272-4332/95/06W-0421$07.50/1 0 1995 Society for Risk Analysis

422 Webler, Rakel, Renn, and Johnson

about peoples' concerns cannot be done objectively, it should be based on a clearly stated normative purpose, and we propose one model for classification that is oriented toward improving future discussions among expert and lay communities.

2. ELICITING CONCERNS

When concerns are elicited through formal questionnaires, people are offered a list of possible concerns and are asked to rank them in order of importance. Re- sults are often presented as a representative picture of societal values and interest~."~) The major shortcoming of the prestructured questionnaire is that it only measures responses to the concerns listed and only in the context of those listed. How the questions are phrased, which concerns are listed (or not listed), how concerns are phrased, and how the questionnaire is structured can influence the responses, raising questions about data va- lidity.(15)

Fischer et ~ 1 . ' ~ ) attempted to overcome these shortcomings through the use of an open-ended questionnaire, which they suggested produces a more authentic depic- tion of peoples' concerns. They illustrated their point with one case study. While we agree that the open-ended questionnaire addresses problems of the prestructured questionnaire, we also recognize a limitation with questionnaires in general: they do not address the fact that individuals form attitudes and beliefs in a social world. This is important because: (1) If they have not recently had the opportunity to discuss and consult with others, respondents may not have thoroughly thought out their views. (This is particularly a problem with concerns about risks with which people have little experience.) (2) The strength and robustness of opinions expressed in a questionnaire are difficult to infer from a typical survey, in which opinions are never contested. (3) Ques- tionnaires weight every respondent's opinion equally, treating responses of the disinterested or the uninformed as valid as those with more carefully developed opin- i o n ~ . ( ~ ~ J ~ ) (The advantages and disadvantages of surveys for measuring public preferences are discussed in Ref. 16.)

While these shortcomings do not invalidate the use of questionnaires, they do speak to the need to supplement these analytical instruments (such as surveys) with discursive tools. Discursive tools (focus groups are one example) provide a way to incorporate social interaction and learning into the elicitation process. Giving people a chance to learn about the risks and to discuss their

interests, values, fears, and concerns with peers, helps to clarify some of the factual consequences and to test the social acceptance of different opinions. Providing factual knowledge helps eliminate the possibility that concerns are based on misunderstandings.

We describe two techniques to elicit concerns in discursive settings: the citizen panel and the group Del- phi. The idea of citizen panels is similar to the technique of focus group~,C'~J~) but overcomes the most serious shortcoming of focus groups, namely the tendency for the discussion to be too far removed from the political reality of decision making. (One illustration of how focus groups can be used to elicit risk perceptions is given in Ref. 19.) The group Delphi is a modification of the original Delphi exercise designed to promote face-to- face interaction among a small group of experts.'20)

Discursive methods to elicit concerns have their advantages, but they also have shortcomings. Most impor- tantly, they cannot present an objective picture of concerns, and they are not meant to replace surveys and other analytical approaches. Views measured after social interaction may be temporary, having been attenuated or amplified by the context of the setting for discourse. Contesting opinions may cause some people to conform to the opinion of the group (the bandwagon effect). Since it is difficult to gather a sample population that is truly representative of the population (due to factors such as commitment, accessibility, time lost from work), these methods may run contrary to democratic princi- ples. Still, discursive elicitation techniques provide a perspective on existing concerns that differs from the analytical approaches, and opportunities for discussion may actually enable and promote the emergence of shared concerns, simply by offering an opportunity for solidarity to develop.

3. CLASSIFYING CONCERNS

Commonly, concerns are sorted into different conceptual categories. There is, however, no agreement in the literature on the best taxonomy for classifying concerns about risks, presumably because different taxono- mies emphasize different attributes of the dataset. A meaningful analysis of the data reveals something of relevance, and what is relevant to whom is not an objectively answerable question (although we might expect to reach broad agreement on some points). Without explicitly stating the normative objectives of the research, it is difficult to judge the scientific competence or the political relevance of the empirical results.

Eliciting and Classifying Concerns 423

We propose a method of comparing and contrasting concerns that is oriented toward tailoring risk discourses to practical action alternatives. Using the causal taxonomy of a hazard sequence developed by Hohenemser, Kasperson, and Kates,’*’) we map onto a causal model the concerns that citizens and scientific experts hold with respect to a hazard sequence. Our objective was to invent a tool that would help to diagnose misunderstandings in a public discourse. We believe that public involvement in risk management planning is meant to produce agreement on action based on mutual understandings of each party’s expectations, belief, fears, interests, and concerns. This is best accomplished through noncoercive means.’22) Since the causal model classification is oriented toward action-it reveals where in the causal stream risk management interventions can occur-plot- ting the differences in concerns among citizens and experts reveals where communication among these groups can be improved. The presumption is that this information can be fed back into the public discourse to help produce agreement on risk management action and thereby lead to more implementable and successful risk management. We present one case study as an illustration of how these new techniques can be applied.

4. THE ISSUE: LAND APPLICATION OF SEWAGE SLUDGE

The New Jersey Department of Environmental Pro- tection (NJDEP) awarded a contract to our research team to experiment with citizen participation in designing a permit for the application of municipal sewage sludge to farmland? We elicited the concerns and recommenda- tions of citizens about a proposed research project in which municipal sewage sludge was to be applied to the fields of an experimental research farm recently acquired by Rutgers University. Although Rutgers proposed to apply only a small amount of sludge to a small area for a short time, and although much larger and longer-term land application programs had been carried out in other parts of the state, people in this mainly agricultural community were concerned that success in this project might lead to more ambitious efforts at local land application,

This work was done when the first three authors were at the Center for Technology, Environment, and Development (CENTED), Clark University, Worcester, Massachusetts. Views expressed here are those of the authors and do not necessarily reflect those of the New Jersey Department of Environmental Protection.

which could have major impacts on their future liveli- hood.

Land application of municipal sewage sludge as an option for sludge management is controversial. The major advantage is that the sludge is recycled as a fertilizer for crops, reducing the amount of chemical fertilizer needed and reducing the need for other modes of sludge disposal such as incineration. The associated risks per- tain to the health threats that compounds routinely found in the sludge pose to humans.(23) These compounds are added to the soil when the sludge is applied. They may be taken up by plants; be removed by volatilization, runoff, or wind; leach down into the soil or groundwater; adsorb to soil particles; or be destroyed by photo- or biodegradation. Human exposure pathways include, among others: consumption of contaminated crops, groundwater contamination, disease vectors, and direct ingestion of soil. Besides the risks posed by the incor- poration of sludge in soils, additional risks are associated with the transport and storage of sludge. Health and environmental risks are accompanied by the impacts of odor, aesthetic changes, and social and economic costs. NJDEP has responded to these concerns with regulations that include limitations on concentrations of sludge con- taminants; performance requirements for sludge transport, storage, and application; and groundwater monitoring.

5. ELICITATION OF CONCERNS: METHODOLOGY AND RESULTS

5.1 Citizens: Citizen Panels

The major element of the project was the organi- zation of citizen panels.(24) The purpose of the panels was to elicit public concerns prior to the mandatory public hearing and to tailor the site-specific permit conditions to these concerns. Early involvement of citizens in the process was expected to ease tensions between the regulatory agency and the affected public and to enhance the process of municipal control over monitoring the risk management efforts.

Citizen panels consist of a representative sample of citizens in a procedure similar to focus groups, but also comparable to a jury trial.(25,26) First, people are in- structed about the potential consequences of each option by experts and are thus aware of the consequences of their choices. Expert assessments are made during a “group Delphi” (explained below). Formal hearings,


lectures, videotapes, inspection tours, written information, and discussions provide the basis for generating and evaluating decision options.’ Second, people are informed that their preferences are part of the decision process (depending on the issue and the institutional context), so that they realize they might suffer or enjoy the consequences of the citizen panel’s decision. Citi- zens act as “value consultants” to the decision makers and play a role in the decision making process. The method of citizen panels that we applied was developed by Peter Dienel at the University of Wuppertal and suc- cessfully applied in local, regional, and national policy issues in Germany.‘27-29) This was the first application of this method in the United States, although similar panels have been organized by the Jefferson Center in Minne- ~ota,’~O) and by a group of city planners in Denver.(3L)

We assembled a citizen panel of 24 people by di- viding the population into two groups and randomly se- lecting equally from these groups. One group was composed of people who owned land directly abutting the research farm. The second group was the general population of the host and neighboring communities. In- vitation acceptance was nearly 100% for the abutters (N = 31), but extremely low (<2%) for the general population (N = 102). The h a 1 panel was almost completely composed of abutters.

After presentations about land application and specific information on the Rutgers research proposal, we asked the participants to list their major concerns with respect to the proposed land application research project. They were told that these concerns would be used to structure a permit to control the sludge application process. To facilitate discussion, we randomly sorted the panelists into three small groups of six to seven people

Informing people about the likely consequences of their preference in advance involves the danger of triggering biased responses according to the information presented and the basic attitudes of the instructing experts. To avoid biased responses, as part of the group Delphi, experts with different attitudes were asked to review our basic material and to design the curriculum in conjunction with the research team. The material was divided into four categories: (1) basic factual knowledge that all experts agree on, ( 2 ) contested factual knowledge where the different camps are represented, (3) inter- pretation of facts where the range of legitimate viewpoints is conveyed to the participants, and (4) expert beliefs where short es- says are produced to include the context of the argument. In addition to the written material and lecture outlines, experts with different points of view participated in a hearing before the citizen panel. The audience also viewed videotapes of the main arguments made during the group delphi. More details can be found in: 0. Renn, “Premises of Risk Communication: Results of Two Participatory Experiments,” in R. E. Kasperson and P. J. Stallen (eds.), Communicating Risks fo the Public: International Perspectives (Dordrecht, Kluwer Academic, 1991), pp. 457-481.

each, and provided each group with a facilitator, who kept the discussion focused on the task at hand. Each small group was asked to prioritize the list of concerns. Table I presents the citizens’ concerns and priorities as given by the three groups.

The citizen groups chose to rank only a few of their concerns. Topping the list were concerns for property values, limiting the period of application, and long-term monitoring for possible contamination. This theme of distant future consequences was brought up repeatedly in the citizen panels, especially the fear of a transfor- mation of farmland into “wasteland.” People expressed concerns that the State’s outlook was unacceptably short. Some citizens were concerned that, once the sludge had been applied, problems would persist over a period longer than the NJDEP would be capable of handling. One farmer put it succinctly when he said, “What are 40 or 50 years to a farmer if the farmland is in the family for generations? I would like to hand this land over to the next generation without compromising its ability to grow crops. The land is all I can pass on.” Also ranked high were concerns about who would manage and pay for the clean-up should contamination occur; the frequency with which testing would be done; and the possibility that the respected image of Rutgers University might be tarnished. Several people voiced fears that their town could become a “sludge dump” for the county. Concern for odor was minimal.8

Citizens eventually came to a consensus that they would oppose land application of sludge on the experimental farm; the proposer, Rutgers University, withdrew its proposal. The reasons for the citizens’ opposition seemed to vary widely, from disagreement with the experts that land application could be safe to concern about current or future actions of county officials on sludge and other waste issues that could oppress their community.

5.2. Experts: The Group Delphi

In our public participation process, experts provide data and interpretations to inform the citizens of the consequences of the decision options. To identify areas of uncertainty, dissent, and consensus among the expert community, we convened a group Delphi on the risks

* The group of citizens who participated represented only land owners abutting the Rutgers farm. They were predominantly longtime residents of the community and were specifically interested in the impacts the proposed project would have on fanning. Their experience with farming may explain why they considered odor only a minor concern.


Table I. Elicited Concerns and Priorities of Citizen Small Groups for Perceived Risks Associated with the Experimental Sludge Land Application Projecr

Priorities of small groups Weight

Citizens: stated concerns (c) Group 1 Group 2 Group 3 (a,)

Decline of property values 1 10 Limitation of period of application 1 10 Future monitoring of soil and groundwater after application ceases Image of Rutgers 2 9 Frequency of testing 2 9 Responsibility and liability for cleanup 2 9

1 10

Basic distrust of DEPE and other authorities 3 8 Prevention of groundwater contamination 3 8 Runoff water 3 8

7

Transportation of sludge material 5 6 1 1

“Foot in the door” problem (gradually extending the sludge application program) Timing between testing and application 4 7

Pretreatment of sludge (strict parameters and industry specific) Education of local residents (to avoid emotional responses) Monitoring wells on site 1 Soil testing 1 Honesty towards citizens (not just positive information) 1 Removal of industrial chemicals from sludge 1 Removal of household chemicals 1 Uptake of chemicals in the food chain 1 Effective regulation (independent testing) 1 Odor 1 Health effects from pathogens 1 Limitation of applied quantity 1 Containment of the experimental area

4

1 Totals 114

Weighting (a,) were assigned by simple linear ranking. For example, concerns ranked first were given 10 points, secondary concerns were given 9 points, and so on. Unranked concerns received 1 point.

associated with land application of sewage sludge. Re- sults were used to prepare an information package for the citizens.

A group Delphi is a modification of the Delphi exercise developed by the RAND Corporation in the 1950s. It accelerates the Delphi exercise by supple- menting the iterative reevaluation process with face-to- face communi~ation.‘~~.~~) We selected our pool of potential participants by telephoning researchers and practitioners to identify experts recognized by their peers as reputable authorities in this field. We tried to achieve a balance among disciplines represented and among competing opinions about the relative value of land application as a disposal method for sewage sludge. At the workshop, nine experts were asked to evaluate the risks of land application of sludge and to judge the effectiveness of the NJDEP regulations for mitigating risks.

To investigate the concerns of the experts, we divided the nine experts into three groups. We asked them to imagine that they lived near the proposed site and had been asked by their neighbors to participate on the com-

munity’s behalf. They were to compose, on a blank sheet of paper, a prioritized list of ten concerns that they would raise. The results are presented in Table 11.9 As with the citizens, this was a dialogue-intensive process.

Group A chose “odor” as their primary concern, followed by potential outcomes from contamination of groundwater, crops, and soils (places 2, 3, and 4). Mon- itoring and mitigating these possible outcomes were addressed in the fifth, sixth, and seventh places. Property value was at the bottom of the list. Group B saw “insurance and cost of clean-up’’ as the primary concern, followed by five concerns for different forms of contamination. Future land use and property value were also at the bottom of this list, ranking eight and tenth, respectively. Group C agreed with the first group that “odor” was the most important concern. Their second priority

We later verified that we had measured the concerns of the experts, rather than the concerns that the experts thought the citizens would want them to raise, by a second survey sent to the experts who participated.


Table 11. Concerns and Priorities of Expert Small Groups to Perceived Risks Associated with the Experimental Sludge Land Application ProjecP

Priorities of small groups

Weight Experts; stated concerns (c) A B C (a,)

Odor 1 I 1 1 21 Insurance and cost for cleanup 1 10 Pathogens (health concerns, movement into groundwater) 2 9 Ground and surface water contamination Regulation (Who is going to “guarantee” project control?) Heavy metals (crop effects, soils, migration to water) Air Contamination

2 9 2 9

3 8 3 8

Water contamination (potable wells and surface water. What will protect my supply and prevent degradation?) Toxic organics (crops, wildlife, soils, water) 4 7

Pathogens (how to control exposure via air and other means of transport?)

3 8

Food chain contamination 4 7 4 7

Groundwater control (effect on wells, metals/nutrients, pathogens) Soil contamination and mobility Importing sludge Monitoring (who does it, how Frequently, for what reason, recordkeeping, availability of records)

5 6 5 6

5 6 6 5

Environmental persistence of pollutants 6 5 Food chain (how do you prevent wildlife from eating crops?) Surface water control (effect on wells, metalshutrients, pathogens) Recordkeeping and monitoring Traffic (nuisance and safety) Traffic (increase in traffic, road damage) Effect on future land use Vectors (rodents, flies, pets will transport contamination into my home) Wildlife concerns (hunting, consumption of game) Public access to site Property value (resale or development potential of my property in the future) Food processors (if I produce sludge-grown crops, will they accept my product?)

Totals

6 5 7 4

7 4 7 4

8 3 8 3

8 3 9 2

9 2 10 10 9 4

10 I I66

Weightings (a,) were assigned in a linear order as with the citizen concerns.

was for “who will guarantee the regulation?” and their fifth concern was for the social fall-out of importing sludge into a rural community. Water and food chain contamination were given medium emphasis. As with the other two small groups, long-term effects on property value were ranked at the bottom.

During the group Delphi, two other issues surfaced that were related to the concerns of several of the experts: public opposition and uncertainties surrounding organic Several experts were skeptical about the possibility of finding public approval for land application of sewage sludge and blamed the media for con- veying a distorted view of the risks. At the same time, they acknowledged that uncertainties about the potential concentrations and effects of organic toxins in the sludge impeded the construction of an unambiguous message to the public. Whereas some experts believed that the am- biguities created by the scientific uncertainties should not be communicated to the public in order to avoid

misunderstanding and possible overreaction, others felt that trust in science and regulatory institutions could only be generated or sustained through an open discourse between researchers and the public.

5.3. Regulations

As a supplement to the elicitation of concerns from citizens and experts, we analyzed the New Jersey standard permit for site-specific regulations for land application of sewage These data were not elicited through a discursive procedure, as were those from the citizens and experts, but were deduced from the content of the permit text.

As a crude indicator of the priority of regulatory concerns, we measured the number of pages devoted to each topic heading in this document. These measures were then normalized to the total number of pages in


the standard permit and are presented as percentages. To amve at a number of concerns, we distinguished individual concerns by topical headlines. Once the concerns were sorted into the causal categories, we summed the number of entries and normalized these numbers ac- cordingly.

Length of text is certainly one indicator of regulatory emphasis, although it may be a weak or misleading indicator. Well-accepted and effective regulatory strategies may address extremely important hazards, but be non-controversial and expressed concisely. Likewise, minor concerns to regulators may be discussed at great length in order to appease opponents. It is impossible, however, to deduce such motivations from the regulations. Nonetheless, we can conclude that items excluded from the regulations do not receive serious concern from the regulatory body. Table I11 lists the subjects of regulations and the indicators of emphasis. The top concern was clearly for preventing future human exposure to contaminated groundwater through groundwater monitoring. A total of 13 pages were devoted to this problem.'O

6. ANALYSIS OF CONCERNS: METHODOLOGY AND RESULTS

6.1. The Causal Taxonomy

One way to illustrate the differences in concern between citizens and experts, along with regulatory emphasis, is to employ the causal taxonomy of hazard events developed by Hohenemser, Kasperson, and Ka- tes.(21) The model views a hazard event as a causal sequence that can be interrupted and blocked at various stages by effective hazard management activities. In this characterization hazard events represent the potential for harm and hazard consequences the realization of harm. Pathways can be constructed that link original wants and needs with final consequences. Figure 1 is a schematic of the causal model.

The generation of technological hazards begins with a human need from which a demand is expressed as a human want. To act on this want a choice of technology is made. From here the hazard event progresses through an initiating event, outcome, and exposure, end- ing in adverse consequences. Risk management options

lo Although groundwater monitoring itself does not prevent exposure, the intention is that the monitoring will provide information that will trigger an exposure prevention mechanism.

are located in relation to the step in the causal sequence that they seek to block. In the design of a hazard management strategy, it is wise to consider the entire causal chain and design mitigative actions at several nodes along the chain.

The causal model that we used for analyzing and interpreting our data provides a taxonomy for sorting and comparing concerns that is oriented toward action, since it is based on where risk management interventions can occur in the hazard stream. The goal of our risk communication program was to reach an agreement on action-regulating sludge application. The structure of the model serves as a general concept for placing individual or group concerns in a logical order that relates to the purpose of the risk communication exercise. Dif- ferences in concerns that appear mark points of potential misunderstanding or disagreement among conversing parties designing a hazard management plan.

Our causal model for land application of sewage sludge is presented in Table IV as a series of hypothet- ical scenarios that were discussed in the citizen panels. The initiating event is the land application of the sludge to farmland, and one possible outcome of this event is the contamination of soil, water, air, or crops by hazardous substances found in the sludge. Exposure may occur through several avenues and bring about negative consequences such as damage to human health, damage to the ecosystem, or impacts on the quality of life for people living in the community. In addition, secondary consequences may occur, such as devaluation of land values and changes in the community's image and character." The management options are regarded as potential in- tervention points along the chain.

6.2. Classifying and Quantifying Concerns

For the datasets from the citizens, experts, and regulations, we sorted each concern into the categories de- lineated by the causal model. The sorting is shown in Tables V, VI, and VII.

To measure the overall emphasis given to steps in the causal model, we quantified the concerns in each category according to two dimensions: priority weighting and number. Priority weighting represents the relative priority that was placed on the concern and number

II Distinguishing between a primary consequence and a secondary consequence is not always simple. The general rule is that the secondary consequence could not occur without first the primary consequence being felt. Land devaluation and changes to community image could not occur without individuals first perceiving that the impacts of the exposures would exist.


is simply the number of different concerns within each category.

For each source group-citizens and experts-we made the following calculations. Each concern, c, with its associated priority weighting, a,, was classified into a set, xi, that corresponded to a step in the causal model. For each set, we calculated an overall priority weighting measure, A , , which is the sum of the individual priority weightings, and we summed a total number of elements,

A , = E a,. L. E x,

And the total number of elements in set, xi, is:

Nxi = c < E xi

To enable us to compare numbers across the three source groups, we expressed the data as percentages by nor- malizing the values as follows:

AXi A’,t =

2 c a, j = l c E xi

where j is a dummy variable for summing; and

N hr- = - -‘I n

where n represents the total number of concerns listed for the respondent group.

We weighted citizens’ and experts’ concerns by as- signing points in an inverse linear relation to the ranking: ten points to the highest concern, nine for the next highest, and so on. For the citizens, each unranked concern was assigned one point, making a total of 114 points (n = 25) (See Table I).’* A total of 166 points were assigned to expert concerns (n = 27) (see Table II).I3 During the elicitation process, discussion was focused to avoid the phrasing of a concern in more than one way, thus the listings in Tables I and I1 are nonre- dundant. For the regulations, points were assigned based on the amount of space in the standard permit dedicated to that topic. Each page was assigned 4 points, with a

I2Only one citizen concern did not apply to a causal category: #2 “Education of local residents to avoid emotional response.” This concern is neither a hazard event nor a consequence of a hazard event. It is a concern about the effect of the perception of risk among citizens in the community and the effect this might have on the decision malung process. Since it was unranked, dropping it did not substantially distort the relative weighting.

”The eleventh concern of expert group 2: “odor” was assigned 1 point.

total of 48 3/4 pages, or 195 points (n = 29) (see Table 111).

One point needs further clarification. The causal model, as an analytical tool, separates concerns about the causal sequence events from concerns about their proposed solutions. While concerns about steps in the hazard event are likely to be about harm or potential for harm, concerns about interventions are likely to be about the effectiveness of those actions, or simply whether or not there are any mitigative actions at all for that step. Making this distinction clear is one of the strengths of the causal model approach. It is significant if people choose to express their concern in a manner that em- phasizes the danger as opposed to a manner that em- phasizes control over the danger. While this analysis cannot expose the motivation behind the choice, it does reveal whether this should be an important theme in the ensuing public participation process.

6.3. Comparison of Concerns

Figures 2 and 3 depict the expressed concerns for the citizens and experts as classified according to the causal taxonomy. Figure 4 presents the results for the concerns extracted from the text of the boilerplate permit. Above each box are the priority weighting, A:,, and number of concerns, NLZ, respectively, expressed as a percent. Priority weighting is also expressed visually as the thickness of the box border.

6.4. Citizens

With both measures: priority weighting and number, secondary consequences were the most emphasized part of the causal sequence for the citizens living next door to the potential site. High concern for secondary consequences stands in strong contrast to the extremely weak concern for primary consequences. The next two most important concerns of the citizens were to prevent exposure and then to modify the outcome of the potential hazard event.

Nearly one quarter of all concerns that the citizens raised had to do with preventing exposure from occur- ring. In all, over half of their concerns had to do with blocking the causal sequence (the boxes along the bottom of Fig. 2), and here the focus was clearly on upstream actions. No concerns were voiced for mitigating primary or secondary consequences.

The absence of concerns for the far upstream issue-whether land application of sewage sludge is an appropriate technology choice-should not be misinter-


Table 111. Implied Concerns and Priorities of Regulations to Risks Associated Land Application of Sewage Sludge

Weight Regulations: implied concern (c) Pages (a,)

Groundwater monitoring (total) 13 52 Application rates 4 16 Residual stabilization (total) 4 16 Residual storage 4 16 Spillage of residuals 4 16 Recordkeeping and reporting 3 12 Residual quality monitoring Residuals approved for land application Approved application areas Application method Crops Lime stabilization Application periods Soil monitoring Closure conditions Process to significantly reduce pathogens Further reduction of pathogens Residuals acceptable for storage Additional methods to reduce pathogens Residuals planning Applicability Storage closing Stored residual quality monitoring Contingency plan Odor control Public access Property owner’s certification Soil pH

314 314 112 112 112 112 112 112

8 8 4 4 4 4 4 4 4 3 3 2 2 2 2 2 2

1 I4 1 1 I4 1 1 I4 1 1 I4 1 1 I4 1

Residuals requiring no pathogen reduction 1 I4 1 Totals 48 314 195

preted as general acceptance for this waste management practice. A more likely explanation is that we framed the discussion in a too narrowly, disadvantaging this topic from being considered.

6.5. Experts

For the experts, outcomes such as contamination were the most emphasized part of the causal model, both in priority weighting and in number of concerns. Pri- mary consequences from human exposure to the pollutants received the second highest priority weighting, but the second highest number of concerns were about preventing exposure. Significant priority weighting was also given to preventing exposure and to secondary consequences.

6.6. Regulations

Regulations by their very nature seek to block the causal chain at one or several nodes. They are solutions that are not accompanied by explicitly voiced concerns. In the standard NJDEP permit the focus is on assuring that sludge residuals meet minimum quality requirements; modifying outcomes through application standards; and preventing exposure through monitoring. The priority weighting score for preventing hazard events (through making sure sludge content meets standards) and preventing exposure (through monitoring) were about equal at 40% each. Application and site specific standards were weighted half as much. The greatest number of concerns was for source quality regulations, followed by application standards and monitoring.

7. DISCUSSION

Because this analytic technique was developed after the fact, the three sets of data were gathered in differing ways. Experts and citizens were both asked to list and prioritize their concerns, but citizens were free to prioritize as few or as many as they pleased while experts were explicitly asked to list and rank only ten concerns. Regulatory “concerns” were not elicited, but deduced from the text of the existing boilerplate permit. Citizens and experts focused on this specific application, while the permit is a general document meant to be amended for each site application.

Regulations are interventive. This means that they do not express concerns directly, but imply them by specifying actions to mitigate and address concerns. We applied the same analytical techniques to the regulatory data, because regulations “speak” to the concerned parties in a manner that is important for the public participation discourse. Citizens and other parties to the discourse see the permit as an expression of the concerns of the regulatory agency. We are, however, aware of the dangers of drawing conclusions from these dissimilar data, and we offer the following analysis more as an illustration of the new methodology than a definitive analysis of this case study.

Concerns of the citizens who participated in the citizen panels, the experts who participated in the group Delphi, and the standard regulatory permit for land application of sewage sludge in New Jersey differed in four major ways.

First, citizens emphasized long-term consequences while experts focused on outcomes such as emissions and different kinds of contamination. This was true for


Initiating Event outwme

Fig. 1. Causal model of a hazard sequence.

Table IV. Example Scenarios of a Causal Model for Hazards Sequences Associated with the Land Application of Sewage Sludge

Example Scenarios ~ ~~

Causal step 1 2 3 4 5

Choice of technology land application land application land application land application land application Mitigate event source quality regula- source quality regula- source quality regula- alternative transport inspections

Initiating event sludge is applied sludge is applied sludge is applied truck transport storage tank leaks Modify outcome application standards: application standards: application standards: transit routing and protective barriers

0 u t c o m e s contamination of soil emission of odorous contamination of soil, road congestion contamination of

Prevent exposure monitoring tests,“’ site soil injection, immedi- monitoring remedial widen roads, traffic emergency remedial

Exposure child eats soil, people wind carries odor to drinking water con- use of roads, living on children playing, fish-

Mitigate consequences emergency medical deodorize air with provide alternative noise barriers emergency medical

Consequences sickness or death decreased quality of sickness or death decreased quality of eutrophication, sick-

tions tions tions mode

timing and amounts timing and amounts timing and amounts timing

or crops compounds migration stream

containment ate plowing action control action

or animals eat crops town tamination streets ermen

care masking agent water supplies care

life due to smell life, noise, safety ness Mitigate secondary trust fund consequences Secondary come- liability quences

property value protec- compensation compensation trust fund tion financial hardship costs of cleanup, etc. decreased social at- liability

tractiveness

both measures used: priority weighting and number. Thirty percent of citizens’ total priority weighting were for secondary consequences. Experts emphasized these only moderately: 14.5% of priority weighting. Both groups allocated about a fifth of all concerns to this category. Comparatively, the category “outcomes” received over one-third of the experts’ priority weighting and the highest number of entries (29.6%). This was more than twice the priority weighting (1 5%) or number (12.5%) of citizens’ Concerns for outcomes.

Second, citizens expressed almost no concern by weight for primary consequences of the sludge land application project, but the experts made this their second highest priority weighting. Citizens listed only two concerns for this category-“health effects from pathogens” and “odor”-but neither of these were ranked. Experts listed three concerns-‘ ‘odor,” “health effects from pathogens,” and ‘‘traffic nuisance and safety”- and ranked these first, third, and twentieth, respectively.

Eliciting and Classifying Concerns 43 1

Table V. Sorting of Citizens’ Concerns into the Categories of the Causal Model

Citizens: concerns sorted into causal model steps A: N , ~

Choice of technology

Prevent event (none)

Basic distrust of DEPE and other authorities (2.66) Effective regulation (independent testing) (0.33) Pretreatment of sludge (strict parameters and industry specific ( I ) Removal of industrial chemicals from sludge (1) Removal of household chemicals (1)

Transportation of sludge material (6)

Basic distrust of DEPE and other authorities (2.66) Effective regulation (independent testing) (0.33) Limitation of period of application (10) Timing between testing and application (7) Limitation of applied quantity (1)

Prevention of groundwater contamination (8) Runoff water (8) Uptake of chemicals in the food chain (1)

Basic distrust of DEPE and other authorities (2.66) Effective regulation (independent testing) (0.33) Future monitoring of soil and groundwater after program ceases (10) Frequency of testing (9) Monitoring wells on site (1 ) Soil testing (1) Containment of the experimental area (1)

Initiating event

Modify outcome

Outcome

Prevent exposure

Exposure

Mitigate consequences

Primary consequences

(none)

(none)

Health effects from pathogens ( I ) Odor (1)

(none) Mitigate secondary consequences

Secondary consequences Decline of property values (10) Responsibility and liability for cleanup (9) Image of Rutgers (9) “Foot in the door” problem (gradually extending the sludge application program (7) Honesty toward citizens (not just positive information) (1)

Education of local residents (to avoid emotional responses) (1) Cannot be classified

5.3 15.3

5.3 4.2

18.6 15.3

15.0 12.5

22.1 23.6

0 0

0 0

1.8 8.3

0 0

31.9 20.8

0.9 4.2

Third, although they were ranked very low, about one in seven of the citizens’ concerns had to do with source quality regulations, while the experts virtually ig- nored this category. In fact, for every mitigative category in the model, citizen concerns (in terms of priority weighting and number) were greater than expert concerns. One explanation of the experts’ lack of concern for source quality standards is their belief in the effect-

iveness of the regulations. During the group Delphi, many experts expressed confidence in the adequacy of the regulations for addressing these concerns. The standard regulatory permit placed strong priority (39.5%) and listed a large number (47.8%) of regulatory concerns on source quality assurance. However, since the citizens were given a presentation on the regulations before we elicited their concerns, the fact that they still retained


Table VI. Sorting of Experts’ Concerns into the Categories of the Causal Model

Experts: concerns sorted into causal model steps A‘, f f x



0

1.8 Regulation (who is going to “guarantee” project control?) (3)

Initiating event 0 (none)

Modify outcome 7.8 Regulation (who is going to “guarantee” project control?) (3) Groundwater control (effect on wells, metalshutrients, pathogens) (6) Surface water control (effect on wells, metals/nutrients, pathogens) (4)

Ground and surface water contamination (9) Heavy metals (crop effects, soils, migration to water) (8) Air contamination (8) Water contamination (potable wells and surface water; what will protect my

Toxic organics (crops, wildlife, soils, water) (7) Food chain contamination (7) Soil contamination and mobility (6) Environmental persistence of pollutants (5)

Outcome 34.9

supply and prevent degradation?) (8)

0

1.2

0

8.6

29.6

Prevent exposure Regulation (who is going to “guarantee” project control?) (3) Pathogens (how to control exposure and other means of transport?) (7) Monitoring (who does it, how frequently, for what reason, recordkeeping,

Food chain (how do you prevent wildlife from eating crops?) (5) Recordkeeping and monitoring (4) Public access to site (2)

Vectors (rodents, flies, pets will bring contamination to my home) (3) Traffic (increase in traffic, road damage) (3) Wildlife concerns (hunting, consumption of game) (2)

availability of records) ( 5 )

Exposure

Mitigate consequences

15.7 19.7

4.8

0 (none)

Odor (21) Pathogens (health concerns, movement into groundwater) (9) Traffic (nuisance and safety) (4)

Primary consequences 20.5

Mitigate secondary consequences

Secondary consequences Insurance and cost for cleanup (1 0) Importing sludge (6) Property value (future resale or development potential of my land) (4) Effect on future land use (3) Food processors (will sludge-grown crops be accepted?) ( I )

(none)

Cannot be classified (none)

11.1

0

11.1

0 0

14.5 18.5

concern about the regulatory activities indicates either a problem of trust, misunderstanding, or disbelief in the adequacy of the regulations.

Fourth, in spite of the length and sophistication of the standard regulatory permit, it is quite apparent that

some of the experts’ concerns and many of the citizens’ concerns are not mentioned or occupy only marginal space in the permit. Most of the citizens’ concerns about long-term secondary effects are not addressed at all. Fur- thermore, the regulations include no mitigative strategies




Residuals acceptable for storage (16) Residual storage (16) Residuals approved for land application (8) Residual quality monitoring (8) Lime stabilization (4) Recordkeeping and reporting (4) Closure conditions (4) Process to significantly reduce pathogens (3) Process to further reduce pathogens (3) Residuals acceptable for storage (2) Additional methods to reduce pathogens (2) Stored residuals quality monitoring (2) Storage closing (2) Residuals planning ( I ) Residuals requiring no pathogen reduction ( I )

Initiating event (none)

Modify outcome Application rates (1 6) Approved application areas (4) Application method (4) Recordkeeping and reporting (4) Crops (4) Application periods (4) Applicability (2) Property owner certification (1) Odor control (1)

Outcome (none)

Prevent exposure Groundwater monitoring (52) Spillage of residuals (1 6) Recordkeeping and monitoring (4) Soil monitoring (4) Contingency plan (1) Public access (1) Soil pH (1)

(none) Exposure and all other categories

Table VII. Sorting of Concerns Implicit in the Regulatory Permit into the Categories of the Causal Model

Regulations: concerns sorted into causal model steps A', N,

to farmland. There are several reasons to be cautious about over-interpreting these results. Both discourses had their strengths and limitations and it would be naive to posit that each individual expressed his or her concerns authentically and sincerely all of the time. How- 0 0

39.5 47.8 ever, in both the- group Delphi and the citizen panel model attempts are made to minimize strategic behavior and other intervening influences (such as the bandwagon effect, moderator persuasion, structural biases).

With these caveats in mind, this analysis suggests one possible solution to this controversy, given the as- sumption that the apparent differences in concerns explain much or most of the conflict over this land application permitting process. (1) Revise or add to regulatory mechanisms as needed to address expert and citizen concerns, particularly on downstream steps (e.g., property value guarantees). (2) Involve citizens in regulation revision, to give them a chance to inform them- selves about the intent and effectiveness of these regulations (e.g., on sludge quality assurances). The fact that a great deal of attention is given in the permit to upstream concerns that the citizens raised may indicate a need to inform the citizens further about the regulations for land application or to involve them in a formal review process of the adequacy of the regulations. (3) Involve citizens in the permit adaptations to address site- specific conditions.

As our case study illustrated, realizing this solution is not as easy as it seems. First of all, the above as- sumption that differences in concerns are at the root of the conflict is not always valid. There are many more factors to consider-factors such as the proper sequence of events, who to involve in the various discussions, how to gain their good faith, how to demonstrate the sincerity of the risk management agency, and whether the cost of reconciling permit conditions with expert and citizen concerns make land application of sewage sludge a fea-

0 0

20.0 31.1

0 0

40.5 21.1

0 0

to block the occurrence of consequences. Such strategies are not difficult to envision. For instance, the permit could contain a clause to guarantee property values or provide emergency medical care in the event of an impact. Clearly, however, the political, administrative, and financial challenges of providing such mitigation options would have to be addressed by a much broader societal debate than is allowed in most regulatory processes.

Clearly, this analysis cannot diagnose all of the elements of the controversy about applying sewage sludge

sible disposal strategy. But the strong advantage of using the causal model to classify concerns seems to be that it pinpoints the substantive risk management issues (and their relative importance) that need to be addressed in public decision making about risks.

8. CONCLUSIONS

This article made two methodological contribu- tions. First, it presented two examples of how discussion-oriented approaches to eliciting concerns could be used to supplement analytical tools such as surveys. The group Delphi was used for the expert community and a form of public participation called citizen panels was


Cnoia 0fTcChoology

APWCATION

t CONIAMlNAnON BMISSIONS

AppLIcAnoN OP SBWAGE SLUWB

1 - 1 -

0.e (0.0)

Expmurc

SOILINGFSIION

wIu)uFB

Fig. 2. Classification of citizens' concerns using a causal model.

Fig. 3. Classification of experts' concerns using a causal model.

used for the lay community. Both techniques make use of face-to-face interaction in structured communicative settings to elicit concerns, judgments, and justifications and then attempt to press for convergence of agreement or disagreement. These techniques are ways to realize advantages of discourse in risk decision making processes. We recommend using these techniques in asso- ciation with more analytical ones, especially when decisions have a high potential to be controversial.

Second, this article presented a taxonomy for classifying concerns that is directly associated with the substantive needs of public discussions about risk. Our use of this taxonomy was inspired by the conviction that public involvement in risk management planning can only produce voluntary and informed agreements through constructing shared understandings of underly- ing beliefs and concerns. Reaching an agreement on risk management actions through negotiation and compen-

Eliciting and Classifying Concerns

Initding Event oulmmc Exposure Cwroqueosr s-yCaueq==

S O M L

EWNOMlCAL

OWR. NOISE

IMAGE HEALTH WVlRONMPNl

SOU INGISTION m c c u m m CO~AhUNAnON AppzlcATloN OP SEWAGE SLUDOE EMISSIONS VBClORS

wIu)uFB

435

Gwia d Technology

APPLICATION

I ModifyCuouuxm

I L

Fig. 4. Classification of implicit concerns in standard permit using a causal model.

sation without informed consent may produce effective outcomes, but is not consistent with broadly shared values for public involvement. Risk communication and public participation discourses must facilitate the emergence of such understanding. One way to do this effi- ciently is to diagnose potential differences in risk perceptions or concerns by using the causal model. The model draws attention to the opportunities for interven- tion in the hazard chain. Moreover, this taxonomy is applicable to any technological risk issue and, consistent with the goal of an open and inclusive discourse, spans the range of topics from human needs, human wants, and choice of technology, to secondary consequences.

Using the causal model taxonomy for classifying peoples’ concerns constitutes a novel application of this tool, originally developed as a nonnative instrument for policy making. Our objective in developing this technique for analytical purposes was to better understand why the risk communication process between the citizen panels and the regulators broke down, and why expert and citizen concerns had little common ground. Major conceptual differences in how the problem and potential solutions were perceived contributed to the difficulties, as did trust problems exemplified by disbelief that other parties were acting in good faith. Our results show that the causal taxonomy, when combined with an interactive elicitation of concerns, could function as a diagnostic tool for predicting difficulties that risk communication and public participation exercises are likely to encounter. Sequential risk communication might benefit from this analysis by focusing on producing regulatory provisions that address these difficulties and differences.

ACKNOWLEDGMENTS

We would like to thank Debra Levine for her con- tributions to the case study work in New Jersey and Tim O’Riordan for extensive comments on an earlier version of this paper.

REFERENCES

1. P. Slovic, “Perception of Risk,” Science 236, 280-285 (1987). 2. E. Vaughan and M. Seifert, “Variability in the Framing of Risk

Issues,” Journal of Social Issues 48: 1 19-1 35 (1 992). 3. A. Vari, R. Kemp, and J. Mumpower, “Public Concerns About

LLRW Facility Siting: A Comparative Study,” Journal of Cross Cultural Psychology 22(1), 83-102 (1991).

4. G. W. Fischer, M. G. Morgan, B. Fischhoff, I. Nair. and L. B. Lave, “What Risks are People Concerned About?” Risk Analysis

5 . D. MacGregor, “Worry Over Technological Activities and Life Concerns,” Risk Analysis 11(2), 315-324 (1991).

6. P. J. Stallen and A. Tomas, “Public Concern About Industrial Hazards,” Risk Analysis 8(2). 237-245 (1988).

7. P. Stem, “Learning Through Conflict: A Realistic Strategy for Risk Communication,” Policy Sciences 24: 99-1 19 (1991).

8. F. W. Allen, “Towards a Holistic Appreciation of Risk: The Chal- lenge for Communicators and Policymakers,” Science, Technol- ogy, andHuman Values 12, 138-143 (1987).

9. W. Freudenberg, “Perceived Risk, Real Risk Social Science and the Art of Probabalistic Risk Assessment,” Science 242, 4 4 4 9 (1 988).

10. T. Dietz “Theory and Method in Social Impact Assessment,” Sociological Inquily 17: 5 4 6 9 (1987).

11. T. Dietz and R. Rycroff, “Definitions of Conflict and the Legiti- mation of Resources. The Case of Environmental Risk,” Socio- logical Forum 4: 47-70 (1989).

12. D. Fiorino, “Citizen Participation and Environmental Risk: A Sur- vey of Institutional Mechanisms,” Science, Technology. & Human Values 15(2): 226243 (1990).

11(2), 303-314 (1991).


13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

P. Slovic, “Perceived Risk, Trust, and Democracy” Risk Analysis 13, 675-682 (1993). C. Bailey, C. E. Faupel, and S. F. Holland, “Hazardous Wastes and Differing Perceptions of Risk in Sumter County, Alabama,” Sociey and Natural Resources 5 , 21-36 (1992). A. Tversky and D. Kahneman, “The Framing of Decisions and the Psychology of Choice,” Science 211,453458 (1981). H. Harty and L. Blair, “Citizen Surveys for Local Governments: A Copout, Manipulative Tool, or a Policy Guidance and Analysis Aid?” in T. Clark (ed.), Citizen Preferences and Urban Public Policy (Beverly Hills, Sage, 1976). L. Milbrath, “Citizen Surveys as Citizen Participation Mecha- nisms,” Journal of Applied Behavioral Science 17(4), 478496 (1981). R. A. Krueger, Focus Groups: A Practical Guide for Applied Re- search (Beverly Hills, Sage, 1988). D. Golding, S. Krimsky, and A. Plough, “Evaluating Risk Com- munication: Narrative vs. Technical Presentations of Information About Radon,” Risk Analysis 12(1), 27-35 (1992). T. Webler, D. Levine, H. Rakel, and 0. Renn, “The Group Del- phi: A Novel Attempt at Reducing Uncertainty,” Technological Forecasting and Social Change 39,253-263 (1991). C. Hohenemser, R. E. Kasperson, and R. W. Kates, “Causal Structure,” in R. W. Kates, C. Hohenemser, and J. X . Kasperson (eds.), Perilous Progress (Boulder, Westview Press, 1985), pp. 25-42. 0. Renn and T. Webler, “Anticipatory Conflict Management,” Gaia: Ecological Perspectives in Science, Humanities, and Eco- nomics 1(2), 84-94 (1992). A. L. Page, T. J. Logan, and J. A. Ryan, Land Application of Sewage Sludge (Lewis Publishers, Chelsea, MI, 1987). 0. Renn, R. Goble, D. Levine, H. Rakel, and T. Webler, Citizen Participation for Sludge Management, Final Report to the New Jersey Department of Environmental Protection (CENTED, Clark University, Worcester, MA, 1989).

25. 0. Renn, T. Webler, and B. Johnson, “Citizen Panels in the US,” RISK: Issues in Health and Safey 1, 197-226 (1991).

26. 0. Renn, T. Webler, H. Rakel, P. Dienel, and B. Johnson, “A Three Step Procedure for Public Participation in Decision Mak- ing,” Policy Sciences 26, 189-214 (1993).

27. P. Dienel, New Options for Participatory Democracy (University of Wuppertal Press, Wuppertal, 1980).

28. P. C. Dienel, Die Planungszelle (Westdeutscher Verlag: Opladen 1978); P. C. Dienel, “Contributing to Social Decision Method- ology: Citizen Reports on Technological Projects,” in C. Vlek and G. Cvetkovich (eds.), Social Decision Methodology for Techno- logical Projects (Kluwer Academic Press, Dordrecht, 1989), pp.

29. 0. Renn, H. U. Stegelmann, G. Albrecht, U. Kone, and H. P. Peters, “An Empirical Investigation of Citizens’ Preferences Among Four Energy Scenarios,” Technological Forecasting and Social Change 26, 1146 (1984).

30. N. Crosby, J. M. Kelly and P. Schaefer, “Citizen Panels: A New Approach to Citizen Participation,” Public Administration Review 46, 170-178 (1986).

31. L. Kathlene and J. A. Martin, “Enhancing Citizen Participation: Panel Designs, Perspectives, and Policy Formation,” Journal of Policy Analysis and Management 10, 4 6 6 3 (1991).

32. 0. Renn and U. Kotte, “Umfassende Bewertung der vier Pfade der Enquet-Kommission auf der Basis eines Indikatorkatal- ogs,” in G. Albrecht and H. U. Stegelmann (eds.), Energie im Brennpunkt, HTV Edition (Technii und Sozialer Wandel, March, 1984), pp. 190-232.

33. H. Rakel, Regulatory Management of Land Application of Sewage Sludge, Master’s thesis ETS Program, Clark University, Worces- ter, MA (1990).

34. State of New Jersey, The Statewide Sludge Management Plan (NJ Department of Environmental Protection, Trenton, NJ, November 1987).

133-1 50.

Eliciting and classifying concerns: A methodological critique

Documents

Transcript of Eliciting and classifying concerns: A methodological critique