Virtual Experiments and Environmental Policy

by

Stephen M. Fiore, Glenn W. Harrison, Charles E. Hughes and E. Elisabet Rutström*

June 2007

Abstract.

We develop the concept of virtual experiments and consider their application toenvironmental policy. A virtual experiment combines insights from virtual reality incomputer science, naturalistic decision-making from psychology, and fieldexperiments from economics. The environmental policy applications of interest to usinclude traditional valuation tasks and less traditional normative decision-making.The methodological objective of virtual experiments is to bridge the gap between theartefactual controls of laboratory experiments and the naturalistic domain of fieldexperiments or direct field studies. This should provide tools for policy analysis thatcombine the inferential power of replicable experimental treatments with the natural“look and feel” of a field domain.

* Cognitive Sciences Program, Department of Philosophy (Fiore); Department of Economics,College of Business Administration (Harrison and Rutström); and School of Electrical Engineeringand Computer Science (Hughes), all at the University of Central Florida, Orlando, Florida. E-mail:sfiore@ist.ucf.edu, gharrison@research.bus.ucf.edu, ceh@cs.ucf.edu, and erutstrom@bus.ucf.edu.We thank the U.S. National Science Foundation for research support under NSF/HSD 0527675;Emiko Charbonneau, Mark Colbert, Jared Johnson, Jaakko Konttinen, Greg Tener and Shabori Senfor research assistance; and James Brenner, Don Carlton, Zach Prusak and Walt Thomson forhelpful comments. We are also grateful for comments from the organizers and participants of theRFF/EPA Conference Frontiers of Environmental Economics, Washington, February 26-27, 2007.

Table of Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -1-

2. Virtual Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -6-2.1 Virtual Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -6-2.2 Virtual Trees, Forests and Wild Fires . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -8-2.3 Mixed Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -11-2.4 Virtual Reality and Expert Decision-Making . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -13-

3. The Value Added of Virtual Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -15-3.1 Frontier of Visualization in Environmental Valuations . . . . . . . . . . . . . . . . . . . . . . . . -15-3.2 Naturally Occurring Risk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -18-3.3 Scenario Rejection: the “R” in VR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -19-3.4 VR Simulations as Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -21-

4. A Case Study: Evaluating the Risks of Wildfires . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -22-4.1 Representation as a Policy Lottery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -24-4.2 Affecting Perceived Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -26-4.3 The Virtual Experimental Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -29-4.4 A Pilot Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -35-

5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -38-

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -50-

Appendix A: Presence Questions and Factors Identified . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -A1-

Appendix B: Instructions and Decision Sheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -A3-

-1-

1. Introduction

We develop the concept of virtual experiments and consider their application to

environmental policy. A virtual experiment combines insights from virtual reality simulations in

computer science, naturalistic decision-making and ecological rationality from psychology, and field

and lab experiments from economics. The environmental policy applications of primary interest to

us are traditional valuation tasks, but the concept is easily extended to include less traditional

normative decision-making. The methodological objective of virtual experiments is to combine the

strengths of the artefactual controls of laboratory experiments with the naturalistic domain of field

experiments or direct field studies. This should provide tools for policy analysis and research on

decision making that combine the inferential power of replicable experimental treatments with the

natural “look and feel” of a field domain.

We start by reviewing the technological frontier provided by virtual reality (section 2). That

general review is then related to some major issues in environmental economics (section 3), and then

finally illustrated in an application to wildfire risk management (section 4). One surprising theme for

economists is that applications of this technology stress the importance of having an underlying

model that simulates the natural processes defining the environment: this is not just cool graphics

untethered to the laws of Nature. Indeed, this connection is one of the methodological insights from

this frontier, requiring attention to how psychologists define a “naturalistic” decision-making

environment (section 2.4).

The frontier we examine also has relevance for broader debates in economics, beyond the

applications to environmental economics. It is now well-known and accepted that behavior is

sensitive to the cognitive constraints of participants. It has been recognized for some time that field

referents and cues are essential elements in the decision process, and can serve to overcome such

constraints (e.g., Ortmann and Gigerenzer [1997]), even if there are many who point to “frames” as

the source of misbehavior from the perspective of traditional economic theory (Kahneman and

-2-

Tversky [2000]). The concept of “ecological rationality” captures the essential idea of those who see

heuristics as potentially valuable decision tools (Gigerenzer and Todd [1999], Smith [2003]).

According to this view, cognition has evolved within specific decision environments. If that

evolution is driven by ecological fitness then the resulting cognitive structures, such as decision

heuristics, are efficient and accurate within these environments. But they may often fail when applied to

new environments.

At least two other research programs develop similar views. Glimcher [2003] describes a

research program, following Marr [1982], that argues for understanding human decision making as a

function of a complete biological system rather than as a collection of mechanisms. As a biological

system he views decision making functions as having evolved to be fit for specific environments.

Clark [1997] sees cognition as extended outside not just the brain but the entire human body,

defining it in terms of all the tools used in the cognitive process, both internal and external to the

body. Field cues can be considered external aspects of such a process. Behavioral economists are

paying attention to these research programs and what they imply for the understanding of the

interactions between the decision maker and his environment. For our purposes here, it means we

have to pay careful attention to the role of experiential learning in the presence of specific field cues

and how this influences decisions.

The acceptance of the role of field cues in cognition provides arguments in favor of field

rather than lab experiments (Harrison and List [2004]). Where else than in field experiments can you

study decision makers in their natural environment using field cues that they have come to depend

on? We actually challenge this view, if it is taken to argue that the laboratory environment is

necessarily unreliable (Levitt and List [2007]). While it is true that lab experiments traditionally use

artefactual and stylized tasks that are free of field cues, in order to generate the type of control that is

seen as essential to hypothesis testing, field experiments have other weaknesses that a priori are

equally important to recognize (Harrison [2005]). Most importantly, the ability to implement

-3-

necessary controls on experimental conditions in the field is much more limited than in the lab. In

addition, recruitment is often done in such a way that it is difficult to avoid and control for sample

selection effects; indeed, in many instances the natural process of selection provides the treatment of

interest (e.g., Harrison and List [2007]). However, that means that one must take the sample with all

of the unobservables that it might have selected on, and just assume that they did not interact with

the behavior being measured. Finally, the cost of generating observational data can be quite

significant in the field, at least in comparison to the lab.

For all these reasons we see lab and field experiments as complementary, a persistent theme

of Harrison and List [2004]. A proper understanding of decision making requires the use of both.

While lab experiments are better at generating internal validity, imposing the controlled conditions

necessary for hypothesis testing, field experiments are better at generating external validity, including

the natural field cues.

We propose a new experimental environment, the Virtual Experiment (VX), that has the

potential of generating both the internal validity of lab experiments and the external validity of field

experiments. A VX is an experiment set in a controlled lab-like environment, using either typical lab

or field participants, that generates synthetic field cues using Virtual Reality (VR) technology. The

experiment can be taken to typical field samples, such as experts in some decision domain, or to

typical lab samples, such as student participants. The VX environment will generate internal validity

since it is able to closely mimic explicit and implicit assumptions of theoretical models, and thus

provide tight tests of theory; it is also able to replicate conditions in past experiments for robustness

tests of auxiliary assumptions or empirically generated hypotheses. The VX environment will

generate external validity because observations will be made in an environment with cues mimicking

those occurring in the field. In addition, any dynamic scenarios can be presented in a realistic and

physically consistent manner, making the interaction seem natural for the participant. Thus the VX

builds a bridge between the lab and the field, allowing the researcher to smoothly go from one to the

-4-

other and see what features of each change behavior. VX is a methodological frontier enabling new

levels of understanding via integration of laboratory and field research in ways not previously

possible. Echoing calls by others for such an integration, we argue that “research must be conducted

in various settings, ranging from the artificial laboratory, through the naturalistic laboratory, to the

natural environment itself” (Hoffman and Deffenbacher [1993; p. 343]).

Two necessary requirements for a successful VX are “presence” and “coherency.” Presence

is the degree to which participants have a sense of “being there.” When participants are present,

their sensory inputs are dominated by those generated in the VR environment. We will use the term

“naturalistic” environment for a synthetically generated environment that induces presence, to

contrast it with genuinely natural or artefactual non-synthetic environments. Compared to textually

and pictorially generated descriptions in artefactual environments, VX includes dynamically

generated experiences. For presence to occur it is important that these experiences are physically and

scientifically coherent. Therefore, a scientifically accepted model generating the temporal sequence

of cues must underlie the VR environment experienced by the decision-maker.

If we do not have observations from a natural, non-experimental environment to compare

behavior with, how will we know if VX is better than the typical field or lab experiment? While it is

straightforward to test for internal validity, external validity tests require an understanding of natural

decisions. We propose testing the external validity of VX by relying on measurements of behavioral

convergence. As a result of the perceptual “scaffolding” provided by a VX, we hypothesize that

identifiable groups of respondents will display less noise and bias in their decisions in a VX than in

an artefactual experiment. We also hypothesize that non-expert decision makers will make decisions

more like expert decision makers in a VX than in an artefactual experiment. Finally, we hypothesize

that experts will make the same decisions in a VX as they will in a field experiment.

We stress that the ability to test for external validity is intrinsically very limited. This is as true

for field experiments as it is for VX, and does not mean that the latter is a priori inferior to the

-5-

former. The ability to make observations on behavior in the natural field without influencing

decisions is limited, making comparisons between experimental and actual behavior imprecise.

The potential applications for VX are numerous. Apart from simulating actual policy

scenarios, such as the wild fire prevention policies investigated here, it can also be used to mimic

environments assumed in a number of field data analyses. For example, popular ways of estimating

valuations for environmental goods include the Travel Cost Method (TCM), the Hedonic Pricing

Method (HPM), and the Stated Choice Method (SCM). To mimic TCM the simulation can present

participants with different travel alternatives and observe which ones are chosen under different

naturalistic conditions. To mimic HPM the simulation can present participants with different real

estate options and observe purchasing behavior, or simply observe pricing behavior for alternative

options (Castronova [2004]). Finally, to mimic SCM participants can experience the different options

they are to choose from through naturalistic simulation. For all of these types of scenarios, some of

the most powerful applications of VX will involve continuous representations of dynamically

generated effects of policy changes. Visualizing and experiencing long-term effects correctly should

improve short-run decisions with long-run consequences.

In our application to wild fire prevention policies we use the SCM. In this application we

present participants with two options: one simply continues the present fire prevention policies, and

the other increases the use of prescribed burns. Participants get to experience a fire season under

each policy and are then asked to make a choice between them. The scenario that simulates the

continuation of the present fire prevention policies will realistically generate fires that cause more

damage on average and that also vary substantially in intensity. This option therefore presents the

participant with a risky gamble with low expected value. The alternative option presents a relatively

safe gamble with a higher expected value, but there will be a non-stochastic cost involved in

implementing the expansion of prescribed burns. It is possible in VX to set the payoff parameters in

such a way that one can estimate Willingness To Pay (WTP) for the burn expansion option that is

-6-

informative to actual fire policy. These values of WTP could then be compared to those generated

through a popular Contingent Valuation Method (CVM) to test the hypothesis that they should be

different. Alternatively, it is possible to manipulate the payoff parameters in such a way that one

estimates parameters of choice models such as risk attitudes, loss aversion, and probability weights.

In summary, we review and illustrate the use of VX as a new tool in environmental

economics, with an emphasis on the methodological issues involved. In section 2 we introduce VR

to economists with lessons for the design of VX. We also discuss what can be learned by comparing

behavior by experts and non-experts in a VX. In section 3 we review the state-of-the-art in using

visualization technologies in applications to environmental economics. Section 4 then presents a

case study in which these techniques are applied to the assessment of the consequences of wildfire

risks and fire management options. Section 5 draws conclusions.

2. Virtual Environments

2.1 Virtual Reality

In a broad sense, a VR is any computer-mediated synthetic world that can be interactively

experienced through sensory stimuli. The range of senses supported nearly always includes sight and

very often hearing. Touch, smell and taste are less often supported, unless required by the VR

system’s application. For example, touch, or haptic feedback, is often provided when using VR to

develop surgical skills (e.g., Çakmak et al. [2005] and Sewell et al. [2007]).

Experiencing a VR requires that the user have some means to navigate through the virtual

world. Although not strictly required, such experiences typically allow interaction, and not just

observation, such that the actions of the user affect the state of the virtual environment. For all cases

of allowed interaction, including navigation, the actions of the user must result in a response that

occurs as quickly as in the physical world. Thus, navigation should be smooth and natural. Where

interaction is allowed, the action of a user must result in a reaction in the virtual world that is

-7-

appropriate, within the framework of the virtual world’s model of truth (its physics, whether real or

fanciful), and as responsive as interaction would be with the virtual object’s real world counterpart, if

such a counterpart exists.

An immersive VR refers to one that dominates the affected senses. As VR is typically visual,

the primary sense of vision is often dominated by having the user wear a head-mounted display

(HMD) or enter a CAVE (a “box” in which the surrounding visual context is projected onto flat

surfaces). The user’s position and orientation are then tracked so that the visual experience is

controlled by normal user movement and gaze direction. This supports a natural means of

navigation, as the real movements of the user are translated into movements within the virtual space.

The realistic navigation paradigm described above works well within constrained spaces and

simple movements, but is not as appropriate for motions such as rising up for a god’s eye view and

then rapidly dropping to the ground for a more detailed view. There are HMD (see Figure 1) and

CAVE-based solutions here, but such experiences are more often provided with either a dome

screen, illustrated in Figure 2, or a flat panel used to display the visual scene. Since dome screens and

flat panels are clearly less immersive than HMDs and CAVEs, experiences that are designed for

these lower-technology, lower-cost solutions need to employ some means to overcome the loss of

physical immersion and therefore presence. This is generally done through artistic conventions that

emotionally draw the user into the experience. In effect, the imagination of the user becomes one of

the tools for immersion (Stapleton and Hughes [2006]). This is somewhat akin to the way a

compelling book takes over our senses to the point that we “jump out of our skins” when a real

world event occurs that seems to fit in with the imagined world in which we find ourselves.

Essentially, a successful VR leads to what is referred to in literature and theater as a “willing

suspension of disbelief” (Ryan [1991]).

To provide an exemplar of the extreme VR environment that might be used for VX in the

future, consider the online community known as Second Life (see http://secondlife.com/). This is a

1 These online games are known as massively multiplayer online role-playing games (MMORPGs),since many humans can be online at the any time and assume certain roles in the game. There are someextreme statements about how MMORPGs differ from VR environments, such as Castronova [2005; p.286-294].

-8-

3D virtual world where people can perform many of the same social functions as they can in real life

but behind the facade of an avatar allowing the real person a high degree of anonymity. This online

community has experienced an explosive growth since its inception in 2003 and the marketplace that

it hosts currently supports millions of US dollars in monthly transactions. Many other such

environments exist, although most are set in some mythical world in which the laws of physics and

body proportionality do not apply. Castronova [2005] provides an entertaining review of the

industry of these environments, with some eye to the economics of the synthetic environment.1

These online worlds can quickly stir the imagination of experimenters, but pose significant problems

for implementing critical controls on participants and the environment. Considerable work is needed

before experiments can be run in these environments.

In the experiment we describe later experimental participants interact with 3D simulations

using a mouse, a keyboard and a really large, flat panel monitor.

2.2 Virtual Trees, Forests and Wild Fires

Our case study considers wildfire prevention policies, and allows the user to view the state of

forests and trees with respect to the fuel load and the moisture conditions. Participants can move

through a forest, close up on specific trees and shrubs, or do a fly-over from a god’s eye view. The

principal means to display the simulations will be large flat panels, but dome screens will also be

employed. Flat panels are very portable, allowing us to bring them to participants in the field, while

dome screens can only be used for participants willing to travel to specific locations where they are

installed. The only sensory signal that will be used initially is vision. There is a potential to add sound

or smell. Adding sound is relatively easy and will be done at a later stage. Smell, however, would

-9-

require additional equipment and is not viewed as essential to this application.

The simulations demand some consideration to the rendering of virtual trees and forests.

Computer modeling of trees has been an active research area for a number of decades. Cohen

[1967] and Honda [1971] pioneered computer models for tree generation. Prior to that, Ulam [1962]

and then Lindenmayer [1968] studied purely mathematical models of plant growth. Lindenmayer

[1968] proposed a set of rules for generating text strings as well as methods for interpreting these

strings as branching structures. His method, now referred to as L-systems, was subsequently

extended to allow random variations, and to account for the interaction of growing plants with their

environment (Mech and Prusinkiewicz [1996]; Prusinkiewicz, James and Mech [1999]; Deussen et al.

[1998]). A wide range of environmental factors, including exposure to sunlight, distribution of water

in the soil, and competition for space with other plants, were incorporated into the model, leading to

biologically accurate ecosystems.

The above approaches focus on biologically accurate generation of vegetation, not

necessarily on the graphical rendering. A number of methods have been proposed for generating

plants with the goal of visual quality without relying on botanical knowledge. Oppenheimer [1986]

used fractals to design self-similar plant models. Bloomenthal [1985] assumed the availability of

skeletal tree structures and concentrated on generating tree images using splines and highly detailed

texture maps. Weber and Penn [1995] and Aono and Kunii [1984] proposed various procedural

models. Chiba et al. [1997] utilized particle systems to generate images of forest scenes.

These and other approaches have found their way into commercial products over the last

decade. In general, L-systems are used when each model must be unique and when a single user

experience can involve noticeable tree growth, such as when simulated years pass by during a single

visit to the virtual world. When the degree of reality that is needed can be provided without

uniqueness, and where dynamic changes are limited, virtual forests are generally created using a

limited number of trees, with instances of these trees scaled, rotated and otherwise manipulated to

2 For our case study, we have selected a commercial library of trees and its associated renderer,SpeedTree, to provide a realistic-enough forest. However, we plan to revisit this issue, with an eye towardsextending the experiences we develop to allow not only fire spread but also forest changes based on policiesthat can affect nutrients, pollution, tree diversity and other disruptions to the natural cycles. Our confidence insuccessfully using this seemingly more computationally intensive approach is based on the results we haveobtained in a related forest walk-through project (Micikevicius and Hughes [2007]; Micikevicius et al. [2004]).

-10-

give the appearance of uniqueness.2

Computer visualizations of fire and smoke are active areas of research: for example, Balci

and Foroosh [2006], Adabala and Hughes [2005], Nguyen et al. [2002] and Stam [2000]. Some of

these efforts focus on physical correctness and others on real-time performance with visually

acceptable results. Our goal is the latter since we require the fire to cover a large area, with no

constraints on the point-of-view from which the user views the fire (close-up or far-away, in the air

or on the ground). The need for interactivity drives much of what we do, so long as this goal does

not interfere with the primary requirement that the experience be perceived as realistic.

Our approach to visualization of the fire and its attendant effects of illumination, charring,

denuding and smoke is centered on computing the illumination of an area. Our lighting model is

based on the time of arrival of the fire in a given cell of the area being modeled. The entire area is

broken up into 30×30 meter cells, consistent with the fire spread model we are using (see §4.2). For

the terrain appearance, the illumination is determined by a simple Gaussian falloff function. In

effect, we use a normal distribution with its peak at the time-of-arrival. Thus, the terrain starts to

light up prior to the arrival of the fire and retains a glow for a period of the time after the fire leaves.

Its brightest red tint occurs when the fire first arrives. This arrival is accompanied by the display of

flames, the height of which is determined by the state of the fire (ground or crown). The burned

region is computed in a similar manner, but there is an inverse squared falloff function determining

the darkness of the charred remains.

The illumination of the trees and leaves is done in a manner nearly identical to that of the

terrain, but the distance relative to the ground attenuates the effect of the light from the fire.

-11-

Depending upon whether or not the burn causes a crown fire determines if the leaves disappear

after the burn is complete. Although not currently integrated into the test system, we have developed

algorithms that visualize the burning of individual leaves: the current test system instantaneously

denudes an entire tree after it is consumed by a crown fire.

Fire and smoke particles appear if the ground illumination is above a certain threshold. New

particles are spawned by selecting one emitting cell (i.e., a cell on fire) at uniform probability. The

list of emitting cells is recomputed at each tick of simulated time, where an hour is currently

partitioned into 300 ticks. Smoke and fire particles have fixed lifetimes of 20 and 3 seconds,

respectively.

The required performance for the above visual effects would be hard to achieve in a purely

CPU-based solution, so the illumination is actually handled using the graphics processing unit. This

means that our implementation requires a relatively high-end graphics card. At present, we get

acceptable performance on an NVIDIA 7900 GT card and good performance on an NVIDIA 8800

GTX card. Of course, references to hardware performance reflect what can be purchased on readily

available systems as of mid-2007.

2.3 Mixed Reality

Recent extensions of the VR concept include environments where elements from both real

and virtual worlds are combined into a so-called Mixed Reality (MR) (Hughes et al. [2005a][2005b]).

In MR we capture the real world and merge its sensory inputs with those of a virtual world. When

this is done in such a way as to make the setting real, with virtual augmentation, the experience is

called an Augmented Reality (AR). When an essentially synthetic world is augmented with objects

extracted from the real world, the experience is called an Augmented Virtuality (AV). The

continuum from purely real to purely virtual is the domain in which MR operates (Milgram and

Kishino [1994]; Tamura et al. [2001]). While MR is an exciting prospect for future experiments it

-12-

poses additional technical problems; a more productive approach is to first develop an experimental

paradigm in completely virtual worlds before moving on to mixed ones. We include a brief review of

this technology here, since it is bound to become accessible to experimenters in the near future.

There are some experiences that just beg for a physical presence. Fidopiastis et al. [2006]

provide a compelling example of this in an application to cognitive rehabilitation, depicted in Figure

3. The left monitor shows one view through the HMD; the middle monitor shows real-time

tracking of the movements of the patient; and the right monitor shows the participant at home. In

this experience, the goal is to help a person with traumatic brain injury return home and regain some

measure of independence, for instance in the kitchen. Because a pure VR experience is disembodied,

it cannot support activities that require subtle senses of touch, such as making coffee. In contrast, an

MR experience can contain a real coffeemaker, grinder, cups, etc. The use of virtuality here is to

make a generic kitchen look like the home environment of the participant. The hypothesis is that

this makes transfer of learning easier since the practice is occurring in context. Moreover, with no

technical effort, this MR experience can satisfy the olfactory and taste senses (you smell and taste the

coffee).

The technical infrastructure of MR is illustrated in Figure 4, and provides a useful summary

of the components involved. First we start with some video capture of reality, in this case acquired

through cameras set into the front of an HMD. Second, we have to track the position and

orientation of the participant in the environment. This tracking must be done very precisely in order

to properly merge virtual and real content. Third, we have to generate the experience using one or

more software “engines” to modify the environment. This is the stage in which the real environment

is blended with virtual content. Underlying this can be 3D computer-generated images, video of

some physical environment, 2D computer-generated images, special effects of various kinds (e.g.,

real water vapor emanating from a virtual tree or actual warning lights going on and off), and

additional audio enhancements (e.g., ambient sounds like trucks in the distance or spatial sounds like

-13-

burning trees crackling nearby). Fourth, the visual aspects of the merged scene are delivered to the

participant through some rendering device, in this case LCDs embedded on the inside of the HMD.

Of course, not all of these components are needed in every application, but all these capabilities

must be present in any system that purports to provide MR experiences.

MR is not needed to create the experiences we are developing for our case study, but it can

bring another dimension to the task: collaboration (Hughes and Stapleton [2005]). In MR we can

place people in the rich context of a forest that surrounds them, a physical table on which they see a

top-down view of the same forest, and a social setting in which they can discuss decisions and

consequences with neighbors and family who are jointly setting policies and experiencing their

consequences. The hypothesis is that this immersion and the social responsibility of discussing

decisions with others might lead naturally to decisions with more consideration given to the

preferences of others. There is abundant evidence for this hypothesis from laboratory and field

experiments conducted with common pool resource extraction problems (e.g., Ostrom et al. [1994]).

2.4 Virtual Reality and Expert Decision-Making

As part of the methodology of VX we are proposing comparing the decisions made by

experts and non-experts. The decision making process of expert decision makers have been studied

by researchers in the field of Naturalistic Decision Making (NDM; Klein [1998] and Salas and Klein

[2001]). The term “naturalistic” here refers to the desire to study these decision makers in their

natural environment to learn how the decisions depend on naturally occurring cues. Experts are

particularly good at making efficient and accurate decisions in their field of expertise under dynamic

conditions of time pressure and high stakes. Crucial elements in such decision making are contextual

cues, pattern recognition, and template decision rules. Contextual cues are critical to this pattern

matching process in which the expert engages in a type of mental simulation. Theories emerging out

of this literature suggest that experts employ forward-thinking mental simulations in order to ensure

3 Although NDM research suggests that option generation and evaluation is not used by experts inthe classical sense of decision making, that research has explored the rapid decision processes engaged byexperts in time-stressed situations. Although experts in such situations may rely more heavily on perceptualand pattern-recognition processes, the simulation process thought to be engaged by experts after the firstoption is generated, based upon pattern matching, can be construed of as a form of evaluation. Our pointhere is that NDM studies of experts suggests that there is a heavy reliance on perceptual information in thedecision process. Our interest is in the extent to which a lack of this knowledge, for naïve or novice decisionmakers, can be overcome through the use of advanced technologies such as VR.

-14-

that the best option is selected. We hypothesize that the VR environment can be used to generate

cues that are sufficiently natural and familiar that decisions will be significantly more like those that

would be generated in the field with sufficient expertise.3

We are interested in experts for two reasons. First, rather than having non-experts rely on

text or static images, as is the case in traditional CVM, we want to compare their decisions to those

of experts to see if the synthetic environment of a VR can be a partial substitute for actual field

experience and make non-experts’ decisions more like those of experts. This would be consistent

with VR generating a sufficiently immersive environment for detecting and processing the cues that

are essential for efficient decisions and would therefore imply that VX has external validity. Second,

a better understanding of the decisions made by experts themselves can be facilitated by generating

much larger samples through VX than what can be done through on-site observations.

Comparisons of experts and non-experts are important because in many economic policy

issues the preferences and recommendations expressed by these two groups are often in conflict.

Nevertheless, both provide important inputs into many policy making processes. The conflict that

arises between the two sets of recommendations is at least partially due to differences in the

perceptions of the problem at hand, and we hypothesize that generating experiences through VR

may serve to generate converging perceptions and therefore also recommendations.

-15-

3. The Value Added of Virtual Experiments

The main use of VX is to generate counter-factual dynamic scenarios with naturalistic field

cues and scientific realism. This contrasts sharply to the standard presentation frames of the

mainstay technology of environmental valuation. The most popular method is the CVM that

presents respondents with textual and pictorial descriptions of hypothetical counter-factual

scenarios. A CVM presents information in a static manner, providing cues that are more artefactual

than the dynamic cues provided in VX. Due to the presentation frame, a CVM is also weaker in

terms of scientific realism, for the simple reason that such realism is often quite complex and

difficult to convey in text form. Further, in VX the information acquisition process is active rather

than passive: the respondent chooses what to view and how to view it by moving through the

environment. In the case of forest fires, for example, respondents can choose whether to view the

fire during a fly-over or by walk-through, from a “safe” distance close to the burning forest.

3.1 Frontier of Visualization in Environmental Valuations

Many environmental valuation techniques provide respondents with some visualization of

the choice options. VX provides a rich extension to such practices by adding dynamic rendering

over time, properly constrained by scientific theories that predict the future paths, possibly in a

stochastic manner.

Bateman et al [2006] and Jude et al. [2006] use visualization methods for coastal areas to

design hypothetical choice tasks for the respondent, with two basic designs. In the first respondents

are shown 2D photo-realistic static side-by-side images of “before and after” scenarios. In the

second they are shown 3D visualizations where they can do fly-overs of the landscapes but that have

less realism (see Figure 6). The idea in this method is to import Geographic Information Systems

(GIS) data for a naturally occurring area, render that area to the respondent in some consistent

manner, and then use a simulation model to generate a counter-factual land use. Figure 5 shows

4 In VR one important distinction made is between egocentric and exocentric views. For ourpurposes, the egocentric view is most like the walk-through perspective and the exocentric view is most likethe fly-over. These distinctions are explored in order to understand their impact on general presence in theenvironment, as well as the impact on performance in a variety of tasks (Darken and Cevik [1999]).

-16-

some of the introductory stimuli presented to their respondents, to help locate the property on a

map, and to have a photographic depiction of the property. Figure 6 shows the critical “before and

after” comparisons of the property, using their VR rendering technology. The top panel in Figure 6

shows the property as it is now, from three perspectives. The bottom panel in Figure 6 shows the

property in the alternative land use, rendered from the same perspective. In this manner the

respondent can be given a range of counter-factual alternatives to consider. From interviews with

coastal management organizers that were presented with these visualizations, Bateman et al. [2006]

conclude that VR adds value over traditional methods.

Using VX extends this methodology by allowing participants to manipulate the perspective

of the visual information even further. Participants can wander around the property and examine

perspectives that interest them. Some people like the perspective of a helicopter ride, some like to

walk on the ground, and most like to be in control of where they go.4 It is desirable not to “force

feed” the participant with pre-ordained perspectives so that they can search and find the cues that

are most valuable to their decision making process. Of course, this puts a much greater burden on

the underlying VR rendering technology to be able to accommodate such real-time updates, but that

technology is available (as modern gaming illustrates). To an economist it also raises some

interesting questions of statistical methodology, in which participants self-select the attributes of the

choice set to focus on. There is a rich literature on choice-based sampling, and such methods will be

needed for environments that are endogenously explored (e.g., see Cameron and Trivedi [2005;

§14.5]).

In fact, such extensions have in part been undertaken. Bishop [2001] and Bishop et al. [2001]

allowed their respondents to visualize a recreational area in Scotland using 3D rendering, and walk

5 An equally interesting possibility might be to turn the ecological clock backwards, allow participantsto change past policies, and then run the clock forward conditional on those policies instead of the ones thatwere actually adopted. Then the participant can see the world as it is now and compare it to what it mighthave been, perhaps with greater salience than some notional future world (however beautifully rendered).

-17-

virtually wherever they wanted within that area. They recorded the attributes of the locations the

respondents chose, and then modeled their choices statistically. The attributes of interest to them,

such as how many pixels are in the foreground trees as compared to the background trees, are not of

any interest to environmental economists, and we have our own way of modeling latent choices such

as these, but the methodological contribution is clear. Moreover, these exercises were undertaken as

one part of a broader conception of what we would call a VX (see Bishop and Gimblett [2000]).

One general elicitation problem that is common to many environmental issues is the

temporally latent nature of the impacts of choices made today. Examples in terms of health impacts are

clear, as we know all too well from tobacco, where the decision to start smoking is invariably made

in the teens, and the consequences invariably felt after the age of 50. One role for VR is to be able to

speed up such processes in a credible, naturalistic manner. It is one thing to tell a teen smoker that

there will be consequences from a lifetime of smoking, but if one can show the teen these

consequences then one would expect, or at least hope for, a more informed decision.

Perhaps a simpler example, at least in terms of the demands on rendering technology, would

be to display the longer-run consequences for the environment of certain policies. Many researchers

are providing numerical projections of the effects of failing to mitigate the risk of global warming,

but it has been a standard complaint from advocates of the need for action that people find it hard

to comprehend the nature or scope of the possible consequences. If we have access to VR systems

that can display forests, for example, and we have projections from climate change models of the

effects of policies on forests, then one can use the VR setting to “fast forward” in a manner that

may be more compelling than numbers (even monetary numbers).5 The need for a simulation model

of the economic effects of climate change is stressed in the Stern Review on the Economics of Climate

-18-

Change (Stern [2006; p. 253]). Of course, one positive side-benefit of undertaking this activity

properly would be a reflection of the uncertainty over those future predictions.

Finally, the immersive capacity of a full-scale VX is much greater than that in any of these

studies, especially if dome screens or HMDs are used. Since the immersive capacity depends

crucially on the synchronization of the actions by the participant and the response of the simulation,

the amount of real-time updating that is required can be quite large. This requirement increases

substantially with the amount of endogenous changes that the environment is undergoing in real

time. Fast computers and high-end graphics cards will be essential for these exercises, but the

advantages in terms of the natural field cues generated and the immersiveness of the simulations will

justify such expense. Moreover, the rapidly increasing power of commodity processors and graphics

cards makes this cost minimal, as the performance of today’s high-end machines becomes the

standard for next year’s entry level systems.

The potential value-added of including VR in the information given to participants is clear.

Nevertheless, many problems that are present in traditional valuation exercises are not resolved by

the use of VR, and in some cases responses may become even more sensitive to the presence of

these problems. We discuss some of these issues next.

3.2 Naturally Occurring Risk

Many economic policy assessments involve a great deal of uncertainty, either because the

impact is stochastic in nature or because of uncertainty in the estimation of impacts. VX can

facilitate the representation of uncertain prospects and can do so in a naturalistic way that allows

respondents to use cues that are familiar to them.

One of the most severe problems in the communication and assessment of risk is the

existence of “background risk” that might be correlated with the “foreground risk” of interest. To

take a canonical case in environmental policy, consider the elicitation of the value that a person

6 Many studies now jointly consider mortality and morbidity risk. The need to do so was an earlytheme from Gerking et al. [1988], who elicited information on morbidity and mortality risk in their surveys.

-19-

places on health, a critical input in the cost-benefit assessment of environmental policy such as the

Clean Air Act (U.S. Environmental Protection Agency [1997]). Conventional procedures to measure

such preferences focus on monetary values to avoid mortality risk, by asking respondents to value

scenarios in which they face different risks of death. The traditional interpretation of responses to

such questions ignores the fact that it is hard to imagine a physical risk that could kill you with some

probability but that would have no effect whatsoever on your health if you survived.6 Of course,

such risks exist, but most of the environmental risks of concern for policy do not fall into such a

category. In general, then, responses to the foreground risk question should allow for the fact that

the participants likely perceived some background risk (Harrison, List and Towe [2007]). In many

field settings it is not possible to artificially identify attitudes towards one risk source without

worrying about how the respondents view that risk as being correlated with other risks. It is

implausible to ask respondents their attitude toward one risk without some coherent explanation as

to why a higher or lower level of that risk would not be associated with a higher or lower risk of the

other.

A virtue of VX is to encourage richer lab designs by forcing the analyst to account for

realistic features of the natural environment that have been placed aside and that involve the

correlations of foreground with background risks. In a VX one can hope to gain some control over

this characteristic of risk in the field, since VX introduces a natural way to simulate the presence of

background risk in ways that are salient.

3.3 Scenario Rejection: the “R” in VR

For some valuation scenarios the counterfactuals may produce perceptually very similar

consequences. In order to generate greater statistical power researchers may then simulate physical

7 This term is an unfortunate short-hand for situations in which the participant views the scenariopresented in an instrument as incredible for some reason, and hence rejects the instrument. There is animportant semantic problem, though. If “incredible” just means “with 0 probability,” then that is somethingthat deserves study and can still lead to rational responses. There is nothing about a zero probability thatcauses expected utility theory to be invalid at a formal level. But from an experimental or survey perspective,the term refers to the instrument itself being rejected, so that one does not know if the participant hasprocessed the information in it or not. This semantic issue is important for our methodology, since we aretrying to mitigate scenario rejection in artefactual instruments. Hence we do not want to view it as an all ornothing response, as it is in some literature in environmental valuation using surveys.

-20-

or economic changes that are not consistent with scientific predictions. That is, the counterfactual

scenarios may not make sense from a physical and economic perspective. Violating such constraints,

however, can lead participants to “scenario rejection.”7 This problem is bound to be especially

severe in situations with which the participant is very familiar, such as when expert decision makers

are put into VR simulations. Underlying the rendering of the visual cues there must therefore be a

scientifically consistent simulation model: in effect, putting the “R” into VR.

Another reason for “scenario rejection” may be an inappropriate, or vaguely stated, choice

process. This is not a problem specific to the VR setting but occurs for all valuation instruments.

Why am I being asked to choose between these two options? Is this a referendum vote? If so, am I

the pivotal voter? Or am I the King (it is good to be King), who gets to say what the land use will be

no matter what others think? Every one of these questions needs some answer for the responses to

be interpretable (Harrison and Kriström [1996]). In addition, it is important to remove auxiliary

motivations for specific answers, such as moral approval by the experimenter. Carson et al. [1998]

and Krosnick et al. [2002] report on a replication and extension of an earlier Exxon Valdez oil spill

survey. They find that, when the response format is completely anonymous in the sense that it is

clear the experimenter cannot identify the response of any individual, the vote in favor of the policy

proposal presented drops by a significant amount. Specifically, they considered naturalistic features

of the voting environment in America, in which respondents were given a voting box to place their

private vote, and had the option of simply not voting. The interaction of these treatments had a

dramatic effect on willingness to vote for the referendum, such that the implied valuation for the

-21-

Exxon Valdez clean-up policy plummeted (Harrison [2006]). One could readily imagine such

referenda occurring in Second Life. The general point is that such naturalistic features of a task

environment are exactly the sorts of things that VR can handle well, and make the environment less

artefactual.

3.4 VR Simulations as Data

There is also a valuable role for simulations of some latent process, even if one does not take

the extra step of rendering the implied environment as VR. Bin et al. [2006] illustrate how one can

use detailed GIS information on a set of parcels of land to simulate attributes of the view that is

likely correlated with the land and a building of known height. In effect this is simulating a virtual

person looking out of the property, in this case in the direction of an amenity, the ocean, although

the user of the simulation software cannot view the amenity. They measure the number of degrees

of ocean view associated with each property from the perspective of this virtual viewer, and use that

information in a traditional hedonic model to identify the value of the amenity. This also provides

better estimates of the value of disamenities, such as flood surge risk, that tend to be highly

correlated with that amenity. The fact that the virtual viewer takes into account virtual obstacles to

the view makes these statistical controls significantly less correlated than if one did not use the VR

simulation, enhancing the power of statistical inference.

More generally, undertaking simulations of stochastic processes can allow one to study

decision-making in environments where outcomes are well defined but the probabilities are the

result of some physical process. In a trivial case one could use this procedure to simulate the rolling

of a die in a standard lottery experiment, although it would be redundant since we “know” the

probabilities without having to simulate the process. But the same procedure generalizes to settings

in which the physical process can be written down and simulated on a computer even if the specific

probabilities are not known directly to the decision-maker (or the observer). If one then makes the

-22-

testable assumption that the decision-maker behaves as if he knows the true probabilities, it is

possible to examine behavior using standard econometric methods. One striking example is

provided by television game shows, which offer contestants large prizes as a function of stochastic

processes that can be relatively easy to simulate; see Andersen et al. [2007b] for a review of such

environments, and an example of the use of simulations when estimating risk attitudes.

4. A Case Study: Evaluating the Risks of Wildfires

The policy context we consider, to illustrate the application of virtual experiments, is the

assessment of how ordinary households and trained experts evaluate the risks and consequences of

forest fire. Fires can occur naturally, such as from lightning, or they can occur in a prescribed burn.

Recent policy of the U.S. Forest Service has been to undertake prescribed burns as a way of reducing

the “fuel” that allows uncontrolled fires to become dangerous and difficult to contain, although

there has been considerable variation in policy towards controlling wild fire over the decades (Pyne

[2004]). Additionally, many private citizens and local communities undertake such prescribed burns

for the same purpose. The benefit of a prescribed burn is the significant reduction in the risk of a

catastrophic fire; the costs are the annoyance that smoke causes to the local population, along with

the potential health risks, the scarring of the immediate forest environment for several years, and the

low risk of the burn becoming uncontrolled. So the policy decision to proceed with a prescribed

burn is one that involves balancing uncertain benefits against uncertain costs. These decisions are

often made by individuals and groups with varying levels of expertise and inputs, often pitting

political, scientific, aesthetic and moral views against each other.

Decisions about prescribed burns are not, however, typically made directly by residents.

Instead, there are teams of experts from various disciplines and government agencies involved in

these decisions. To be sure, they are often charged with listening to residents and taking their

concerns into account. Ultimately, however, residents exert significant influence on the policy

8 The notion of a “prescribed burn” also includes allowing a naturally occurring fire to spread but in acontrolled manner. Some of the most controversial wildfires have resulted from such fires not being subjectto a clear plan for control before the fire began. Two particularly well-known fires that got out of control havegiven prescribed burns a bad name in some circles: the 1998 Yellowstone fire and the 2000 Los Alamos fire.

-23-

outcome through the political process. The use of prescribed burns as a forest management tool is

extremely controversial politically, and for understandable reasons.8 Many homeowners oppose

prescribed burns if they are too close to their residences, since it involves some risk of unplanned

damage and ruins the aesthetics of the area for several years. Conversely, other residents want such

burns to occur because of the longer-term benefits of increased safety from catastrophic loss, which

they are willing to balance against the small chance of localized loss if the burn becomes

uncontrolled. And there are choices to be made about precisely where the burns should go, which

can have obvious consequences for residents and wildlife. Thus, the tradeoffs underlying household

decisions on whether to support prescribed burns involve their risk preferences, the way they

trade-off short-term costs with long-term gains, their perception of the risks and aesthetic

consequences of the burns, and their view of the moral consequences of animal life lost in human-

prescribed burns versus the potentially larger losses in uncontrolled fires.

So the policy context of this case study has most of the characteristics that one finds in other

environmental policy settings. The standard economic model of decision making, EUT, suggests

that decisions are determined by the subjective valuation of each possible outcome, perceptions of

the value of outcomes over time as well as their subjective discounted values, perceptions of the

extent and distribution of the risk, and subjective risk preferences. A key question is how the

experts’ decisions differ from those that residents make and what influences these differences. Is it

due to differences in their risk preferences, their willingness to trade-off short-term costs (to others)

and longer-term benefits (to others), or their perception of the risks and aesthetic consequences of

the burns? Or is it due to the extensive experience that the experts may have in forest management,

making it perhaps easier for them to conceive the range of consequences and the risks involved in

-24-

various actions? As discussed earlier, NDM has shown that experts employ contextual cues, pattern

recognition, and template decision rules in a very efficient manner, engaging in forward-thinking

mental simulations to choose the best option. By understanding this process better we can formulate

models of how non-experts may learn through synthetic experiences similar to the real experiences

of experts.

4.1 Representation as a Policy Lottery

Forest fire management options can be viewed as choices with uncertain outcomes. In the

language of economists these options are “lotteries” or “prospects” that represent a range of final

outcomes, each with some probability, and hence can be viewed as a policy lottery. One final

outcome in this instance might be “my house did not burn down in the 5 years after I made my

choice” or “my house did burn down in the 5 years after I made my choice.” Many more outcomes

are obviously possible: these are just examples to illustrate concepts. Contrary to decisions involving

actual lottery tickets, a policy lottery has outcomes and probability distributions that are not

completely known to the agent. We therefore expect the choices to be affected by individual

differences not just in risk preferences, but also in risk perceptions.

The canonical laboratory choice task that experimental economists use to identify risk

preferences in a setting like this is to present individuals with simple binary choices. One choice

option might be a “safe” lottery that offers outcomes that are close to each other but different: for

example, $16 and $20. Another choice option would then be a “risky” lottery that offers outcomes

that are more extreme: for example, $1 or $38.50. If each outcome in each lottery has a ½ chance of

occurring, and the decision-maker is to be paid off from a single realization of the lottery chosen,

then one cannot say which is the better choice without knowing the risk preferences of the decision

maker. In this example a risk neutral individual would pick the risky lottery, since the expected value

is $19.75, compared to the expected value of $18 for the safe lottery. So someone observed picking

-25-

the safe lottery might just be averse to risk, such that the expected increment in value of $1.75 from

picking the risky lottery is not enough to compensate for the chance of having such a wide range of

possible outcomes. A risk averse person is not averse to having $38.50 over $1, but just to the

uncertain prospect of having one or the other prior to the uncertainty being realized.

Armed with a sufficient number of these canonical choice tasks, where the final outcomes

are monetary payments and the probabilities are known, economists can estimate the parameters of

alternative latent choice models. These models generally rely on parametric functional forms for key

components, but there is now a wide range of those functional forms and a fair understanding of

how they affect inferences (e.g., Holt and Laury [2002] and Harrison, Lau and Rutström [2007]).

Thus one can observe choices and determine if the individual is behaving as if risk neutral, risk

averse, or risk loving in this situation. And one can further identify what component of the decision

making process is driving these risk preferences: the manner in which final outcomes are valued by

the individual, and/or the manner in which probabilities are transformed into decision weights.

There remains some controversy in this area, but the alternative approaches are now well

established.

Our research may be viewed as considering the manner in which the lottery is presented to

respondents, and hence whether that has any effect on the way in which they, in turn, perceive and

evaluate it. Outcomes in naturally occurring environments are rarely as crisp as “you get $20 today.”

And the probability of any one outcome is rarely as simple and discrete as ½, a, ¼ or c.

One difficulty with using more natural counterparts to prizes or probabilities is that control

is lost over the underlying stimuli, with consequent loss of internal validity. This makes it hard, or

impossible, to identify and estimate latent choice models. The advantage of VX is that one can

smoothly go where no lab experiment has gone before: the same virtual stimuli can be set to be

artefactual and crisp, and then steadily one can make the stimuli more and more natural in affect and

comprehension. For example, if the probability distribution is actually a smooth density defined over

-26-

a continuous set of possible outcomes, one can “discretize” it and present that to the respondents.

By varying the coarseness of the discrete representation one can see how that affects behavior.

4.2 Affecting Perceived Probabilities

One of the things that is immediately apparent when working with VR, but unexpected to

outsiders, is the “R” part. Mimicking nature is not a matter left solely to the imagination. When a

participant is viewing some scene, it is quite easy to generate images in a manner that appear unusual

and artefactual. That can occasionally work to one’s advantage, as any artist knows, but it can also

mean that one has to work hard to ground the simulation in certain ways. At one affectual level

there might be a concern with “photo-realism,” the extent to which the images look as if they were

generated by photographs of the naturally occurring environment, or whether there is rustling of

trees in the wind if one is rendering a forest.

We utilize software components that undertake some of these tasks for us. In the case of

trees and forests, a critical part of our application, we presently use SpeedTree to undertake most of

the rendering (http://www.speedtree.com/). This commercial package requires that one specify the

tree species (including miscellaneous weeds), the dimensions of the tree, the density of the trees in a

forest, the contour structure of the forest terrain, the prevailing wind velocity, the season, and a

number of other characteristics of the forest environment. We discuss how that is done in a

moment. This package is widely used in many games, including Tiger Woods PGA Tour, Call of Duty 3,

and of course numerous fantasy games. The forest scene in Figure 7 illustrates the capabilities we

needed for a walk-through, and is from the game The Elder Scrolls IV: Oblivion. The image that is

rendered allows for real-time updating as the participant moves through the forest, and handles light

and shade particularly well from this walking perspective. As the participant gets closer to a specific

tree, it renders that in greater detail as needed.

One can also take a helicopter tour of a forest. Figures 8 and 9 were each produced by our

-27-

software that uses SpeedTree as its underlying renderer. These are each scenes from the same forest

used in our application, first from a distance, and then close-up. Figure 8 displays many thousands

of trees, but is based on a virtual forest covering 18.8 square miles and over 5 million trees and

shrubs. So the helicopter could travel some distance without falling off the end of this virtual earth.

Figure 9 is a close-up of the same forest. The point is that this is the same forest, just seen from two

different perspectives, and the “unseen forest” would be rendered in a consistent manner if the user

chose to go there.

This last point is not trivial. There is one underlying computer representation of the entire

forest, and then the “window” that the participant looks at is rendered as needed. In the case of the

sample forest shown in Figures 8 and 9, the rest of the forest is rather boring and repetitive unless

you are a squirrel, but one can add numerous variations in vegetation, topography, landmarks, and

so forth. All of these factors may play important roles in determining the dynamics of the VR

experience.

To add such factors and their effects on the evolution of the scenarios, one links a separate

simulation software package to the graphical rendering software that displays the evolution of a

landscape. Since we are interested in forest fires, and how they spread, we can use one of several

simulation programs used by professional fire managers. There are actually many that are available,

for different fire management purposes. We wanted to be able to track the path of a wildfire in real

time, given GIS inputs on vegetation cover, topography, weather conditions, points of ignition, and

so on. Thus, the model has to be able to keep track of the factors causing fires to accelerate, such as

variations in slope, wind or vegetation, as well as differences between surface fires, crown fires, and

the effects of likely “spotting” (when small blobs of fire jump discretely, due to the effects of wind

or fleeing animals).

For our purposes the FARSITE software due to Finney [1998] is ideal. It imports GIS

information for a given area and then tracks the real-time spread of a fire ignited at some coordinate,

9 This is an important feature of the data underlying the Florida Fire Risk Assessment System of theDivision of Forestry of the Florida Department of Agriculture & Consumer Affairs, documented athttp://www.fl-dof.com/wildfire/wf_fras.html.

-28-

say by lightning or arson. Figure 10 shows the GIS layers needed to define a FARSITE simulation.

Some of these, such as elevation and slope, are readily obtained for most areas of the United States.

The “fuel model” and vegetation information (the last four layers) are the most difficult to specify.

Vegetation information can be obtained for some sites, although it should ideally be calibrated for

changes since the original survey.9 The fuel model defines more precisely the type of vegetation that

is present at the location. The most recent tabulation of 40 fuel models is documented by Scott and

Burgan [2005]. One formal type of vegetation is water, for example, which does not burn. One of

the other fuel models is shown in Figure 11: photographs help one identify what is described, and

then the fuel model implies certain characteristics about the speed with which a simulated fire burns

and spreads. In this case we have an example of the type of fuel load that might be expected the year

after hurricanes had blown down significant forest growth, a serious problem for the wildfire cycle

in Florida.

The output from a typical FARSITE simulation is illustrated in Figure 12. The fire is

assumed to be ignited at the point indicated, and the path of the fire as of a certain time period is

shown. The output from this simulation consists of a series of snapshots at points in time defining

what the location and intensity of the fire is. We can take that output and use it for detailed visual

rendering of the fire as it burns. The output from FARSITE is information on the spread of the

forest fire, but our rendering also uses the inputs to FARSITE to “set the stage” in a manner that is

consistent with the assumptions used in FARSITE. For example, we use the same topography and

wind conditions as assumed in FARSITE. Thus the underlying numerical simulation of the path and

severity of the fire are consistent with the visual rendering to the decision-maker.

We can then take that output and render images of the forest as it burns. Using SpeedTree as

-29-

the basic rendering module, we have adapted its software for our needs. This has included the

incorporation of other landscape components, smoke, fire and the effects that fire has on the

appearance of the forest, both in altering illumination and causing damage. The images in Figure 13

display the evolution of a wildfire, initially from a distance, then close up, and finally after the front

of the fire has passed. These images have been generated using SpeedTree and our own programming,

based on the simulated path of the fire from FARSITE, conditional on the landscape. Thus we have

simulated a GIS-consistent virtual forest, with a fire spreading through it in a manner that is

consistent with one of the best models of fire dynamics that is widely used by fire management

professionals, and then rendered it in a naturalistic manner. The participant is free to view the fire

from any perspective, and we can track that; or we can force feed a series of perspectives chosen

beforehand, to study the effect of allowing endogenous information accumulation.

The static images in Figure 13 are actually snapshots of an animation which plays out in real

time for the participant. This dynamic feature of the evolution of the fire is one reason that a model

such as FARSITE is needed, since it computes the path and intensity of the fire, given the

conditions of the environment. The dynamic feature also adds to the experience that the participant

has, in ways that can again be studied orthogonally in a VX design. In other words, it is a relatively

simple matter to study the effects of presenting static images versus a dynamic animation on

behavioral responses.

4.3 The Virtual Experimental Task

We now have all of the building blocks to put together the task that will be given to

participants in our VX. The logic of the instrument we have developed can be reviewed to see how

the building blocks fit together. We focus on our value added from a methodological perspective,

building on existing contingent valuation surveys of closely related matters due to Loomis et al.

[2002][2005]. Detailed instructions are available in an appendix (available on request).

-30-

The initial text of the instrument is standard, in the sense of explaining the environmental

and policy context, the design of forest fire management in Florida. We explain the risks from wild

fires, as well as the opportunity cost of public funds allocated to prescribed burns already. The

participant is introduced to a policy option of expanding the prescribed burn policy from 4% to 6%

of the forest area. Typical damages from the devastating 1998 wild fires in Florida are explained, and

represented as roughly $59,000 per lost home. Other damages are discussed, such as lost timber,

health costs, and lost tourist income.

We then specifically introduce prescribed burning as a fire management tool that can reduce

the frequency and severity of fires. The idea of VR computer simulation is introduced, with

explanations that the predicted path depends on things such as topography, weather, vegetation, and

ignition points. Participants are made familiar with the idea that fires and fire damages are stochastic

and can be described through frequency distributions. The distributions that are presented to them

are generated through Monte Carlo simulations using the FARSITE model, as described earlier. The

participants then experience several dynamic VR simulations of specific wild fires, rendered from the

information supplied by FARSITE simulations that vary weather and fuel conditions. We selected

these simulations to represent extremes in the distribution of fire damage, and the participants are

told this.

Participants are told that they have a property in this area that is subject to wild fire, and that

they must make a decision whether to pay for an enhanced prescribed burn policy or not, which

would reduce the risk that their property would burn. The information about risks to their property

that participants receive is threefold.

First, they are told that the background uncertainties are generated by (a) temperature and

humidity, (b) fuel moisture, (c) wind speed, (d) duration of the fire, and (c) the location of the

ignition point. They are also told that these uncertainties are binary for all but the last, which is

ternary; hence that there are 48 background scenarios. Reasonably specific values for these

-31-

conditions are provided (e.g., low wind speed is 1 mph, and high wind speed is 5 mph). And it is

explained that these background factors will affect the wild fire using a computer simulation model

developed by the U.S. Forest Service to predict the spread of wild fire. Thus participants could use

this information, and their own sense of how these factors play into wildfire severity, to form some

probability judgments about the risk to their property. We fully appreciate that only fire experts

would likely have the ability to translate such information into relatively crisp probabilities. But the

objective is to provide information in a natural manner, akin to what would be experienced in the

actual policy-relevant choice, even if that information does not directly “tell” the participant the

probabilities.

Second, the participant is shown some histograms displaying the distribution of acreage in

the whole area that is burnt across the 48 scenarios. Figure 14 shows the histograms presented to

participants. The instructions provided larger versions, and explained slowly how to read these

graphs. The scaling on the vertical axis is deliberately in terms of natural frequencies defined over

the 48 possible outcomes, and the scaling of the axes of the two histograms is identical to aid

comparability. The qualitative effect of the enhanced prescribed burn policy is clear, to reduce the

risk of severe wild fires. Of course, the information here is about the risk of the entire area burning,

and not the risk of the property burning, and that is pointed out clearly in the instructions.

Third, they are allowed to experience several of these scenarios in a VR environment that

renders the landscape and fire as naturally as possible. We provide some initial training in navigating

in the environment, which for this software is essentially the same as in any so-called “first person

shooter” video game. The mouse is used to change perspective and certain keys are designated for

forward, backward, and sideward movements. The participant practices how to walk or fly around

the landscape, and navigate back to their property. This instruction occurs within an environment

that is stationary in time, and thus there is no wild fire spreading. To encourage navigation skills, and

to get a measurement of the individual’s skill at navigation, a test is done at the end of this training

-32-

where a small prize is offered if the participant can navigate back to their property from one

predetermined corner of the landscape within a certain period of time ($5 within 30 seconds, $2

within 1 minute, and $1 within 2 minutes). After this navigational training, the participant is

presented with 4 scenarios, which are described as high-risk and low-risk environments, each with or

without the enhanced prescribed burn policy in effect. The participant is then free to explore the

environment, the path of the fire, and the fate of their property.

Participants are then introduced to the choice problem they face. In order to make the

choice salient, we pay them according to damages to one particular area in the region being

simulated. They are assigned this area as their property, and told that they have a credit of $80. If

their property burns, they would lose $59, which is simply scaled down from the $59,000 value of an

average home lost to fire in Florida. They have the option of contributing to the enhanced

prescribed burn policy, which would reduce the risk of their property burning, but it would cost

them some money up front – non-refundable in the event that their property does not burn, of

course. We presented the participant with an array of 10 possible costs for the policy, between $2

and $20 dollars in increments of $2. In each case the participant was asked to simply say “yes” or

“no” as to whether they wanted to contribute to the policy at that cost. This elicitation procedure is

recognized in the experimental economics literature as a multiple price list (Andersen et al.

[2006][2007a]), and is well known to environmental economists from contingent valuation surveys

(Mitchell and Carson [1989; p. 100, fn. 14]). The participant was told that one of these costs would

be selected at random, using a 10-sided dice, and the policy that had been selected would be put in

place. After the cost was selected at random, the participant was told that we would then select each

of the background factors using the roll of a die for each factor, so that one of the 48 background

scenarios would be selected. Thus the participant’s choices, plus the uncertainty over the

background factors affecting the severity of a fire conditional on the chosen policy, would determine

final earnings.

-33-

Panel A of Table 1 displays the payoff matrix in a conventional instrument for eliciting risk

attitudes, and is a scaling by a factor of 10 of the baseline instrument used by Holt and Laury [2002].

The participant picks option A or B in each row, and one row is selected at random to be played out

in accord with these choices. A risk neutral participant would select the safe option A until row 5,

and then switch to the risky option B, as can be seen from the expected value (EV) information on

the right of the table. A risk averse (risk loving) participant would switch from option A after

(before) row 5.

Panel B of Table 1 shows the comparable payoff matrix that is implied by our VX

instrument, assuming that the participant accurately infers the true probabilities of his own property

burning. Panel C shows the matrix if we assume that the participant infers the probability from the

incidence of fire as displayed in Figure 14 instead. Of course, these are implied payoffs, and the

participant does not get to see a decision sheet in this artefactual manner: that is actually the whole

point of the VX exercise. And we do not know if or how the participant uses the information in

Figure 14 in conjunction with heuristics or cues obtained from the VR experience. Again, this is one

of the reasons we implement the VX task in the first place.

In panel B of Table 1 we see that the true probabilities of the house burning are 0.06 if there

is an enhanced prescribed burn policy in effect, and 0.29 if the participant chooses not to contribute

to the policy. In panel C these probabilities are 0.013 and 0.16, respectively. The reason for the

difference is the location of the property, and the manner in which randomly generated fires “bear

down on the property” in this location. This is entirely expected and natural, and could have been

inferred to varying degrees from the information in the VR experience. In fact, it provides a good

test of the extent to which participants process the information in the VR experience, as distinct

from the “words, numbers and pictures” in the written instructions. Based on the difference in

expected values of the two options shown in Table 1 we can see that, conditional on their risk

attitude, the more a participant adjusts his perception of the probability of his property burning in

10 We did not include items that referred to auditory and other sensory experiences since they werenot part of our VR environment.

-34-

the direction of the true one, the more he is willing to pay for the prescribed burn policy. The

difference in EV between the two options is much higher in panel B than in panel C.

In order to control for the risk attitude of the participant we also give them a standard

lottery task after they are finished with the fire policy task. The lottery choices we offer are those

presented in panel A of Table 1, although participants are not given the information on the expected

values. Participants are paid for both of these tasks, as well as for the navigation training task, in cash

at the end of the experiment.

In order to conduct a preliminary assessment of the user experience with the virtual

environment we also collected subjective data on perceptions of presence. In VR research the

concept of presence is used to assess the degree to which the artificial environment engages

participants’ senses, while capturing attention and supporting an active involvement on the part of

the user (Insko [2003]; Witmer et al. [1998][2005]). Research in this area has sought to understand

how the virtual environment mediates this experience through factors such as the fidelity of the

presented world, the task executed in the environment and the interactions required by the task.

Concepts such as involvement and immersion are additionally used to capture the degree to which the

user’s experience is altered by the VR environment. Involvement has to do with user attention on

the presented stimuli, and immersion describes “a psychological state characterized by perceiving

oneself to be enveloped by, included in, and interacting with an environment that provides a

continuous stream of stimuli and experiences.” (Witmer et al. [2005; p. 299]). For our initial

assessment, we used 23 appropriate tasks from a 32-item questionnaire reported in Witmer et al.

[2005].10 Using principal-components analysis, they found a best fit with a four-factor model

consisting of Involvement, Adaptation/Immersion, Sensory Fidelity, and Interface Quality. An

appendix (available on request) lists the factors and the corresponding questions used to identify the

-35-

factor.

We also ask some questions about prior exposure to VR simulations via video games. We

suspect that such experience influences the ease with which a user navigates the environment, and

that this has consequences for the extent to which participants feel a sense of presence. In addition

we have observations on the time that the participant uses in the paid navigation test as a measure of

the ease of navigation.

4.4 A Pilot Study

We report here the findings from a pilot study undertaken using GIS data from the Ashley

National Forest in Utah. Our pilot participants were recruited from faculty and staff at the

University of Central Florida, and from residential sites in the urban interface of Orlando. All

procedures were as described above. The complete session consisted of (a) a navigational training

task; (b) the wild fire policy task described above; (c) a traditional risk aversion instrument in the

form of a lottery choice; and (d) a demographic survey and the Presence questionnaire. The purpose

of the traditional risk aversion task was to provide independent estimates of risk attitudes, so that we

could draw inferences in the main wild fire task about beliefs without the two being confounded.

That is, we assume for the purposes of identification that the risk attitudes used in the two tasks are

the same, and then draw inferences about the joint interaction of beliefs and risk attitudes in the wild

fire task. The same logic of combining experimental tasks has been used elsewhere to draw

inferences about the discount rate defined over the present value of utility streams, and is becoming

common in behavioral econometrics (e.g., see Andersen et al. [2005]).

Our objective is not to test treatments, but to evaluate the instrument developed since it

contains so many new features. In this sense our sample can be best viewed by environmental

economists as akin to one focus group for the development of a survey instrument. For that reason

we limit ourselves to a small sample of 12 subjects, but collected relatively extensive information on

11 We used the power function version of the CRRA function: u(y) = yr, where r is the coefficient ofrisk aversion. When r=1 a person is risk neutral, 0<r<1 indicates risk aversion, and r>1 indicates risk loving.

12 Andersen et al. [2006] explain that a single multiple switch, in which someone selectsAABABBBBBB, for example, is plausible evidence for indifference between the underlined options. Thissuggests a widening of the implied CRRA interval to match this behavior. However, behavior such asBAABAABABB is much more likely due to misconceptions (or an attempt to spell out a nursery rhyme). Aformal statistical analysis of such data would correctly allow for relatively large standard errors on theestimated risk attitudes of such a subject, or for some modeled heteroskedasticity to explain occurrences ofsuch behavior. In the prescribed burn task, subject #2 chose 2211212221.

-36-

their perception of the VR instrument.

Table 2 displays the raw data of immediate interest. We sort each subject by the number of

prescribed burn choices they made. Each subject could make up to 10 such choices, and the

observed sample ranges from 2 up to 10. The modal choice was 6, which is consistent with the

choices expected of a risk-neutral decision maker who based their beliefs on the true probabilities

(Panel B of Table 1). However, these subjects were generally not risk-neutral, as the data in columns

3-5 of Table 2 shows. The number of safe choices refers to the choice of lottery A in the risk

aversion instrument (Panel A of Table 1), and the implied measures of risk aversion using the

characterization of Holt and Laury [2002; Table 3] is then shown.11 Only one subject (#8) was risk

neutral by this measure, and most were risk averse. Two subjects (#12 and #11) were risk loving,

but from direct observation we do not believe that they properly understood the tradeoffs involved.

These two subjects also appeared not to comprehend the tradeoffs in the prescribed burn

instrument either, and had considerable difficulty with the VR interface (explained below). One

subject (#2) exhibited multiple switches in the risk aversion task and the prescribed burn task, which

is evidence of misconception of the task.12

Thus we would characterize most subjects as risk averse, consistent with the literature using

experimental tests of this kind. This suggests that the rational decision-maker would actually choose

the prescribed burn option more than 6 or 4 times in Panels B and C of Table 1, respectively,

implying a higher willingness to pay (WTP) for prescribed burns. Figure 15 collates the information

we have on risk attitudes with decisions on WTP for the prescribed burn option. The solid line

-37-

shows how risk attitudes would be associated with choices about the prescribed burn option if a

decision maker had used the VX environment to form true beliefs about the risk to the property.

The dashed line shows a comparable set of predictions if beliefs about the risk to the property had

been based solely on the information provided in the written instrument. We see some tendency for

decisions to move towards those predicted with true beliefs, although we stress that these are pilot

experiments with a small sample. This tendency suggests that the VR experience did have some

effect on shifting beliefs towards the true risks that the property would burn.

We find that those subjects who had some prior experience with video games had the least

problems with the VR environment, as expected. Subjects with no time shown in the Time Test

column of Table 2 were unable to find their way “home” after about 3 minutes in the navigation

test, and in most cases abandoned the effort. In three cases (#7, #11 and #12), the subject was

effectively unable to use the VR environment, and the experimenter had to do that for them. The

results in the Time Test column correlate well with video game experience. One implication is that

the VR environment should have some option for such subjects to rely on “movie clips” of the

environment, rather than actively explore it themselves, since this would not require navigational

skills.

Figure 16 shows histograms of the results from asking subjects about the virtual

environment, using the Presence Questionnaire. We translate the responses to the indices measuring

the four factors explained earlier; hence these indices correspond to the scale of 1 through 7 used by

the subjects on the original survey instrument, where 7 indicates the strongest acceptance of the VR.

Research on presence has used indices of this kind in a variety of ways, ranging from comparisons of

differing VR environments, to manipulations within particular environments, to investigations of

how a specific environment produces differences across a given set of presence questions. For

example, Lin et al. [2002] found significantly increasing reports of experienced presence with

increases in field of view presented in a driving simulator-based virtual environment. Sutcliffe et al.

13 One subject, #7, had considerable problems as well, but appeared informally to be providingexperimenter-induced responses to the Presence questionnaire.

-38-

[2005] compared a different set of virtual environments and evaluated the experienced presence

across these environments to make claims about which produced more. They compared a highly

immersive VR CAVE, an Interactive Workbench, and a Reality Room; they found overall average

presence scores ranging from 4.9 for the CAVE, 4.6 for the Interactive Workbench, and 4.4 for the

Reality Room, with the data showing that presence scores in the CAVE were significantly greater.

Others have used the presence indices to determine whether their VR system produced scores that

differ significantly from the neutral value of the scale, with the neutral value being “4.” For

example, in a study of a simulator to train aircraft fault inspection Vora et al. [2001] found that their

system only scored significantly greater than the neutral value on approximately 50% of the

questions; see Vembar et al. [2005] for a similar analysis using VR for medical training. As such, we

are encouraged that our scores produced values ranging from 5.1 to 5.7 for the sub-components,

suggesting that participants did report robust levels of presence within our VX. Three subjects that

did not manage the environment well (#10, #11 and #12), in terms of our “time test” and informal

observation, provided correspondingly low indicators of presence, lending credibility to the accuracy

of the instrument.13

5. Conclusions

We have introduced the concept of virtual experiments (VX) as a way to bridge the

methodological gap between lab and field experiments. VX has the potential to generate the internal

validity of controlled lab experiments with the external validity of field experiments. In particular,

VX can generate the kind of field cues that have been shown to be important inputs in complex

decision making. The central aspect of the VX methodology is a VR environment that makes

participants experience a sense of presence, a psychological state of “being there.” This sense

-39-

depends on the degree of involvement that participants experience as a consequence of focusing

attention on the set of stimuli and activities generated by the VR simulation. Presence also depends

on immersion, the perception that one is enveloped by, included in, and interacting with a

continuously stimulating environment.

VX holds promise as a method for experimentation in the field of environmental and

resource economics where actual field experiments are rare, and where field events often evolve over

long time horizons. In VX participants can experience long run scenarios in a very short amount of

time, and many counterfactual scenarios can be generated. The application we discuss here is the

prevention of forest fire risks, a significant economic problem in many parts of the world. Using VX

we can elicit valuations as well as risk attitudes under conditions that generate important field cues.

Our pilot study results are encouraging, and demonstrate the feasibility and application of this new

methodological frontier.

-40-

Figure 1: Head Mounted Display Figure 2: A Dome Screen

Figure 3: Mixed Reality Used in Cognitive Rehabilitation

-41-

Figure 4: The Technical Infrastructure of Mixed Reality

Figure 5: Introductory Displays from Bateman et al. [2006]

-42-

Figure 6: GIS Rendering of Alternative Land Used, from Bateman et al. [2006]

Figure 7: SpeedTree Rendering from The Elder Scrolls IV: Oblivion

-43-

Figure 8: Rendering of a forest from a distance Figure 9: Rendering of the same forest up close

Figure 10: GIS Input Layers for FARSITE Simulation of Path of a Fire

-44-

Figure 11: Illustrative fuel load model from Scott and Burgan [2005]

Figure 12: Illustrative FARSITE Simulation

-45-

Figure 13: Authors’ Rendering of Simulated Forest Fire

-46-

0

10

20

30

40

Numberof TimesOut of 48

0 5 10 15 20 25 30 35 40 45 50 55 60

Percent of Ashley National Forest that Burned

Effect of Wildfire on Ashley National ForestWith No Prescribed Burn Policy

0

10

20

30

40

Numberof TimesOut of 48

0 5 10 15 20 25 30 35 40 45 50 55 60

Percent of Ashley National Forest that Burned

Effect of Wildfire on Ashley National ForestWith A Prescribed Burn Policy in Place

Figure 14: Representation of Risk Provided to Participants

$0

$2

$4

$6

$8

$10

$12

$14

$16

$18

$20

WTP forPrescribed

Burn

0 .5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5

CRRA

True beliefs Beliefs based on acreage Actual choices

Figure 15: Comparison of Predicted Choices and Actual Choices

-47-

0

2

4

6

8

Freq

uen

cy

1 2 3 4 5 6 7

Index Value

Involvement

0

2

4

6

8

Freq

uen

cy

1 2 3 4 5 6 7

Index Value

Adaption/Immersion

0

2

4

6

8

Freq

uen

cy

1 2 3 4 5 6 7

Index Value

Sensory Fidelity

0

2

4

6

8

Freq

uen

cy

1 2 3 4 5 6 7

Index Value

Interface Quality

Histograms of factors derived from Presence Questionnaire; N=12 subjects

Figure 16: Indices of Presence in the Virtual Environment

-48-

Table 1: Explicit and Implicit Payoff Matrices in Experimental Tasks

A. The Holt and Laury [2002] Risk Aversion Instrument

Lottery A Lottery B EVA EVB Differencep($20) p($16) p($38.50) p($1.00)0.1 $20 0.9 $16 0.1 $38.50 0.9 $1.00 $16.40 $4.80 $11.700.2 $20 0.8 $16 0.2 $38.50 0.8 $1.00 $16.80 $8.50 $8.300.3 $20 0.7 $16 0.3 $38.50 0.7 $1.00 $17.20 $12.30 $4.900.4 $20 0.6 $16 0.4 $38.50 0.6 $1.00 $17.60 $16.00 $1.600.5 $20 0.5 $16 0.5 $38.50 0.5 $1.00 $18.00 $19.80 -$1.700.6 $20 0.4 $16 0.6 $38.50 0.4 $1.00 $18.40 $23.75 -$5.100.7 $20 0.3 $16 0.7 $38.50 0.3 $1.00 $18.80 $27.30 -$8.400.8 $20 0.2 $16 0.8 $38.50 0.2 $1.00 $19.20 $31.00 -$11.800.9 $20 0.1 $16 0.9 $38.50 0.1 $1.00 $19.60 $34.80 -$15.201 $20 0 $16 1 $38.50 0 $1.00 $20 $38.50 -$18.50

B. The Implied Forest Fire Instrument Assuming True Beliefs

Lottery A Lottery B EVA EVB Differencep(safe) p(burn) p(safe) p(burn)0.94 $78 0.06 $19 0.71 $80 0.29 $21 $74.31 $62.79 $11.520.94 $76 0.06 $17 0.71 $80 0.29 $21 $72.31 $62.79 $9.520.94 $74 0.06 $15 0.71 $80 0.29 $21 $70.31 $62.79 $7.520.94 $72 0.06 $13 0.71 $80 0.29 $21 $68.31 $62.79 $5.520.94 $70 0.06 $11 0.71 $80 0.29 $21 $66.31 $62.79 $3.520.94 $68 0.06 $9 0.71 $80 0.29 $21 $64.31 $62.79 $1.520.94 $66 0.06 $7 0.71 $80 0.29 $21 $62.31 $62.79 -$0.480.94 $64 0.06 $5 0.71 $80 0.29 $21 $60.31 $62.79 -$2.480.94 $62 0.06 $3 0.71 $80 0.29 $21 $58.31 $62.79 -$4.480.94 $60 0.06 $1 0.71 $80 0.29 $21 $56.31 $62.79 -$6.48

C. The Implied Forest Fire Instrument Assuming Beliefs Based on Acreage Information

Lottery A Lottery B EVA EVB Differencep(safe) p(burn) p(safe) p(burn)0.987 $78 0.013 $19 0.84 $80 0.16 $21 $77.24 $70.68 $6.560.987 $76 0.013 $17 0.84 $80 0.16 $21 $75.24 $70.68 $4.560.987 $74 0.013 $15 0.84 $80 0.16 $21 $73.24 $70.68 $2.560.987 $72 0.013 $13 0.84 $80 0.16 $21 $71.24 $70.68 $0.560.987 $70 0.013 $11 0.84 $80 0.16 $21 $69.24 $70.68 -$1.440.987 $68 0.013 $9 0.84 $80 0.16 $21 $67.24 $70.68 -$3.440.987 $66 0.013 $7 0.84 $80 0.16 $21 $65.24 $70.68 -$5.440.987 $64 0.013 $5 0.84 $80 0.16 $21 $63.24 $70.68 -$7.440.987 $62 0.013 $3 0.84 $80 0.16 $21 $61.24 $70.68 -$9.440.987 $60 0.013 $1 0.84 $80 0.16 $21 $59.24 $70.68 -$11.44

-49-

Table 2: Raw Responses from Pilot Experiments

Number of PlayedNumber of Safe Implied Video Time Test Presence Factors †

ID PB Choices Choices CRRA > Risk Attitude ‡ Age Male Games? (seconds) I SF A/I IQ

8 2 4 0.9 < r < 1.15 risk neutral 39 y y 29 5.6 5.5 6 66 4 7 0.11 < r < 0.35 very risk averse 46 y y 67 5.5 6 5.9 6.39 5 6 0.35 < r < 0.6 risk averse 47 y y 106 6.5 7 6.3 77 5 6 0.35 < r < 0.6 risk averse 57 6.8 7 6.4 73 6 5 0.6 < r < 0.9 slightly risk averse 20 y y 100 4.1 6 6.6 6.34 6 6 0.35 < r < 0.6 risk averse 19 168 5.4 6.5 5.5 3.32 6 5 0.6 < r < 0.9 slightly risk averse 23 y y 23 6.6 6 6.5 510 8 7 0.11 < r < 0.35 very risk averse 24 3.1 4.5 3.9 5.31 10 5 0.6 < r < 0.9 slightly risk averse 18 y 29 5.3 6 6.4 4.35 10 6 0.35 < r < 0.6 risk averse 51 5.9 6.5 5.9 612 10 2 1.5 < r < 1.95 very risk loving 37 3.6 1.5 3.4 611 10 0 r > 1.95 highly risk loving 57 3.1 4.5 3.9 5.3

Notes:> Assuming no multiple switches, inferred using the value for the CRRA coefficient r that makes the decision-maker indifferent

between switching. The CRRA functional form for utility u assumed is u(y) = yr for r>0 and y the lottery prize. So r>1implies risk-loving, r=1 implies risk neutrality, and 1>r>0 implies risk aversion.

‡ From Holt and Laury [2002; Table 3], given the number of safe choices (and assuming no multiple switches).† The indices are Involvement (I), Sensory Fidelity (SF), Adaptation/Immersion (A/I), and Interface Quality (IQ). Indices range

from 1 (poor) to 7 (excellent). See the text for explanation of these factors.

-50-

References

Adabala, Neeharika, and Hughes, Charles E., “Gridless Controllable Fire,” in K. Pallister (ed.), GameProgramming Gems 5 (Boston: Charles River Media, 2005).

Aono, Masaki, and Kunii, Tosiyasu L., “Botanical tree image generation,” IEEE Computer Graphicsand Applications, 4(5), 1984, 10-34.

Andersen, Steffen; Harrison, Glenn W.; Lau, Morten Igel, and Rutström, E. Elisabet, “Eliciting Riskand Time Preferences,” Working Paper 05-24, Department of Economics, College of BusinessAdministration, University of Central Florida, 2005.

Andersen, Steffen; Harrison, Glenn W.; Lau, Morten Igel, and Rutström, E. Elisabet, “ElicitationUsing Multiple Price Lists,” Experimental Economics, 9(4), December 2006, 383-405.

Andersen, Steffen; Harrison, Glenn W.; Lau, Morten Igel, and Rutström, E. Elisabet, “ValuationUsing Multiple Price List Formats,” Applied Economics, 39(6), April 2007a, 675-682.

Andersen, Steffen; Harrison, Glenn W., Lau, Morten I., and Rutström, E. Elisabet, “Risk Aversionin Game Shows,” in J.C. Cox and G.W. Harrison (eds.), Risk Aversion in Experiments(Greenwich, CT: JAI Press, Research in Experimental Economics, Volume 12, 2007bforthcoming).

Balci, Murat, and Foroosh, Hassan, “Real-time 3D Fire Simulation Using a Spring-Mass Model,”Proceedings of IEEE International Multimedia Modeling Conference, 2006.

Bateman, Ian J.; Jones, Andrew P.; Jude, Simon, and Day, Brett H., “Reducing Gain/LossAsymmetry: A Virtual Reality Choice Experiment (VRCE) Valuing Land Use Change,”Unpublished Manuscript, School of Environmental Sciences, University of East Anglia,December 2006.

Bin, Okmyung; Crawford, Thomas W.; Kruse, Jamie B, and Landry, Craig E., “Viewscapes andFlood Hazard: Coastal Housing Market Responses to Amenities and Risk,” Working Paper,Center for Natural Hazards Research, East Carolina University, November 2006.

Bishop, Ian D., “Predicting Movement Choices in Virtual Environments,”Landscape and UrbanPlanning, 56, 2001, 97-106.

Bishop, Ian D.; Wherrett JoAnna R., and Miller, David R., “Assessment of Path Choices on aCountry Walk using a Virtual Environment,” Landscape and Urban Planning, 52, 2001, 225-237.

Bishop, Ian D., and Gimblett, H.R., “Management of Recreational Areas: Geographic InformationSystems, Autonomous Agents and Virtual Reality,” Environment and Planning B, 27, 2000, 423-435.

Bloomenthal, Jules, “Modeling the mighty maple,” Proceedings of SIGGRAPH 85, 1985, 305-311.

Çakmak, H.K.; Maas, H., and Kühnapfel, U, “VSOne: A virtual reality simulator for laparoscopic

-51-

surgery,” Minimally Invasive Therapy and Allied Technologies (MITAT), 14(3), 1005, 134-144.

Cameron, A. Colin, and Trivedi, Pravin K., Microeconometrics: Methods and Applications (New York:Cambridge University Press, 2005).

Carson, Richard T; Hanemann, W. Michael; Krosnick, Jon A.; Mitchell, Robert C.; Presser, Stanley;Ruud, Paul A., Smith, V. Kerry, “Referendum Design and Contingent Valuation: TheNOAA Panel’s No-Vote Recommendation,” Review of Economics and Statistics, 80(2), May1998, 335-338; reprinted with typographical corrections in Review of Economics and Statistics,80(3), August 1998.

Castronova, Edward, Synthetic Worlds: The Business and Culture of Online Games (Chicago: University ofChicago Press, 2005).

Castronova, Edward, “The Price of Bodies: A Hedonic Pricing Model of Avatar Attributes in aSynthetic World,” Kyklos, 57(2), 2004, 173-196.

Chiba, Norishige; Muraoka, Kazunobu; Doi, Akio, and Hosokawa, Junya, “Rendering of forestscenery using 3D textures,” The Journal of Visualization and Computer Animation 8, 1997,191-199.

Clark, Andy, Being There: Putting Brain, Body, and World Together Again (Cambridge, MA: MIT Press,1997).

Cohen, Dan, “Computer simulation of biological pattern generation processes,” Nature, 216, 1967,246-248.

Darken, Rudolph P., and Cevik, Helsin, “Map Usage in Virtual Environments: Orientation Issues,”Proceedings of IEEE Virtual Reality, 1999, 133-140.

Desvousges, William H.; Johnson, F. Reed, and Banzhaf, H. Spencer, Environmental Policy AnalysisWith Limited Information: Principles and Applications of the Transfer Method (New York: Elgar,1999).

Deussen, Oliver; Hanrahan, Patrick; Lintermann, Bernd; Mech, Radomir; Pharr, Matt, andPrusinkiewicz, Przemyslaw, “Realistic modeling and rendering of plant ecosystems,”Proceedings of SIGGRAPH 98, 1998, 275-286.

Fidopiastis, Cali M.; Stapleton, Christopher B.; Whiteside, Janet D.; Hughes, Charles E.; Fiore,Stephen M.; Martin, Glenn A.; Rolland, Jannick P., and Smith, Eileen M., “HumanExperience Modeler: Context Driven Cognitive Retraining to Facilitate Transfer ofTraining,” CyberPsychology and Behavior, 9(2), Mary Ann Liebert Publishers, New Rochelle, NY,2006, 183-187.

Finney, Mark A., FARSITE: Fire Area Simulator – Model Development and Evaluation, Research PaperRMRS-RP-4, Rocky Mountain Research Station, Forest Service, United States Departmentof Agriculture, March 1998.

-52-

Gerking, Shelby; de Haan, Menno, and Schulze, William, “The Marginal Value of Job Safety: AContingent Value Study,” Journal of Risk & Uncertainty, 1, 1988, 185-199.

Gigerenzer, Gerd, and Peter M. Todd, Simple Heuristics that Make Us Smart (New York: OxfordUniversity Press, 1999).

Glimcher, Paul W., Decisions, Uncertainty, and the Brain: The Science of Neuroeconomics, The MIT Press,Cambridge, Massachusetts, 2003.

Harrison, Glenn W. “Field Experiments and Control,” in J. Carpenter, G.W. Harrison and J.A. List(eds.), Field Experiments in Economics (Greenwich, CT: JAI Press, Research in ExperimentalEconomics, Volume 10, 2005, 17-50).

Harrison, Glenn W., “Making Choice Studies Incentive Compatible,” in B. Kanninen (ed)., ValuingEnvironmental Amenities Using Stated Choice Studies: A Common Sense Guide to Theory and Practice(Boston: Kluwer, 2006, 65-108).

Harrison, Glenn W., and Kriström, Bengt, “On the Interpretation of Responses to ContingentValuation Surveys”, in P.O. Johansson, B. Kriström and K.G. Mäler (eds.), Current Issues inEnvironmental Economics (Manchester: Manchester University Press, 1996).

Harrison, Glenn W.; Lau, Morten I., and Rutström, E. Elisabet, “Estimating Risk Attitudes inDenmark: A Field Experiment,” Scandinavian Journal of Economics, 109(2), June 2007,forthcoming.

Harrison, Glenn W., and List, John A., “Naturally Occurring Markets and Exogenous LaboratoryExperiments: A Case Study of the Winner’s Curse,” Economic Journal, 117, 2007 forthcoming.

Harrison, Glenn W., and List, John A., “Field Experiments,” Journal of Economic Literature, 42(4),December 2004, 1013-1059.

Harrison, Glenn W.; List, John A., and Towe, Charles, “Naturally Occurring Preferences andExogenous Laboratory Experiments: A Case Study of Risk Aversion,” Econometrica, 75(2),March 2007, 433-458.

Hoffman, R. R., and Deffenbacher, K. A., “An analysis of the relations of basic and appliedscience,” Ecological Psychology, 5, 1993, 315-352.

Holt, Charles A., and Laury, Susan K., “Risk Aversion and Incentive Effects,” American EconomicReview, 92(5), December 2002, 1644-1655.

Honda, Hisao,.“Description of the form of trees by the parameters of the tree-like body: effects ofthe branching angle and the branch length on the shape of the tree-like body,” Journal ofTheoretical Biology, 31, 1971, 331-338.

Hughes, Charles E., and Stapleton, Christopher B., “The Shared Imagination: Creative Collaborationin Augmented Virtuality,” HCI International 2005, Las Vegas, NV, July 22-27, 2005a.

-53-

Hughes, Charles E.; Stapleton, Christopher B.; Hughes, Darin E., and Smith, Eileen M., “MixedReality in Education, Entertainment and Training: An Interdisciplinary Approach,” IEEEComputer Graphics and Applications, 26(6), November/December 2005b, 24-30.

Insko, B. E., “Measuring presence: Subjective, behavioral and physiological methods,” in G. Riva, F.Davide & W. A. IJsselsteijn (eds.), Being There: Concepts, Effects and Measurement of User Presencein Synthetic Environments (Amsterdam: Ios Press, 2003).

Jude, Simon; Jones, Andrew P.; Andrews, J.E., and Bateman, Ian J., “Visualisation for Participatory Coastal Zone Management: A Case Study of the Norfolk Coast, England,” Journal of CoastalResearch, 22(6), 2006, 1527-1538.

Kahneman, Daniel, and Tversky, Amos (eds.), Choices, Values, and Frames (New York: CambridgeUniversity Press, 2000).

Klein, Gary, Sources of Power: How People Make Decisions (Cambridge, MA: MIT Press, 1998).

Krosnick, Jon A.; Holbrook, Allyson L.; Berent, Matthew K.; Carson, Richard T.; Hanemann, W.Michael; Kopp, Raymond J.; Mitchell, Robert C.; Presser, Stanley; Ruud, Paul A.; Smith, V.Kerry; Moody, Wendy R.; Green, Melanie C., and Conaway, Michael, “The Impact of ‘NoOpinion’ Response Options on Data Quality: Non-Attitude Reduction or an Invitation toSatisfice?” Public Opinion Quarterly, 66, 2002, 371-403.

Levitt, Steven D., and List, John A., “What Do Laboratory Experiments Measuring SocialPreferences Reveal About the Real World?” Journal of Economic Perspectives, 21(2), Spring 2007,153-174.

Lin, J.W.; Duh, B.L.; Abi-Rached, H.; Parker, D.E., and Furness, T.A., “Effects of field of view onpresence, enjoyment, memory and simulator sickness in a virtual environment,” Proceedings ofIEEE Virtual Reality, 164-171, 2002.

Lindenmayer, Astrid, “Mathematical models for cellular interactions in development, I & II,” Journalof Theoretical Biology, 18, 1968, 280-315.

Loomis, John B.; Bair, Lucas S., and Gonzáles-Cabán, Armando, “Language-Related Differences ina Contingent Valuation Study: English Versus Spanish,” American Journal of AgriculturalEconomics, 84(4), November 2002, 1091-1102.

Loomis, John B.; Le, Hung Trong, and Gonzáles-Cabán, Armando, “Testing Transferability ofWillingness to Pay for Forest Fire Prevention Among Three States of California, Florida andMontana,” Journal of Forest Economics, 11, 2005, 125-140.

Marr, David, Vision (San Francisco: Freeman, 1982).

Mech, Radomir, and Prusinkiewicz, Przemyslaw, “Visual models of plants interacting with theirenvironment,” Proceedings of SIGGRAPH 96, 1996, 397-410.

Micikevicius, Paulius, and Hughes, Charles E., “Visibility-based Forest Walk-through Using Inertial

-54-

Level of Detail Model,” Journal of Defense Modeling and Simulation, 4, 2007, in press.

Micikevicius, Paulius; Hughes, Charles E.; Moshell, J. Michael; Sims, Valerie, and Smith, Hannah,“Perceptual Evaluation of an Interactive Forest Walk-through,” VR Usability Workshop:Designing and Evaluating VR Systems, Nottingham, England, January 22-23, 2004.

Milgram, Paul, and Kishino, A. Fumio, “Taxonomy of Mixed Reality Visual Displays,” IEICETransactions on Information and Systems, E77-D (12), 1994, 1321-1329.

Mitchell, Robert C., and Carson, Richard T., Using Surveys to Value Public Goods: The ContingentValuation Method (Baltimore: Johns Hopkins Press, 1989).

Nguyen, Duc Q.; Fedkiw, Ronald, and Jensen, Heinrich W., “Physically based modeling andanimation of fire,” Proceedings of SIGGRAPH 02, 2002, 721-728.

Oppenheimer, Peter E., “Real time design and animation of fractal plants and trees,” Proceedings ofSIGGRAPH 86, 1986, 55-64.

Ortmann, Andreas, & Gigerenzer, Gerd, “Reasoning in Economics and Psychology: Why SocialContext Matters,” Journal of Institutional and Theoretical Economics, 153, 1997, 700–710.

Ostrom, Elinor; Gardner, Roy, and Walker, James, Rules, Games, & Common-Pool Resources (AnnArbor, MI: Michigan University Press, 1994).

Prusinkiewicz, Przemyslaw; James, Mark, and Mech, Radomir, “Synthetic topiary,” Proceedings ofSIGGRAPH 94, 1999, 351-358.

Pyne, Stephen J., Tending Fire: Coping with America’s Wildland Fires (Washington, DC: Island Press,2004).

Ryan, Marie-Laure, Possible Worlds, Artificial Intelligence, and Narrative Theory (Bloomington, IN: IndianaUniversity Press, 1991).

Salas, E. & Klein, G. (Eds.), Linking expertise and naturalistic decision making. (Mahwah, NJ: Erlbaum,2001).

Scott, Joe H., and Burgan, Robert E., Standard Fire Behavior Fuel Models: A Comprehensive Set for Use withRothermel’s Surface Fire Spread Model, Research Paper RMRS-GTR-153, Rocky MountainResearch Station, Forest Service, United States Department of Agriculture, June 2005.

Sewell, C.; Blevins, N.S.; Peddamatham, S.; Tan, H.Z.; Morris, D., and Salisbury, K., “The Effect ofVirtual Haptic Training on Real Surgical Drilling Proficiency,” Proceedings of the IEEE WorldHaptics, March 2007.

Smith, Vernon L., “Constructivist and Ecological Rationality,” American Economic Review, 93(3), 2003,465-508.

Stam, Jos, “Interacting with smoke and fire in real time,” Communications of the ACM, 43(7), 2000,

-55-

76-83.

Stapleton Christopher B., and Hughes, Charles E., “Believing is Seeing,” IEEE Computer Graphics andApplications 27(1), January/February 2006, 80-85.

Stern, Nicholas, The Stern Review of the Economics of Global Warming, available from web site athttp://www.hm-treasury.gov.uk/independent_reviews/stern_review_economics_climate_change/sternreview_index.cfm, 2006.

Sutcliffe, A.; Gault, B., and Shin, J., “Presence, memory and interaction in virtual environments,”International Journal of Human-Computer Studies, 62(3), 2005, 307-327.

Tamura, Hideyui; Yamamoto, Hiroyuki, and Katayama, Akihiro, “Mixed Reality: Future DreamsSeen at the Border between Real and Virtual Worlds,” IEEE Computer Graphics andApplications, 21 (6), 2001, 64-70.

U.S. Environmental Protection Agency, The Benefits and Costs of the Clean Air Act, 1970 to 1990. United States Environmental Protection Agency, Office of Policy Analysis and Review,Document # A.97.20, October 1997.

Ulam, Stanislaw, “On some mathematical properties connected with patterns of growth of figures,” Proceedings of Symposia on Applied Mathematics, 14, 1962, 215-224.

Vembar, D.; Sadasivan, S.; Duchowski, A. T.; Stringfellow, P., and Gramopadhye, A. K., “Design ofa Virtual Borescope: A Presence Study,” Proceedings of HCI International, Las Vegas, NV, July22-27, 2005.

Vora, J.; Nair, S.; Meldin, E.; Gramopadhye, A. K.; Duchowski, A.; Melloy, B., and Kanki, B.,“Using Virtual Reality Technology to Improve Aircraft Inspection Performance: Presenceand Performance Measurement Studies,” Proceedings of the Human Factors and Ergonomics SocietyMeeting, 1867-1871, 2001.

Weber, Jason, and Penn, Joseph, “Creation and rendering of realistic trees,” Proceedings ofSIGGRAPH 95, 1995, 119-128.

Witmer, B. G., Jerome, C. J., and Singer, M. J., “The Factor structure of the Presence Questionnaire,” Presence, 14, 2005, 298-312.

Witmer, B. G., and Singer, M. J., “Measuring presence in virtual environments: A presencequestionnaire,” Presence: Teleoperators and Virtual Environments, 7, 1998, 225–240.

-A1-

Appendix A: Presence Questions and Factors Identified (NOT FOR PUBLICATION)

Factor 1: Involvement

1. How much were you able to control events?2. How responsive was the environment to actions that you initiated (or performed)?3. How natural did your interactions with the environment seem?4. How much did the visual aspects of the environment involve you?5. How natural was the mechanism which controlled movement through the environment?6. How much did your experiences in the virtual environment seem consistent with your realworld experiences?8. How completely were you able to actively survey or search the environment using vision?9. How compelling was your sense of moving around inside the virtual environment?12. How involved were you in the virtual environment experience?23. How easy was it to identify objects through physical interaction, like touching an object,walking over a surface, or bumping into a wall or object?

Factor 2: Sensory Fidelity

10. How closely were you able to examine objects?11. How well could you examine objects from multiple viewpoints?

Factor 3: Adaptation/Immersion

7. Were you able to anticipate what would happen next in response to the actions that youperformed?14. How quickly did you adjust to the virtual environment experience?15. How proficient in moving and interacting with the virtual environment did you feel at theend of the experience?18. How well could you concentrate on the assigned tasks or required activities rather than onthe mechanisms used to perform those tasks or activities?19. How completely were your senses engaged in this experience? 24 Were there moments during the virtual environment experience when you felt completelyfocused on the task or environment?25 How easily did you adjust to the control devices used to interact with the virtualenvironment?26 Was the information provided through vision in the virtual environment consistent?

Factor 4: Interface Quality

13. How much delay did you experience between your actions and expected outcomes?16. How much did the visual display quality interfere or distract you from performing assignedtasks or required activities?17. How much did the control devices interfere with the performance of assigned tasks or withother activities?

-A2-

Not assigned to factor

20. To what extent did events occurring outside the virtual environment distract from yourexperience in the virtual environment?21. Overall, how much did you focus on using the display and control devices instead of thevirtual experience and experimental tasks?22. Were you involved in the experimental task to the extent that you lost track of time?

-A3-

Appendix B: Instructions and Decision Sheets (NOT FOR PUBLICATION)

PDF versions of all instructions, as provided to subjects, are available on request.

ID:Some Questions About You

In this survey most of the questions asked are descriptive. We will not be grading youranswers and your responses are completely confidential. Please think carefully about each questionand give your best answers.

1. What is your AGE? ____________ years

2. What is your sex? (Circle one number.)

01 Male 02 Female

3. Which of the following categories best describes you? (Circle one number.)

01 White 06 Hispanic-American02 African-American 07 Hispanic 03 African 08 Mixed Race04 Asian-American 09 Other05 Asian

4. What is your major? (Circle one number.)

01 Accounting02 Economics03 Finance04 Business Administration, other than Accounting, Economics, or Finance05 Education06 Engineering07 Health Professions08 Public Affairs or Social Services09 Biological Sciences10 Math, Computer Sciences, or Physical Sciences11 Social Sciences or History12 Humanities13 Psychology14 Other Fields15 Does not apply

5. What is your class standing? (Circle one number.)

01 Freshman 04 Senior02 Sophomore 05 Masters03 Junior 06 Doctoral07 Does not apply

6. What is the highest level of education you expect to complete? (Circle one number)

01 Bachelor’s degree02 Master’s degree03 Doctoral degree04 First professional degree05 High school diploma or GED06 Less than high school

-A4-

7. What was the highest level of education that your father (or male guardian) completed?(Circle one number)

01 Less than high school02 GED or High School Equivalency03 High school04 Vocational or trade school05 College or university

8. What was the highest level of education that your mother (or female guardian) completed?(Circle one number)

01 Less than high school02 GED or High School Equivalency03 High School04 Vocational or trade school05 College or university

9. What is your citizenship status in the United States?

01 U.S. Citizen02 Resident Alien03 Non-Resident Alien04 Other Status

10. Are you a foreign student on a Student Visa?

01 Yes02 No

11. Are you currently...

01 Single and never married?02 Married?03 Separated, divorced or widowed?

12. On a 4-point scale, what is your current GPA if you are doing a Bachelor’s degree, or whatwas it when you did a Bachelor’s degree? This GPA should refer to all of your coursework,not just the current year. Please pick one:

01 Between 3.75 and 4.0 GPA (mostly A’s)02 Between 3.25 and 3.74 GPA (about half A’s and half B’s)03 Between 2.75 and 3.24 GPA (mostly B’s)04 Between 2.25 and 2.74 GPA (about half B’s and half C’s)05 Between 1.75 and 2.24 GPA (mostly C’s)06 Between 1.25 and 1.74 GPA (about half C’s and half D’s)07 Less than 1.25 (mostly D’s or below)08 Have not taken courses for which grades are given.

13. How many people live in your household? Include yourself, your spouse and anydependents. Do not include your parents or roommates unless you claim them asdependents.

___________________

-A5-

14. Please circle the category below that describes the total amount of INCOME earned in 2006by the people in your household (as “household” is defined in question 13).[Consider all forms of income, including salaries, tips, interest and dividend payments,scholarship support, student loans, parental support, social security, alimony, and childsupport, and others.]

01 $15,000 or under02 $15,001 - $25,00003 $25,001 - $35,00004 $35,001 - $50,00005 $50,001 - $65,00006 $65,001 - $80,00007 $80,001 - $100,00008 over $100,000

15. Please circle the category below that describes the total amount of INCOME earned in 2006by your parents. [Consider all forms of income, including salaries, tips, interest and dividendpayments, social security, alimony, and child support, and others.]

01 $15,000 or under02 $15,001 - $25,00003 $25,001 - $35,00004 $35,001 - $50,00005 $50,001 - $65,00006 $65,001 - $80,00007 $80,001 - $100,00008 over $100,00009 Don’t Know

16. Do you work part-time, full-time, or neither? (Circle one number.)

01 Part-time02 Full-time03 Neither

17. Before taxes, what do you get paid? (fill in only one)

01 _____ per hour before taxes02 _____ per week before taxes03 _____ per month before taxes04 _____ per year before taxes

18. Do you currently smoke cigarettes? (Circle one number.)

00 No01 Yes

If yes, approximately how much do you smoke in one day? _______ packs

-A6-

Exercises for using the visual simulation of forest and fire

When you move the arrow across the screen by using the mouse it is as if you aremoving your head around.

You can move your whole body forward by clicking “w”Move backward by clicking “s”Sidestep to the left by clicking “a”Sidestep to the right by clicking “d”You can always return to your house by hitting the “Home” key

Follow these exercises:

1. Move arrow right = look right2. To go faster and slower = move further away and closer to center of screen3. Return to center of screen to stop moving head around4. Move arrow to the left = look left and return gaze to house5. stop by moving arrow to center of screen when you see the house6. look diagonally left of house and click on “w” to move forward7. use head motions and “w” to move all the way around the house8. after completing the circle around the house, look at the house and return arrow to center ofscreen9. look upward somewhat – this is in preparation for “flying” up10. return arrow to center of screen while still looking up – this stops the head tilting motion11. use “w” to move forward/upward a bit12. look slightly down until finding horizon, return arrow to center13. look around 180 degrees – then return arrow to center14. the house is somewhere below you – try to find it15. look up again16. find horizon and move forward to edge of the simulated landscape – avoid hitting the groundor flying up in the air – use up and down movement of head17. turn around until you are facing the mountains18. return and try to find the house again19. practice using backward and sidestepping motions20. move freely around the landscape, keep returning to house

Next I will move you over to one particular location on the landscape and you will find your wayback to the house from there as quickly as possible without using the “Home” key. In fact, I willgive you an incentive to get back quickly. If you make it back (crashing into or through thehouse) in 30 seconds or less I will give you $5, in 1 minute or less I will give you $2 and in 2minutes or less I will give you $1.

-A7-

[The following text was provided in a formatted manner, akin to modern CVM survey instruments.The PDF version, available on request, shows that formatted version.]

FLORIDA FOREST FIRES

5000 wild fires each year in Florida

What are your views?

Florida typically has about 5000 wild fires each year, with the most active period betweenMarch and June. During the fires season of 2006 the Division of Forestry of Florida Department ofAgriculture and Consumer Services reported over 4,000 wild fires covering more than 200,000 acresby the end of July.

Managing and preventing these wild fires is important. Significant economic costs, in terms ofdamage to property, as well as serious health and safety concerns from fire and smoke, make this anissue of interest to everyone in the state. On the other hand, there are many other important uses foryour taxes. Some examples include the education system, health care, crime prevention, otherenvironmental activities, or tax relief. Your views on this topic are important to Florida fire managers and policy planners as theydecide how best to proceed. This study is funded by the National Science Foundation in an effort toprovide scientists and policy makers with better information. This study is not commissioned by agovernment agency or by a private corporation.

How severe were the 1998 fires?

To give you perspective on the economic damages caused by wild fire, here are someestimates from the 1998 fires in Florida. Over 340 homes, 33 businesses, and several cars and boatswere destroyed, totaling between $10 million and $20 million. Lost homes accounted for the bulk ofthe direct costs to households living in Florida. The average cost per destroyed house is estimated tobe roughly $59,000.

About 500,000 acres of timber forest burned in Florida that year. Net timber damages areestimated at between $300 million and $500 million, implying a loss between $600 and $1,000 peracre. In addition, there were additional health expenditures associated with increases in asthmaincidences, particularly among children. The estimated total health costs for asthma was between$210,000 and $650,000. Another significant loss in Florida was the lost tourism income, estimated at $138 million.

The total economic cost estimated for the 1998 wild fires was at least $448 million for the500,000 acres burnt. An average fire year, burning about 200,000 acres, therefore costs about $180million.

Possible solutions to the wild fire problem

A build-up of wild fire fuel in the form of brush, dead branches, logs and pine needles onthe forest floor has occurred over several decades. As a result, when wild fires are ignited theyspread faster and burn longer than would otherwise be the case. During drought years the fuelmoisture levels are low and compound the risks. In addition, when winds are strong the spread isfaster and the damages more extensive.

Recently, prescribed burning has been used to prevent wild fires from becoming intensive andspreading quickly. Prescribed burning involves having fire professionals periodically set fires to clear

-A8-

the forest floor of the excess brush, dead branches and pine needles. These prescribed fires areeasier to manage than wild fires since prescribed fires do not burn as intensely and they can bedirected away from structures.

While prescribed fires do result in an increase in air pollution, they generally produce far less airpollution than would be expected from a wild fire on the same acreage. By timing prescribed fireswith favorable weather and wind conditions, smoke can be directed away from the majority of thepopulation.

Prescribed burns occur during winter, and therefore precede the wild fire season, which starts inMarch each year. Because the prescribed burns take place in the winter, when the vegetation ismoist, the risk of an uncontrolled spread is extremely low.

Who would fund the fire prevention program?

Prescribed burns cost roughly $25 per acre. An average of half a million acres in Florida arealready treated with prescribed burn each year. This corresponds to about 4% of the total forestarea, and costs approximately $12.5 million statewide. To put the cost in perspective: The Floridastate budget during 2005-2006 was $64.7 billion. For the 2006-7 budget year $80 million has beenallocated to beach restoration after increased hurricane damages.

In this study we are investigating a proposed expansion of the prescribed burn program thatincreases the treated acreage from 4% to 6% of the total forest area, from ½ million to ¾ millionacres. This is expected to cost an additional $6.25 million per year.

The State of Florida is considering using some of the state revenue as matching funds to helpcounties finance fire prevention programs. If a majority of residents vote to pay the county share ofthis program, the Expanded Florida Prescribed Burning Program would be implemented in yourcounty, other counties in Florida, in state forest and rangelands, and with private land owners thatchoose to do so.

Funding of the Expanded Florida prescribed Burning Program would require that all users ofFlorida’s forest and rangelands, such as timber companies, recreation visitors, and Floridahouseholds, pay the additional cost of the program. If this expanded program were to beimplemented, by law the money would be deposited in a separate Florida Prescribed Burning Fundwhich could only be used to carry out the prescribed burning program described above. A citizenadvisory board would review the expenditures from the fund annually.

Your task

Several computerized simulation programs exist that predict the spread and severity of wildfires with high accuracy. They are used by fire fighters during live wild fires to predict the spread offire and the best actions to contain the fire and minimize the damages. These simulations are basedon the landscape and urban settlements in specific regions of the US and therefore provide goodexamples of potential fire risks. We are going to employ one such program in this study. Thisprogram has been developed by the United States Forest Service.

The forest area that we are simulating is based on the topography and vegetation of AshleyNational Forest in Utah. The vegetation in this forest differs by location, and so does the elevation.You will be able to explore these aspects of the forest in the simulations. We will first introduce youto a series of simulations based on the fire fuel situation before an expansion of prescribed burns.You will see several fires that start in different locations, each randomly determined.

We are going to let you run through two series of simulations using this software so that youcan experience the effect of wild fires with and without the expanded prescribed burn program. Thedifference between the simulations with and without expanded prescribed burn is entirely due todifferences in the fuel load of the forests. The fuel load with the expanded prescribed burn is muchlower. In the background, we will vary other factors that affect the severity of the fires, such as fuelmoisture, winds, rain and temperatures. These background factors will vary in the same way acrossthe simulations with and without the expanded prescribed burn program. The purpose of theseinitial simulations is to familiarize you with how the fire reacts to changing conditions.

-A9-

Your property

In the simulations we will designate one property as yours. This property has one house onit, valued at $59. This is simply scaled down from the average house damage in Central Florida firesthat was estimated at $59,000. Thus, the most you stand to lose in a fire is $59. These are roughnumbers, of course, but they are about what people actually lose during wild fires in Florida if theylose their home.

You are asked to make a choice of fire management policy for yourself. You may choose one ofthese two policies:

1. Keep the present prescribed burn program. This leaves more fuel for fire in the forests.

2. Expand the prescribed burn program. Expanding the prescribed burn program from 4% to 6% ofthe total forest area reduces the fuel for fire in the forests. If you select this option you will have topay a certain amount out of your initial credit to pay for it. We explain how later.

After experiencing a number of fires under different risk conditions in the two series ofsimulations we will let you run one more simulation. This final simulation will be used to determinehow much you will be paid for your participation and it will be based on your choice of firemanagement policy. At the start of this simulation you are credited with $80. Based on your choiceof policy (choosing policy 2 costs money) and of the outcome of the fire in the final simulation(your property may burn) you may be paid less than this amount. We will explain this to you next.

Your earnings

We are going to simulate a wild fire based on your choice of fire management policy 1 or 2to determine your payment. The background factors that determine the severity of the fire, such asweather conditions or the location of lightning strikes, will be determined completely by chance. Weexplain these factors later. These background factors will be the same no matter which policy youchoose.

However, if you select policy 1 the amount of fuel for any fire will be greater than if you selectpolicy 2. The difference in fuel loads between policy 1 and 2 will be the same as the difference usedin the first two series of simulations where you learn about how the fire reacts to changingconditions. Thus you will have a chance to experience what the effect of the policy is on the severityof a fire, before you make your choice. We will simply deduct any damage to your property from your initial credit line of $80, aftertaking out the cost of expanded prescribed burn if you choose that option. You will then be paidthe. The less wild fire damage there is to your property in this final simulation, the more you will getpaid. But you can affect the risk of a wild fire damaging your property by choosing to pay up frontfor the expanded prescribed burn policy. The choice is yours.

If you choose not to expand the program, and select option 1, there will be no additional coststo you. We will simply deduct the wild fire damages from your initial credit. It is expected, though,that there will be a higher risk of more extensive damages than under option 2. You will be able tomake your own assessment of the two fire management options during the initial series ofsimulations.

What are the risks?

We have generated a number of fires by varying several background factors for the initialseries of simulations that you will experience. These background factors reflect the normal thingsthat determined if a wild fire is severe or not.

The first background factor is the temperature and humidity. In one case we have a hightemperature and low humidity, which makes for a more severe fire. In this case the morning lowtemperature of the day is 60 and the afternoon high is 99, with morning high humidity of 70% andafternoon low of 10%. And in another case we have a relatively low temperature and high humidity,which makes for a less severe fire. This time the morning low temperature is 40 and the afternoonhigh is 85, with morning high humidity of 90% and afternoon low of 60%. Each of these cases isequally likely to occur in the final simulation. In fact, for the final simulation that is used to

-A10-

determine your payments the temperature and humidity will be set randomly by the throw of a dice. The second background factor is the wind. This will be either high or low, corresponding to 5miles per hour or 1 mile per hour. Higher winds, as you might expect, generate a more severe wildfire. We maintain the same variations in the direction of the wind, no matter what the wind speed.Again, when we pick the scenario to actually pay you, these two wind speeds will be selected by thethrow of a dice. The third background factor is the fuel moisture in the vegetation. This will either be high orlow, with expected effects on the severity of the wild fire. When fuel moisture is high the moisturecontent of all dead vegetation is set to 20%, when it is low it is set to 3-5%, depending on the typeof vegetation. Further, when the dead fuel moisture is high the proportion of the vegetation that isdead is low (0-10%) and when the dead fuel moisture is low the proportion of the vegetation that isdead is higher (25-50%). The choice between these two options will be determined by the throw of adice. The fourth background factor is the duration of the fire, which will either be 1 day or 2 days.You can think of this as reflecting rainfall or fire-fighting activities. The longer a fire lasts, the moreit will spread across the landscape. Again, the duration will be selected by a the throw of a dice. The final background factor is the location of the initial lightning strike that ignites the fire. Thiscould be in the center of Ashley National Forest, in the north (in the mountains), or in the south.Each is equally likely when we select a scenario to actually pay you. Your property is in the center ofAshley, slightly to the east and close to the foothills. For the final simulation all of these factors will be selected randomly by the throw of a dice. Thissimulates our inability to perfectly forecast which conditions will apply in any one fire season. Forthe initial two series of simulations, where you will have a chance to understand the nature of thefires and how it depends on these background factors as well as on which fire management optionyou select, you will experience both benign and severe conditions.

Calculating the risks

In a minute we will show you the risks of different wild fires in Ashley National Park, usingsome pictures to statistically describe the spread of wild fires generated by the simulation model andhow it varies due to varying background factors. If we consider each of the possible backgroundfactors, there are 48 scenarios that are possible. Each scenario is a combination of temperature andhumidity (high or low), wind (high or low), fuel moisture (high or low), duration (short or long), andignition location (center, north, or south).

The graph below shows you what happens when prescribed burn is not expanded. This isoption 1 in terms of your choices. The horizontal or bottom axis shows the total acreage of AshleyNational Forest that burns, in percentages. So the most severe fires burn slightly more than 60% ofthe whole area (in the bottom right hand side of the graph). But many of the fires burn less than10% of the whole area (in the bottom left hand side of the graph). The vertical axis shows thenumber of times that the fire burns each percent of the whole area. So we see that the fire burnedbetween 0% and 2½% of the whole area in 11 of the 48 scenarios (the first bar at the bottom left)and over 60% of the area in 3 of the 48 scenarios (the last two bars).

[Top part of Figure 14 of the text inserted here]

The effect of prescribed burns on risks

We can generate a similar graph, assuming the same background risk factors, in the eventthat the enhanced prescribed burn policy is in place. In this case we reduced the fuel load to reflectwhat would be expected if there had been an expanded prescribed burn policy.

The results are clear: having the enhanced prescribed burn policy in place has changed therisks of having more severe wild fires. It does not eliminate those risks, but now almost every wildfire burns less than 5% of the whole area of Ashley National Forest. There are some wild fires thatare more severe, even with the enhanced prescribe burn policy in place.

-A11-

You should also keep in mind that these displays refer to the percent of the whole area ofAshley National Forest that is burned. Even if it is only 1%, if that 1% happens to be your property,then your property will burn. Of course, you can judge the risk of that for yourself.

[Bottom part of Figure 14 of the text inserted here]

Experiencing the risks

You will now be given a chance to experience the risks we have been describing, and whichhave been simulated by the computer model developed for Ashley National Forest by the UnitedStates Forest Service.

We will first give you a short tutorial on how to use the visualization software that has beenprovided for this exercise. This should just take a few minutes, and will help you learn how tonavigate around the landscape.

We will then let you experience several scenarios to help you understand the risks of a severewild fire, and how these are affected by background conditions and your choice of an enhancedprescribed burn policy. We will show you 4 scenarios:

* A high risk scenario in which there is no policy in place.* A low risk scenario in which there is no policy in place.* A high risk scenario in which there is a policy in place.* A low risk scenario in which there is a policy in place.

After you have had a chance to view these scenarios, you will be asked to make your decision aboutoption 1 or 2. We will then pick a background scenario and play out your choice, to see if yourproperty is destroyed or not.

Make your decision

Recall that you will be paid according to the damages to your designated property only. Youare given an initial credit of US $80, and we will pay you amount of this remaining after deductingwild fire damages and any costs for the expanded burn program. Each time your house burns downwe will deduct US $59. We will pay you your earnings in cash today, and this will be on top of anyother earnings you might receive from other tasks.

You now get to make a choice between paying a certain amount of money towards theexpansion of prescribed burning program or maintaining the present fire prevention program at noadditional cost to yourself. We ask you to indicate YES or NO to each the 10 choices shown on thedecision sheet you have received. After you have done this, we will roll a 10-sided die to pick one ofthese rows. That row will then be the one that is used to determine your final earnings. We would like to know your choice for this range of costs for prescribed burn since the actualfuture cost of such management options are always uncertain. Thus, you should answer YES or NO to the expanded prescribed burn program (firemanagement option 2) for each row, where each row shows you a different cost of implementingthe prescribed burn. As you think about your answer for each row you should realize that each rowis equally likely to be the one determining your earnings. If a row is picked where you have answeredYES, your earnings will be determined by one of the 48 simulations that we generate under thelower fuel load that is the result of expanded prescribed burn. If a row is picked where you answeredNO, one of the 48 simulations with a higher fuel load (the result of fire management option 1) willbe used instead. After finishing these choices you will throw the dice several times to randomly select thebackground factors for the final simulation. This will determine which one of the 48 scenarios thatwill be implemented. Do you have any questions before you make your decision?

-A12-

Decision Sheet for Enhanced Prescribed Burn Policy Option

Please circle one choice for each cost shown below. Once you have done this we will roll a 10-sideddie to pick the cost that applies to you today:

Cost of Policy to You in US

dollars

Yes, I choose option 2, the

expanded prescribed burn.

No, I do not choose option

2. I prefer option 1, to not

expand prescribed burn.

$2 Yes No

$4 Yes No

$6 Yes No

$8 Yes No

$10 Yes No

$12 Yes No

$14 Yes No

$16 Yes No

$18 Yes No

$20 Yes No

Then the background factors are selected using the throw of a 6 sided dice:

1. Temperature and humidity (low is 1, 2 or 3, and high is 4, 5 or 6) _________

2. Wind speed (low is 1, 2 or 3, and high is 4, 5 or 6) _________

3. Fuel moisture (low is 1, 2 or 3, and high is 4, 5 or 6) _________

4. Duration (short is 1, 2 or 3, and long is 4, 5 or 6) _________

Location (Central is 1 or 2, North is 3 or 4, and South is 5 or 6) _________

-A13-

PRESENCE QUESTIONNAIRE

Please characterize your experience in the environment, by marking an "X" in theappropriate box of the 7-point scale, in accordance with the question content and descriptive labels. Please consider the entire scale when making your responses, as the intermediate levels may apply. Answer the questions independently in the order that they appear. Do not skip questions or returnto a previous question to change your answer.

WITH REGARD TO THE EXPERIENCED ENVIRONMENT 1. How much were you able to control events?

|________|________|________|________|________|________|________|NOT AT ALL SOMEWHAT COMPLETELY

2. How responsive was the environment to actions that you initiated (or performed)?

|________|________|________|________|________|________|________|NOT MODERATELY COMPLETELY RESPONSIVE RESPONSIVE RESPONSIVE

3. How natural did your interactions with the environment seem?

|________|________|________|________|________|________|________|EXTREMELY BORDERLINE COMPLETELY ARTIFICIAL NATURAL

4. How much did the visual aspects of the environment involve you?

|________|________|________|________|________|________|________|NOT AT ALL SOMEWHAT COMPLETELY

5. How natural was the mechanism which controlled movement through the environment?

|________|________|________|________|________|________|________|EXTREMELY BORDERLINE COMPLETELY ARTIFICIAL NATURAL

-A14-

6. How much did your experiences in the virtual environment seem consistent with your real worldexperiences?

|________|________|________|________|________|________|________|NOT MODERATELY VERY CONSISTENT CONSISTENT CONSISTENT

7. Were you able to anticipate what would happen next in response to the actions that youperformed?

|________|________|________|________|________|________|________|NOT AT ALL SOMEWHAT COMPLETELY

8. How completely were you able to actively survey or search the environment using vision?

|________|________|________|________|________|________|________|NOT AT ALL SOMEWHAT COMPLETELY

9. How compelling was your sense of moving around inside the virtual environment?

|________|________|________|________|________|________|________|NOT MODERATELY VERY COMPELLING COMPELLING COMPELLING

10. How closely were you able to examine objects?

|________|________|________|________|________|________|________|NOT AT ALL PRETTY VERY

CLOSELY CLOSELY

11. How well could you examine objects from multiple viewpoints?

|________|________|________|________|________|________|________|NOT AT ALL SOMEWHAT EXTENSIVELY

-A15-

12. How involved were you in the virtual environment experience?

|________|________|________|________|________|________|________|NOT MILDLY COMPLETELY INVOLVED INVOLVED ENGROSSED

13. How much delay did you experience between your actions and expected outcomes?

|________|________|________|________|________|________|________|NO DELAYS MODERATE LONG

DELAYS DELAYS

14. How quickly did you adjust to the virtual environment experience?

|________|________|________|________|________|________|________|NOT AT ALL SLOWLY LESS THAN

ONE MINUTE

15. How proficient in moving and interacting with the virtual environment did you feel at the endof the experience?

|________|________|________|________|________|________|________|NOT REASONABLY VERY PROFICIENT PROFICIENT PROFICIENT

16. How much did the visual display quality interfere or distract you from performing assigned tasksor required activities? |________|________|________|________|________|________|________|NOT AT ALL INTERFERED PREVENTED SOMEWHAT TASK PERFORMANCE

17. How much did the control devices interfere with the performance of assigned tasks or withother activities?

|________|________|________|________|________|________|________|NOT AT ALL INTERFERED INTERFERED

SOMEWHAT GREATLY

-A16-

18. How well could you concentrate on the assigned tasks or required activities rather than on themechanisms used to perform those tasks or activities?

|________|________|________|________|________|________|________|NOT AT ALL SOMEWHAT COMPLETELY

19. How completely were your senses engaged in this experience?

|________|________|________|________|________|________|________|NOT MILDLY COMPLETELY ENGAGED ENGAGED ENGAGED

20. How easy was it to identify objects through physical interaction, like touching an object, walkingover a surface, or bumping into a wall or object?

|________|________|________|________|________|________|________|NOT AT ALL MODERATELY VERY EASY

21 Were there moments during the virtual environment experience when you felt completelyfocused on the task or environment?

|________|________|________|________|________|________|________|NOT AT ALL MODERATELY VERY MUCH

22 How easily did you adjust to the control devices used to interact with the virtual environment?

|________|________|________|________|________|________|________|NOT AT ALL MODERATELY VERY EASILY

23 Was the information provided through vision in the virtual environment consistent?

|________|________|________|________|________|________|________|NOT CONSISTENT MODERATELY COMPLETELY

CONSISTENT

-A17-

24. Overall, how much did you focus on using the display and control devices instead of the virtualexperience and experimental tasks?

|________|________|________|________|________|________|________|NOT AT ALL SOMEWHAT VERY MUCH

25. Were you involved in the experimental task to the extent that you lost track of time?

|________|________|________|________|________|________|________|NOT AT ALL SOMEWHAT COMPLETELY

-A18-

Some additional questions: ID: __________

Have you ever played video games in which you can move around the landscape?

YES NO NOT SURE

Have you ever played online video games, such as World of Warcraft?

YES NO NOT SURE

Have you ever participated in online virtual worlds, such as Second Life?

YES NO NOT SURE

Would you call yourself an avid player of video games?

YES NO NOT SURE

Have you ever been close to a wild fire?

YES NO NOT SURE

Have you ever lived in a forested area during the summer, when there was some risk?

YES NO NOT SURE

Have you ever done any camping or trekking in areas prone to wild fires?

YES NO NOT SURE

-A19-

Decision Task

Your decision sheet shows ten decisions listed on the left. Each decision is a paired choice between “Option

A” and “Option B.” You will make a choice on each row and record these in the final column.

Here is a ten-sided die that will be used to determine payoffs. The faces are numbered from 1 to 10.

Look at Decision 1 at the top. Option A pays $20.00 if the throw of the ten sided die is 1, and it

pays $16.00 if the throw is 2-10. Option B yields $38.50 if the throw of the die is 1, and it pays $1.00

if the throw is 2-10.

The other Decisions are similar, except that as you move down the table, the chances of the higher payoff

for each option increase. In fact, for Decision 10 in the bottom row, the die will not be needed since

each option pays the highest payoff for sure, so your choice here is between $20.00 or $38.50.

After you have finished making your choices, you will throw this die twice, once to select one of the ten decisions to

be used, and a second time to determine what your payoff is for the option you chose, A or B, for

the particular decision selected. Even though you will make ten decisions, only one of these will end

up affecting your earnings, but you will not know in advance which decision will be used.

-A20-

ID: ______________

Decision Option A Option BYour Choice(Circle A or B)

1 $20.00 if throw of die is 1$16.00 if throw of die is 2-10

$38.50 if throw of die is 1$1.00 if throw of die is 2-10

A B

2 $20.00 if throw of die is 1-2$16.00 if throw of die is 3-10

$38.50 if throw of die is 1-2$1.00 if throw of die is 3-10

A B

3 $20.00 if throw of die is 1-3$16.00 if throw of die is 4-10

$38.50 if throw of die is 1-3$1.00 if throw of die is 4-10

A B

4 $20.00 if throw of die is 1-4$16.00 if throw of die is 5-10

$38.50 if throw of die is 1-4$1.00 if throw of die is 5-10

A B

5 $20.00 if throw of die is 1-5$16.00 if throw of die is 6-10

$38.50 if throw of die is 1-5$1.00 if throw of die is 6-10

A B

6 $20.00 if throw of die is 1-6$16.00 if throw of die is 7-10

$38.50 if throw of die is 1-6$1.00 if throw of die is 7-10

A B

7 $20.00 if throw of die is 1-7$16.00 if throw of die is 8-10

$38.50 if throw of die is 1-7$1.00 if throw of die is 8-10

A B

8 $20.00 if throw of die is 1-8$16.00 if throw of die is 9-10

$38.50 if throw of die is 1-8$1.00 if throw of die is 9-10

A B

9 $20.00 if throw of die is 1-9$16.00 if throw of die is 10

$38.50 if throw of die is 1-9$1.00 if throw of die is 10

A B

10 $20.00 if throw of die is 1-10 $38.50 if throw of die is 1-10 A B

DECISION ROW CHOSEN BY FIRST THROW OF THE DIE: _________

THROW OF THE DIE TO DETERMINE PAYMENT: ____________

EARNINGS: ________________

-A21-

Payment Record Date: __________________

Task 1 __________________

Task 2 __________________

Task 3 Credit: $80

Cost: _________

Damage: _________

Net: _________ __________________

Total: __________________

I verify that I have been paid the above stated Total amount of money for participation in anexperiment on the above date.

Print Name: ___________________________________________

Signature: ___________________________________________

Last 4 SSN: ___________________