“Learning Outcomes of Game-Theoretic Computer Simulation: An Evaluation.” Social Science...

367 Learning Outcomes of Game-TheoreticComputer Simulation: An EvaluationFrançois Pétry

This article reports an experimental use of Conflict and Cooperation (C&C) in a largeundergraduate class and evaluates the effect of using the program on student learningas compared with the next best conventional teaching method. Keywords: prisoner’sdilemma, computer simulation, game theory, collective goods, cooperation.

A recent article in Social Science Computer Review by Professor PeterSenn discusses the use of computer software in political education. Hedeplores that &dquo;no well-qualified and professional political scientistshave contributed their knowledge and expertise to the field&dquo; (1989, p.277). Professor Senn also observes that for reasons of time and financialconstraint evaluations of educational software tend to focus on contentand technical quality rather than on the effect of the programs on stu-dent learning. Yet learning outcomes are &dquo;the ultimate criterion for thequality of computer software ... when compared with the next bestmethod of achieving these outcomes&dquo; (Senn, pp. 289-9o).

This article addresses Professor Senn’s concerns by reporting an ex-perimental use of c & c: Conflict and Cooperation-a software programdeveloped by expert political scientists-and evaluating the program’seffect on student learning as compared with the next best conventionalteaching method. 1The idea of the experiment was motivated by the disappointing re-

sults achieved with conventional techniques in teaching strategic de-cisions.2 The assumption was that participation in a well-designed com-puter simulation would produce a discemable difference over

conventional methods of education in the learning outcomes. Other-wise, there would be no justification for the extra expenditure and timespent in the computer simulation activity.The objectives of this article are

i. to describe the nature of prisoner’s dilemma simulation

Social Science Computer Review 8:3, Fall 1990. Copyright C iggo by Duke UniversityPress. ccc o894-4393~90~$1.50.

at UNIVERSITE LAVAL on September 14, 2014ssc.sagepub.comDownloaded from

http://ssc.sagepub.com/

368 2. to describe the c & c module and the procedure used to administerthe simulation

3. to present the pedagogical advantages of using c & c

4. to present findings on the impact of the experiment on students’ per-formances

Prisoner’s Dilemma

The c & c module is useful to model a variety of strategic situations, butits main target is the simulation of problems involving the supply of col-lective goods or benefits. Collective benefits are a problem because it isdifficult to make people pay for them. A private benefit is exclusive; itonly goes to those who have paid for it. But a collective benefit is nonex-clusive. This means that once the benefit has been acquired by one per-son, others may take a free ride and get the same benefit without payingfor it. If people do free nde, they will not supply themselves with opti-mal amounts of collective benefits; in fact, they may gain none. Theparadox is that people want these collective benefits, but they do notwant to pay their cost, hoping that someone else will. But since no onedoes, the benefits are not available.The paradox of cooperation described above occurs when the pursuit

of self-interest by each leads to a perverse outcome for all. Many deci-sion problems in the social sciences and management have this para-doxical property in common. These are referred to as prisoner’s dilem-mas. Prisoner’s dilemmas are used as a tool to analyze processes asdiverse as tariff negotiation (Evans, 1971), the taxation of multinationalfirms (Laver, 1977), truck weight regulation on highways (Prentice &

Hildebrand, 1988) industrial relations bargaining and negotiations(Rubin & Brown, 1975), or the failure of oligopolists to maximize jointprofit (Sherer, 1980).

This partial list should suffice to demonstrate the central role of theprisoner’s dilemma in social science education. Yet students often havedifficulties leaming how to make decisions in prisoner’s dilemma-likesituations. Class assignments, exercises, and case studies involvingstrategic decisions are a source of anxiety for many students. The cen-tral point of these assignments is often not clearly understood. At best,a student may accidentally stumble upon a good strategic plan, but thisoften seems to be due to personal creativity rather than a disciplinedstrategic approach to the assigned problem.

Conventional Prisoner’s Dilemma Class Simulation

Instructors who wish to familiarize their students with prisoner’s di-lemma-like situations may rely upon class demonstration exercisessuch as the one described below. The matrix of Table 1 depicts a four-person prisoner’s dilemma game that can be played in a classroom fordemonstration purpose. The numerical entries in the matrix are the

possible payoffs to the row player. A row player has exactly two strate-



369

gies : to contribute or to defect. The player’s payoff varies dependingupon his choice of strategy and the number of other (column) playerswho contribute = m, with m varying from o to 3. The row player essen-tially contemplates two rows of payoii+-one for contributing, the otherfor defecting-for a variable number of contributing others.One particular feature of this game is immediately apparent. A player

is always better off by defecting than by contributing regardless of thechoice of the other players. To put it another way, players in a Prisoner’sDilemma Game have a uniformly dominant strategy of defecting.The game depicted in Table i may be played in class as follows: The

instructor calls for student volunteers willing to gamble a small amountof money for a possible gain that triples their stake. Each volunteer isgiven a sheet of paper on which to write the amount of his gamble. Sim-ulations which do not involve real payoffs are often suspect because par-ticipants have little incentive to make what in their mind is the best in-dividual choice. To forestall this possible objection, participants aregiven a monetary payoff at the end of the experiment. Only two gamblesare admissible: give 50 cents or give nothing. After the student volun-teers have written in their contributions, the instructor collects them,calculates their sum m, with $o < m < $2, and matches that sum withan equal amount taken out of his pocket. The total collective good may,therefore, reach $4. Each student volunteer receives from the instructor(or gives to other players) a payoff equal to his share of the collectivegood (sum of students’ contributions + instructor’s matching contri-bution) minus his contribution. Thus, if two players contribute in thegame depicted, their payoff is zero, while the payoffs of the two defec-tors amount to 50 cents each (and the instructor pays out one dollar).The advantage of a classroom simulation of this type is the small

amount of time and comparatively low cost required to run the experi-ment. The drawback, of course, is that only a small group of studentsparticipate in the simulation while the rest of the class only experiencethe simulation secondhand; this passive learning experience is likely tolead to sub-optimal learning (Dennis, 1979). By contrast, with a com-puter simulation, every student can be offered the opportunity to max-imize his learning potential.



370 Computer Simulation

The setting for the c & c simulation is the computer room of the Facultyof Management, University of Manitoba. Individual players are con-nected to one of three local area networks using their own Pc compati-ble work station. The networked simulations are run via a control ma-chine which can be connected to a video projector for instructionalpurposes. Thus, for a simulation with n real persons, n + i machines arerequired.

c & c is totally menu-driven and participants play by flashing a seriesof screens on their monitor:

- The instruction screen tells a short story tailored to the particularsimulation being run.

- The decision screen asks the participants to choose a strategy. In theexperiment reported here, the possible strategies were limited to abinary choice (contribute or defect). c & c also allows the possibilityof a continuous choice of strategies in proportion of one’s budget tobe donated.

- After all participants have selected a strategy, an outcome screenflashes which reports the payoffs obtained. The results are deter-mined by each participant’s choice together with the choices of theother participants. A person’s payoff is, therefore, very much depen-dent upon the choices of others over which he has no control. In anygame, participants can only earn one of two payoffs. While payoffsvary from session to session depending upon the ratio of defectors tototal number of players, a defector (contributor) in one session re-ceives the same payoff as all other defectors (contributors) in thatsession.

- To assist participants in making their choices, it is possible to lookat any time before calling the decision screen at the worksheetscreen. This allows a participant to visually check and modify thepayoffs associated with any combination of contribute vs. defect de-cisions.

- Finally, a message screen is available for networked simulations. Itcan be used to exchange messages with other participants. In this ex-periment, the message facility was not made available to the partic-ipants.

It can be argued that a game theoretic simulation similar to c & c

could be set up without computers in a classroom environment. Stu-dents would be divided into small groups, with each group playing itsown game as depicted in Table i. However, this method would haveseveral disadvantages when compared with the use of c & c.

- A conventional simulation in which every student participateswould be difficult to administer in a large class; by contrast the com-puter simulation being user-friendly requires little supervision fromthe experimenter.



371 - Experimenter’s influence on simulation results, while it can neverbe totally eliminated, can be controlled better in a computer simu-lation than in a classroom simulation. Different groups of playersare more likely to be treated differently by the experimenter in aclassroom context than in a computer simulation.

- Unlike c & c simulations, a classroom simulation does not provideautomatic data memory recording.

- A conventional simulation would take place in the classroom set-ting. By contrast, the computer simulation takes place in a differentsetting and has a dynamics of its own. This is likely to improve stu-dent identification with the simulation and, consequently, studentlearning.

Experimental Procedure

This article seeks to establish whether, in addition to the above advan-tages, participation in the c & c simulation had an impact on student’sresponses to selected test items. It is hypothesized that the quality of an-swers on these items will be higher for students who participated in thecomputer simulation than for those who did not. To test this hypothe-sis, an experiment was designed involving 197 undergraduate manage-ment students. These were students in several sections of a business-

government relations course taught by the same instructor during the1988-89 academic year.The following procedure was used in the experiment:

i. The 104 students who took the course in the first term did not par-ticipate in the c & c simulation; they were passively exposed (withthe exception of student volunteers) to the conventional classroomexercise depicted in Table i. The first term students are referred to asthe control group.

2. In anticipation of the experiment, specially targeted essay and mul-tiple choice questions were included in the first term final exam.These questions were designed to test students’ knowledge and un-derstanding of prisoner’s dilemma-like situations (the questions arereported in the Appendix). Identical questions were included in thesecond term final exam. There were no other questions concerningprisoner’s dilemmas in either final exam.

3. Second term students (experimental group) were required to partici-pate in the c & c simulation or to write an extra assignment (as an-nounced in the course outline). The experimental group is comprisedof 81 out of the 93 students who took the course in the second termand chose to take part in the computer simulation. Second term stu-dents were exposed to the same conventional classroom simulationas their first term counterparts. The instructor took care that in allclasses the same amounts of time and lecture material were used toteach the part of the course related to the final exam items in ques-tion.



372 Table 2 Regression results

t - values in parentheses.

4. Participants in the computer simulation were given a payoff at theend of the experiment in the form of a mark (up to 10% of the finalmark) proportional to the payoff obtained in the game played. Giventhe number of students participating in the experiment, the instruc-tor had to rule out monetary payoffs.

5. Nine c & c simulation sessions/groups were organized with eachparticipant taking part in one session only. There were two gamesplayed in each session. The size of the groups varied from six to ten.The average duration of a session was forty-five minutes.

The hypothesis that the leaming outcome in the experimental group ex-ceeds that of the control group was tested by regressing a dependent var-iable score on the independent variable simulation. Two additional in-dependent variables were included in the regression equation to controlfor the likely impact of a student’s final exam performance and the timeof instruction on the dependent variable. The full model is specified bythe following equation:

score = a + b1 simulat + b2midterm + b3 final + b4 Term + e

where score = the average of a student’s score on the selected exam

questions (in % of maximum score); final = the student’s mark in thefinal exam on a similar scale with the score on selected exam questionssubtracted from the mark; simulat = a dichotomous variable coded iwhen a student participated in the simulation and o otherwise; andterm = a dichotomous variable coded i for students in the experimentalgroup and o for those in the control group.3

FindingsTable 2 shows three different multiple regressions, each using a differ-ent combination of independent variables. Note that the coefficients forthe variables midterm and final change substantially from regression ito regression 2. This, combined with the low t-values for the coeffi-



373 cients associated with these variables is an indication of multicolinear-

ity between the variables midterm and final on one hand, and the vari-ables term and simulation on the other. This is indeed confirmed by theexistence of strong Pearson correlation coefficients between these pairsof variables (.66 and .88 respectively).The best predictor variables are contained in the equation of model 3

(with t ratios in parentheses):Score = .341 + .504 Final Mark + .i19 Simulation

b.03) (4.11) (3-78)where all coefficients are of the right sign and significant at the o.oilevel.The final mark coefficient indicates that a change of one percent in

the variable is associated with a change of one-half of a percent in thevariable score. The simulation coefficient can be interpreted to meanthat participation in the simulation improved a student’s score on theselected test items by a margin of 12%. In spite of the low RSQ andthe relatively small value of the simulation coefficient, the regressionevidence does lend support to the hypothesis that participation in thec & c simulation had a positive impact on test scores.

Recall that the dependent variable in the regression equation is acomposite variable made of four distinct scores. Perhaps some testscores have been more sensitive to student participation in the simula-tion than others. To verify this, a t-test analysis of the data was per-formed, using each test score as a distinct dependent variable. The find-ings are reported in Table 3. In the analysis, the entire student

population was divided into three distinct groups: control group, exper-imental group, and defectors group (this last group representing stu-dents who were offered a chance to play c & c but did not). The t-testprocedure seeks to measure if the differences in mean test scores be-tween the three groups are of the right sign and statistically significant.As the entries in Table 3 indicate, in two out of three dependent varia-bles (test question #1 and multiple choice questions) there is a statisti-cally significant difference between the scores obtained by students inthe control and experimental groups.

It can be inferred from these results that the c & c simulation is amore effective teaching tool than conventional classroom methods.However, the findings that are reported should be interpreted with cau-tion. This is an exploratory test which cannot be considered truly ex-perimental. First, the subjects were not assigned randomly in the con-trol and experimental groups; it is, therefore, possible that some of thedifference in student performance across the two groups is due to differ-ences in student characteristics. However, the possibility is unlikelygiven that the business-government relations course in which the ex-periment was run is a required course for all second year students. Stu-dent population characteristics such as age, cumulative GPA or sex arenormally quite similar from one term to the next. There is also the pos-sibility that, in the absence of a double blinded test, the instructor



374 Table 3 Results of analysis of variance

&dquo;Significant at the .oi level.* * Numbers of answers vary as questions were non-compulsory.

treated the control and experimental groups differently. However, itshould be emphasized that the instructor took great care that the onlydifference in student treatment between the control and experimentalgroups was the participation of the latter in the computer simulation.

Evaluation and Discussion

Prisoner’s dilemma simulations and experiments have been used forpurposes of testing theories about human behavior (Axelrod, 1984) orimproving class attendance (Broder, 1989). But there is also a pedagogicside to such experiments. This study was motivated by the idea thatcomputer simulations such as the c & c program can be cost-effectivetools to aid students in solving decision problems in which the state ofnature is not fixed. The need for such pedagogic aid is all the more ur-gent given ( i ) the difficulty that strategic decisions seem to pose for alarge number of students and (2) the increasing relevance of strategicthought in social science and business education.



375 While the study shows that there exist statistically significant differ-ences in the exam scores of students who took part in the c & c simu-

lation vs. those who did not, the low regression RSQs suggest the possi-bility that other factors have affected students’ scores. However, it ispossible that the simulation that was used in this study, with its partic-ular story line, payoff configuration, and production function, failed tomaximize the pedagogic potential that lies in c & c. Only further test-ing, using a variety of c & c simulation scenarios and payoff structures,and involving many more groups of students over a longer period of timewill tell.The c & c experiment will prove successful in other settings only in-

sofar as students feel free to participate or not in the simulation. Thisinevitably raises the question of the opportunity cost associated withparticipating in the simulation. This consideration is especially rele-vant if the number of players is large. c & c can only accommodate tenplayers at a time on a same network. The instructor and the students aretherefore prisoners of the number of networks available. Assuming twonetworks and a student population of sixty, it would be necessary to runthree sessions, preferably outside peak-use time-which probablymeans that the simulations must be run outside of regular class hoursas well. With only twenty students, however, a single simulation exer-cise can be run easily during class time. As a rule, therefore, it is to beexpected that the smaller the number of participants the smaller theiropportunity costs and the more successful the experiment.The success of the simulation also depends upon students’ expecta-

tions that they will benefit from the exercise. The expected benefit fromparticipating in the simulation is twofold: A student is likely to benefitpersonally through more efficient learning, leading to a higher testscore; he may also receive a high simulation mark as a result of a coop-erative attempt by himself and his fellow players to maximize joint pay-off. While it is true that an expected private benefit (better learning lead-ing to higher test mark) may be realized irrespective of the payoff aplayer receives in the simulation, this expected private benefit likelywill depend in part upon the expected collective benefit of cooperativebehavior in the simulation. A group’s ability to behave cooperatively islargely a function of group size. Thus, although c & c was designed toaccommodate up to ten players, a smaller number of participants ineach group may be more consistent with expectation of high benefit.

Appendix: Exhibit-Test Questions

Essay Questionsi. Explain why laws exist which force all members of a trade or profes-

sion to be a member of some association or pay equivalent dues to acharity.

2. Evaluate the following statement: &dquo;Collective action is merely thesum of individual actions.&dquo;



376 Multiple Choice Questionsi. A free-rider problem exists in the case where:

(a) all citizens must pay for a few to enjoy some benefits(b) all persons pay exactly according to the benefits they receive(c) a few pay all the cost of a program which all the community en-

joys(d) a small group pays for a program from which noncontributors are

excluded

(e) all of the above are different types of free-rider problems

2. Which of the following facilitates oligopolistic coordination?(a) different cost structures among firms

(b) the industry is characterized by large buyers of its output(c) the industry is made out of a small number of equal-size firms(d) a high ratio of fixed to total costs(e) the ability to keep prices secret

Notes

François Pétry is assistant professor of business-government relations in the Faculty ofManagement, University of Mamtoba.1. Joe Oppenheimer et al., c & c: Conflict and Cooperation, an authoring systemfor game theoretic simulations, PDS Software Company, 3210 Pauline Drive, ChevyChase, MD 20815. c & c is available for institutional use by site license in either stand-alone or networked versions. c& c: Conflict and Cooperation was reviewed by R.G.Peckens in Social Science Computer Review 7(3), Fall 1989, pp. 377-78.2. The difficulty with strategic decision lies primarily in the fact that there is a veil ofuncertainty clouding the outcome of such decision. Too often students in social sci-ence are exclusively trained to make linear or probabilistic decisions, that is, decisionswhere the prevailing state of nature can be predicted with either certainty or a mean-ingful probability assignment. On the other hand, strategic decisions are made underconditions of conflict where the state of nature one faces is no longer a known or pre-dictable umverse but a rival decision maker whose choice of action itself depends uponone’s decision. The resulting infinite-regress process renders futile any attempt at as-signing probability that a particular state of nature will occur.3. The term variable was included to control for the few second term students whochose not to participate in the computer simulation.

References

Axelrod, Robert. (1984). The evolution of cooperation. New York: Basic Books.Broder, Josef M. (1989). Laboratory experiments for maintaining class attendance. Pa-

per presented to the annual meeting of the Public Choice Society, Orlando, FL.Denms, J. Richard. (1979). Computer simulation and its instructional uses. (The Illi-

nois Series on Educational Application of Computers, Monograph No. 8e). Illinois:The College of Education, University of Illinois.

Evans, John W. (1971). The Kennedy round in American trade policy. Cambridge: Har-vard University Press.

Laver, Michael. (1977). Intergovernmental policy on multinational corporations, asimple model of tax bargaining. European Journal of Political Research, 5, 363-380.

Prentice, Barry E. & Hildebrand, M. D. (1988). An economic approach to truck weightregulation enforcement. (mimeo) Transport Institute, The University of Mamtoba.



377 Rubin, Jeffrey & Brown, Bert. (1975). The social psychology of bargaining and negoti-ation. New York: Academic Press.

Scherer, F. M. (1980). Industrial market structure and economic performance. Boston:Houghton Mifflin Co.

Senn, Peter R. (1989). Obstacles to computer software in political education. SocialScience Computer Review, 7, 273-285.



“Learning Outcomes of Game-Theoretic Computer Simulation: An Evaluation.” Social Science...

Documents

Transcript of “Learning Outcomes of Game-Theoretic Computer Simulation: An Evaluation.” Social Science...