Punishment Leads to Cooperative Behavior in Structured Societies

Punishment Leads to Cooperative Behaviorin Structured Societies

Shade T. Shutters [email protected] of Sustainability and Center for Social Dynamicsand Complexity, Arizona State University, Tempe, Arizona, 85282

AbstractAltruistic punishment occurs when an agent incurs a cost to punish another but receivesno material benefit for doing so. Despite the seeming irrationality of such behavior, hu-mans in laboratory settings routinely pay to punish others even in anonymous, one-shotsettings. Costly punishment is ubiquitous among social organisms in general and isincreasingly accepted as a mechanism for the evolution of cooperation. Yet if it is truethat punishment explains cooperation, the evolution of altruistic punishment remainsa mystery. In a series of computer simulations I give agents the ability to punish oneanother while playing a continuous prisoner’s dilemma. In simulations without socialstructure, expected behavior evolves—agents do not punish and consequently no coop-eration evolves. Likewise, in simulations with social structure but no ability to punish,no cooperation evolves. However, in simulations where agents are both embedded ina social structure and have the option to inflict costly punishment, cooperation evolvesquite readily. This suggests a simple and broadly applicable explanation of coopera-tion for social organisms that have nonrandom social structure and a predisposition topunish one another. Results with scale-free networks further suggest that nodal degreedistribution plays an important role in determining whether cooperation will evolvein a structured population.

KeywordsContinuous prisoner’s dilemma, agent-based model, altruistic punishment, strongreciprocity, game theory, small-world network, scale-free network.

1 Introduction

The history of life is punctuated by the periodic emergence of new hierarchical levelsof organization, including eukaryotic cells, multicellular life, eusociality, and humaninstitutions (Michod, 1997). These emergent entities typically have global attributesand evolutionary trajectories not predictable through observation of the componentindividuals. This emergence is attributed to the phenomenon of cooperation (Michod,1997; Maynard Smith and Szathmary, 1997).

In addition to theoretical advancement, there are pressing practical reasons forunderstanding cooperative behavior. Many environmental problems today require co-ordinated, collective action among nations of the world if they are to be solved (Oldero,2002; Rees, 2002; Kaul and Mendoza, 2003; Beddoe et al., 2009). As societies, drivenby a desire to avoid their own “tragedy of the commons” (Hardin, 1968), increasinglypush for sustainable management of the earth’s resources, international cooperation isrequired. Given that sustainability is a desirable policy course for a growing number

C© 2012 by the Massachusetts Institute of Technology Evolutionary Computation 20(2): 301–319

S. T. Shutters

of nations (Beddoe et al., 2009), and that cooperation is often a prerequisite for coor-dinated global sustainable management, a fundamental understanding of when andunder what conditions cooperation will emerge becomes imperative.

Yet, the evolution of cooperation remains largely unexplained and is today amongthe greatest questions facing evolutionary theorists (West et al., 2007). Following rejec-tion of the long-held explanation of group selection in the 1960s, two theories emergedthat were thought to explain most instances of cooperation: Hamilton’s (1964) the-ory of inclusive fitness, explaining cooperation between related individuals (especiallybetween nonhuman animals), and Trivers’ (1971) theory of direct reciprocity, explain-ing cooperation between unrelated individuals. These theories were so influential thatmany today still assume, though incorrectly, that nearly all instances of cooperation andaltruism can be explained by them (West et al., 2007). In fact, several theories contributeto our understanding of cooperative and altruistic behavior, but they typically apply tolimited circumstances.

1.1 Inclusive Fitness

Hamilton’s theory of inclusive fitness, or kin selection, defines an individual’s fitnessas a function of how well that individual’s genes are propagated, regardless of whocarries the genes (Hamilton, 1964). Thus an individual acting altruistically toward closerelatives can pass on copies of genes through those relatives’ offspring in addition tothe individual’s own offspring. Relatedness, in this context, can apply to any set ofindividuals that share particular genes of interest, regardless of ancestry (West et al.,2007). Despite its applicability to certain insect and animal groups, inclusive fitness isunsatisfactory for understanding cooperative behavior that is common between unre-lated individuals in human societies (Di Paolo, 1999; Wilson and Holldobler, 2005). Inaddition, long-time proponents of inclusive fitness have recently challenged the validityof the entire theory (Nowak et al., 2010).

1.2 Direct Reciprocity

Direct reciprocity (or reciprocal altruism) is embodied in the phrase “you scratch myback now, I’ll scratch yours later.” This theory asserts that when a population of agentsreciprocates each other’s cooperative behavior, the population will resist invasion by aselfish strategy (Trivers, 1971). Axelrod and Hamilton’s (1981) work with agents playingthe iterated prisoner’s dilemma is one example. However, because its underlying re-quirements and assumptions are so restrictive, direct reciprocity has fallen out of favoras a general theory of cooperation and few researchers still believe it has applicabilitybeyond certain situations involving humans (Dugatkin, 1997; West et al., 2007). In ad-dition, direct reciprocity typically requires long-term repeated interactions and cannotexplain cooperation in anonymous one-shot interactions—a phenomenon growing evermore prevalent in human societies (Nowak and Sigmund, 2005).

1.3 Indirect Reciprocity

In contrast to direct reciprocity, the idea of indirect reciprocity can be summarized as“you scratch my back, I’ll scratch someone else’s” (Nowak and Sigmund, 2005). Accord-ing to this theory, after an agent unconditionally confers a benefit on a second agent, thissecond agent will at some later time confer an unconditional benefit on a third agent

302 Evolutionary Computation Volume 20, Number 2

Punishment Leads to Cooperative Behavior in Structured Societies

and so on (Leimar and Hammerstein, 2001). Like direct reciprocity, indirect reciprocityrequires a lengthy period of repeat interactions, though unlike direct reciprocity, theseinteractions must only be within the same group and not with the same individual. Thismakes indirect reciprocity too a less satisfying explanation of cooperation in anonymousone-shot interactions.

A related concept is that of tag recognition, or the so-called “green-beard” phe-nomenon, in which a benefit is unconditionally conferred on another, but only to recipi-ents displaying a suitable, heritable trait (Macy and Skvoretz, 1998). While previoussimulations have shown that high levels of altruism may evolve in such models(Riolo et al., 2001; Spector and Klein, 2006), others have demonstrated that resultsare highly dependent on model parameters and the conditions under which tags canactually facilitate cooperation may be limited (McDonald and Sen, 2006; Roberts andSherratt, 2002; Hales, 2005; Hales and Edmonds, 2005).

1.4 Multi-Level Selection

Multi-level selection theory asserts that a trait’s frequency can increase in a populationbecause it confers a benefit on a group of individuals, not just the individuals exhibit-ing the trait (West et al., 2007). In contrast to the discredited classical notion of groupselection, the contemporary theory of multi-level selection is theoretically groundedand more general (Wilson and Wilson, 2007) and provides explanatory power for manysocial insect phenomena as well as cultural patterns in isolated human populations(Wilson and Holldobler, 2005; Holldobler and Wilson, 2009). Initially thought to re-quire periods of group isolation and group mixing (Wilson, 1975), the theory was notconsidered a serious explanation of cooperation among organisms that do not formclearly defined groups, including dynamic human societies. Studies of population vis-cosity sought to remedy this, but with mixed results (Queller, 1992; Mitteldorf andWilson, 2000).

Recently, theorists have modified the requirement of clearly defined groups, assert-ing that “groups need not have discrete boundaries; the important feature is that socialinteractions are local, compared to the size of the total population” (Wilson and Wilson,2007). Thus, contemporary multi-level selection theory identifies social structure as aprerequisite for the evolution of cooperative behavior.

1.5 Altruistic Punishment

A promising explanation for the evolution of cooperation is the phenomenon of altruisticpunishment, which occurs when an individual incurs a cost to punish another withoutreceiving a material benefit in return (Gintis, 2000; Bowles and Gintis, 2004). ThoughWest et al. (2007) claim nothing is altruistic about the punishment in these cases, the ter-minology is retained here for consistency with contemporary literature. This mechanismhas repeatedly induced cooperative behavior in laboratory experiments with humans(Fehr and Gachter, 2000, 2002; Boyd et al., 2003; Fehr and Fischbacher, 2004; Gardnerand West, 2004; Fowler, 2005; Fowler et al., 2005), where subjects often pay to punishplayers that are not even in the same game as the punishee (Ostrom et al., 1992; Fehrand Gachter, 2000; Andreoni et al., 2003; Gurerk et al., 2006). Those engaged in statecraftconcur, asserting that punishment mechanisms are prerequisites for cooperation amongnations (Barrett, 2003a, 2003b).

Evolutionary Computation Volume 20, Number 2 303

S. T. Shutters

Punishment is ubiquitous among social organisms. Wherever cooperating indi-viduals have an incentive to cheat, punishment mechanisms often exist as a deterrent(Frank, 1995). This includes restriction of cancer cell growth through preprogrammeddeath or senescence (Sharpless and DePinho, 2005), toxin release in colonial bacteria thataffects only noncooperators (Travisano and Velicer, 2004), the destruction of eggs laid byworkers in social insect colonies (Foster and Ratnieks, 2001), and enforcement of cooper-ation and mating hierarchies in nonhuman mammals (Clutton-Brock and Parker, 1995).Even the process of cellular meiosis can be viewed as a form of policing selfish genes(Michod, 1996). In humans, punishment and policing are pervasive among societies andcultural groups (Marlowe et al., 2008) and neurological research suggests they are tosome degree genetically coded behaviors (de Quervain et al., 2004; Spitzer et al., 2007).

Yet, from a rational choice perspective the phenomenon of altruistic punishment isjust as irrational as cooperation and, as an explanation for cooperation, it only shifts thequestion from “why should an individual cooperate?” to “why should an individualinflict costly punishment?” Thus, a causal mechanism for altruistic punishment is lack-ing and must be explored if altruistic punishment is to serve as an explanation for theevolution of cooperation.

1.6 Social Structure

Like theories of cooperation, explanations of altruistic punishment are often limited bybeing framed in terms of evolutionary game theory (Killingback and Doebeli, 1996) andfail to address the social structure governing interactions between individuals (Jacksonand Watts, 2002). Such explanations assume a system is homogeneous or well-mixed andthat members of the system interact randomly with each other with equal probability.Previous simulations have shown that adding simple two-dimensional space leadsto very different evolutionary outcomes compared to well-mixed population models(Schelling, 1971; Nowak and May, 1992; Nowak et al., 1994). While incorporating socialstructure in simulation models is becoming more accepted, a bias for using regulartwo-dimensional lattices lingers. Real-world network structures of cooperating agentsare known to be far from well-mixed, yet neither do they conform neatly to the regularpattern of a lattice (Amaral et al., 2000; Barabasi and Albert, 1999; Dorogtsev andMendes, 2003).

Social scientists, on the other hand, have long acknowledged the association be-tween complex networks and cooperation (Oliver, 1984; Marwell et al., 1988; Gould,1993; Chwe, 1999, 2000). Yet only recently has the simulation and modeling communitybegun to routinely move beyond regular lattice structures to explore the important rolethat complex social networks play in the evolution of cooperation (Santos and Pacheco,2005; Santos et al., 2006; Chen et al., 2007; Olfati-Saber et al., 2007; Ohtsuki et al., 2006).

It remains then to explore the combined effects of punishment and social structureand whether their combination leads to a broadly applicable explanation of cooperation.As Chwe (1999, p. 128) observed,

Collective action has been studied in two largely disjoint approaches, one focusing on the influ-ence of social structure and another focusing on the incentives for individual participation. Theseapproaches are often seen as competing or even opposed.

Instead of treating these views as competing, this study melds the two approachesby demonstrating that altruistic punishment is a viable mechanism for the evolution of



cooperation but only in structured societies. To test the ability of altruistic punishmentto induce cooperation, a punishment option was incorporated into simulations of thecontinuous prisoners dilemma run on a variety of social structures.

2 Methods

2.1 The Continuous Prisoners Dilemma (CPD)

In the classic prisoner’s dilemma, players are limited to two choices, cooperate ordefect. Here, that requirement is relaxed and players may select a level of cooperationat any point on a continuum between full cooperation and full defection. This presentsan arguably more realistic picture of choices facing those in social dilemmas (Sandler,1999; Killingback and Doebeli, 2002) and is known as the continuous prisoner’s dilemma(CPD). Previous studies have suggested a general payoff function for the CPD take theform

pi = B(xj ) − C(xi) (1)

where the payoff to agent i equals a benefit function B of j ’s contribution less a costfunction C of i’s contribution (Ifti et al., 2004; Killingback and Doebeli, 2002; Doebeliet al., 2004).

In this study, the CPD takes the form of a public goods game between only twoplayers (Santos et al., 2008; Rand et al., 2009) in which both the benefit and cost functionsare linear. In a game between players i and j , each is given an endowment e from whicha portion x is contributed to a public good pool. The total pool is multiplied by a publicgood multiplier r ∈ (1, 2), representing the synergistic effects of cooperation, and thendivided evenly between the players. Here initial endowments are standardized to e = 1and contributions are limited to x ∈ [0, 1], so that x = 0 represents full defection andx = 1 indicates full cooperation. In a simple CPD between players i and j , i’s payoff p

is calculated as

pi = 1 − xi + r(xi + xj )/2 (2)

For any given contribution by j , i’s payoff is maximized by contributing 0. Thusxi = xj = 0 is the expected rational outcome or Nash equilibrium of the prisoner’sdilemma (Binmore, 1992). A dilemma arises, however, because total social welfare,measured as the sum of payoffs earned by both players, is maximized when bothagents cooperate fully.

The addition of altruistic punishment introduces a third player, the observer k,further modifying the game’s payoffs (see Table 1). As in the laboratory setting of Fehrand Gachter (2000), the observer is drawn from the same neighborhood from whichthe observed agent selects game partners. Note that it is possible for the observer k

to be the same agent j selected to play the CPD with i and the probability of thisoccurrence increases as the average network degree decreases. Just as theory predictsrational agents will contribute 0 in a CPD, it also predicts that, regardless of the amountof an agent’s contribution, an observing neighbor will never pay to punish (Fehr andGachter, 2000).


S. T. Shutters

Table 1: Payoffs p in a CPD game between i and j with k observing and potentiallypunishing i∗.

Agent xi ≥ tk xi < tkpayoff (k does not punish i) (k punishes i)

pi 1 − xi + r(xi + xj )/2 1 − xi + r(xi + xj )/2 − ckM

pj 1 − xj + r(xi + xj )/2 1 − xi + r(xi + xj )/2pk 0 −ck

∗See text for descriptions of simulation variables and parameters.

2.2 Game Play

A single simulation run is initiated with the creation of a network. Each node of thenetwork is populated with a single agent i consisting of strategy (xi, ti , ci) where xi =the contribution i makes to the public good when playing against j ; ti = the offer belowwhich the agent will punish another agent in a game being observed by i; and ci = thecost that i is willing to incur to punish the observed agent when the observed agent’scontribution is too low. In other words, when i is acting as an observer of a game, tidetermines if i will punish and ci determines how much i will punish. Each strategycomponent xi, ti , ci ∈ [0, 1] is represented as a floating point value generated randomlyfrom a uniform distribution at the beginning of each simulation. To control for otherfactors that might contribute to the maintenance of cooperation, such as interactionhistory or reputation, agents have no recognition of or memory of other agents withinthe population. Each game is one-shot and anonymous.

During a single CPD game, i initiates an encounter by randomly selecting from itsneighborhood a partner j . The neighborhood consists of all nodes one link away from i

in a given network type. Agents are given their endowment of one unit from which eachsimultaneously contributes to the public good pool. Payoffs are calculated using Table1. The initiating player i then randomly selects a second neighbor k who is tasked withobserving and evaluating i’s contribution. Since selection is without replacement, theobserver may also be the player j . If k judges the contribution to be too low, xi < tk , thenk pays ck to punish i by an amount ckM , where M is the relative strength of punishmentreferred to here as the punishment multiplier. During a single generation, every agentselects three neighbors randomly with replacement, with which it initiates a CPD gameand, on average, every agent is selected three times as a game observer and potentialpunisher. A single simulation run proceeds for 10,000 generations.

Each generation consists of three routines: game play; observation and punishment;and selection and reproduction. During each routine, an agent interacts only with its im-mediate neighbors as defined by the network type and all interactions are synchronous.A payoff variable for each agent P tallies the costs and rewards an agent accumulatesfrom all games during a generation,

∑p. Because this model depicts the simplest case

in which parents do not differentially provision resources for their offspring, P = 0 foreach agent at the beginning of a new generation.

Following game play and punishment, agents compete with one another for theability to pass offspring to the next generation. During reproduction, each agent i ran-domly selects a neighbor j with which to compare P . If Pi > Pj , i’s strategy remainsat i’s node in the next generation. If Pi < Pj , j ’s strategy is copied onto i’s node forthe next generation. In the event that Pi = Pj , a coin toss determines the child strategy.



As with game play, strategy updating is synchronous. As strategies are copied to thenext generation, each strategy component is subject to mutation with a probabilitym = 0.10. If selected for mutation, Gaussian noise with mean = 0 and SD = 0.01 isadded to the component. Should mutation drive a component’s value outside [0, 1],the value is adjusted back to the closer boundary value. Simulation results did not varysignificantly under different mutation parameter values, though the number of gener-ations required before a population converged to either cooperation or defection wasaffected.

2.3 Networks

Following are descriptions of networks used in this study along with details of howthey were generated. All networks are unweighted and undirected. Algorithms used togenerate networks for this study as well as the simulation engine were written in Javaand the code is freely available upon request.

Random networks were generated by starting with a fixed number of nodes (N =400 in this study) and then connecting pairs of nodes randomly until the desired totalnumber of links in the network is reached. If the random generation process resulted ina nonconnected graph, additional links were added until it was connected, resulting inan insignificant variation in total number of links among these networks.

Regular networks, typically lattice structures, are composed of nodes arranged ina regular repeating pattern having the same number of neighbors (nodal degree). Inaddition, such structures are toroidal, meaning that they have no edges but insteadloop around onto themselves in a fashion similar to the surface of a sphere. Two regularnetworks used commonly in simulations, including this study, are the von Neumannnetwork, in which each node has four neighbors, and the Moore network, with eachnode connected to eight neighbors. Hexagonal networks are also used here as well aslinear networks (rings).

Random regular networks retain one element of regular networks in that all nodesare linked to an identical number of neighbors. However, neighbors are selected ran-domly, without regard for spatial proximity. In this study, random regular networkshave degree = 4.

Small-world networks have features of both random and regular networks and rep-resent interaction patterns commonly observed in human societies (Watts and Strogatz,1998). In particular, they exhibit the high clustering of regular networks and the smalldiameter of random networks. Small-world networks were generated starting with alinear network and then cutting and randomly reconnecting each link in the networkwith a probability of 0.05. Each node in the initial linear substrate was linked to the twonodes on either side (neighborhood radius = 2).

Scale-free networks, ubiquitous in real-world systems (Barabasi and Albert, 1999),are characterized by a power law distribution of nodal degrees with a small number ofhighly connected nodes and many nodes that are minimally connected to the network.These networks are generated by growth through preferential attachment. That is, anetwork is grown by adding single nodes to an initial seed network of three connectednodes, until the desired number of total nodes is reached. As a new node is added,the points within the existing network at which it is attached is probability-based.The probability that a specific node in the existing network will be the point of newattachment is proportional to the number of existing links each node already has. Inthis study, each node was added to the growing network by two new links.


S. T. Shutters

Figure 1: Results using four different social structures. Mean CPD contributions versusM are presented for populations on (a) complete, (b) regular (Moore), (c) small-world,and (d) scale-free networks. Each point represents a population’s mean contribution inthe 10,000th generation of a single simulation run. Simulations on small-world networksclearly demonstrate a transition to cooperative behavior as M increases while scale-freenetworks exhibit no such effect.

2.4 Simulation Variables and Output

The important parameter governing altruistic punishment is the ratio of costs incurredby the punishing party to those of the party being punished (Casari, 2005; Shutters,2009). Defined above as the punishment multiplier M , this parameter is an indication ofthe strength of punishment and, along with network type, is an independent variablein these simulations. The dependent variable of interest is the mean contribution towhich a population evolves by the 10,000th generation and is a measure of the generalcooperative level of the population.

Data were collected in two sets. In the first, 100 simulation replications were con-ducted at M = 0.0 and at subsequent values of M in increments of 0.5, up to M = 6.0.This allowed for an analysis of variances in outcomes for a given simulation parameterset. In the second set, a parameter sweep of M was conducted so that a single simulationwas run at 0.0 and at subsequent values of M in increments of 0.01 up to M = 6.0 (seeFigure 1). This allowed for an analysis of the effect of M at higher resolution but atthe cost of no replications. Descriptions of simulation parameters and their values arepresented in Table 2.



Table 2: Simulation parameters and their values used in the CPD.

Parameter Value

Population size (N ) 400Generations per simulation run 10,000Games initiated by each agent per generation 3Value range for strategy components (x, t, c, s) [0, 1]Probability of strategy component mutation (m) 0.1Mean ± SD of Gaussian noise added during mutation 0 ± 0.01Punishment multiplier (M) [0, 6]∗

Public good multiplier (r) 1.5∗In increments of 0.01 (one replicant per) and in increments of 0.5 (100 replicants per).

Table 3: Mean (SD) of ending contributions on different networks, without punishment(M = 0.0) and with punishment (M = 4.0).

Mean contribution (SD)

Network Type M = 0.0 M = 4.0

Complete graph 0.003 (0.001) 0.030 (0.010)

Regular networksMoore 0.004 (0.001) 0.990 (0.017)Hexagonal 0.005 (0.001) 0.997 (0.002)von Neumann 0.005 (0.001) 0.998 (0.001)Linear 0.006 (0.001) 0.996 (0.002)

Complex, real-world networksSmall-world 0.006 (0.001) 0.997 (0.001)Scale-free 0.490 (0.310) 0.666 (0.232)

Other networksRandom 0.023 (0.012) 0.455 (0.284)Random regular 0.005 (0.001) 0.999 (0.001)

3 Results

3.1 Control Case: No Social Structure, No Punishment

For control purposes, the initial population was simulated on a complete graph with noaltruistic punishment. As predicted by rational choice theory, the population evolvedcontribution rates of approximately 0. In the absence of both punishment and socialstructure, no cooperation was exhibited.

3.2 Either Punishment or Social Structure Alone

In the second set of simulations, populations were subjected to alternate treatmentsof either social structure or punishment. First, with punishment disabled, simulationswere run on a variety of network structures (see Table 3) with the result that socialstructure alone did not drive outcomes from the Nash equilibrium and no cooperation


S. T. Shutters

evolved. Only in the anomalous case of scale-free networks did contributions deviatefrom the expected contribution rate of approximately 0.

Next, using a well-mixed complete graph analogous to no social structure, simu-lations were run with punishment enabled. Despite having the ability to punish eachother, populations lacking structure continued to evolve to the Nash equilibrium withincreasing M (see Figure 1(a)). Neither social structure alone nor punishment alonewas sufficient to induce the population to evolve away from noncooperative behavior.Again, these results concur with rational expectations.

3.3 Punishment and Social Structure Together

In the final round of simulations, the CPD was played using both structured popula-tions and altruistic punishment while the punishment multiplier M was systematicallyvaried. The results are presented in Figure 1. With increasing M , punishment eventu-ally led to nearly full cooperation on Moore and small-world networks. In each case,as M increased, populations underwent a rapid transition from contributions of ap-proximately 0 to contributions near 1 (Figure 1(b) and 1(c)). This flip from nearly fulldefection to nearly full cooperation also occurred in simulations run on von Neumann,hexagonal, and linear (ring) networks (see Table 3).

Interestingly, populations using scale-free networks neither evolved to the Nashequilibrium nor showed any significant response to the introduction of altruistic pun-ishment. Though Figure 1(d) reveals a slight positive trend in mean contributions un-der scale-free networks with increasing M (R2 = 0.007), the trend is not significant(Spearman rank order correlations, p = .086).

4 Discussion

4.1 The Absence of Cooperation in Structured Populations WithoutPunishment

Other than simulations with scale-free networks discussed below, social structure alonewas not adequate to facilitate cooperation. However, other studies have suggested thatthe mere addition of social structure, without punishment, is sufficient for cooperationto evolve.

Ifti et al. (2004) show that agents playing a CPD in a regular lattice evolve coopera-tive contribution levels. However, important differences exist between their model andthe current model. First, the Ifti version of the CPD uses a nonlinear function to calcu-late agent payoffs while the current model uses a linear function. This fact alone maymake the comparability between models limited. In addition, Ifti used asynchronousupdating of strategies while the current model employs synchronous game play andstrategy updating. Finally, it is important to note that populations in the Ifti studyevolved contributions of about 9% of the theoretical maximum contribution and so theconclusion that cooperation evolved is debatable.

Similarly, Nowak et al. (1994) use a simulation with asynchronous updating todemonstrate that cooperation may evolve in structured populations. However, in ad-dition to the different updating mechanism, the authors use an interesting concept ofcontinuous space, in which agents are arrayed randomly across a continuous landscape.This is quite different from the idea of a regular lattice but may have relevance to agentsembedded in complex networks.



In another study, Boyd and Richerson (2002) show that cooperative traits prop-agating via replicator dynamics can persist in populations if those populations arestructured. However, structure in this case is more aptly described as a metapopulationmodel with distinct groups of agents that interact at times within group and at othertimes between groups. The authors simulate a plausible scenario of multilevel selectionbut this scenario may not be comparable to models in which social structure does notinclude distinct groups.

Recently, Taylor et al. (2007), extending the work of Ohtsuki et al. (2006), demon-strated the spread of cooperation under certain conditions on homogeneous networks,but allowed weights along network edges to vary, while weights in the current studyare all equivalent. In addition, Taylor generated different edge weights depending onwhether the linked nodes were engaged in game interactions or reproductive interac-tions. Each node was effectively a member of two networks, an interaction network anda dispersal network, while in the current study there is no distinction between routineinteractions and dispersal.

4.2 Structure as a Prerequisite for the Evolution of Altruistic Punishment

Once social structure was added to punishment, and agents were restricted to localinteractions, cooperation typically evolved at M above some specific value. This resultsuggests one possible explanation for the rise of the seemingly irrational phenomenon ofaltruistic punishment. Results are robust when moving from artificial social structures,such as regular lattices, to stochastically generated small-world networks that morerealistically represent complex human interaction patterns. Furthermore, because socialnetworks are not exclusive to human societies (Fewell, 2003; Lusseau and Newman,2004; Flack et al., 2006) this finding may have broad applicability wherever socialorganisms engage in costly punishment.

Under the parameters of these simulations it is clear that punishment, as a mech-anism for the evolution of cooperation, is only a viable explanation within structuredpopulations. This result suggests that Chwe (1999) is correct in the call for a meldingof those exploring social structure and those studying individual incentives as mecha-nisms for collective action. However, it should be noted that this study demonstrateshow punishment may thrive in a population and subsequently leads to cooperativebehavior, but it does not directly address how punishment behavior might invade anonpunishing population.

Others have similiarly demonstrated that cooperative behavior requires local inter-actions facilitated by social structure. In their evolutionary simulations of tag-mediatedaltruism, Spector and Klein (2006) found that in unstructed populations, altruistic actsoccured in less than 4% of interactions. However, when interactions were restricted tonearest neighbors in a regular lattice, the rate of altruistic acts evolved to more than 84%.

4.3 Cooperation and Network Density

These results further shed light on a contemporary debate among network scientistsregarding the role that social networks play in facilitating cooperation. In particular,scholars disagree on the effect that network density has on the ability of a networkedpopulation to evolve cooperative behavior. It has long been asserted that when a pop-ulation is more densely connected, the likelihood of cooperation is increased (Marwelland Oliver, 1993; Opp and Gern, 1993). This point is supported in simulations using


S. T. Shutters

Table 4: Approximate value of M at which populations transitioned from defection tocooperation on regular networks, which have no variance in nodal degrees.

Network type Number of neighbors Approximate transition value of M

Linear 2 1.5von Neumann 4 1.8Hexagonal 6 2.2Moore 8 2.8Complete N − 1 —∗No transition to cooperation occurred on complete networks even at values as high as M = 5,000.

structured populations in which cost-benefit ratios are manipulated in a game with anonlinear payoff function (Jun and Sethi, 2007). Yet recent research suggests the oppo-site, demonstrating that dense networks inhibit cooperation in structured populations(Flache, 2002; Takacs et al., 2008; Flache and Macy, 1996) and that increasing averagedegree requires increasing the relative benefit of cooperative acts (Ohtsuki et al., 2006).Results from this study strongly support the latter view. Simulations using the max-imally dense complete network never evolved cooperation, even with a punishmentmultiplier at the unrealistic value of M = 5,000.

This view is further supported by the trend among regular networks. Though fullcooperation eventually evolved on all regular networks (excluding complete networks),the severity of punishment, measured as the magnitude of M , required to move the pop-ulation from defectors to cooperators increased as the density of the network increased(see Table 4). The denser the network, measured as the number of neighbors per agent,the stronger the punishment required to evolve cooperation. This finding is similar tothat of Ifti et al. (2004), which showed that as neighborhood size increases beyond acritical threshold, cooperation collapses.

4.4 The Anomaly of Scale-Free Networks

An unexpected result was the response to punishment of populations embedded inscale-free networks. Unlike other networks used, populations on scale-free networksappear to be unresponsive to punishment as M increases. This is an especially impor-tant result given that scale-free architecture is pervasive in both natural and human-engineered systems (Barabasi and Albert, 1999). Though some studies of cooperativeevolution on networks have shown no qualitative difference in simulation outcomesbetween scale-free and other networks (Ohtsuki et al., 2006; Taylor et al., 2007), oth-ers have noted unique properties of scale-free networks with respect to cooperativeoutcomes (Santos and Pacheco, 2005).

To ensure that the results were not due to an inadequate sampling of punishmentstrength, simulations were run on scale-free networks at M = 5,000 but again resultedin no convergence. To further confirm that populations on scale-free networks simplyconverge to fixation more slowly than other networks, simulations with M = 1.5 wereextended to 200,000 generations. However, even after increasing the evolutionary timeperiod 20 fold, no convergence in contribution rates occurred.

One explanation of this result is suggested by the relationship between cooperativethresholds of M and network degree (see Table 4). Because scale-free networks display



Figure 2: Population response to increasing punishment strength on different networktypes.

a power law distribution of nodal degrees, some parts of the networks will exhibit lowdegree and others very high, giving a mixture of areas that are locally conducive tocooperation and others that are not. This does not, however, explain the emergence ofintermediate levels of cooperation when M = 0 (i.e., population mean ending offer ofabout 0.50), as all networks in Table 4 evolve no cooperation at M = 0. On the contrary,this suggests that cases of cooperation evolving on scale-free networks (see Figure 1(d))are due less to punishment and more to the peculiarities of scale-free architecture (Santosand Pacheco, 2005, 2006).

In particular, it may be that the few highly connected nodes are driving the behaviorin scale-free simulations and that ending levels of cooperation merely reflect the initialtrait values of those highly connected agents. Because fitness during a generation is thesum of payoffs of all games, highly connected agents will, on average, play substantiallymore games than the large number of agents with few connections. Thus, a highlyconnected agent may earn a higher payoff despite an inferior strategy simply because itplays more games. The inferior strategy may then persist among other highly connectednodes.

Future work should seek to understand how cooperative outcomes are affected bythe nodal degree distribution of a population’s network structure.

4.5 Evolutionary Dynamics

The results presented thus far have consisted of a population’s mean contribution at theend of 10,000 simulated generations. Though data processing demands preclude a fullanalysis of population dynamics across time, generation-level data were collected forthree simulation runs using small-world networks. Trajectories were selected to giveexamples of a population that evolved to full defection at low strength of punishment(M = 0.5), a population that evolved to full cooperation at higher strength of punishment(M = 3.0), and a population that evolved to an intermediate level of cooperation at M =1.75, corresponding to the chaotic transition range in Figure 1(c).

A summary of these dynamics are presented in Figure 3. Simulations run at M =0.5 and M = 3.0 converged to full defection and full cooperation, respectively, within


S. T. Shutters

Figure 3: Evolutionary dynamics on small-world networks. Though simulations runat M = 0.5 and M = 3.0 converge to full defection and full cooperation, respectively,within 200 generations (a), the simulation run at M = 1.75 did not converge even after10,000 generations, at which point population level data are collected for analysis.Extending simulations with M = 1.75 to 100,000 generations (not shown) still resultedin no convergence. The end point of each curve in (b) corresponds to a single point inFigure 1(c).

200 generations. At values of M near the transition between defection and coopera-tion (M = 1.75 in this case), the population’s mean contribution rate did not convergeover time to either cooperation or defection but instead drifted unpredictably. To de-termine whether the population was simply converging more slowly at punishmentstrengths near the transition point, the simulation run at M = 1.75 was extended to100,000 generations (not shown) but still exhibited no convergence in contributionrate.

5 Conclusion

Despite years of research on the evolution of cooperation, there remains a need for abroadly applicable explanation. The mechanism of altruistic punishment has shownpromise as a general theory, yet it lacks an explanation of its own. Simple systems-level models predict that altruistic punishment should not evolve. However, thesesimple models often fail to incorporate the existence of complex social structure. Inthis study, populations embedded in complex networks were given the option ofpunishing one another while playing the continuous prisoner’s dilemma. In controlcases without social structure, the results matched expectations and only defectionevolved. However, simply embedding agents in a network, which localized social in-teractions, allowed punishment to drive the evolution of fully cooperative populations.Because a recently broadened theory of multilevel selection requires that social in-teractions be local, this study supports this idea and suggests that renewed interestin multilevel selection as an evolutionary mechanism for group-level adaptation iswarranted.



Acknowledgments

David Hales, Ann Kinzig, Kim Hill, and three anonymous reviewers provided criticalfeedback on this work. This material is based upon work supported in part by the Na-tional Science Foundation, through a Graduate Research Fellowship, and in part by anIGERT Fellowship in Urban Ecology from Arizona State University. Code will be madefreely available upon request. Any opinions, findings, and conclusions or recommen-dations expressed in this material are those of the author and do not necessarily reflectthe views of the National Science Foundation.

References

Amaral, L. A. N., Scala, A., Barthelemy, M., and Stanley, H. E. (2000). Classes of small-worldnetworks. Proceedings of the National Academy of Sciences of the United States of America,97(21):11149–11152.

Andreoni, J., Harbaugh, W., and Vesterlund, L. (2003). The carrot or the stick: Rewards, punish-ments, and cooperation. American Economic Review, 93(3):893–902.

Axelrod, R., and Hamilton, W. D. (1981). The evolution of cooperation. Science, 211(4489):1390–1396.

Barabasi, A. L., and Albert, R. (1999). Emergence of scaling in random networks. Science,286(5439):509–512.

Barrett, S. (2003a). Creating incentives for cooperation: Strategic choices. In I. Kaul, P. Conceicao, K. Le Goulven, and R. U. Mendoza (Eds.), Providing global public goods (pp. 308–328).Oxford, UK: Oxford University Press.

Barrett, S. (2003b). Environment and statecraft: The strategy of environmental treaty-making. Oxford,UK: Oxford University Press.

Beddoe, R., Costanza, R., Farley, J., Garza, E., Kent, J., Kubiszewski, I., Martinez, L., McCowen,T., Murphy, K., Myers, N., Ogden, Z., Stapleton, K., and Woodward, J. (2009). Overcomingsystemic roadblocks to sustainability: The evolutionary redesign of worldviews, institutions,and technologies. Proceedings of the National Academy of Sciences of the United States of America,106(8):2483–2489.

Binmore, K. (1992). Fun and games: A text on game theory. Lexington, MA: D. C. Heath andCompany.

Bowles, S., and Gintis, H. (2004). The evolution of strong reciprocity: Cooperation in heteroge-neous populations. Theoretical Population Biology, 65(1):17–28.

Boyd, R., Gintis, H., Bowles, S., and Richerson, P. J. (2003). The evolution of altruistic punishment.Proceedings of the National Academy of Sciences of the United States of America, 100(6):3531–3535.

Boyd, R., and Richerson, P. J. (2002). Group beneficial norms can spread rapidly in a structuredpopulation. Journal of Theoretical Biology, 215(3):287–296.

Casari, M. (2005). On the design of peer punishment experiments. Experimental Economics,8(2):107–115.

Chen, X. J., Fu, F., and Wang, L. (2007). Prisoner’s dilemma on community networks. Physica A:Statistical Mechanics and Its Applications, 378(2):512–518.

Chwe, M. S. Y. (1999). Structure and strategy in collective action. American Journal of Sociology,105(1):128–156.


S. T. Shutters

Chwe, M. S. Y. (2000). Communication and coordination in social networks. Review of EconomicStudies, 67(1):1–16.

Clutton-Brock, T. H., and Parker, G. A. (1995). Punishment in animal societies. Nature,373(6511):209–216.

de Quervain, D. J. F., Fischbacher, U., Treyer, V., Schelthammer, M., Schnyder, U., Buck, A., andFehr, E. (2004). The neural basis of altruistic punishment. Science, 305(5688):1254–1258.

Di Paolo, E. A. (1999). A little more than kind and less than kin: The unwarranted use ofkin selection in spatial models of communication. In D. Floreano, J. D. Nicoud, andF. Mondada (Eds.), Proceedings of Advances in Artificial Life, ECAL ’99. Lecture Notes inArtificial Intelligence, Vol. 1674 (pp. 504–513). Berlin: Springer.

Doebeli, M., Hauert, C., and Killingback, T. (2004). The evolutionary origin of cooperators anddefectors. Science, 306(5697):859–862.

Dorogtsev, S. N., and Mendes, J. F. (2003). Evolution of networks: from biological nets to the Internetand WWW. Oxford, UK: Oxford University Press.

Dugatkin, L. A. (1997). Cooperation among animals: An evolutionary perspective. Oxford, UK: OxfordUniversity Press.

Fehr, E., and Fischbacher, U. (2004). Third-party punishment and social norms. Evolution andHuman Behavior, 25(2):63–87.

Fehr, E., and Gachter, S. (2000). Cooperation and punishment in public goods experiments.American Economic Review, 90(4):980–994.

Fehr, E., and Gachter, S. (2002). Altruistic punishment in humans. Nature, 415(6868):137–140.

Fewell, J. H. (2003). Social insect networks. Science, 301(5641):1867–1870.

Flache, A. (2002). The rational weakness of strong ties: Failure of group solidarity in a highlycohesive group of rational agents. Journal of Mathematical Sociology, 26(3):189–216.

Flache, A., and Macy, M. W. (1996). The weakness of strong ties: Collective action failure in ahighly cohesive group. Journal of Mathematical Sociology, 21(1-2):3–28.

Flack, J. C., Girvan, M., de Waal, F. B. M., and Krakauer, D. C. (2006). Policing stabilizes construc-tion of social niches in primates. Nature, 439(7075):426–429.

Foster, K. R., and Ratnieks, F. L. W. (2001). Convergent evolution of worker policing by eggeating in the honeybee and common wasp. Proceedings of the Royal Society of London. SeriesB: Biological Sciences, 268(1463):169–174.

Fowler, J. H. (2005). Altruistic punishment and the origin of cooperation. Proceedings of the NationalAcademy of Sciences of the United States of America, 102(19):7047–7049.

Fowler, J. H., Johnson, T., and Smirnov, O. (2005). Egalitarian motive and altruistic punishment.Nature, 433(7021):E1.

Frank, S. A. (1995). Mutual policing and repression of competition in the evolution of cooperativegroups. Nature, 377(6549):520–522.

Gardner, A., and West, S. A. (2004). Cooperation and punishment, especially in humans. AmericanNaturalist, 164(6):753–764.

Gintis, H. (2000). Strong reciprocity and human sociality. Journal of Theoretical Biology, 206(2):169–179.

Gould, R. V. (1993). Collective action and network structure. American Sociological Review,58(2):182–196.



Gurerk, O., Irlenbusch, B., and Rockenbach, B. (2006). The competitive advantage of sanctioninginstitutions. Science, 312(5770):108–111.

Hales, D. (2005). Change your tags fast! A necessary condition for cooperation? In P. Davidsson,B. Logan, and K. Takadama (Eds.), Multi-agent and multi-agent-based simulation. Lecture Notesin Computer Science, Vol. 3415 (pp. 89–98). Berlin: Springer.

Hales, D., and Edmonds, B. (2005). Applying a socially inspired technique (tags) to improvecooperation in p2p networks. IEEE Transactions on Systems Man and Cybernetics Part A—Systems and Humans, 35(3):385–395.

Hamilton, W. D. (1964). Genetical evolution of social behaviour 1, 2. Journal of Theoretical Biology,7(1):1–52.

Hardin, G. (1968). Tragedy of commons. Science, 162(3859):1243–1248.

Holldobler, B., and Wilson, E. O. (2009). The superorganism: The beauty, elegance, and strangeness ofinsect societies. New York: W. W. Norton.

Ifti, M., Killingback, T., and Doebelic, M. (2004). Effects of neighbourhood size and connectivityon the spatial continuous prisoner’s dilemma. Journal of Theoretical Biology, 231(1):97–106.

Jackson, M. O., and Watts, A. (2002). The evolution of social and economic networks. Journal ofEconomic Theory, 106(2):265–295.

Jun, T., and Sethi, R. (2007). Neighborhood structure and the evolution of cooperation. Journal ofEvolutionary Economics, 17(5):623–646.

Kaul, I., and Mendoza, R. U. (2003). Advancing the concept of public goods. In I. Kaul, P. Conceio,K. L. Goulven, and R. U. Mendoza (Eds.), Providing global public goods: Managing globalization,(pp. 78–111). Oxford, UK: Oxford University Press.

Killingback, T., and Doebeli, M. (1996). Spatial evolutionary game theory: Hawks and dovesrevisited. Proceedings of the Royal Society of London. Series B: Biological Sciences, 263(1374):1135–1144.

Killingback, T., and Doebeli, M. (2002). The continuous prisoner’s dilemma and the evolutionof cooperation through reciprocal altruism with variable investment. American Naturalist,160(4):421–438.

Leimar, O., and Hammerstein, P. (2001). Evolution of cooperation through indirect reciprocity.Proceedings of the Royal Society of London. Series B: Biological Sciences, 268(1468):745–753.

Lusseau, D., and Newman, M. E. J. (2004). Identifying the role that animals play in their socialnetworks. Proceedings of the Royal Society of London. Series B: Biological Sciences, 271:S477–S481.

Macy, M. W., and Skvoretz, J. (1998). The evolution of trust and cooperation between strangers:A computational model. American Sociological Review, 63(5):638–660.

Marlowe, F. W., Berbesque, J. C., Barr, A., Barrett, C., Bolyanatz, A., Cardenas, J. C., Ensminger,J., Gurven, M., Gwako, E., Henrich, J., Henrich, N., Lesorogol, C., McElreath, R., and Tracer,D. (2008). More “altruistic” punishment in larger societies. Proceedings of the Royal Society ofLondon. Series B: Biological Sciences, 275(1634):587–590.

Marwell, G., and Oliver, P. E. (1993). Critical mass in collective action. Cambridge, UK: CambridgeUniversity Publishing.

Marwell, G., Oliver, P. E., and Prahl, R. (1988). Social networks and collective action: A theory ofthe critical mass III. American Journal of Sociology, 94(3):502–534.

Maynard Smith, J., and Szathmary, E. (1997). The major transitions in evolution. Oxford, UK: OxfordUniversity Press.


S. T. Shutters

McDonald, A., and Sen, S. (2006). The success and failure of tag-mediated evolution of cooper-ation. In K. Tuyls, P. J. Hoen, K. Verbeeck, and S. Sen (Eds.), Learning and adaption in multi-agent systems. Lecture Notes in Computer Science, Vol. 3898 (pp. 155–164). Berlin: Springer-Verlag.

Michod, R. E. (1996). Cooperation and conflict in the evolution of individuality II: Conflictmediation. Proceedings of the Royal Society of London. Series B: Biological Sciences, 263:813–822.

Michod, R. E. (1997). Cooperation and conflict in the evolution of individuality I: Multilevelselection of the organism. American Naturalist, 149(4):607–645.

Mitteldorf, J., and Wilson, D. S. (2000). Population viscosity and the evolution of altruism. Journalof Theoretical Biology, 204(4):481–496.

Nowak, M. A., Bonhoeffer, S., and May, R. M. (1994). Spatial games and the maintenance ofcooperation. Proceedings of the National Academy of Sciences of the United States of America,91:4877–4881.

Nowak, M. A., and May, R. M. (1992). Evolutionary games and spatial chaos. Nature,359(6398):826–829.

Nowak, M. A., and Sigmund, K. (2005). Evolution of indirect reciprocity. Nature, 437(7063):1291–1298.

Nowak, M. A., Tarnita, C. E., and Wilson, E. O. (2010). The evolution of eusociality. Nature,466(7310):1057–1062.

Ohtsuki, H., Hauert, C., Lieberman, E., and Nowak, M. A. (2006). A simple rule for the evolutionof cooperation on graphs and social networks. Nature, 441(7092):502–505.

Oldero, K. K. (2002). Collective action, inaction, and the global commons. In The Commons inan age of globalization. Proceedings of the Ninth Conference of the International Association for theStudy of Common Property.

Olfati-Saber, R., Fax, J. A., and Murray, R. M. (2007). Consensus and cooperation in networkedmulti-agent systems. Proceedings of the IEEE, 95(1):215–233.

Oliver, P. (1984). If you don’t do it, nobody else will: Active and token contributors to localcollective action. American Sociological Review, 49(5):601–610.

Opp, K. D., and Gern, C. (1993). Dissident groups, personal networks, and spontaneouscooperation—The East-German revolution of 1989. American Sociological Review, 58(5):659–680.

Ostrom, E., Walker, J., and Gardner, R. (1992). Covenants with and without a sword: Self-governance is possible. American Political Science Review, 86(2):404–417.

Queller, D. C. (1992). Does population viscosity promote kin selection? Trends in Ecology &Evolution, 7(10):322–324.

Rand, D. G., Dreber, A., Ellingsen, T., Fudenberg, D., and Nowak, M. A. (2009). Positive interac-tions promote public cooperation. Science, 325:1272–1275.

Rees, W. E. (2002). An ecological economics perspective on sustainability and prospects for endingpoverty. Population and Environment, 24(1):15–46.

Riolo, R. L., Cohen, M. D., and Axelrod, R. (2001). Evolution of cooperation without reciprocity.Nature, 414(6862):441–443.

Roberts, G., and Sherratt, T. N. (2002). Does similarity breed cooperation? Nature, 418(6897):499–500.



Sandler, T. (1999). International cooperation and the international commons. Duke EnvironmentalLaw & Policy Forum, 10:131–145.

Santos, F. C., and Pacheco, J. M. (2005). Scale-free networks provide a unifying framework forthe emergence of cooperation. Physical Review Letters, 95(9/098104):1–4.

Santos, F. C., and Pacheco, J. M. (2006). A new route to the evolution of cooperation. Journal ofEvolutionary Biology, 19(3):726–733.

Santos, F. C., Rodrigues, J. F., and Pacheco, J. M. (2006). Graph topology plays a determinant rolein the evolution of cooperation. Proceedings of the Royal Society of London. Series B: BiologicalSciences, 273(1582):51–55.

Santos, F. C., Santos, M. D., and Pacheco, J. M. (2008). Social diversity promotes the emergenceof cooperation in public goods games. Nature, 454(7201):213–216.

Schelling, T. C. (1971). Dynamic models of segregation. Journal of Mathematical Sociology, 1(2):143–186.

Sharpless, N. E., and DePinho, R. A. (2005). Cancer: Crime and punishment. Nature, 436(7051):636–637.

Shutters, S. T. (2009). Strong reciprocity, social structure, and the evolution of fair allocations in asimulated ultimatum game. Computational and Mathematical Organization Theory, 15(2):64–77.

Spector, L., and Klein, J. (2006). Genetic stability and territorial structure facilitate the evolutionof tag-mediated altruism. Artificial Life, 12(4):553–560.

Spitzer, M., Fischbacher, U., Herrnberger, B., Gron, G., and Fehr, E. (2007). The neural signatureof social norm compliance. Neuron, 56(1):185–196.

Takacs, K., Janky, B., and Flache, A. (2008). Collective action and network change. Social Networks,30(3):177–189.

Taylor, P. D., Day, T., and Wild, G. (2007). Evolution of cooperation in a finite homogeneous graph.Nature, 447(7143):469–472.

Travisano, M., and Velicer, G. J. (2004). Strategies of microbial cheater control. Trends in Microbi-ology, 12(2):72–78.

Trivers, R. L. (1971). Evolution of reciprocal altruism. Quarterly Review of Biology, 46(1):35–57.

Watts, D. J., and Strogatz, S. H. (1998). Collective dynamics of “small-world” networks. Nature,393(6684):440–442.

West, S. A., Griffin, A. S., and Gardner, A. (2007). Social semantics: Altruism, cooperation,mutualism, strong reciprocity and group selection. Journal of Evolutionary Biology, 20(2):415–432.

Wilson, D. S. (1975). Theory of group selection. Proceedings of the National Academy of Sciences ofthe United States of America, 72(1):143–146.

Wilson, D. S., and Wilson, E. O. (2007). Rethinking the theoretical foundation of sociobiology.Quarterly Review of Biology, 82(4):327–348.

Wilson, E. O., and Holldobler, B. (2005). Eusociality: Origin and consequences. Proceedings of theNational Academy of Sciences of the United States of America, 102(38):13367–13371.


Punishment Leads to Cooperative Behavior in Structured Societies

Documents

Transcript of Punishment Leads to Cooperative Behavior in Structured Societies