Anchors of Strategic Reasoning in the Traveler's Dilemma

44

Transcript of Anchors of Strategic Reasoning in the Traveler's Dilemma

Anchors of Strategic Reasoning in the Traveler's

Dilemma

Hanh T. Tong and David J. Freeman *

August 17, 2021

Abstract

We experimentally study players' initial beliefs about non-strategic play that an-

chors their strategic reasoning in the traveler's dilemma, a game in which each player

chooses a number and has the incentive to undercut their opponent by the minimal

amount possible. In a within-subject design, each subject repeatedly plays variations

of the traveler's dilemma game without feedback. To identify their strategic reasoning,

we vary the upper and lower bounds of the strategy space in each round, and also vary

the reward/penalty for undercutting. We �nd that players are both heterogeneous in

the amount that they reason, and in their beliefs about non-strategic play. Notably,

few players anchor their strategic reasoning on non-strategic uniform random play. We

also �nd ample evidence of non-strategic play. Our results caution against the common

practice of assuming the same anchor of initial reasoning for all players when estimating

players' depths of strategic reasoning.

*Tong: Theory+Practice, [email protected]. Freeman: Simon Fraser University,[email protected]. We thank SSHRC-SFU Institutional Grant �Initial Anchors of Strategic Rea-soning� for funding this work. This study was approved by the SFU ORE study #2016s0380. We thankAzraf Ahmad, Yoram Halevy, Erik Kimbrough, Terri Kneeland, Sabine Kroger, Kevin Laughren, ShihEn Lu, Alia McCutcheon, Luba Petersen, David Rojo Arjona, James Wright, participants at the REBEL2019, 4th Coller Conference on Behavioral Economics, ESA World Meeting 2019, and ESA North AmericanMeeting 2019, and especially two anonymous referees for comments and suggestions that greatly improvedthe paper.

1

1 Introduction

Decades of lab experiments have documented that play by human subjects often violates the

predictions of Nash equilibrium in games without feedback. The most prominent class of

models in behavioral game theory explains these deviations as a result of limited strategic

reasoning (Nagel 1995, Stahl and Wilson 1995). In these models, a player anchors their

strategic reasoning with initial beliefs about the play of a non-strategic player. To select

an action, they iteratively calculate best-replies a �nite number of times starting with non-

strategic play. Much of the behavioral game theory literature has focused on estimating

players' strategic sophistication while controlling for initial beliefs. It does so by studying

games where di�erent plausible initial models of non-strategic play lead to identical best

replies (e.g. Nagel 1995, Arad and Rubinstein 2012) or using an identi�cation strategy

that is insensitive to the details of how a player anchors their strategic reasoning (Kneeland

2015). In general, when a player only reasons a �nite number of steps, the anchor of a player's

strategic reasoning may matter. However, the extant literature in behavioral game theory

has made few attempts to uncover how players form the initial beliefs about non-strategic

play that anchor their strategic reasoning.

We thus conduct an experiment to test the separate predictions of three plausible models

of non-strategic play in the traveler's dilemma game (Basu 1994). In the traveler's dilemma

game, two players each make a monetary �claim� that must lie between an upper and a

lower bound. If the claims are equal, each player is paid their claim. When claims are

unequal, each player is paid the lower of the two claims and the player who made the lower

claim receives a transfer from their opponent. In the traveler's dilemma, the upper bound,

lower bound, and middle of the strategy space are salient and plausible speci�cations of

non-strategic play, as is uniform randomization across the strategy space. Each player has

the incentive to undercut the claim they expect the other player to make by as little as

possible. Given a lack of knowledge of the opponent's claim, a player thus faces a trade-o� �

a lower claim wins the transfer against more opponent claims, but conditional on being the

2

lower of the two claims, lowering one's claim reduces one's payo�. Given a �xed number of

steps of strategic reasoning, each of these models of initial beliefs about non-strategic play

makes distinct predictions about play given �xed game parameters. By observing how an

individual's play varies as we vary the upper and lower bounds of the strategy space and the

transfer, we can test the competing predictions of these three models of how they anchor

their strategic reasoning while simultaneously measuring their strategic sophistication.

We thus classify each subject based on their play in 30 rounds of the traveler's dilemma

with di�erent game parameters. Assuming the level k model for our initial analysis, we �nd

heterogeneity in how subjects anchor their strategic reasoning, with some subjects anchoring

at the top of the claim space, fewer on the middle of the claim space, and even fewer on

uniform random play, while a notable minority are best described by Nash equilibrium

(which is observationally equivalent to anchoring on the bottom of the claim space). We

show that if we instead pick one of the �rst three models of initial beliefs and assume that all

subjects anchor on that model, we substantially underestimate the fraction of strategically

sophisticated subjects. Our preferred analysis considers multiple di�erent models of limited

strategic reasoning and �nds that 27% of players tend to play Nash equilibrium, 33% are

boundedly strategic, while 38% are non-strategic. A majority of boundedly strategic players

(55%) anchor their strategic reasoning at the upper bound, and a similar fraction of non-

strategic players (58%) tend to non-strategically play the upper bound. Some strategic

players (35%) anchor their reasoning on and some non-strategic players (33%) tend to play

the middle of the strategy space. Notably, in our preferred classi�cation, we classify only

one strategic subject as anchoring on uniformly random initial beliefs. We thus conclude

that players are both heterogeneous in the amount that they reason, and how they anchor

their strategic reasoning, with anchoring on uniform randomization being rare.

Related Literature. Our paper contributes to the existing literature in behavioral game

theory by providing individual-level estimates of how players initiate their strategic rea-

3

soning. Our work takes leading models of limited strategic reasoning as a starting point,

namely, the level k model (Nagel 1995, Stahl and Wilson 1994, 1995) and the noisy introspec-

tion model (Goeree and Holt 2004).1 These models assume that each player anchors their

strategic reasoning on beliefs about how a non-strategic player, denoted L0, would play. In

this literature �L0 is usually assumed to be uniform random over others' possible decisions�

(Crawford, Costa-Gomes, and Iriberri 2013; examples include Stahl and Wilson 1994, 1995),

or estimates players' strategic sophistication in games where most plausible models of L0

generate the same best responses (Nagel 1995, Costa-Gomes and Crawford 2006, Arad and

Rubinstein 2012, Kneeland 2015). This literature �nds that the majority of players behave

as if they compute 1-4 levels of reasoning, while almost no one is classi�ed as L0. We instead

study the traveler's dilemma (Basu 1994) because it has multiple plausible L0 speci�cations

that each typically generates a di�erent best reply, allowing us to test between di�erent L0

speci�cations. Past work showed that subjects frequently play non-rationalizable strategies

in this game, even after playing multiple rounds with feedback (Capra et al., 1999). Our

study of how players anchor their strategic reasoning complements three existing strands of

experimental work that estimate how people reason in strategic settings.

A �rst strand of work estimates how players reason in two player normal form games

using econometric speci�cations of level k models. This literature typically assumes that L0

uniformly randomizes. Early work by Stahl and Wilson (1994) showed that the vast majority

of players are best classi�ed as L1 or L2. Costa-Gomes, Crawford, and Broseta (2001) use

players' information look-ups to disentangle a broader class of models, but nevertheless �nd

similarly. Wright and Leyton-Brown (2019) estimate alternative speci�cations for L0 using

data from experimental studies of normal form games. They �nd that the most predictive L0

speci�cation deviates from uniform randomization by putting additional weight on actions

based on their minmax, maxmax, and maxmax fairness evaluations, and by their distance

from the symmetric maxmax action when available. Their best performing model puts weight

1Crawford, Costa-Gomes, and Iriberri (2013) surveys this literature.

4

of at least 45% on uniform randomization and results in an approximately 80% weight on

uniform randomization when applied to the traveler's dilemma.2 However, unlike our study,

none of the aforementioned papers study heterogeneity in L0 across individuals. In addition,

our focus on a particular class of games� the traveler's dilemma� allows us to specify a class

of L0 speci�cations a priori.

A second strand of work uses a non-neutral framing to induce L0 (following Rubinstein,

Tversky, and Heller 1997) and estimates that L0 is sensitive to salience (Crawford and

Iriberri 2007). Bardsley et al. (2010) measure non-strategic play and beliefs about non-

strategic play in coordination games by studying play in two ancillary games, one where

players �pick� an action from the game's strategy space but without any incentives, another

where players are paid to �guess� the actions of the pickers. They compare the actions

chosen by pickers, guessers, and subjects who play the underlying coordination game to

test for alignment between non-strategic play, guessers' beliefs thereof, and coordinators'

beliefs. Hargreaves-Heap, Rojo Arjona, and Sugden (2014) study experimental hide-and-

seek games, where, following Rubinstein, Tversky, and Heller (1997) and Crawford and

Iriberri (2007), non-neutral action labels are designed to induce a non-uniform L0. They

reject the assumption that players' implied anchors of their strategic reasoning depend only

on non-strategic features of the game. Penczynski (2016) infers L0 and levels of reasoning

from text communication between partners in hide-and-seek games and �nds evidence of

role-asymmetric L0s that respond to non-neutral frames. In contrast, the neutrally-framed

traveler's dilemma motivates a di�erent class of L0 speci�cations � uniform random, the

lower bound, the upper bound, and the middle of the action space.

Our paper also complements a third strand of work on belief formation in experimental

guessing games and related games with incentives to undercut, following Nagel's (1995) pio-

neering study of the 2/3 beauty contest. Costa-Gomes and Crawford (2006) use guesses and

2Approximate percentages are from Figure 6 of Wright and Leyton-Brown (2019). Since any individualaction can achieve a symmetric payo� when the other player takes the same action, their notion of maxmaxfairness puts equal weight on all actions in the traveler's dilemma.

5

information look-ups in a variety of two player guessing games to estimate levels of sophisti-

caton while assuming a uniform random L0, and classify most subjects as L1 or L2, but with

up to 38% of subjects not well classi�ed by any strategic type. In a similar class of games,

Fragiadakis, Knoep�e, and Niederle (2016) fail to classify 70% of subjects and �nd that most

of these subjects cannot mimic or best-reply to their own past play in an identical game,

and suggest that they may be poorly described by any model based around deterministic

strategic reasoning. Burchardi and Penczynski (2014) apply the method used in Penczyn-

ski (2016) to Nagel's (1995) 2/3 beauty contest and �nd that the modal communicated L0

belief is in the middle of strategy space (50), but the average is slightly higher (55); 20% of

their subjects are classi�ed as L0. Agranov, Caplin, and Tergiman (2015) use incentivized

choice process data to study the process of reasoning in the 2/3 beauty contest, and classify

45% of subjects as L0 � in contrast to earlier �ndings (reviewed in Crawford, Costa-Gomes,

and Iriberri 2013) that classify almost no one to L0. Fragiadakis, Kovaliukaite, and Arjona

(2019) elicit subjects' beliefs about others' in an undercutting game. They �nd that most

subjects put positive weight on multiple other behavioral types, consistent with the cognitive

hierarchy model of Camerer, Ho, and Chong (2004) but inconsistent with models like level

k in which each player makes a point prediction about their opponent's play. Our paper

contributes by providing individual-level tests of concrete alternative speci�cations of how

players anchor their strategic reasoning in an undercutting game where a uniform random

L0 generates di�erent predictions for behavior than a L0 that plays in the middle of the

action space.

The rest of the paper is organized as follows. Section 2 presents theoretical predictions.

Section 3 presents the experimental design and procedure. Section 4 presents the experi-

mental results. Section 5 presents our preferred classi�cation analysis. Section 6 concludes

with a discussion.

6

2 Theoretical Predictions

In the traveler's dilemma game, each of two subjects simultaneously and independently

makes a �claim�, which is an integer xi ∈ {x,. . . , x̄} from a range speci�ed by a lower bound

x and an upper bound x̄. Subject i's payo�, πi, is given by

πi =

xi +R if xi < x−i

xi if xi = x−i

x−i −R if xi > x−i

where R is a reward/penalty parameter. When the claims are equal, both players receive

their claims. When the claims are unequal, both players receive the lower of the two claims

and the player who made the higher claim transfers R to the player who made the lower

claim. Thus, if R > 0, each player has the incentive to undercut the other player's claim by

1. A triple 〈x, x̄, R〉 fully describes a parameterization of the traveler's dilemma game.

We initially consider the Nash Equilibrium and level k (Nagel, 1995; Stahl and Wilson,

1995) as alternative models of strategic reasoning.

Nash Equilibrium. Since R > 0, each player has the incentive to undercut the other's

claim. That is, given the conjecture that the other player never plays actions above x̂, then

x̂ is dominated by x̂ − 1. By induction, the game is dominance solvable and claiming x is

the unique rationalizable strategy (and unique Nash Equilibrium strategy) for each player.

Level k. The level k model is a non-equilibrium model of limited strategic reasoning

wherein a player iteratively best-replies to a model of non-strategic play, referred to as

L0. A L1 player best-replies to a L0 play, and more generally, a Lk player best-replies to

L(k−1) play. The parameter k captures the number of steps, or level, that a player reasons.

7

Unlike in most games studied in lab experiments, the predictions of the level k model across

di�erent parameterizations of the traveler's dilemma depend on the speci�cation of L0. This

gives us the ideal setting to test the contrasting predictions of di�erent models of L0.

We consider three models of L0:

1. �Top�, where L0 plays x̄;

2. �Middle�, where L0 plays x+x2; and

3. �Uniform�, where L0 uniformly randomizes.

Each model of L0 makes a separate set of predictions for strategic players:3 under L0-top,

Lk claims x̄− k. Under L0-middle, Lk claims x̄+x2− k. Under L0-uniform, Lk is indi�erent

between claiming x + 1 − 2R − k and x + 2 − 2R − k. We note that a fourth model of L0

where L0 plays the lower bound, x, makes identical predictions to Nash Equilibrium.

3 Experimental Design and Procedures

Each subject was randomly and anonymously matched with another subject to play thirty

rounds of the traveler's dilemma game with di�erent parameters, divided into three blocks.

Each block has the same ten pairs of lower bound-upper bound parameters but a di�erent

reward/penalty parameter R (Table 1); the �rst and third blocks have R = 5, while the

second block has R = 20. Within each block, we vary the lower and upper bounds across

rounds in the following four ways: varying the upper bound only (e.g. round 1 versus 6),

varying the lower bound only (e.g. round 5 versus 8), varying both the lower and upper

bounds while keeping the middle of the range comparable to another pair of lower and

upper bounds (e.g round 4 versus 9), and varying both the lower and upper bounds while

keeping the gap between the bounds comparable to another pair of lower and upper bounds

3We provide full derivations in the Appendix.

8

(e.g. round 2 versus 3). Subjects received no feedback between decisions. One round was

randomly chosen for payment at the end of the experiment.

Table 1: Round parameters

Round Lower bound Upper bound

1, 11, 21 20 1202, 12, 22 80 2003, 13, 23 40 1604, 14, 24 20 1805, 15, 25 50 2006, 16, 26 20 1607, 17, 27 60 1808, 18, 28 100 2009, 19, 29 50 15010, 20, 30 40 200

R=5 for rounds 1-10 and 21-30;R=20 for rounds 11-20.

We recruited 60 subjects from the experimental economics recruitment pool at Simon

Fraser University to participate in experimental sessions between July 2018 and February

2019. Each session lasted approximately 45 minutes. Each subject received a minimum $7

(CAD) show-up fee in addition to their experiments earnings, which were converted from

experimental currency units to dollars at a rate of 1 ECU = $0.10; the average payment was

$15.50. We report additional details in the Supplementary Appendix.

4 Main Results

To compare claims across rounds with di�erent lower and upper bounds, we compute a

normalized claim xnig =xigx̄−x ∈ [0, 1] in each game g for each subject i. Figure 1 shows that

normalized claims are spread out over the feasible ranges with three spikes at exactly the

lower bound (19% of the data), the middle (6% of the data) and the upper bound (21% of

the data). These three spikes account for 46% of the data.

9

Figure 1: Frequency of normalized claims

Block 1

Re

lative

Fre

qu

en

cy

0.0 0.2 0.4 0.6 0.8 1.0

0.0

00

.05

0.1

00

.15

0.2

00

.25

0.3

0

Block 2

0.0 0.2 0.4 0.6 0.8 1.0

0.0

00

.05

0.1

00

.15

0.2

00

.25

0.3

0

Block 3

0.0 0.2 0.4 0.6 0.8 1.0

0.0

00

.05

0.1

00

.15

0.2

00

.25

0.3

0

Claims of the lower bound x are exactly consistent with Nash equilibrium play, although

we cannot distinguish whether a subject follows a heuristic of choosing the lower bound or

strategically plays the Nash equilibrium action. The remaining data are di�cult to reconcile

with leading behavioral models of limited strategic reasoning. This is because any models of

strategic behavior, including level k and related models (like noisy introspection, considered

in Section 5) predict that when k > 0, subjects will undercut below a deterministic model

of non-strategic (i.e. L0) play � which would lead to spikes in the claim distribution below,

rather than on focal claims like the top and the middle. Similarly, level k with k > 0

does not predict claims exactly at focal points under a model of uniformly random L0. We

also note that quantal response equilibrium (considered in Section 5) always predicts that

a person's modal claim will be strictly below x̄, and also predicts a smooth distribution of

responses without isolated spikes at focal points. These models are thus inconsistent with our

observation that the top and middle claims are frequently made. This observation suggests

that some of our subjects may follow a heuristic of choosing a focal point instead of being

10

strategic (in the sense of undercutting). We will further analyze this behavior at the subject

level.

Figure 4 plots median normalized claims across all rounds by subject, in increasing order

of the median, and their corresponding 25th and 75th percentiles. Of the 60 subjects, there are

11, 4 and 9 subjects whose medians are exactly at the three spikes - upper bound, middle,

and lower bound, respectively. We also notice the variation of choices within subject, a

signi�cant portion of subjects seem to have claims spread out across the normalized range.

Figure 2: Median normalized claims in increasing order, by subject

0.00

0.25

0.50

0.75

1.00

0 20 40 60Subject

Med

ian

To understand heterogeneity in how subjects anchor their strategic reasoning, we estimate

a version of the level k model at the subject-level separately for each of the three L0 speci-

�cations. Following Goeree, Louis, and Zhang (2018), we model each subject's level of rea-

soning in each round as drawn from a truncated Poisson distribution with individual-speci�c

11

Tremble level kL0 # non-strategic # strategic Median τ Median τ (strategic only)

Top 13 10 0.081 0.49Middle 7 4 0.11 1.30Uniform 4 2 0.34 5.54

Bottom/Nash 20 � �We assign each subject to the L0 speci�cation that achieves the highest likelihood.

We separate subject in each model-row into �non-strategic� or �strategic� based

on a test of statistical signi�cance of τ at the 5% level, one-sided test.

Table 2: Classi�cation to L0 the tremble level k model

strategic sophistication parameter, τ . Thus the probability that a subject with parameter τ

exhibits k levels of reasoning in a round is given by f(k; τ) =( e

−τ ∗τkk!

)∑l(e−τ ∗τl

l!). We model depar-

tures from the theory by incorporating �tremble� errors, that is, with probability ε, a player

makes a uniformly distributed error, and otherwise, plays according to the level k model. We

can then form a grand likelihood function for each model given parameters θ = (τ, ε), a set of

games G, and a subject's vector of claims x, as: L (θ | x,G) =∑

k f (k; τ)∏

g∈G p (x | θ, g),

where p (x | θ, g) denotes the probability of claiming x in game g when the model under

consideration has parameters θ. Our estimation thus �ts τ and ε for each subject and each

L0 speci�cation (except for Bottom/Nash, for which τ is not identi�ed).

We then classify each subject to the L0 model with the highest likelihood (Table 2). We

further subclassify each subject as �strategic� if their estimated τ is signi�cantly di�erent

from 0 at the 5% level (one-sided test), and as �non-strategic� otherwise (except for the

Bottom/Nash group). For each L0 speci�cation, we report the median of τ of all subjects

we classify to that L0 model (including non-strategic subjects). This procedure classi�es

23 subject to Top, 11 to Middle, 20 to Bottom/Nash, and 6 to Uniform. We classify 24 of

subjects as non-strategic, 16 as (boundedly) strategic, and 20 to the Bottom/Nash model.

Finding 1: Non-strategic play is prevalent.

We classify a large proportion � 40% � of subjects as non-strategic (Table 2). Speci�cally,

22% tend to play the top and 12% tend to make the claim in the middle of the range. We

classify 33% as playing the lower bound (i.e. Nash equilibrium) and 27% as (boundedly)

12

strategic. This �nding suggests that non-strategic play is prevalent and not just a �ctitious

anchor for strategic reasoning.

Finding 2: There is heterogeneity in non-strategic behavior, but uniform

randomization is relatively rare.

We observe heterogeneity in non-strategic play. The most common (54% of non-strategic

subjects) is to claim at the upper bound of the range. Furthermore, we �nd weak evidence for

the common assumption of non-strategic play as uniform randomization. We only categorize

7% of subjects to the non-strategic heuristic of uniformly randomizing. At the aggregate

level, the choice distribution of non-strategic subjects is di�erent from a uniform distribution,

as depicted in Figure 3 (p < 0.01, Uniform (0, 1) Kolmogorov-Smirnov test).

Figure 3: Choice distribution of subjects classi�ed as non-strategic

Relat

ive Fr

eque

ncy

0.0 0.2 0.4 0.6 0.8 1.0

0.00.1

0.20.3

0.40.5

Finding 3: There is heterogeneity in how boundedly strategic players anchor

their strategic reasoning.

Among boundedly strategic subjects, 63% anchor their reasoning at the top of the strat-

13

egy space, 25% anchor at the middle of the strategy space, and only 13% anchor their

reasoning with uniform random play. The heterogeneity we observe suggests that in similar

structural estimation exercises, imposing the same L0 assumption on all people may lead to

a misidenti�cation of strategic reasoning.

Finding 4: Any a priori homogeneous assumption about L0 leads us to dra-

matically overestimate the prevalence of non-strategic players.

Based on Finding 3, analysis that assumes that all players anchor their strategic reasoning

on the same L0 is likely to be misspeci�ed. To evaluate the consequences of making an a

priori assumption about L0 when estimating the degree of strategic sophistication, we again

estimate the tremble Lk model for each subject with each possible pre-speci�ed L0. Each row

of Table 3 reports how the assumed L0 choice a�ects inference about strategic sophistication

by reporting the results of tests of strategic sophistication (i.e. signi�cance test of τ for each

subject) and the median τ obtained under each speci�cation.4 For any of the speci�cations

for which τ is identi�ed, this exercise implies that more players are non-strategic as compared

to our estimates in which each subject is classi�ed to the best �tting L0 model. For any

a priori assumption about L0, the only subjects classi�ed as strategic are those who were

classi�ed to that L0 model and as strategic in Table 2 (except for one subject in the Top

speci�cation). As a result, any of the three a priori assumptions would lead us infer that

82%-97% of subjects are non-strategic � a dramatic overestimate as compared to when we

allow for heterogeneity in L0 across subjects.

Finding 4 may seem surprising given that most prior work assumes a homogeneous L0

speci�cation. However, many existing studies that estimate subjects' strategic sophistica-

tion do so from play in games like the p-beauty contest (Nagel, 1995), the 11-20 game (Arad

and Rubinstein, 2012), and ring games (Kneeland, 2015) in which multiple plausible L0

have identical best-replies. Thus, unlike in our study, these studies' estimates of strategic

4Note that τ is unidenti�ed for a completely non-strategic player (ε = 1) and so comparing values of τmay be misleading. We �nd that the median value of τ varies across speci�cations � though the τ estimateis statistically insigni�cant for the vast majority of subjects in each L0 speci�cation, precluding reliableinferences about how the choice of L0 a�ects the estimated τ .

14

Tremble Level kL0 # non-strategic # strategic Median τ Median τ (strategic only)

Top 49 11 0.49 0.57Middle 56 4 1.00 1.30Uniform 58 2 1.46 5.54

Bottom/Nash � � �

Table 3: Tests of strategic sophistication for each L0

sophistication are relatively robust to assumptions about L0. In other classes of games, ho-

mogeneous L0 speci�cations may be descriptively accurate, unlike in the traveler's dilemma

� but our results caution that this should not be taken for granted, but rather be evaluated

empirically.

E�ect of the reward penalty parameter. In line with previous studies, we �nd a

negative correlation between the reward/penalty parameter and average choices. Average

normalized choices in Block 1, Block 2 and Block 3 are 0.595, 0.482 and 0.624, respectively;

subject average normalized claims are similar in Block 1 and Block 3 (paired Wilcoxon test,

p = 0.15). That is, we �nd a moderate e�ect of varying the reward/penalty parameter on

claims (paired Wilcoxon test for Block 1 and Block 2, p < 0.01 in favor of the alternative

�greater� for Block 1, also p < 0.01 for Block 3 and Block 2.), but we do not �nd evidence

of learning between Block 1 and Block 3.

5 Classifying L0 for alternative models of strategic rea-

soning

Alternative empirical models beyond level k that have been proposed to capture failures of

Nash equilibrium in games di�er in how they capture limits to strategic reasoning, what they

15

assume about noise in subjects' behavior, and in how subjects anticipate the noise of others.

One possible concern is that our preceding classi�cation results may be driven by our choice

of the level k model in which errors are modeled as uniform trembles � which might lead

to misleading results if it is the wrong model of strategic reasoning for some subjects. We

thus consider three additional models: logit level k, quantal response equilibrium (QRE),

and noisy introspection (NI). Like Nash Equilibrium, QRE is an equilibrium model in which

play is a stochastic best-reply to the equilibrium distribution of actions of other players. In

QRE, as in Nash equilibrium, the anchor of strategic reasoning does not matter. In contrast,

level k and NI are models in which a player respectively best replies to a deterministic or

stochastic less sophisticated player(s). Our interest is in the distinction between di�erent

models of anchors of strategic reasoning, and not on distinguishing between level k and NI,

so we now consider both models.

We reclassify each subject to the best-�tting model of noisy strategic play. We consider

12 models of strategic reasoning in total: tremble level k for L0 ∈ {Top,Middle,Uniform},

logit level k for L0 ∈ {Top,Middle,Uniform}, NI for NI0 ∈ {Top,Middle,Uniform}, QRE,

logit Nash, and tremble Nash. We also consider three models of non-strategic play: we

estimate a tremble model for Top and Middle, and obtain the likelihood for Uniform (which

does not require estimation). Note that level k with L0 as the bottom of the claim space is

observationally equivalent to Nash play.

Logit Level k. In logit level k, subjects are assumed to compute the expected distribution

of other's behavior, p, according to the level k model, but stochastically best reply to this

according to the logit model, that is, Prob(xi = c|p) = exp(γU(xi=c|p))x̄∑j=x

exp(γU(xi=j|p)).

Quantal Response Equilibrium. QRE assumes that each player noisily best responds

to the equilibrium distribution of play. Speci�cally, let p denote a probability distribution

over actions, and let U(xi = c|p) denote the expected payo� to playing xi = c if the other

player's distribution over actions is given by p. Then, p is a QRE if

16

pc ≡ Prob(xi = c|γ) =exp (γU(xi = c|p))x̄∑j=x

exp (γU(xi = j|p))

The parameter γ captures the precision of play � with higher γ indicating less noisy play.5

In the QRE model, a player's claims will have a smooth distribution and centered on their

modal claim, which must be strictly below x̄.

Noisy Introspection. NI is a non-equilibrium model and can be viewed as a noisy version

of level k where a player iteratively and noisily best-replies to a model of non-strategic play,

referred to as NI0 (which plays an identical role to L0 in the level k model). An NI1 player

noisily best-replies to NI0 play, and more generally, a NIk player noisily best-replies to the

distribution of NI(k− 1) play. As in QRE, noisy behavior is modeled using a logit formula.

Let p denote a probability distribution over actions. Then, for a NIk player,

pkc ≡ Prob(xi = c|γ) =exp

(γU(xi = c|pk−1)

)x̄∑j=x

exp (γU(xi = j|pk−1))

As in level k model, we consider three models of NI0: �top�, where NI0 plays x̄; �middle�,

where NI0 plays x̄+x2; and �uniform�, where NI0 uniformly randomizes. Each model of NI0

also makes a separate set of predictions for higher levels, for reasons analogous to the level k

model. Figure 4 illustrates the predictions of QRE and of NIk with k = 1 for γ ∈ {0.1, 0.2}

in the traveler's dilemma parameterized by 〈x = 20, x̄ = 120, R = 5〉.

Classifying subjects to best-�tting models. Among the �fteen models (including uni-

form randomization), for each subject, we pick the model with the highest log-likelihood and

classify that subject to the best �tting of the above speci�cations whenever we can reject the

null hypothesis of random or non-strategic behavior, according to the procedure we describe

5As γ → ∞, the model converges to Nash Equilibrium play, whereas lower levels of γ induce noisierbehaviors.

17

Figure 4: Illustration of QRE and NIk predictions (k = 1)

below. For each subject, we use a likelihood ratio test to compare the best-�tting strate-

gic model to uniform randomization, and we use Vuong (1989) tests to compare it to the

other two models of non-strategic play. When the best-�tting strategic model is a signi�cant

improvement (at the 5% level) over the models of non-strategic play, we classify a subject

to that strategic model � though we only consider them strategic if the estimated τ is at

least 0.5.6 Otherwise, if either of the two non-strategic models is a signi�cant improvement

over uniform randomization, we classify them to the best �tting non-strategic model. In the

event of a tie between strategic models, we classify the subject to the model with the fewest

parameters. We detail this procedure in the Supplementary Appendix.

Table 4 summarizes our subject classi�cation.

6If τ < .5, NIk and Lk generate mostly non-strategic play. Whereas Tables 2 and 3 focus on statisticalinferences about strategic reasoning, we believe that this added restriction is desirable for classifying subjects'strategic reasoning.

18

Table 4: Subject classi�cation with heterogeneous models of noisy strategic reasoning

Best-�tting modelL0 # non-strategic # strategic

Top 14 11Middle 8 8Uniform 1 1

Bottom/Nash 16QRE � 1

We use Vuong tests to to test between Uniform, the best-�tting L0

speci�cation, and the best-�tting model overall � only assigning the

subject to a model with an additional parameter if it is signi�cantly better

(at the 5% level). Additionally, we classify a subject as �non-strategic�

if a strategic model passes these tests, but is best �t with τ < .5.

Finding 5: Considering alternative models of limited strategic reasoning changes

the subject classi�cation slightly and reinforces Findings 1-3.

Compared to our classi�cation using only the tremble level k model, we now classify only

two subjects (instead of 6) to uniform random L0 (one strategic and one non-strategic), only

16 subjects (instead of 20) to the Bottom/Nash group, and 8 subjects to middle � strategic

(instead of 4). These slightly di�erent classi�cation results nevertheless broadly support

Findings 1-3. The revised classi�cation suggests that the tremble level k analysis may very

slightly overstate the prevalence of non-strategic players, overestimate the prevalence of non-

strategic uniform randomization, and underestimate the prevalence of strategic reasoning

anchored at the middle.7

6 Discussion

By carefully varying the game parameters in the traveler's dilemma, we uncover how strategic

players anchor their strategic reasoning and how non-strategic players play. We classify 27-

33% of subjects as tending to play Nash equilibrium (or tending to choose the lower bound),

7Compared to the classi�cation using only the tremble level k model in Table 2, Table 4 classi�es oneadditional subject to each of non-strategic middle and top. This is due to the slight di�erences in procedures.

19

27-33% as boundedly strategic, and 38-40% as non-strategic. Contrary to our initial ex-

pectations, we �nd substantial evidence for non-strategic play and we �nd that few players

are well-described by the common assumption of non-strategic play as uniform randomiza-

tion. In addition, we �nd substantial heterogeneity in how subjects anchor their strategic

reasoning.

Our results suggest that the common practice of assuming a homogeneous and prespec-

i�ed L0 to identify levels of strategic reasoning may lead to misleading estimates whenever

the L0 speci�cation matters. Since the way that people anchor their strategic reasoning

matters in many games, we thus believe that future work in experimental game theory on

this topic will be important for understanding how real people play games and how to model

it.

20

References

Agranov, Marina, Andrew Caplin, and Chloe Tergiman (2015). �Naive play and the process of

choice in guessing games�. In: Journal of the Economic Science Association 1.2, pp. 146�

157.

Arad, Ayala and Ariel Rubinstein (2012). �The 11-20 money request game: A level-k reason-

ing study�. In: American Economic Review 102.7, pp. 3561�73.

Bardsley, Nicholas et al. (2010). �Explaining focal points: cognitive hierarchy theory versus

team reasoning�. In: Economic Journal 120.543, pp. 40�79.

Basu, Kaushik (1994). �The traveler's dilemma: Paradoxes of rationality in game theory�.

In: American Economic Review 84.2, pp. 391�395.

Burchardi, Konrad B and Stefan P Penczynski (2014). �Out of your mind: Eliciting individual

reasoning in one shot games�. In: Games and Economic Behavior 84, pp. 39�57.

Camerer, Colin F, Teck-Hua Ho, and Juin-Kuan Chong (2004). �A cognitive hierarchy model

of games�. In: Quarterly Journal of Economics 119.3, pp. 861�898.

Capra, C Monica et al. (1999). �Anomalous behavior in a traveler's dilemma?� In: American

Economic Review 89.3, pp. 678�690.

Chen, Daniel L, Martin Schonger, and Chris Wickens (2016). �oTree�An open-source plat-

form for laboratory, online, and �eld experiments�. In: Journal of Behavioral and Exper-

imental Finance 9, pp. 88�97.

Costa-Gomes, Miguel and Vincent Crawford (2006). �Cognition and behavior in two-person

guessing games: An experimental study�. In: American Economic Review 96.5, pp. 1737�

1768.

Costa-Gomes, Miguel, Vincent Crawford, and Bruno Broseta (2001). �Cognition and behav-

ior in normal-form games: An experimental study�. In: Econometrica 69.5, pp. 1193�

1235.

21

Crawford, Vincent, Miguel Costa-Gomes, and Nagore Iriberri (2013). �Structural models of

nonequilibrium strategic thinking: Theory, evidence, and applications�. In: Journal of

Economic Literature 51.1, pp. 5�62.

Crawford, Vincent and Nagore Iriberri (2007). �Fatal attraction: Salience, naivete, and so-

phistication in experimental "hide-and-seek" games�. In: American Economic Review

97.5, pp. 1731�1750.

Fragiadakis, Daniel E, Daniel T Knoep�e, and Muriel Niederle (2016). Who is strategic?

Working Paper.

Fragiadakis, Daniel E, Ada Kovaliukaite, and David Rojo Arjona (2019). �Belief-Formation

in Games of Initial Play: an Experimental Investigation�. In.

Goeree, Jacob and Charles Holt (2004). �A model of noisy introspection�. In: Games and

Economic Behavior 46.2, pp. 365�382.

Goeree, Jacob, Philippos Louis, and Jingjing Zhang (2018). �Noisy introspection in the 11�20

game�. In: Economic Journal 128.611, pp. 1509�1530.

Hargreaves-Heap, Shaun, David Rojo Arjona, and Robert Sugden (2014). �How Portable Is

Level-0 Behavior? A Test of Level-k Theory in Games With Non-Neutral Frames�. In:

Econometrica 82.3, pp. 1133�1151.

Kneeland, Terri (2015). �Identifying Higher-Order Rationality�. In: Econometrica 83.5, pp. 2065�

2079.

Nagel, Rosemarie (1995). �Unraveling in guessing games: An experimental study�. In: Amer-

ican Economic Review 85.5, pp. 1313�1326.

Penczynski, Stefan P (2016). �Strategic thinking: The in�uence of the game�. In: Journal of

Economic Behavior & Organization 128, pp. 72�84.

Rubinstein, Ariel, Amos Tversky, and Dana Heller (1997). �Naive strategies in competitive

games�. In: Understanding strategic interaction. Ed. by W Albers et al. Springer, pp. 394�

402.

22

Stahl, Dale O. and Paul W. Wilson (1994). �Experimental evidence on players' models of

other players�. In: Journal of Economic Behavior & Organization 25.3, pp. 309�327.

� (1995). �On players' models of other players: Theory and experimental evidence�. In:

Games and Economic Behavior 10.1, pp. 218�254.

Vuong, Quang H (1989). �Likelihood ratio tests for model selection and non-nested hypothe-

ses�. In: Econometrica, pp. 307�333.

Wright, James R and Kevin Leyton-Brown (2019). �Level-0 models for predicting human

behavior in games�. In: Journal of Arti�cial Intelligence Research 64, pp. 357�383.

23

Appendix A

Derivation of L1's best response to a Uniform L0

Consider a traveler's dilemma game with a discrete choice range [x, x] and a reward/penalty

parameter R. Let L0 be a uniform distribution over possible choices.

If player i holds the belief that L0 plays a mixed strategy, speci�cally, uniformly randomly

x−i ∼ U [x, x], then i chooses xi to maximize expected payo� πei :

Prob(xi > x−i)[E(x−i|x−i < xi)−R] + Prob(xi < x−i)(xi +R) + Prob(xi = x−i)xi

= xi−xx−x+1

(xi−1+x

2−R

)+ (x−xi)(xi+R)

x−x+1+ xi

x−x+1

= −12(x−x+1)

x2i +

x+ 12−2R

x−x+1xi − x2+x

2(x−x+1)+ R(x+x)

x−x+1

F.O.C. −xi + (x+ 12− 2R) = 0

⇒ x∗i = x+ 12− 2R

When R is an integer, this yields a non-integer; since the original objective function is

quadratic and concave, x− 2R and x̄+ 1− 2R will both be best replies.

For R = 0, we obtain a corner solution at x∗i = x.

This solves the problem for L1. Since Lk best-replies to L(k − 1), we can solve for best

replies for L2, L3, and so on, and obtain that x − 2R − k + 1 and x̄ + 2 − 2R − k will be

best replies for Lk.

24

Supplementary Appendices

Appendix B - Experimental Instructions

Experimental Implementation

At the beginning of each session, paper instructions were distributed and read aloud; subjects

followed along and could use a pen or pencil to write notes on the instructions. Also, an elec-

tronic summary of the instructions appeared on-screen in each round. Copies or screenshots

of all materials used in the experiment are provided in the supplementary material. After

the instructions, subjects completed a comprehension quiz. The experimenter checked the

answers privately, and when they encountered incorrect answers, the experimenter pointed

the subject to the relevant part of the instructions and gave the subject the opportunity to

revise their answers. After all subjects had answered all questions correctly, the experiment

commenced. The experiment was conducted through a web interface based on oTree (Chen,

Schonger, and Wickens 2016), with a 10 second forced delay before a subject could submit

their choice in each round.

1

Instructions

Overview

You are going to take part in an experimental study of decision making.

During the experiment, you are not allowed to talk or communicate with other participants.

If at any time you have any questions, please raise your hand and the experimenter will come

to your desk to answer it.

Your earnings in the experiment will depend on your choices, the choices of other

participants, and an element of chance. By following the instructions and making decisions

carefully, you may earn a considerable amount of money.

At the end of the experiment, the number of Experimental Currency Units (ECU) that you

earn will be converted to Canadian dollars at the exchange rate 1 ECU = $0.10.

You will be given a $7 show up payment in addition to your earnings for the experiment.

Your total earnings will be paid to you in cash today at the end of this experiment.

General Description

You will be randomly and anonymously matched with another participant in the room for the

duration of the experiment. Your identity will never be revealed to your opponent, and their

identity will never be revealed to you.

The experiment consists of a number of rounds. At the end of the experiment, one round will

be randomly selected, and the decisions that you and that other participant made in that round

will determine the amount earned by each of you.

o Since you do not know which round will be selected to determine your payment, you

should treat each round as if it will determine your final payment.

The choices that you and the other subject will make, and the corresponding results, will not

be communicated to you at the end of each round.

At the end of the whole experiment, you will be informed of your choice, the other person’s

choice and the result of only the payment round.

2

Task and Earnings

In each round, at the same time, you and the person you are matched with will each choose a

number or "claim" between a specified minimum claim and a specified maximum claim

(inclusive).

o The minimum and maximum claim will be different in each round.

If the claims are equal, then you and the other person each receive the amount claimed.

If the claims are not equal, then each of you receives the lower of the two claims. In

addition, the person who makes the lower claim earns a reward, and the person with the

higher claim pays a penalty of the same amount as the reward.

o The amount of the reward/penalty will change twice during the experiment, and you

will be notified in advance of these changes.

Thus you will earn an amount that equals the lower of the two claims, plus a reward if you

are the person making the lower claim, or minus a penalty if you are the person making the

higher claim. There is no penalty or reward if the two claims are exactly equal, in which case

each person receives what they claimed.

Example

Suppose that your claim is X and the other's claim is Y, and the reward/penalty is T.

If X = Y (both claim the same amount), you get X, and the other gets Y.

If X > Y (your claim is higher), you get Y minus T, and the other gets Y plus T.

If X < Y (your claim is lower), you get X plus T, and the other gets X minus T.

3

Interface

The following screenshot shows how the minimum and maximum claims, the penalty, and your

claim input box will be displayed on each decision screen.

4

Summary

You have been randomly and anonymously paired with another participant.

In each round, you and the other participant each simultaneously and independently choose a

number between the round’s minimum and maximum claim (inclusive).

If both choose the same number, then this amount will be paid to both.

If you choose different amounts, then the lower amount will be paid to both. Additionally,

the one with the lower claim will receive a reward; the one with the higher claim will receive

a penalty.

The reward and penalty are the same magnitude, and this will be specified each round.

You will be paid your earnings in cash at the end of the experiment based on one randomly

selected round. Your actual decisions and those of the participant with whom you are paired

in that round will determine your earnings for the experiment.

During the experiment, you are not permitted to speak or communicate with the other

participants. If you have a question while the experiment is going on, please raise your hand and

the experimenter will come to your desk to answer it.

5

Comprehension Quiz

To verify your comprehension of the instructions, please complete the following comprehension

quiz. Your answers will not affect your earnings in any way. We just want to ensure that you

understand how the experiment works and how your earnings will be calculated. We will come

around and check your responses.

For questions Q1-Q5 below, suppose that the minimum claim is 20, the maximum claim is 200,

and the reward/penalty is 5.

Q1. What is the highest number that you can claim? _____________

Q2. What is the lowest number that you can claim? _____________

Below, write any claim for your opponent in the first blank, and any claim of your own in the

second blank, such that your claim is higher than your opponent’s claim. You will use these

numbers to answer Q3, Q4, and Q5.

Suppose that your opponent claims ____ and you claim ____.

Q3. The lower of the two claims is: _____________

Q4. Your opponent will earn: _____________

Q5. You will earn: _____________

6

For questions Q6-Q10 below, suppose that the minimum claim is 80, the maximum claim is 120,

and the reward/penalty is 10.

Q6. What is the highest number that you can claim? _____________

Q7. What is the lowest number that you can claim? _____________

Below, write any claim for your opponent in the first blank, and any claim of your own in the

second blank, such that your claim is equal to your opponent’s claim. You will use these

numbers to answer Q8, Q9, and Q10.

Suppose that your opponent claims ____ and you claim ____.

Q8. The lower of the two claims is: _____________

Q9. Your opponent will earn: _____________

Q10. You will earn: _____________

7

For questions Q11-Q15 below, suppose that the minimum claim is 70, the maximum claim is

150, and the reward/penalty is 2.

Q11. What is the highest number that you can claim? _____________

Q12. What is the lowest number that you can claim? _____________

Below, write any claim for your opponent in the first blank, and any claim of your own in the

second blank, such that your claim is lower than your opponent’s claim. You will use these

numbers to answer Q13, Q14, and Q15.

Suppose that your opponent claims ____ and you claim ____.

Q13. The lower of the two claims is: _____________

Q14. Your opponent will earn: _____________

Q15. You will earn: _____________

8

Appendix C � Classi�cation Procedure

We apply the following steps for each individual subject. We classify each subject to the

best �tting model whenever we can reject the null hypothesis of random or non-strategic

behavior according to the procedure we describe below. All tests use the 5% signi�cance

level.

1. Fit all 15 models (12 strategic and 3 non-strategic) using the maximum likelihood

estimation.

2. Find the model with the highest log-likelihood (labeled m herein).

(a) In case of a tie with uniform randomization model, select uniform randomization

model.

(b) In case of a tie with one of the other two non-strategic models, select the non-

strategic model.

3. Compare the best-�tting model m with the non-strategic model of uniform random-

ization using a likelihood ratio test.

4. If we do not reject H0 in step 3, we assign this subject to uniform randomization.

5. When we reject H0 in step 3:

(a) If best-�tting model m is a non-strategic model, assign that subject to m.

(b) If best-�tting model m is Nash, assign that subject to m.

(c) If best-�tting model m is a strategic model Lk, NIk, or QRE. Compare m to

the corresponding non-strategic model.

i. Use a likelihood ratio test1 when the non-strategic model is nested within

a strategic model, e.g. non-strategic model of playing the top with action

1This is equivalent to using a Vuong test when the two models are nested.

9

trembles versus a best-�tting strategic model level k (k ≥ 0) with L0 as the

top and action trembles.

ii. Use Vuong tests when a non-strategic model is not nested within a strategic

model, e.g. non-strategic model of playing the top with action trembles versus

a best-�tting strategic model level k (k ≥ 0) with L0 as the top and payo�

trembles.

iii. If we cannot reject the null hypothesis in likelihood ratio tests or in Vuong

tests, classify that subject to the best �tting non-strategic model that is not

rejected. If we reject the null in each of these tests, classify the subject to m.

6. Among subjects classi�ed to a strategic model Lk or NIk, further classify subjects

into strategic (k ≥ 1) and non-strategic based (k = 0). A subject's level distribution is

modeled as a truncated Poisson distribution with parameter τ 2. The probability of a

player playing type k is f(k; τ) =( e

−τ ∗τkk!

)∑l( e

−τ ∗τll!

). The lower τ is, the higher the probability

that a subject plays L0 is. τ < 0.5 assigns more than 50% weight on a subject being

L0. For a subject with τ < 0.5, we classify that subject as non-strategic under the

corresponding Lk or NIk and for a subject with τ ≥ 0.5,we classify that subject as

strategic under the corresponding Lk or NIk.

7. If there is a tie in the likelihood ratios between two strategic models, classify the subject

to the model with the fewest parameters.

2Our structural estimation approach is similar to that of Goeree, Louis, and Zhang (2018).

10

Appendix D - Estimation Results

We estimate 15 models for each subject. The estimation results are recorded below. Note

that a model of uniform randomization play yields a log-likelihood of −145 for all subjects,

we therefore omit a column for uniform randomization.

�S� denotes subject, going from 1 to 60 for 60 subjects in our data set. �Para� denotes

estimated parameters: ε for action tremble speci�cation (uniform error), γ for payo� tremble

speci�cation (logit error), and τ for level distribution. �LL� stands for log-likelihood.

For 14 models, we list the abbreviations and parameters of interest below.

Table 1: AbbreviationsAbbreviation Model Parameters

T Non-strategic play at the top εM Non-strategic play at the middle εN_at Nash equilibrium with action trembles εN_pt Nash equilibrium with payo� trembles γQRE Quantal response equilibrium γLK_at_T Level k with action trembles, L0 at the top ε, τLK_pt_T Level k with payo� trembles, L0 at the top γ, τLK_at_M Level k with action trembles, L0 at the middle ε, τLK_pt_M Level k with payo� trembles, L0 at the middle γ, τLK_at_U Level k with action trembles, L0 as uniform randomization ε, τLK_pt_U Level k with payo� trembles, L0 as uniform randomization γ, τNI_T Noisy introspection L0 at the top γ, τNI_M Noisy introspection L0 at the middle γ, τNI_U Noisy introspection L0 as uniform randomization γ, τ

11

SP

ara

TM

N_a

tN

_pt

QR

ELK

_at_

TLK

_pt_

TLK

_at_

MLK

_pt_

MLK

_at_

ULK

_pt_

UN

I_T

NI_

MN

I_U

0.6

72

10

.97

50

.04

60

.07

60

.59

20

.07

10

.01

50

0.0

57

0.0

78

0.0

76

0.1

16

(0.0

87

)(0

.18

3 )

(0.0

33

)(0

.11

0 )

(0.0

16

)(0

.09

5 )

(0.0

16

)(0

.18

2 )

(0.0

14

)(2

.88

5 )

(0.0

25

)(0

.01

9 )

(In

f )

(0.0

27

)

0.3

12

1.2

05

15

.79

70

.05

92

.37

20

.98

32

03

.10

4

(0.1

99

)(0

.29

7 )

(NA

)(3

.49

3 )

(0.1

76

)(1

.21

8 )

(0.2

28

)(I

nf

)(1

.25

8 )

LL

-11

5.2

61

-14

5.0

85

-14

4.4

33

-14

5.0

17

-12

9.0

3-1

13

.24

3-9

2.7

06

-14

5.0

85

-14

4.5

22

-14

2.4

07

-13

1.8

54

-95

.81

5-1

29

.03

-12

6.8

47

10

.84

11

00

.00

61

00

.84

10

.02

71

0.0

02

0.0

06

0.0

11

0.0

06

(0.1

83

)(0

.06

9 )

(0.1

83

)(0

.16

7 )

(0.0

11

)(0

.18

2 )

(0.0

05

)(0

.06

7 )

(0.0

15

)(0

.03

4 )

(0.0

06

)(0

.00

0 )

(0.0

12

)(0

.00

0 )

11

2.1

81

01

.89

61

.45

89

.26

21

0.5

47

1.7

88

4.0

58

(NA

)(6

.11

3 )

(NA

)(0

.45

5 )

(NA

)(N

A )

(NA

)(0

.42

7 )

(NA

)

LL

-14

5.0

85

-13

4.9

32

-14

5.0

85

-14

5.0

85

-14

4.9

57

-14

5.0

85

-14

5.0

85

-13

4.9

33

-13

3.0

52

-14

5.0

85

-14

5.0

1-1

44

.95

9-1

34

.59

3-1

44

.95

8

01

10

0.1

00

10

.02

41

0.0

49

00

.10

8N

A

(0.1

84

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

23

)(0

.15

6 )

( 0

.03

9)

(0.1

82

)(0

.01

4 )

(NA

)(0

.01

2 )

(0.0

39

)(0

.02

5 )

(NA

)

00

15

.34

31

.45

83

.99

10

13

.78

5N

A

(NA

)(0

.00

2 )

(NA

)(2

.44

2 )

(NA

)(1

.65

5 )

(0.0

01

)(8

.39

4 )

(NA

)

LL

0-1

45

.08

5-1

45

.08

5-1

45

.08

5-1

23

.98

50

0-1

45

.08

5-1

43

.64

1-1

45

.08

5-1

28

.96

60

-12

3.9

41

NA

01

10

0.1

00

10

.02

41

0.0

49

00

.10

8N

A

(0.1

84

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

23

)(0

.15

6 )

(0.0

39

)(0

.18

2 )

(0.0

14

)(N

A )

(0.0

12

)(0

.03

9 )

(0.0

25

)(N

A )

00

15

.34

31

.45

83

.99

10

13

.78

5N

A

(NA

)(0

.00

2 )

(NA

)(2

.44

2 )

(NA

)(1

.65

5 )

(0.0

01

)(8

.39

4 )

(NA

)

LL

0-1

45

.08

5-1

45

.08

5-1

45

.08

5-1

23

.98

50

0-1

45

.08

5-1

43

.64

1-1

45

.08

5-1

28

.96

60

-12

3.9

41

NA

11

03

.73

70

10

10

10

00

0

(0.1

83

)(0

.18

3 )

(0.1

84

)(2

3.7

27

)(0

.00

6 )

(0.1

82

)(0

.00

5 )

(0.1

82

)(0

.00

0 )

(NA

)(N

A )

(0.0

06

)(0

.00

6 )

(0.0

00

)

16

17

.53

61

.45

83

5.8

12

8.6

22

1.0

01

(NA

)(3

.44

5 )

(NA

)(N

A )

(NA

)(N

A )

(3.5

68

)(5

.93

2 )

(NA

)

LL

-14

5.0

85

-14

5.0

85

00

-14

5.0

85

-14

5.0

85

-14

5.1

59

-14

5.0

85

-14

5.1

01

-14

5.0

85

-14

5.0

85

-14

5.1

75

-14

5.0

9-1

45

.08

5

0.9

75

0.9

07

0.3

70

.92

50

0.9

75

00

.90

70

10

00

0

(0.0

33

)(0

.05

5 )

(0.0

89

)(0

.09

0 )

(0.0

07

)(0

.02

8 )

(0.0

05

)(0

.05

3 )

(0.0

12

)(N

A )

(0.0

00

)(0

.00

8 )

(0.0

08

)(0

.00

0 )

03

.67

20

4.8

64

1.4

58

3.0

01

3.6

76

2.3

78

1.0

15

(NA

)(1

.33

2 )

(NA

)(N

A )

(NA

)(N

A )

(1.4

75

)(0

.59

8 )

(NA

)

LL

-14

4.4

33

-14

0.1

25

-72

.99

9-6

6.9

51

-14

5.0

85

-14

4.4

33

-14

4.4

33

-14

0.1

25

-14

3.2

62

-14

5.0

85

-14

5.0

85

-14

4.4

33

-14

0.1

25

-14

5.0

85

0.6

72

11

00

.10

.67

20

.08

10

.02

40

0.1

68

0.1

90

.12

80

.12

7

(0.0

87

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

21

)(0

.08

6 )

(0.0

18

)(0

.18

2 )

(0.0

14

)(0

.63

1 )

(0.0

80

)(0

.10

2 )

(0.0

00

)(0

.00

0 )

01

.26

51

5.3

43

0.2

59

1.3

59

1.5

17

18

.91

81

1.9

62

(NA

)(0

.32

0 )

(NA

)(2

.67

9 )

(0.1

82

)(0

.41

2 )

(0.5

25

)(N

A )

(NA

)

LL

-11

5.6

57

-14

5.0

85

-14

5.0

85

-14

5.0

85

-11

9.4

16

-11

5.6

57

-90

.08

6-1

45

.08

5-1

43

.64

1-1

25

.14

2-1

27

.92

2-8

8.7

15

-11

8.8

52

-11

8.8

59

10

.97

60

.70

60

.31

30

10

0.9

76

01

00

00

(0.1

83

)(0

.03

3 )

(0.0

84

)(0

.04

0 )

(0.0

08

)(0

.18

2 )

(0.0

05

)(0

.02

8 )

(0.0

12

)(0

.00

0 )

(0.0

00

)(0

.00

8 )

(0.0

09

)(N

A )

16

03

.72

71

.45

86

6.8

13

5.8

75

1.0

57

(NA

)(4

.23

1 )

(NA

)(1

.44

5 )

(NA

)(N

A )

(6.0

37

)(N

A )

(NA

)

LL

-14

5.0

85

-14

4.5

65

-12

0.1

33

-11

0.4

46

-14

5.0

85

-14

5.0

85

-14

5.1

59

-14

4.5

66

-14

4.5

65

-14

5.0

85

-14

5.0

85

-14

5.1

18

-14

4.9

19

-14

5.0

85

1

ε o

r γ

τ

2

ε o

r γ

τ

3

ε o

r γ

τ

4

ε o

r γ

τ

5

ε o

r γ

τ

6

ε o

r γ

τ

7

ε o

r γ

τ

8

ε o

r γ

τ

12

SP

ara

TM

N_a

tN

_pt

QR

ELK

_at_

TLK

_pt_

TLK

_at_

MLK

_pt_

MLK

_at_

ULK

_pt_

UN

I_T

NI_

MN

I_U

0.3

36

0.7

73

10

.00

00

.10

00

.33

60

.00

00

.74

50

.11

01

0.0

84

0.0

00

0.2

49

0.2

52

(0.0

87

)(0

.07

8 )

(0.1

83

)(0

.16

7 )

(0.0

23

)(0

.08

6 )

(0.0

09

)(0

.08

2 )

(0.0

34

)(0

.00

0)

(0.0

19

)(0

.01

0 )

(0.0

60

)(0

.08

4 )

0.0

00

0.3

99

0.1

09

1.4

10

1.4

58

10

.23

50

.46

02

.00

73

.62

5

(NA

)(0

.12

7 )

(0.1

29

)(0

.31

8 )

(NA

)(3

.25

5 )

(0.1

47

)(0

.36

7 )

(0.6

85

)

LL

-67

.37

6-1

27

.54

4-1

45

.08

5-1

45

.08

5-1

17

.76

3-6

7.3

76

-67

.38

0-1

26

.55

0-1

17

.08

2-1

45

.08

5-1

18

.41

2-6

7.4

44

-10

3.2

32

-11

3.2

85

0.9

07

0.9

41

0.7

40

0.2

54

0.0

16

0.9

06

0.0

02

0.9

41

0.0

00

0.0

01

0.0

15

0.0

06

0.0

12

0.1

53

(0.0

55

)(0

.04

6 )

(0.0

81

)(0

.03

2 )

(0.0

11

)(0

.05

3 )

(0.0

05

)(0

.04

3 )

(0.0

12

)(6

.10

8 )

(NA

)(0

.01

0 )

(0.0

14

)(0

.12

8 )

0.0

00

2.3

82

0.0

00

2.8

32

0.0

23

2.2

49

2.3

93

2.9

85

0.4

65

(NA

)(0

.60

0 )

(NA

)(0

.77

9 )

(0.1

46

)(N

A )

(0.6

05

)(0

.85

2 )

(0.3

01

)

LL

-13

9.8

17

-14

2.6

09

-12

4.2

08

-12

4.5

12

-14

4.2

86

-13

9.8

17

-13

9.7

17

-14

2.6

09

-14

2.6

09

-14

4.5

74

-14

3.4

35

-13

9.6

66

-14

2.2

91

-14

2.8

29

0.9

08

10

.10

11

.31

50

.00

00

.90

80

.00

01

.00

00

.00

01

0.0

00

0.0

00

0.0

00

0.0

00

(0.0

55

)(0

.18

3 )

(0.0

55

)(0

.12

6 )

(0.0

07

)(0

.05

3 )

(0.0

05

)(0

.18

2 )

(0.0

00

)(N

A )

(0.0

00

)(0

.00

7 )

(0.0

07

)(0

.00

1 )

0.0

00

2.2

71

1.0

00

8.7

99

1.4

58

2.0

00

2.3

78

10

.20

61

.00

2

(NA

)(0

.55

8 )

(NA

)(N

A )

(NA

)(N

A )

(0.6

88

)(5

.93

2 )

(NA

)

LL

-14

0.2

91

-14

5.0

85

-24

.04

0-2

2.9

80

-14

5.0

85

-14

0.2

91

-14

0.3

10

-14

5.0

85

-14

5.0

89

-14

5.0

85

-14

5.0

85

-14

0.2

91

-14

5.0

86

-14

5.0

85

0.3

36

11

0.0

00

0.1

00

0.3

36

0.1

38

1.0

00

0.0

24

10

.05

00

.17

80

.11

1N

A

(0.0

87

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

22

)(0

.08

6 )

(0.0

42

)(0

.18

2 )

(0.0

14

)(N

A )

(0.0

13

)(0

.08

7 )

(0.0

25

)(N

A )

0.0

00

0.4

53

1.0

00

5.3

47

1.4

58

3.6

99

0.4

23

17

.19

6N

A

(NA

)(0

.14

7 )

(NA

)(2

.44

2 )

(NA

)(1

.68

2 )

(0.1

44

)(N

A )

(NA

)

LL

-67

.37

6-1

45

.08

5-1

45

.08

5-1

45

.08

5-1

22

.12

0-6

7.3

76

-48

.75

8-1

45

.08

5-1

43

.64

1-1

45

.08

5-1

28

.82

6-4

9.6

32

-12

2.0

12

NA

0.9

06

11

0.0

00

0.1

00

0.3

41

0.1

18

1.0

00

0.0

24

10

.05

50

.41

90

.14

1N

A

(0.0

55

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

21

)(0

.09

8 )

(0.0

25

)(0

.18

2 )

(0.0

14

)(N

A )

(0.0

17

)(0

.04

7 )

(0.0

30

)(N

A )

4.6

28

2.8

02

1.0

00

5.3

47

1.3

30

3.1

09

2.5

48

25

.34

8N

A

(0.5

78

)(0

.73

7 )

(NA

)(2

.44

2 )

(NA

)(1

.44

6 )

(0.3

56

)(8

.40

2 )

(NA

)

LL

-13

9.7

52

-14

5.0

85

-14

5.0

85

-14

5.0

85

-11

8.3

05

-11

2.2

11

-97

.28

3-1

45

.08

5-1

43

.64

1-1

45

.08

5-1

28

.34

2-8

8.4

46

-11

7.2

81

NA

0.9

40

0.9

75

0.6

72

0.2

80

0.0

00

0.9

40

0.0

00

0.9

75

0.0

00

10

.00

00

.00

00

.00

00

.00

0

(0.0

46

)(0

.03

3 )

(0.0

87

)(0

.03

4 )

(0.0

08

)(0

.04

3 )

(0.0

05

)(0

.02

8 )

(0.0

08

)(N

A )

(0.0

01

)(0

.00

8 )

(0.0

09

)(0

.00

0 )

0.0

00

2.0

00

0.0

00

13

.31

01

.45

82

.00

02

.79

73

.64

81

.02

1

(NA

)(0

.44

9 )

(NA

)(N

A )

(NA

)(N

A )

(0.8

53

)(1

.26

6 )

(NA

)

LL

-14

2.3

52

-14

4.4

33

-11

5.5

26

-11

7.8

43

-14

5.0

85

-14

2.3

52

-14

3.1

88

-14

4.4

33

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

2.3

53

-14

4.4

33

-14

5.0

85

0.9

75

11

0.0

00

0.0

31

0.9

18

0.0

66

0.6

82

0.0

91

10

.04

40

.03

10

.15

3N

A

(0.0

33

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

09

)(0

.05

6 )

(0.0

26

)(0

.08

8 )

(0.0

23

)(N

A )

(0.0

15

)(0

.00

9 )

(0.0

31

)(N

A )

0.6

55

57

.31

81

.00

32

.43

11

.45

83

4.8

89

12

.79

31

.77

8N

A

(0.5

31

)(7

.42

8 )

(0.3

28

)(0

.58

6 )

(NA

)(4

.34

6 )

(5.9

32

)(0

.30

6 )

(NA

)

LL

-14

4.4

33

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

0.6

50

-14

2.4

99

-13

6.1

40

-12

5.9

01

-13

6.0

49

-14

5.0

85

-13

7.7

97

-14

0.6

51

-13

5.9

69

NA

10

.97

31

0.0

00

0.0

48

0.9

31

0.0

64

0.9

73

0.0

62

0.0

00

0.0

52

0.0

48

0.0

82

0.0

48

(0.1

83

)(0

.03

3 )

(0.1

83

)(0

.16

7 )

(0.0

11

)(0

.00

0 )

(0.0

20

)(0

.03

0 )

(0.0

18

)(1

.89

0 )

(0.0

15

)(0

.01

1 )

(0.0

26

)(0

.00

0 )

8.7

85

61

.47

70

.00

04

.23

10

.09

22

5.3

74

13

.15

22

.97

86

.84

4

(NA

)(6

.56

7 )

(NA

)(2

.02

8 )

(0.1

80

)(4

.48

6 )

(NA

)(0

.69

1 )

(NA

)

LL

-14

5.0

85

-14

4.2

07

-14

5.0

85

-14

5.0

85

-13

5.6

29

-14

4.4

67

-13

5.9

12

-14

4.2

07

-13

6.1

03

-14

0.2

57

-13

3.5

94

-13

5.6

30

-13

2.6

49

-13

5.6

24

15

ε o

r γ

τ

ε o

r γ

τ 1

6

13

ε o

r γ

τ

14

ε o

r γ

τ

11

ε o

r γ

τ

12

ε o

r γ

τ

9

ε o

r γ

τ

10

ε o

r γ

τ

13

SP

ara

TM

N_a

tN

_pt

QR

ELK

_at_

TLK

_pt_

TLK

_at_

MLK

_pt_

MLK

_at_

ULK

_pt_

UN

I_T

NI_

MN

I_U

0.8

40

0.9

75

0.9

42

0.1

40

0.0

17

0.8

40

0.0

00

0.9

75

0.0

64

10

.05

90

.00

00

.02

0N

A

(0.0

69

)(0

.03

3 )

(0.0

46

)(0

.04

9 )

(0.0

10

)(0

.06

7 )

(0.0

06

)(0

.02

9 )

(0.0

22

)(0

.00

0 )

(0.0

20

)(0

.00

9 )

(0.0

12

)(N

A )

0.0

00

2.0

00

0.0

00

11

.12

41

.46

95

8.1

74

1.8

29

3.3

96

NA

(NA

)(0

.48

3 )

(NA

)(3

.29

7 )

(NA

)(3

.54

9 )

(0.4

35

)(1

.17

6 )

(NA

)

LL

-13

3.7

62

-14

4.4

33

-14

2.7

66

-14

2.9

19

-14

4.0

58

-13

3.7

62

-13

3.8

35

-14

4.4

33

-13

8.8

67

-14

5.0

85

-13

9.4

10

-13

3.7

62

-14

3.1

73

NA

10

.94

00

.97

30

.04

60

.00

01

.00

00

.00

00

.94

00

.00

01

0.0

00

0.0

00

0.0

00

0.0

00

(0.1

83

)(0

.04

6 )

(0.0

33

)(0

.11

0 )

(0.0

09

)(0

.18

2 )

(0.0

05

)(0

.04

3 )

(0.0

00

)(0

.00

0 )

(0.0

00

)(0

.00

9 )

(0.0

10

)(N

A )

1.0

00

6.0

00

0.0

00

14

.95

51

.45

85

.99

26

.22

92

.81

01

.10

6

(NA

)(3

.44

5 )

(NA

)(N

A )

(NA

)(N

A )

(3.4

74

)(0

.75

7 )

(NA

)

LL

-14

5.0

85

-14

2.2

16

-14

4.2

59

-14

5.0

17

-14

5.0

85

-14

5.0

85

-14

5.1

59

-14

2.2

16

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.1

44

-14

2.2

16

-14

5.0

85

10

.84

21

0.0

00

0.0

04

1.0

00

0.0

00

0.8

14

0.1

40

10

.00

00

.00

40

.01

90

.00

4

(0.1

83

)(0

.06

9 )

(0.1

83

)(0

.16

7 )

(0.0

11

)(0

.18

2 )

(0.0

00

)(0

.07

4 )

(0.0

34

)(0

.00

0 )

(0.0

06

)(0

.01

1 )

(0.0

14

)(0

.01

1 )

1.0

00

8.9

87

0.1

46

15

.99

81

.45

85

.99

51

0.4

63

1.6

25

3.1

47

(NA

)(N

A )

(0.1

75

)(2

.16

3 )

(NA

)(6

.16

0 )

(NA

)(0

.42

1 )

(5.9

39

)

LL

-14

5.0

85

-13

5.2

77

-14

5.0

85

-14

5.0

85

-14

5.0

33

-14

5.0

85

-14

5.0

89

-13

4.3

72

-12

6.5

66

-14

5.0

85

-14

5.0

85

-14

5.0

34

-13

4.3

16

-14

5.0

33

11

0.9

42

0.1

69

0.0

00

1.0

00

0.0

00

1.0

00

0.1

09

10

.00

00

.00

00

.00

00

.00

0

(0.1

83

)(0

.18

3 )

(0.0

46

)(0

.04

0 )

(0.0

09

)(0

.18

3 )

(0.0

05

)(0

.00

0 )

(0.0

33

)(N

A )

(0.0

00

)(0

.00

9 )

(0.0

09

)(0

.00

0 )

0.9

70

7.0

00

3.8

02

34

.11

71

.45

85

.99

47

.44

48

.10

01

.19

1

(NA

)(4

.21

8 )

(NA

)(3

.64

1 )

(NA

)(N

A )

(5.9

68

)(5

.93

2 )

(NA

)

LL

-14

5.0

85

-14

5.0

85

-14

2.7

66

-14

0.5

92

-14

5.0

85

-14

5.0

85

-14

5.1

12

-14

5.0

85

-13

7.1

10

-14

5.0

85

-14

5.0

85

-14

5.1

02

-14

5.0

94

-14

5.0

85

0.9

75

10

.23

51

.09

00

.00

00

.97

50

.00

01

.00

00

.00

01

0.0

00

0.0

00

0.0

00

0.0

00

(0.0

33

)(0

.18

3 )

(0.0

78

)(0

.09

4 )

(0.0

07

)(0

.02

8 )

(0.0

05

)(0

.18

2 )

(NA

)(N

A )

(0.0

00

)(0

.00

6 )

(0.0

07

)(0

.00

1 )

0.0

00

6.0

00

1.0

00

6.3

72

1.4

58

6.0

01

5.8

13

12

.07

31

.00

3

(NA

)(N

A )

(NA

)(N

A )

(NA

)(N

A )

(NA

)(5

.93

2 )

(NA

)

LL

-14

4.4

33

-14

5.0

85

-50

.29

1-4

6.8

19

-14

5.0

85

-14

4.4

33

-14

4.8

96

-14

5.0

85

-14

5.1

36

-14

5.0

85

-14

5.0

85

-14

4.8

65

-14

5.0

85

-14

5.0

85

0.5

38

11

0.0

00

0.1

00

0.5

38

0.0

71

1.0

00

0.0

24

0.0

00

0.0

55

0.1

08

0.1

22

0.1

22

(0.0

92

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

21

)(0

.09

1 )

(0.0

18

)(0

.18

2 )

(0.0

14

)(0

.84

8 )

(0.0

17

)(0

.03

2 )

(0.0

00

)(0

.00

0 )

0.0

00

0.8

19

1.0

00

5.3

35

0.1

92

3.0

15

0.9

01

15

.71

31

2.7

61

(NA

)(0

.21

4 )

(NA

)(2

.44

2 )

(0.1

77

)(1

.42

0 )

(0.2

37

)(N

A )

(NA

)

LL

-97

.81

3-1

45

.08

5-1

45

.08

5-1

45

.08

5-1

20

.46

5-9

7.8

13

-79

.27

5-1

45

.08

5-1

43

.64

1-1

31

.30

0-1

28

.53

4-7

8.0

04

-12

0.0

85

-12

0.0

83

11

0.1

01

1.3

15

0.0

00

1.0

00

0.0

00

1.0

00

10

.00

00

.00

00

.00

00

.00

0

(0.1

83

)(0

.18

3 )

(0.0

55

)(0

.12

6 )

(0.0

06

)(0

.18

2 )

(0.0

05

)(0

.18

2 )

()(N

A )

(0.0

00

)(0

.00

7 )

(0.0

06

)(0

.00

1 )

1.0

00

6.0

00

1.0

00

1.4

58

6.0

00

6.2

52

16

.83

51

.00

3

(NA

)(4

.23

1 )

(NA

)()

(NA

)(N

A )

(6.3

18

)(N

A )

(NA

)

LL

-14

5.0

85

-14

5.0

85

-24

.62

2-2

2.9

80

-14

5.0

85

-14

5.0

85

-14

5.1

59

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.1

43

-14

5.0

85

-14

5.0

85

11

10

.00

00

.00

01

.00

00

.00

01

.00

00

.00

01

0.0

00

0.0

00

0.0

00

0.0

00

(0.1

83

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

07

)(0

.18

2 )

(0.0

05

)(0

.18

3 )

(NA

)(N

A )

(0.0

00

)(0

.00

7 )

(0.0

07

)(0

.00

1 )

1.0

00

6.0

00

1.0

00

6.8

48

1.4

58

6.0

00

5.6

68

7.7

61

1.0

52

(NA

)(2

.97

9 )

(4.1

95

)(N

A )

(NA

)(N

A )

(3.6

03

)(5

.93

2 )

(NA

)

LL

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.1

59

-14

5.0

85

-14

5.1

17

-14

5.0

85

-14

5.0

85

-14

5.1

89

-14

5.0

98

-14

5.0

85

23

ε o

r γ

τ

24

ε o

r γ

τ

21

ε o

r γ

τ

22

ε o

r γ

τ

19

ε o

r γ

τ

20

ε o

r γ

τ

17

ε o

r γ

τ

18

ε o

r γ

τ

14

SP

ara

TM

N_a

tN

_pt

QR

ELK

_at_

TLK

_pt_

TLK

_at_

MLK

_pt_

MLK

_at_

ULK

_pt_

UN

I_T

NI_

MN

I_U

11

10

.00

00

.00

01

.00

00

.00

01

.00

00

.09

31

0.0

00

0.0

00

0.0

00

0.0

00

(0.1

83

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

09

)(0

.18

2 )

(0.0

05

)(N

A )

(0.0

35

)(N

A )

(NA

)(0

.00

9 )

(0.0

09

)(0

.00

0 )

1.0

00

10

.00

03

.36

82

9.5

27

1.4

58

6.0

00

12

.31

79

.00

01

.11

4

(NA

)(N

A )

(NA

)(4

.84

1 )

(NA

)(N

A )

(NA

)(5

.93

2 )

(NA

)

LL

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

86

-14

5.0

85

-13

9.5

27

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

88

-14

5.0

85

0.4

70

11

0.0

00

0.0

76

0.4

70

0.0

52

1.0

00

0.0

24

10

.04

00

.05

90

.07

6N

A

(0.0

92

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

15

)(0

.09

1 )

(0.0

15

)(0

.18

2 )

(0.0

14

)(N

A )

(0.0

11

)(0

.01

7 )

(0.0

00

)(N

A )

0.0

00

0.6

59

1.0

00

5.3

35

1.4

58

3.9

71

0.6

45

12

.49

9N

A

(NA

)(0

.18

1 )

(NA

)(2

.44

2 )

(NA

)(2

.10

1 )

(0.1

76

)(N

A )

(NA

)

LL

-88

.26

4-1

45

.08

5-1

45

.08

5-1

45

.08

5-1

28

.05

6-8

8.2

65

-75

.99

4-1

45

.08

5-1

43

.64

1-1

45

.08

5-1

32

.62

4-7

6.1

98

-12

8.0

58

NA

0.9

73

0.7

73

0.8

08

0.2

13

0.0

00

0.9

73

0.0

00

0.7

45

0.0

00

10

.00

10

.00

00

.00

00

.00

0

(0.0

33

)(0

.07

8 )

(0.0

74

)(0

.03

3 )

(0.0

13

)(0

.03

0 )

(0.0

05

)(0

.08

2 )

(0.0

13

)(0

.00

0 )

(NA

)(0

.01

1 )

(0.0

15

)(0

.01

9 )

0.0

00

3.4

43

0.1

09

1.2

85

1.4

58

9.3

81

3.8

80

1.4

82

1.2

00

(NA

)(1

.10

5 )

(0.1

28

)(0

.28

9 )

(NA

)(N

A )

(1.6

24

)(0

.34

3 )

(6.2

40

)

LL

-14

4.2

07

-12

7.4

14

-13

1.8

19

-13

3.8

59

-14

5.0

85

-14

4.2

07

-14

4.2

16

-12

6.4

20

-12

7.6

02

-14

5.0

85

-14

5.0

56

-14

4.2

30

-12

7.4

14

-14

5.0

85

0.7

73

0.9

07

0.9

73

0.1

27

0.0

35

0.7

73

0.0

11

0.9

07

0.0

09

0.0

45

0.0

87

0.0

16

0.0

36

0.2

35

(0.0

78

)(0

.05

5 )

(0.0

33

)(0

.05

4 )

(0.0

10

)(0

.07

6 )

(0.0

06

)(0

.05

3 )

(0.0

13

)(4

.14

9 )

(0.0

62

)(0

.00

8 )

(0.0

14

)(0

.17

3 )

0.0

00

1.5

07

0.0

00

2.3

47

0.0

63

1.2

34

1.4

98

2.7

56

1.0

34

(NA

)(0

.35

3 )

(NA

)(0

.58

2 )

(0.2

80

)(0

.62

7 )

(0.3

49

)(0

.68

4 )

(0.5

01

)

LL

-12

7.5

03

-14

0.1

25

-14

4.2

07

-14

3.5

86

-14

0.0

84

-12

7.5

04

-12

5.6

62

-14

0.1

25

-13

9.8

62

-14

2.4

10

-13

7.1

53

-12

5.7

25

-13

6.6

34

-13

5.7

88

0.0

00

11

0.0

00

0.1

00

0.0

00

0.0

00

1.0

00

0.0

24

10

.04

90

.00

00

.10

8N

A

(0.1

84

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

23

)(0

.15

6 )

( 0

.03

9)

(0.1

82

)(0

.01

4 )

(NA

)(0

.01

2 )

(0.0

39

)(0

.02

5 )

(NA

)

0.0

00

0.0

00

1.0

00

5.3

43

1.4

58

3.9

91

0.0

00

13

.78

5N

A

(NA

)(0

.00

2 )

(NA

)(2

.44

2 )

(NA

)(1

.65

5 )

(0.0

01

)(8

.39

4 )

(NA

)

LL

0.0

00

-14

5.0

85

-14

5.0

85

-14

5.0

85

-12

3.9

85

0.0

00

0.0

00

-14

5.0

85

-14

3.6

41

-14

5.0

85

-12

8.9

66

0.0

00

-12

3.9

41

NA

0.7

73

0.9

75

10

.00

00

.10

00

.54

60

.05

40

.97

50

.02

50

.00

00

.06

60

.11

00

.11

6N

A

(0.0

78

)(0

.03

3 )

(0.1

83

)(0

.16

7 )

(0.0

19

)(0

.09

4 )

(0.0

12

)(0

.02

8 )

(0.0

14

)(1

.85

9 )

(0.0

35

)(0

.02

8 )

(0.0

22

)(N

A )

0.5

70

1.5

07

0.0

00

3.5

15

0.0

93

2.5

15

1.7

52

6.4

97

NA

(0.2

19

)(0

.36

2 )

(NA

)(1

.16

4 )

(0.1

81

)(1

.52

4 )

(0.3

84

)(1

.58

5 )

(NA

)

LL

-12

7.1

53

-14

4.4

33

-14

5.0

85

-14

5.0

85

-12

0.3

59

-11

1.0

64

-10

6.7

15

-14

4.4

33

-14

2.7

13

-13

9.8

06

-12

8.0

28

-10

2.9

42

-11

9.8

52

NA

0.9

74

11

0.0

00

0.0

29

0.9

74

0.0

09

0.8

70

0.0

39

0.0

01

0.0

18

0.0

28

0.0

29

NA

(0.0

33

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

09

)(0

.02

9 )

(0.0

05

)(0

.07

9 )

(0.0

16

)(N

A )

(0.0

07

)(0

.00

0 )

(NA

)(N

A )

0.0

00

3.9

45

4.1

78

5.4

34

0.0

25

5.4

79

5.6

74

19

.29

7N

A

(NA

)(1

.67

0 )

(1.3

44

)(2

.65

3 )

(NA

)(5

.93

3 )

(NA

)(N

A )

(NA

)

LL

-14

4.3

14

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

1.1

24

-14

4.3

14

-14

2.9

41

-14

2.6

59

-14

1.4

84

-14

4.4

42

-14

1.3

15

-14

1.1

29

-14

1.1

24

NA

10

.37

00

.87

40

.21

90

.00

01

.00

00

.00

00

.30

50

.00

01

0.0

00

0.0

00

0.0

00

0.0

01

(0.1

83

)(0

.08

9 )

(0.0

63

)(0

.03

2 )

(0.0

12

)(0

.18

2 )

(0.0

05

)(0

.08

5 )

(0.0

19

)(0

.00

0 )

(0.0

06

)(0

.01

2 )

(0.0

20

)(0

.01

7 )

1.0

00

8.9

96

0.0

93

0.4

61

1.4

58

9.9

99

9.9

44

0.4

63

1.1

03

(NA

)(6

.07

1 )

(0.0

68

)(0

.14

1 )

(NA

)(N

A )

(5.9

32

)(0

.14

1 )

(NA

)

LL

-14

5.0

85

-73

.22

4-1

37

.38

9-1

32

.78

0-1

45

.08

4-1

45

.08

5-1

45

.08

8-6

8.4

54

-73

.22

4-1

45

.08

5-1

45

.08

5-1

45

.08

5-7

3.2

24

-14

5.0

84

32

τ

29

τ

30

τ

31

τ

ε o

r γ

ε o

r γ

25

τ

26

τ

27

τ

28

τ

ε o

r γ

ε o

r γ

ε o

r γ

ε o

r γ

ε o

r γ

ε o

r γ

15

SP

ara

TM

N_a

tN

_pt

QR

ELK

_at_

TLK

_pt_

TLK

_at_

MLK

_pt_

MLK

_at_

ULK

_pt_

UN

I_T

NI_

MN

I_U

0.6

72

0.9

76

0.9

41

0.1

69

0.0

95

0.6

43

0.0

18

0.9

76

0.0

24

0.0

00

0.0

80

0.0

28

0.2

10

0.2

29

(0.0

87

)(0

.03

3 )

(0.0

46

)(0

.04

0 )

(0.0

21

)(0

.08

9 )

(0.0

07

)(0

.02

8 )

(0.0

14

)(0

.73

7 )

(0.0

26

)(0

.01

0 )

(0.0

10

)(0

.06

5 )

0.0

81

1.1

37

0.0

00

3.5

54

0.2

25

2.5

35

1.2

03

11

.02

92

.69

7

(0.0

92

)(0

.27

3 )

(NA

)(1

.18

8 )

(0.1

82

)(0

.83

0 )

(0.2

96

)(1

.50

5 )

(0.6

55

)

LL

-11

6.0

69

-14

4.5

65

-14

2.5

69

-14

0.5

92

-12

6.5

01

-11

4.8

84

-11

2.0

66

-14

4.5

66

-14

2.9

70

-12

7.9

70

-12

6.0

31

-11

1.3

31

-11

8.3

38

-12

0.8

46

11

0.0

00

3.7

37

0.0

00

1.0

00

0.0

00

1.0

00

0.0

00

10

.00

00

.00

00

.00

00

.00

0

(0.1

83

)(0

.18

3 )

(0.1

84

)(2

3.7

27

)(0

.00

6 )

(0.1

82

)(0

.00

5 )

(0.1

82

)(0

.00

0 )

(NA

)(N

A )

(0.0

06

)(0

.00

6 )

(0.0

00

)

1.0

00

6.0

00

1.0

00

7.5

36

1.4

58

3.0

00

5.8

12

8.6

22

1.0

01

(NA

)(3

.44

5 )

(NA

)(N

A )

(NA

)(N

A )

(3.5

68

)(5

.93

2 )

(NA

)

LL

-14

5.0

85

-14

5.0

85

0.0

00

0.0

00

-14

5.0

85

-14

5.0

85

-14

5.1

59

-14

5.0

85

-14

5.1

01

-14

5.0

85

-14

5.0

85

-14

5.1

75

-14

5.0

90

-14

5.0

85

0.1

68

11

0.0

00

NA

0.0

00

5.9

65

1.0

00

0.0

24

10

.04

95

.96

50

.10

80

.10

8

(0.0

69

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(NA

)(0

.18

6 )

(In

f )

(0.1

82

)(0

.01

4 )

(NA

)(0

.01

2 )

(NA

)(0

.00

0 )

(0.0

26

)

0.1

67

0.1

67

1.0

00

5.3

34

1.4

58

3.9

80

0.1

67

13

.79

71

1.5

71

(0.0

75

)(I

nf

)(N

A )

(2.6

79

)(N

A )

(1.6

58

)(0

.07

2 )

(NA

)(N

A )

LL

-38

.11

5-1

45

.08

5-1

45

.08

5-1

45

.08

5N

A-1

3.9

59

-13

.97

2-1

45

.08

5-1

43

.64

1-1

45

.08

5-1

28

.95

9-1

3.9

72

-12

3.8

25

-12

3.8

24

0.9

07

10

.10

10

.88

40

.00

00

.90

70

.00

01

.00

00

.00

01

0.0

00

0.0

00

0.0

00

0.0

00

(0.0

55

)(0

.18

3 )

(0.0

55

)(0

.09

2 )

(0.0

06

)(0

.05

3 )

(0.0

05

)(0

.18

2 )

(0.0

00

)(N

A )

(0.0

00

)(I

nf

)(0

.00

0 )

(0.0

00

)

0.0

00

2.3

74

1.0

00

8.3

36

1.4

58

3.0

00

13

.69

21

0.2

66

1.0

01

(NA

)(0

.59

3 )

(NA

)(N

A )

(NA

)(N

A )

(In

f )

(NA

)(N

A )

LL

-13

9.9

59

-14

5.0

85

-24

.40

1-7

1.4

74

-14

5.0

85

-13

9.9

59

-13

9.9

59

-14

5.0

85

-14

5.0

92

-14

5.0

85

-14

5.0

85

-14

5.0

84

-14

5.0

86

-14

5.0

85

11

10

.00

00

.04

51

.00

00

.01

71

.00

00

.07

50

.64

50

.03

30

.04

50

.17

90

.04

5

(0.1

83

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

11

)(I

nf

)(0

.00

7)

(0.1

83

)(0

.02

3 )

(0.1

11

)(0

.00

0 )

(In

f )

(0.0

13

)(0

.00

0 )

1.0

00

17

.90

71

.00

01

8.6

38

10

.68

79

.10

61

8.8

46

3.8

74

12

.22

4

(In

f )

(6.0

91

)(5

.93

2 )

(4.5

70

)(1

.79

1 )

(NA

)(I

nf

)(0

.58

0 )

(NA

)

LL

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

85

-13

7.4

29

-14

5.0

85

-14

1.0

98

-14

5.0

85

-13

7.7

90

-13

6.2

94

-13

6.0

06

-13

7.4

29

-12

9.0

10

-13

7.4

31

0.9

75

0.8

73

0.8

06

0.1

96

0.0

00

0.9

75

0.0

00

0.8

73

0.0

00

0.0

00

0.0

00

0.0

00

0.0

00

0.0

00

(0.0

33

)(0

.06

3 )

(0.0

74

)(0

.03

5 )

(0.0

10

)(0

.02

8 )

(0.0

05

)(0

.06

1 )

(0.0

13

)(1

.97

0 )

(0.0

05

)(0

.00

9 )

(0.0

11

)(0

.00

4 )

0.0

00

3.6

05

0.0

00

2.0

72

0.0

92

3.0

00

3.6

81

2.0

65

1.0

64

(NA

)(1

.23

8 )

(NA

)(0

.49

7 )

(0.1

88

)(N

A )

(1.3

35

)(0

.49

5 )

(NA

)

LL

-14

4.4

33

-13

7.0

49

-13

0.2

50

-13

6.9

32

-14

5.0

85

-14

4.4

33

-14

4.4

34

-13

7.0

49

-13

7.0

49

-14

0.2

57

-14

5.0

85

-14

4.4

33

-13

7.0

49

-14

5.0

85

11

10

.00

00

.00

01

.00

00

.00

01

.00

00

.24

21

0.0

00

0.0

00

0.0

00

0.0

00

(0.1

83

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

10

)(0

.18

2 )

(0.0

05

)(0

.18

3 )

(0.0

99

)(N

A )

(0.0

00

)(0

.01

0 )

(0.0

10

)(0

.00

3 )

1.0

00

8.2

20

1.0

00

24

.72

71

.45

83

.00

06

.33

67

.99

21

.05

2

(NA

)(6

.06

1 )

(NA

)(1

.91

5 )

(NA

)(N

A )

(6.2

20

)(4

.19

5 )

(NA

)

LL

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

93

-14

5.0

85

-12

4.1

69

-14

5.0

85

-14

5.0

85

-14

5.1

38

-14

5.0

95

-14

5.0

85

0.2

69

10

.94

30

.16

9N

A0

.10

20

.02

61

.00

00

.02

21

0.0

48

0.0

29

0.1

76

0.1

80

(0.0

81

)(0

.18

3 )

(0.0

46

)(0

.04

0 )

(NA

)(0

.05

6 )

(0.0

12

)(0

.18

2 )

(0.0

14

)(N

A )

(0.0

12

)(0

.01

3 )

(0.0

13

)(0

.00

9 )

0.1

85

0.3

09

1.0

00

5.7

99

1.4

58

4.1

85

0.3

16

8.1

51

10

.71

1

(0.0

83

)(0

.11

0 )

(NA

)(2

.97

7 )

(NA

)(2

.08

6 )

(0.1

12

)(1

.38

7 )

(2.4

10

)

LL

-56

.03

9-1

45

.08

5-1

42

.92

4-1

40

.59

2N

A-3

7.3

12

-53

.21

4-1

45

.08

5-1

43

.88

5-1

45

.08

5-1

29

.49

4-5

3.0

25

-11

9.9

34

-12

0.8

44

40

τ

37

τ

38

τ

39

τ

ε o

r γ

ε o

r γ

ε o

r γ

33

τ

34

τ

35

τ

36

ε o

r γ

ε o

r γ

ε o

r γ

ε o

r γ

ε o

r γ

τ

16

SP

ara

TM

N_a

tN

_pt

QR

ELK

_at_

TLK

_pt_

TLK

_at_

MLK

_pt_

MLK

_at_

ULK

_pt_

UN

I_T

NI_

MN

I_U

0.9

08

11

0.0

00

NA

0.0

00

5.9

65

1.0

00

0.0

24

10

.04

91

.24

20

.11

40

.11

4

(0.0

55

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(NA

)(0

.18

8 )

(NA

)(0

.18

2 )

(0.0

14

)(N

A )

(0.0

12

)(0

.33

1 )

(0.0

00

)(0

.02

7 )

1.4

33

1.4

31

1.0

00

5.3

34

1.4

58

3.9

16

0.9

79

16

.50

01

3.2

16

(0.2

19

)(0

.21

9 )

(NA

)(2

.67

9 )

(NA

)(1

.67

3 )

(0.1

88

)(N

A )

(NA

)

LL

-14

0.3

92

-14

5.0

85

-14

5.0

85

-14

5.0

85

NA

-38

.61

0-3

8.6

22

-14

5.0

85

-14

3.6

41

-14

5.0

85

-12

8.9

29

-56

.11

9-1

22

.82

8-1

22

.82

9

0.5

38

10

.97

60

.12

7N

A0

.20

40

.04

71

.00

00

.03

31

0.0

45

0.0

55

0.1

72

0.1

74

(0.0

92

)(0

.18

3 )

(0.0

33

)(0

.05

4 )

(NA

)(0

.07

4 )

(0.0

13

)(0

.18

2 )

(0.0

15

)(N

A )

(0.0

12

)(0

.01

5 )

(0.0

09

)(0

.00

8 )

0.4

16

0.7

52

1.0

00

6.0

34

1.4

58

3.9

34

0.7

57

10

.52

49

.10

7

(0.1

32

)(0

.19

4 )

(NA

)(3

.42

6 )

(NA

)(2

.03

8 )

(0.1

94

)(8

.53

6 )

(1.9

24

)

LL

-98

.09

6-1

45

.08

5-1

44

.56

5-1

43

.58

6N

A-6

2.7

87

-85

.48

2-1

45

.08

5-1

42

.38

5-1

45

.08

5-1

30

.74

0-8

5.2

67

-12

1.9

92

-12

1.3

37

0.2

35

11

0.0

00

NA

0.0

00

5.9

65

1.0

00

0.0

24

10

.04

95

.96

50

.10

9N

A

(0.0

78

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(NA

)(0

.18

6 )

(6.8

49

)(0

.18

2 )

(0.0

14

)(N

A )

(0.0

12

)(6

.84

9 )

(0.0

26

)(N

A )

0.2

33

0.2

33

1.0

00

5.3

34

1.4

58

3.9

87

0.2

33

13

.92

2N

A

(0.0

88

)(0

.08

8 )

(NA

)(2

.67

9 )

(NA

)(1

.80

2 )

(0.0

88

)(8

.39

4 )

(NA

)

LL

-50

.17

5-1

45

.08

5-1

45

.08

5-1

45

.08

5N

A-1

7.1

87

-17

.20

5-1

45

.08

5-1

43

.64

1-1

45

.08

5-1

28

.96

2-1

7.2

05

-12

3.7

57

NA

0.9

76

11

0.0

00

NA

0.8

45

0.1

00

1.0

00

0.0

24

0.0

00

0.1

74

0.1

53

0.1

04

NA

(0.0

33

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(NA

)(0

.07

7 )

(0.0

22

)(0

.18

2 )

(0.0

14

)(0

.55

4 )

(0.0

77

)(0

.04

4 )

(0.0

21

)(N

A )

1.2

35

7.8

16

1.0

00

5.3

34

0.2

92

1.3

26

2.4

93

8.8

11

NA

(0.8

73

)(1

.56

5 )

(NA

)(2

.67

9 )

(0.1

82

)(0

.37

5 )

(0.4

87

)(4

.90

2 )

(NA

)

LL

-14

4.5

65

-14

5.0

85

-14

5.0

85

-14

5.0

85

NA

-13

9.3

52

-10

6.0

71

-14

5.0

85

-14

3.6

41

-12

1.6

08

-12

8.1

39

-11

1.6

07

-12

1.9

16

NA

11

0.0

00

3.7

37

0.0

00

1.0

00

0.0

00

1.0

00

0.0

00

10

.00

00

.00

00

.00

00

.00

0

(0.1

83

)(0

.18

3 )

(0.1

84

)(2

3.7

27

)(0

.00

6 )

(0.1

82

)(0

.00

5)

(0.1

82

)(0

.00

0 )

(NA

)(N

A )

(0.0

06

)(0

.00

6 )

(0.0

00

)

1.0

00

6.0

00

1.0

00

7.5

36

1.4

58

3.0

00

5.8

12

8.6

22

1.0

01

(NA

)(3

.44

5 )

(NA

)(N

A )

(NA

)(N

A )

(3.5

68

)(5

.93

2 )

(NA

)

LL

-14

5.0

85

-14

5.0

85

0.0

00

0.0

00

-14

5.0

85

-14

5.0

85

-14

5.1

59

-14

5.0

85

-14

5.1

01

-14

5.0

85

-14

5.0

85

-14

5.1

75

-14

5.0

90

-14

5.0

85

0.9

08

11

0.0

00

0.0

42

0.6

84

0.0

12

1.0

00

0.0

71

0.0

00

0.0

40

0.0

24

0.2

04

0.6

92

(0.0

55

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

11

)(0

.09

7 )

(0.0

06

)(0

.18

3 )

(0.0

24

)(1

.05

7 )

(0.0

35

)(0

.00

9 )

(0.0

21

)(0

.77

4 )

7.1

24

2.4

55

1.0

00

19

.97

40

.15

91

.93

62

.60

34

.37

70

.58

6

(0.9

89

)(0

.63

7 )

(NA

)(6

.81

2 )

(0.1

81

)(1

.90

9 )

(0.7

21

)(0

.60

7 )

(0.2

96

)

LL

-14

0.2

49

-14

5.0

85

-14

5.0

85

-14

5.0

85

-13

8.6

37

-13

5.2

71

-13

7.7

88

-14

5.0

85

-13

8.9

21

-13

4.1

68

-13

8.1

92

-13

6.9

44

-12

1.1

60

-13

7.3

77

11

10

.00

0N

A0

.00

05

.96

51

.00

00

.02

41

0.0

49

5.9

65

0.1

13

NA

(0.1

83

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(NA

)(0

.18

7 )

(3.7

52

)(0

.18

2 )

(0.0

14

)(N

A(0

.01

2 )

(3.6

61

)(0

.02

7 )

(NA

)

1.0

00

1.0

00

1.0

00

5.3

34

1.4

58

3.9

53

1.0

01

16

.47

5N

A

(0.1

83

)(0

.18

3 )

(NA

)(2

.67

9 )

(NA

)(1

.60

2 )

(0.1

83

)(N

A )

(NA

)

LL

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

85

NA

-30

.00

0-3

0.0

77

-14

5.0

85

-14

3.6

41

-14

5.0

85

-12

8.9

46

-30

.07

7-1

23

.13

7N

A

10

.97

60

.06

71

.40

80

.00

01

.00

00

.00

00

.97

60

.00

01

0.0

00

0.0

00

0.0

00

0.0

00

(0.1

83

)(0

.03

3 )

(0.0

46

)(0

.14

9 )

(0.0

06

)(0

.18

2 )

(0.0

00

)(0

.02

8 )

(0.0

07

)(N

A )

(0.0

01

)(0

.00

6 )

(0.0

07

)(0

.00

0 )

1.0

00

17

.96

90

.00

01

2.1

52

1.4

58

3.0

00

7.8

63

3.7

12

1.0

02

(NA

)(N

A )

(NA

)(N

A )

(NA

)(N

A )

(NA

)(1

.33

3 )

(NA

)

LL

-14

5.0

85

-14

4.5

65

-16

.56

2-1

6.1

87

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

4.5

66

-14

5.0

84

-14

5.0

85

-14

5.0

85

-14

5.0

96

-14

4.5

66

-14

5.0

85

48

τ

45

τ

46

τ

47

τ

ε o

r γ

ε o

r γ

41

τ

42

τ

43

τ

44

τ

ε o

r γ

ε o

r γ

ε o

r γ

ε o

r γ

ε o

r γ

ε o

r γ

17

SP

ara

TM

N_a

tN

_pt

QR

ELK

_at_

TLK

_pt_

TLK

_at_

MLK

_pt_

MLK

_at_

ULK

_pt_

UN

I_T

NI_

MN

I_U

11

10

.00

00

.06

41

.00

00

.08

20

.81

90

.08

41

0.0

67

0.0

64

0.0

93

0.0

66

(0.1

83

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(0.0

13

)(I

nf

)(0

.00

0 )

(0.0

75

)(0

.02

2 )

(0.0

00

)(0

.01

7 )

(0.0

13

)(0

.02

5 )

(0.0

00

)

1.0

00

44

.98

71

.00

02

.43

31

.31

21

4.9

88

14

.63

42

.91

36

.11

2

(In

f )

(NA

)(0

.43

0 )

(0.5

67

)(N

A )

(2.8

41

)(N

A )

(0.8

68

)(N

A )

LL

-14

5.0

85

-14

5.0

85

-14

5.0

85

-14

5.0

85

-13

0.5

49

-14

5.0

85

-12

9.1

68

-13

7.0

02

-13

7.7

27

-14

5.0

85

-12

6.6

36

-13

0.5

49

-12

9.7

70

-13

0.5

13

0.9

40

11

0.0

00

NA

0.0

00

5.9

65

1.0

00

0.0

24

10

.04

95

.96

50

.11

3N

A

(0.0

46

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(NA

)(0

.18

7 )

(In

f )

(0.1

82

)(0

.01

4 )

(NA

)(0

.01

2 )

(3.9

54

)(0

.00

0 )

(NA

)

0.9

33

0.9

37

1.0

00

5.3

34

1.4

58

3.9

57

0.9

33

16

.47

7N

A

(0.1

76

)(I

nf

)(N

A )

(2.6

79

)(N

A )

(1.6

01

)(0

.17

6 )

(NA

)(N

A )

LL

-14

2.4

11

-14

5.0

85

-14

5.0

85

-14

5.0

85

NA

-29

.93

2-3

0.0

04

-14

5.0

85

-14

3.6

41

-14

5.0

85

-12

8.9

48

-30

.00

4-1

23

.18

9N

A

10

.97

60

.94

10

.08

70

.00

01

.00

00

.00

00

.97

60

.00

01

0.0

00

0.0

00

0.0

00

0.0

00

(0.1

83

)(0

.03

3 )

(0.0

46

)(0

.07

6 )

(0.0

08

)(0

.18

2 )

(0.0

05

)(0

.02

8 )

(0.0

12

)(0

.00

0 )

(NA

)(0

.00

0 )

(0.0

08

)(0

.00

0 )

1.0

00

7.6

85

0.0

00

3.7

50

1.4

58

3.0

01

9.0

56

3.7

27

1.0

63

(NA

)(6

.05

6 )

(NA

)(1

.44

2 )

(NA

)(N

A )

(NA

)(1

.36

1 )

(NA

)

LL

-14

5.0

85

-14

4.5

65

-14

2.5

68

-14

4.6

74

-14

5.0

85

-14

5.0

85

-14

5.0

99

-14

4.5

66

-14

4.5

65

-14

5.0

85

-14

5.0

85

-14

5.0

88

-14

4.5

65

-14

5.0

85

0.3

36

11

0.0

00

NA

0.0

34

0.4

12

1.0

00

0.0

24

10

.05

00

.70

80

.10

90

.10

9

(0.0

87

)(0

.18

3 )

(0.1

83

)(0

.16

7 )

(NA

)(0

.03

3 )

(0.1

31

)(0

.18

2 )

(0.0

14

)(N

A )

(0.0

12

)(0

.14

5 )

(0.0

25

)(0

.02

6 )

0.3

10

0.3

58

1.0

00

5.3

34

1.4

58

3.8

89

0.4

52

13

.75

61

1.8

63

(0.1

03

)(0

.11

4 )

(NA

)(2

.67

9 )

(NA

)(1

.61

8 )

(0.1

29

)(N

A )

(8.3

95

)

LL

-67

.37

6-1

45

.08

5-1

45

.08

5-1

45

.08

5N

A-2

8.5

12

-38

.97

2-1

45

.08

5-1

43

.64

1-1

45

.08

5-1

28

.91

6-3

5.9

33

-12

3.2

77

-12

3.2

73

10

.84

00

.63

90

.25

40

.00

01

.00

00

.00

00

.84

00

.00

01

0.0

00

0.0

00

0.0

00

0.0

00

(0.1

83

)(0

.06

9 )

(0.0

89

)(0

.03

2 )

(0.0

08

)(0

.18

2 )

(0.0

05

)(0

.06

7 )

(0.0

13

)(N

A )

(0.0

00

)(0

.00

8 )

(0.0

10

)(0

.00

0 )

1.0

00

5.9

08

0.0

00

1.6

79

1.4

58

3.0

01

8.7

98

1.7

90

1.0

64

(NA

)(3

.44

4 )

(NA

)(0

.38

3 )

(NA

)(N

A )

(NA

)(0

.41

6 )

(NA

)

LL

-14

5.0

85

-13

3.8

90

-11

2.0

58

-12

4.5

12

-14

5.0

85

-14

5.0

85

-14

5.1

66

-13

3.8

90

-13

3.9

58

-14

5.0

85

-14

5.0

85

-14

5.0

89

-13

3.8

95

-14

5.0

85

10

.77

31

0.0

00

0.0

19

1.0

00

0.0

00

0.0

74

0.6

47

10

.03

40

.01

90

.84

40

.01

9

(0.1

83

)(0

.07

8 )

(0.1

83

)(0

.16

7 )

(0.0

08

)(0

.18

2 )

(0.0

05

)(0

.05

0 )

(0.1

01

)(N

A )

(0.0

15

)(0

.00

0 )

(0.0

63

)(0

.00

9 )

1.0

00

7.3

15

1.5

91

1.1

40

1.4

58

45

.05

91

1.9

63

1.0

13

5.2

60

(NA

)(6

.05

7 )

(0.2

44

)(0

.23

3 )

(NA

)(4

.27

1 )

(NA

)(0

.19

3 )

(NA

)

LL

-14

5.0

85

-12

7.7

19

-14

5.0

85

-14

5.0

85

-14

3.3

39

-14

5.0

85

-14

5.1

03

-66

.69

5-6

5.2

31

-14

5.0

85

-14

1.3

90

-14

3.3

40

-65

.17

3-1

43

.33

4

10

.43

70

.67

20

.41

50

.00

01

.00

00

.00

00

.37

20

.00

01

0.0

00

0.0

00

0.0

00

0.0

00

(0.1

83

)(0

.09

1 )

(0.0

87

)(0

.08

3 )

(0.0

10

)(0

.18

2 )

(0.0

00

)(0

.08

9 )

(0.0

18

)(N

A )

(0.0

04

)(0

.00

0 )

(0.0

18

)(0

.00

0 )

1.0

00

7.9

73

0.1

02

0.5

74

1.4

58

3.0

00

10

.38

10

.57

41

.09

8

(NA

)(N

A )

(0.0

75

)(0

.16

2 )

(NA

)(N

A )

(NA

)(0

.16

2 )

(NA

)

LL

-14

5.0

85

-83

.43

2-1

15

.65

7-9

8.1

26

-14

5.0

85

-14

5.0

85

-14

5.0

95

-79

.02

3-8

3.4

32

-14

5.0

85

-14

5.0

85

-14

5.0

86

-83

.43

2-1

45

.08

5

0.3

02

0.9

75

0.8

07

0.2

29

NA

0.2

72

0.0

00

0.9

75

0.0

00

10

.05

70

.00

00

.02

70

.25

1

(0.0

84

)(0

.03

3 )

(0.0

74

)(0

.03

2 )

(NA

)(0

.08

2 )

(0.0

10

)(0

.02

8 )

(0.0

13

)(N

A )

(0.0

68

)(0

.01

0 )

(0.0

12

)(0

.23

0 )

0.0

42

0.3

60

0.0

00

8.0

90

1.4

58

1.0

52

0.3

60

4.3

21

0.6

11

(0.0

46

)(0

.12

1 )

(NA

)(N

A )

(NA

)(0

.95

6 )

(0.1

22

)(1

.79

1 )

(0.3

51

)

LL

-61

.20

2-1

44

.43

3-1

31

.43

0-1

30

.54

4N

A-5

9.6

73

-61

.20

2-1

44

.43

3-1

45

.05

8-1

45

.08

5-1

41

.02

3-6

1.2

02

-14

2.3

29

-14

0.1

02

56

τ

53

τ

54

τ

55

τ

ε o

r γ

ε o

r γ

49

τ

50

τ

51

τ

52

τ

ε o

r γ

ε o

r γ

ε o

r γ

ε o

r γ

ε o

r γ

ε o

r γ

18

SP

ara

TM

N_a

tN

_pt

QR

ELK

_at_

TLK

_pt_

TLK

_at_

MLK

_pt_

MLK

_at_

ULK

_pt_

UN

I_T

NI_

MN

I_U

0.9

73

0.9

76

10

.00

0N

A0

.97

30

.06

50

.97

60

.03

70

.67

00

.07

80

.11

80

.12

30

.12

6(0

.03

3 )

(0.0

33

)(0

.18

3 )

(0.1

67

)(N

A )

(0.0

30

)(0

.02

6 )

(0.0

28

)(0

.01

5 )

(0.1

55

)(0

.01

7 )

(0.0

20

)(0

.00

0 )

(0.0

30

)

0.0

00

25

.92

90

.00

03

.47

21

.09

76

.63

76

.76

91

7.7

77

5.4

74

(NA

)(1

4.5

54

)(N

A )

(1.0

84

)(0

.70

9 )

(3.5

20

)(2

.22

3 )

(NA

)(4

.88

0 )

LL

-14

4.2

07

-14

4.5

65

-14

5.0

85

-14

5.0

85

NA

-14

4.2

07

-12

4.7

97

-14

4.5

66

-14

1.0

47

-13

3.9

70

-11

9.3

18

-11

8.6

52

-11

8.6

31

-11

8.6

73

0.8

75

0.9

76

0.5

38

0.3

31

0.0

00

0.8

75

0.0

00

0.9

76

0.0

00

10

.00

00

.00

00

.00

00

.00

0(0

.06

3 )

(0.0

33

)(0

.09

2 )

(0.0

44

)(0

.00

7 )

(0.0

60

)(0

.00

5 )

(0.0

28

)(0

.01

2 )

(NA

)(N

A )

(0.0

07

)(0

.00

7 )

(NA

)

0.0

00

2.0

76

0.0

00

3.7

99

1.4

58

3.0

00

2.1

03

3.7

46

1.0

06

(NA

)(0

.49

8 )

(NA

)(1

.49

1 )

(NA

)(N

A )

(0.5

60

)(1

.44

5 )

(NA

)

LL

-13

7.7

04

-14

4.5

65

-98

.49

5-1

07

.23

0-1

45

.08

5-1

37

.70

4-1

37

.70

4-1

44

.56

6-1

44

.56

6-1

45

.08

5-1

45

.08

5-1

37

.70

5-1

44

.56

5-1

45

.08

5

0.8

73

0.9

75

10

.00

0N

A0

.87

30

.08

30

.97

50

.02

70

.00

00

.28

50

.35

9N

A2

.13

6(0

.06

3 )

(0.0

33

)(0

.18

3 )

(0.1

67

)(N

A )

(0.0

61

)(0

.01

8 )

(0.0

28

)(0

.01

4 )

(0.3

98

)(0

.11

4 )

(0.0

27

)(N

A )

(0.2

90

)

0.0

00

11

.24

60

.00

03

.42

50

.39

21

.37

45

.10

2N

A1

.22

2(N

A )

(2.8

34

)(N

A )

(1.0

68

)(0

.18

1 )

(0.3

66

)(0

.74

1 )

(NA

)(0

.28

8 )

LL

-13

7.1

53

-14

4.4

33

-14

5.0

85

-14

5.0

85

NA

-13

7.1

53

-11

2.5

47

-14

4.4

33

-14

2.4

10

-10

9.8

00

-12

1.6

52

-10

1.0

78

NA

-11

4.2

53

10

.94

11

0.0

00

0.0

11

1.0

00

0.0

00

0.9

41

0.0

51

10

.01

20

.01

10

.01

7N

A(0

.18

3 )

(0.0

46

)(0

.18

3 )

(0.1

67

)(0

.01

0 )

(0.1

82

)(0

.00

5 )

(0.0

43

)(0

.01

7 )

(0.0

34

)(0

.00

8 )

(0.0

00

)(0

.01

2 )

(NA

)

1.0

00

11

.80

70

.00

03

.38

31

.45

82

3.7

68

11

.04

62

.60

3N

A(N

A )

(6.1

06

)(N

A )

(1.2

41

)(N

A )

(5.9

78

)(N

A )

(0.7

38

)(N

A )

LL

-14

5.0

85

-14

2.5

10

-14

5.0

85

-14

5.0

85

-14

4.5

94

-14

5.0

85

-14

5.0

85

-14

2.5

10

-13

6.9

08

-14

5.0

85

-14

3.8

45

-14

4.5

95

-14

1.6

06

NA

60

τ

ε o

r γ

ε o

r γ

ε o

r γ

ε o

r γ

57

τ

58

τ

59

τ

19

References

Chen, Daniel L, Martin Schonger, and Chris Wickens (2016). �oTree�An open-source plat-

form for laboratory, online, and �eld experiments�. In: Journal of Behavioral and Exper-

imental Finance 9, pp. 88�97.

Goeree, Jacob, Philippos Louis, and Jingjing Zhang (2018). �Noisy introspection in the 11�20

game�. In: Economic Journal 128.611, pp. 1509�1530.

20