Dynamic Outside Options in Alternating-Offers Negotiations

Cuihong LiTepper School of BusinessCarnegie Mellon University

5000 Forbes AvenuePittsburgh, PA 15213

cuihong@andrew.cmu.edu

Katia SycaraRobotics Institute

Carnegie Mellon University5000 Forbes Avenue

Pittsburgh, PA 15213katia@cs.cmu.edu

Joseph GiampapaRobotics Institute

Carnegie Mellon University5000 Forbes Avenue

Pittsburgh, PA 15213garof@cs.cmu.edu

Abstract

We present a model for bilateral negotiations that con-siders the uncertain and dynamic outside options. Out-side options affect the negotiation strategies via theirimpact on the reservation price. The model is com-posed of three modules, single-threaded negotiations,synchronized multi-threaded negotiations, and dy-namic multi-threaded negotiations. These three mod-els embody increased sophistication and complexity.The single-threaded negotiation model provides negoti-ation strategies without specifically considering outsideoptions. The model of synchronized multi-threaded nego-tiations builds on the single-threaded negotiation modeland considers the presence of concurrently existing out-side options. The model of dynamic multi-threaded nego-tiations expands the synchronized multi-threaded modelby considering the uncertain outside options that maycome dynamically in the future. Experimental anal-ysis is provided to characterize the impact of outsideoptions on the reservation price and thus on the nego-tiation strategy. The results show that the utility of anegotiator improves significantly if she considers out-side options, and the average utility is higher when sheboth considers the concurrent outside options and fore-sees future options.

1. Introduction

Bilateral negotiations are an important mechanismto achieve distributed conflict resolution when it is inthe common interest of the parties to cooperate [18, 17].Automated negotiation strategies have been one ofthe most fundamental decision models for implement-ing self-interested and autonomous interacting softwareagents [9]. A negotiation strategy is a mapping from in-put information about the environment to a sequence

of decisions. By environment we mean all factors thatimpact the negotiation outcome, for example, valua-tion of agreements, the possible valuation held by the“opponent”1, and the deadline for reaching an agree-ment. Previous work on negotiation decisions has beenfocused on the design of negotiation strategies assum-ing inputs (e.g., reservation prices) are given and theenvironment is static. But in a realistic situation it isnot straightforward to acquire the input informationbecause of a complex, uncertain and dynamic environ-ment.

Usually a negotiator can face more than one candi-date to reach an agreement with. A motivating exam-ple is the Navy detailing system, which allocates sailorson rotation to job vacancies (billets). Bilateral negoti-ations are one of the mechanisms proposed to achievedistributed detailing so that personal preferences canbe considered and matching incentives can be provided.One important feature of bilateral negotiations in theNavy detailing system is that a sailor may have or ex-pect multiple jobs that he is qualified for, and similarlya Navy command may see multiple sailors available fora job. These alternatives are called outside options, andthey contribute to the environment of each negotiation.Existence of outside options is typical in matching mar-kets, and also common in commodity and service mar-kets [6, 17]. Accepting a proposal in one negotiationmeans refusing all outside options. On the other handone may leave a negotiation (called “opt-out” of a ne-gotiation) without reaching an agreement based on theexpectation of reaching a more favorable agreement inoutside options. We call the negotiation between a ne-gotiator and one of the opponents a negotiation thread.For convenience of presentation, in the rest of the pa-per we call the two parties in a negotiation a buyer and

1 Here we refer to the other party against whom a party is nego-tiating as the “opponent”.

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

1

a seller, and the model is presented from a buyer’s per-spective.

Outside options can execute concurrently with a ne-gotiation thread, or present sequentially in the future.A concurrently executing outside option is a negotia-tion thread that the negotiator executes in parallel withanother thread. A sequentially available outside optionis an uncertain exchange opportunity that may comein the future with both the other party (and hence theitem/service to contract) and the timing of presenceuncertain.

Outside options impact the input to a negotiationdecision model as a part of the environment of a ne-gotiation (thread). The existence of outside optionschanges the utility that the buyer expects from the cur-rent negotiation, and hence the agreement that is ac-ceptable for the buyer in the current negotiation [21].We claim that outside options affect the negotiationstrategies via their impact on the reservation price. Thereservation price is the worst agreement that a nego-tiator can accept in a negotiation thread. For exam-ple, in the Navy detailing system the reservation priceof a command (buyer) is the highest salary he is will-ing to pay to a sailor (seller). The reservation priceof the buyer depends on the value of the seller’s ser-vice/item to the buyer, and also on the availability ofother providers. The reservation utility of the buyer in anegotiation is the utility of the worst acceptable agree-ment, which is reached at the reservation price. There-fore we have the following equivalence:

reservation price = value − reservation utility.

If there are no outside options, the reservation utilityis zero, the reservation price held by the buyer is equalto the value of the item/service.2

If there are outside options, the reservation utilityin a negotiation is equal to the expected utility fromoutside options that are available either now or later.The utility from outside options is measured on ex-pectation because of the uncertainty of the availabilityand quality of outside options. We should remark thatthe claim is based on the assumption that a threadis non-resumable. In a resumable thread a negotiatorcan leave the negotiation table temporarily for discov-ering more information in other negotiation threads,and come back to resume the negotiation if necessary.In this work we assume that the negotiation threadsare non-resumable. This assumption is not very restric-tive because prior research on bargaining [7, 14] shows

2 Note that the reservation price is generally not the price in thenegotiation outcome. Therefore a zero reservation utility doesnot imply a final zero utility from the negotiation.

that having the option to suspend and resume a nego-tiation does not improve the utility of a negotiator incommon bargaining situations.

The design of an effective negotiation strategy canbe divided into two parts: the first is the design of a ne-gotiation strategy given the reservation price and otherinputs, the second is to calculate the reservation pricebased on the model of outside options. We call themodel in the first part single-threaded negotiations.

The model of outside options can be built with twolevels of complexity based on the two forms of avail-ability of outside options. On the first level we can as-sume that there will be no outside options in the fu-ture. In other words, all negotiation threads are as-sumed to start at the same time. Therefore there is nouncertainty about outside options in terms of both thenumber of threads and item values. We call this modelof negotiations with only concurrently available out-side options, synchronized multi-threaded negotiations.On the second level we also consider the outside optionsthat may present dynamically in the future. Hence thenumber of threads that the negotiator would be in-volved in is a random variable and changes with time.The items (and hence the values) in the future threadsare also uncertain. We call this model with both con-currently and sequentially available outside options dy-namic multi-threaded negotiations. It builds on the syn-chronized multi-threaded model but introduces uncer-tainty about the threads. In both models of synchro-nized and dynamic multi-threaded negotiations the ne-gotiation strategy in one thread can be derived fromthe single-threaded negotiation model, but the reser-vation price is calculated with different models of out-side options. Figure 1 shows the relationship betweenthese three negotiation models.

Negotiation strategieswithout outside options

Single-threadednegotiations

Sychronized multi-threadednegotiations

Dynamic multi-threadednegotiations

Considering the impact of otherconcurent negotiation threads

Considering the outside optionscoming in the future

Figure 1. A nested view of the model

The inputs to each model are summarized in Table I.Each of these models can be applied separately. For ex-ample, if there are no outside options, then the single-threaded negotiation model is only needed. If the nego-tiator does not expect any outside options in the future,

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

2

Models InputsSingle-threaded 1. Reservation price,

2. Prob. dist. of opponents’ reservation pricesSynchronized multi-threaded 3. Number of threads, 4. Opponents’ values and 1,2Dynamic multi-threaded 5. Arriving prob. of outside options,

6. Prob. dist. of opponents’ values and 1,2,3,4

Table I. Additional inputs of each module

then she can apply the synchronized multi-threaded ne-gotiation model. Based on the environment, the appro-priate model can be chosen to map the inputs into aneffective negotiation strategy.

The rest of the paper is organized as follows: Sec-tion 2 reviews related work, Section 3 presents the threemodels, an illustrative example of the negotiation de-cision support is given in Section 4, Section 5 providesexperimental results, and in Section 6 we conclude.

2. Related work

Research on quantitative bilateral negotiation mod-els has been conducted in the fields of game theory,and artificial intelligence (AI). Research in game the-ory focuses on the outcome that satisfies given axioms,or the strategy equilibrium of agents, based on rigor-ous assumptions. Heuristic AI approaches are usuallyused in complex situations for which game theoreticanalysis is intractable. Li, et al. [12] provides an exten-sive research literature review on quantitative bilateralnegotiation models in both fields of game theory andAI.

For a survey on negotiation models in the AI fieldplease refer to Jennings, et al. [9] and Gerding, et al. [8].Klein, et al. [10] present a model for multi-issue con-tract negotiations based on simulated annealing with-out consideration of outside options. In game-theoreticresearch on bilateral negotiations most of the worksthat consider outside options assume that negotiatorshave complete information about each other’s prefer-ences [2, 20, 3, 14]. Gantner [7] presents a bilateral ne-gotiation model with incomplete information and analternating-offers protocol. The outside option is mod-elled as a standard sequential search process. To sim-plify the analysis the paper assumes that a negotia-tor evaluates the item at only two values: high or low.In this paper we consider a general situation in whichthe type space of a negotiator is continuous, and out-side options are also negotiations that may happen si-multaneously and dynamically.

3. The model

There are T periods over the entire window of op-portunity for a buyer, who is searching for a negotiationagreement, to buy an item from a seller. Let a period bedenoted by t, t = 0, . . . , T − 1. A buyer needs to reachan agreement with a seller before period T . The poten-tial sellers may come at different times, unexpectedly,with different reservation prices, and the buyer can ne-gotiate with the sellers simultaneously. The number ofthreads in period t is denoted by nt, and the collec-tion of threads in period t is denoted by Dt = {di}nt

i=1.The seller in the thread di is denoted by si. For simplic-ity we define the value of a seller as the intrinsic valueof the seller’s item/service to the buyer. A seller’s valueis private information of the buyer. Let the value of aseller si be vi. If the buyer agrees with a seller si at aprice x, then the utility of the buyer is vi − x.

The buyer wants to reach the lowest possible agree-ment with a seller by negotiating via an alternating-offers protocol. In an alternating-offers protocol the ne-gotiators propose and respond alternatively, until oneaccepts an offer or quits the negotiation, or the negoti-ation deadline T is reached. The action at each step ofan alternating-offers protocol can be one of: accept, re-ject and propose an offer, quit. Neither negotiation partyknows the reservation price of the other. Assume thebuyer has an estimate of the reservation price of aseller, and the estimation is characterized by a proba-bility distribution F (·), where F (c) denotes the proba-bility that the reservation price of a seller is no greaterthan c. This probability distribution is called the priorbelief of the buyer. A negotiation strategy specifies theaction at each step of the negotiation, based on thenegotiation history of that thread3, and based on thereservation price and prior beliefs of the negotiators.

The reservation utility OUi in thread di is equal tothe expected utility from the outside options, whichcan be viewed together as a multi-threaded negotia-tion. Given the reservation utility OUi, the reservation

3 The history of a negotiation at time t is a sequence of the nego-tiators’ actions before t.

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

3

price Ri of the buyer in thread di can be calculated byRi = vi − OUi. If the reservation price in a thread isknown, the buyer can apply the single-threaded nego-tiation model to make the negotiation decisions in thethread.

Calculation of the expected utility from the outsideoptions depends on the model of the outside options,and on the approach to estimate the expected utilityfrom a multi-threaded negotiation. In a synchronizedmulti-threaded negotiation model the outside optionsat period t for thread di are the other concurrently ex-isting negotiation threads Dt \ di. The synchronizedmodel maps the current outside options to the reserva-tion utility OUi(Dt\di) of each thread di, i = 1, . . . , ni.The dynamic multi-threaded negotiation model furtherconsiders the outside options that may come in thefuture at uncertain times with uncertain values, andcan be viewed as a synchronized model with uncertainthreads.

In the following sections 3.1, 3.2 and 3.3 these mod-els are presented individually.

3.1. Single-threaded negotiations

The calculation of the optimal negotiation strategyrequires game theoretic analysis of the strategy equi-librium. This analysis is not tractable when both par-ties have incomplete information and the negotiation isbased on an alternating-offers protocol [1]. In AI someeffective heuristic negotiation strategies have been de-veloped to provide formal decision models for auto-mated negotiation agents. Among the generic single-issue quantitative models there are [4], [23], and [18],etc.. Since the focus of this paper is not the designof a single-threaded negotiation strategy, we adopt thetime-dependent negotiation strategy that is developedin Faratin, et al. [4], for its simplicity, to illustrate theintegrative negotiation model with outside options.

In the time-dependent approach, the proposal xtb to

be offered by a buyer and the value xts to be offered by

a seller at time t, t ∈ [0, T − 1], are as follows:

xtb = minb + αb(t)(maxb − minb), (1)

xts = mins + (1 − αs(t))(maxs − mins), (2)

where maxb (mins) is the reservation price of the buyer(seller), minb (maxs) is the lower (upper) bound of avalid offer of the buyer (seller) (reasonably we can as-sume minb = 0), and αi(t) decides the pace of conces-sion at time t. The buyer accepts an offer xt

s at time tif it is not worse than the offer she would submit nexttime, i.e., xt+1

b ≥ xts. Similarly the seller s accepts an

offer xtb at time t if xt+1

s ≤ xtb.

The time-dependent function αi(t), i ∈ {b, s}, canbe defined by a family of polynomial functions4:

αi(t) = (t

T)

1β .

The constant β > 0 determines the concession pacealong with time, or the convexity degree of the curve ofproposals (see Figure 2). By varying β a wide range ofnegotiation strategies can be characterized. Two setsof β can be identified to characterize two classes ofstrategies: Boulware with β < 1 and Conceder withβ > 1. With a Boulware strategy [17] a negotiatortends to maintain the offered value until the time isalmost exhausted, then she concedes to the reserva-tion price quickly. With a Conceder strategy [16] anegotiator goes to the reservation price rapidly andearly. No matter what value β takes, with a constantreservation price, the offer monotonically increases (de-creases) with time for a buyer (seller) based on thetime-dependent negotiation strategy.

0 5 10 15 200

0.2

0.4

0.6

0.8

1

Time

Pro

posa

l

beta=1

beta=10

beta=50

beta=0.1

beta=0.02

Figure 2. Offer curves with different β

3.2. Synchronized multi-threaded negotia-tions

In a synchronized multi-threaded negotiation pro-cess a negotiator participates in multiple bilateral ne-gotiation threads with different, simultaneous negoti-

4 Alternatively we can also use the exponential function fam-

ily, and define αi(t) = e(1− tT

)β. These two families are sim-

ilar in their functionality except that their sensitivity to thechange of time is different with different β. For the same bigvalue of β, the polynomial function concedes faster at the be-ginning than the exponential one; then they behave similarly.For a small value of β, the exponential function waits longerthan the polynomial one before it starts conceding [4].

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

4

ation opponents. The negotiator can reach an agree-ment in at most one of these threads, and is aware of allthe threads at the beginning of the process. From onethread’s perspective the other threads are outside op-tions. The reservation utility that the negotiator shouldset in one thread is equal to the expected utility fromall other threads. The other threads form a synchro-nized multi-threaded negotiation with one less threadthan the original process.

A synchronized multi-threaded negotiation processis different from an auction in which the negotiator isthe auctioneer and the negotiation opponents are thebidders, in that (1) in the former mechanism the com-munications between opponents and the negotiator arenot synchronized as in an auction; (2) the messages be-tween the negotiator and opponents are not observedby other opponents, but in an open auction all bidderssee the bids of other bidders and are informed of theprovisional winner; and (3) in the negotiation mech-anism (with an alternating-offers protocol) both par-ties can propose and respond, but in an auction onlybidders propose and the auctioneer only responds. De-spite the differences, both mechanisms drive competi-tions. The competition among bidders in an open auc-tion is realized via transparent and synchronized com-munications and the winner selection mechanism. In asynchronized multi-threaded negotiation the negotiatorachieves indirect competition among opponents sincethe negotiator sets a reservation price that is more ag-gressive than she would without outside options.

It is reasonable to assume that if any agreement isreached in a multi-threaded negotiation, the agreementis signed with the most competitive opponents amongall opponents of the threads. For a buyer, the sellersi in thread di is more competitive than the sellers inother threads if si can give more utility to the buyer,i.e., yi = vi − ri is greater than yj , dj ∈ D \ di, whereri is the reservation price of the seller in thread di,and D = {d1, . . . , dN} is the collection of threads. Theamount yi is the maximum utility that the buyer canachieve from the negotiation thread di. Let Gi(y) de-note the probability that maximum utility in threaddi is less than y. Let G1(y) and G2(y) be the prob-ability distribution of the highest and second highestmaximum utility. The probability density functions ofGi(y), G1(y) and G2(y) are denoted by gi(y), g1(y)and g2(y) respectively. These probabilities can be cal-culated by the following formulas:

Gi(y) = Pr(vi − ri ≤ y) = 1 − F (vi − y),

G1(y) =∏

di∈D

Gi(y),

G2(y) = G1(y) +N∑

i=1

(1 − Gi(y))∏

dj∈D\di

Gj(y).

The corresponding probability density functions, or thederivatives of these (cumulative) probability distributionfunctions, are as follows:

gd(y) = −f(vd − y), g1(y) =∑

di∈D

gi(y)∏

dj∈D\di

Gj(y),

g2(y) = g1(y) −N∑

i=1

gi(y)∏

dj∈D\di

Gj(y) +N∑

i=1

(1 − Gi(y))

[∑

dj∈D\di

gj(y)∏

dm∈D\{di,dj}Gm(y)].

We provide four heuristic approaches to estimate the ex-pectedutilityOU(D) fromamulti-threadednegotiationcom-posed of the threads D:

• Conservative estimation: A synchronized multi-threaded negotiation is approximated by an auction.The utility of the buyer is equal to the expected sec-ond highest maximum utility, as is the result in a reverseauction in which the buyer is the auctioneer and the op-ponents are the bidders [11]. The expected utility iscalculated by

OU =

∫ y

0

yg2(y)dy

where y is the upper bound of the possible utility thatthe negotiator can achieve. If the lower bound of an ac-ceptable price for a seller is c, and the upper bound of abuyer’s valuation is v, then y = v − c.

• Medium estimation: A synchronized multi-threaded ne-gotiation is approximated by an English auctioncontinued with a single-threaded negotiation be-tween the buyer and the winning seller. Assumethe single-threaded negotiation ends at the mid-dle point between the buyer’s and the winning seller’sreservation price, if the buyer’s reservation priceis higher than the winning seller’s5. In the contin-ued single-threaded negotiation the reservation priceof the buyer is equal to the last offer of the sec-ond most competitive seller in the auction, which bringsa utility equal to the second highest maximum util-ity6. Then the expected utility is the average of theexpected highest and second highest maximum util-ity.

OU = (

∫ y

0

yg2(y)dy +

∫ y

0

yg1(y)dy)/2

5 If the buyer’s reservation price is lower than the seller’s, thereis no “zone of agreement” and the negotiation will fail.

6 In an English auction the optimal strategy of a bidder (seller)is to bid down to the true cost.

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

5

In this estimation we assume an agreement can bereached as long as there is a zone of agreement, i.e., thebuyer’s reservation price is higher than a seller’s. In thenext approachof uniformapproximationwe further con-sider the probability that a negotiation may fail even ifthere is a zone of agreement, when negotiators do notknow each other’s reservation prices.

• Uniform approximation: Previous research has estab-lished an optimal bargaining result between a buyer anda seller based on game theoretic analysis when both par-ties’ reservation prices follow uniform distributions [15].Based on this result, an agreement occurs if and onlyif the buyer’s valuation exceeds the seller’s cost by atleast 1/4, if both parties’ reservation prices distributeuniformly on [0, 1]. We can approximate the probabil-ity distributions of negotiators’ types by uniform distri-butions and apply this result to calculate the probabil-ity of reaching an agreement. In the heuristic we assumean agreement cannot be reached in the continued single-threaded negotiation between the buyer and the win-ning seller if the maximum utility of the winning selleris less than a quarter of the highest possible utility y.In this case the buyer achieves the second highest maxi-mum utility, which is the reservation utility of the buyerin thecontinuedsingle-threadednegotiation. If anagree-ment is reached in the single-threaded negotiation, it isreasonable to assume that it is at the middle point be-tween both parties’ reservation prices. Therefore in thiscase the buyer achieves the medium of the highest andthe second highest maximum utility.

OU =

∫ y

0yg2(y)dy +

∫ y

0yg1(y)dy

2

∫ y

y/4

g1(y)dy +

∫ y

0

yg2(y)dy(1 −∫ y

y/4

g1(y)dy).

• Learning: Learn the probability of reaching an agree-ment and the distribution of agreements based on theprevious negotiations [22]. The result of learning is rep-resented by x(v, c), the expected outcome of the negoti-ation when the buyer’s and seller’s reservation prices arev and c respectively. If the seller si in the thread di is thewinning seller, then the probability distribution of herreservation price is F (c)

∏dj∈D\di

(1 − F (vj − vi + c)),

where theproduct is theprobability thatnoother threaddj has themaximumutility vj−cj greater than themax-imumutility vi−ci in thread di.Given the expected out-come x(v, c), the expected utility from a multi-threadednegotiation can be approximated by

OU =∑

di∈D

∫ c

c

[(vd − x(vi, c))

∏dj∈D\di

(1 − F (vj − vi + c))]dF (c)

If negotiators use the time-dependent strategy andtheparameterβ is chosen randomlywith themean equal

to 1, then we expect negotiators to concede constantlyon average. Then the result of learning is expected to beclose to the result of negotiation when β = 1 for both ne-gotiators:

x(v, c) =

{v

1+v−cif v ≥ c

∅ otherwise,(3)

assuming the upper bound of an offer is 1 and thelower bound is 07.

3.3. Dynamic multi-threaded negotiations

During an agreement searching process negotiation oppo-nents canbediscovered sequentially andnewnegotiations arelaunched dynamically. For an ongoing negotiation thread theoutside options not only include the other simultaneous ne-gotiation threads, but also the threads that may be launchedin the future. Considering the outside options in the future, anegotiator must decide how much to offer in the current nego-tiation, and when to stop searching for future opportunitiesand accept an offer from the current negotiation. If a negotia-tor knows the number of outside options that will come, andthe value of the opponent in each outside option, then the ne-gotiator can apply the synchronized multi-threaded negotia-tion model to calculate the appropriate reservation price ineach thread. But usually a negotiator is not sure about the ar-rival of, and the opponents’ values in, future outside options.The reservation utility of a thread is the expected utility ofa multi-threaded negotiation, including other simultaneousthreads and threads launched in the future, with a stochas-tic thread number and uncertain opponents.

Following the usual way of modelling uncertain arrival, weassume the arrival of outside options follows a Poisson pro-cess [13]. In each period there is probability p that the nego-tiator finds an alternative and launches a negotiation thread.The granularity of each period is small enough so that theprobability that there are more than one arrival in one periodis zero. In a Poisson process the number of arrivals η(τ, p)during an interval with length τ follows a Poisson distribu-tion, Pp,τ (n) = Pr(η(τ, p) = n) = e−pτ (pτ)n

n![19]. Denote by

Φ(y) = Pr(v ≤ y) the probability that an opponent’s valueis no greater than y. A negotiator knows the value of an oppo-nent when the opponent is identified, but not the reservationprice of the opponent.

The state st of the system is defined as the number of pastor existing threads nt

8, and the value of each opponent vd inthe thread, st = {nt, {vd}nt

d=1}. Let Ut(st) be the utility that

7 With β = 1, the proposal by the buyer at time t is xtb = vt/T ,

and by the seller is xts = 1−t(1−c)/T . xt

s = xtb = v/(1+v−c)

when t = T/(1 + v − c).

8 We count the past threads in the state because they affect theprobability distribution of the maximum utilities of the ex-isting threads. The threads that have survived generally havehighermaximumutilities than the threads thathaveendedear-lier.

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

6

the negotiator expects from the dynamic multi-threaded ne-gotiation when she sees the system state st at period t. Fol-lowing Section 3.2 we can calculate U({n, {vd}n

d=1}), the ex-pected utility from a synchronized multi-threaded negotia-tion with n threads and the opponent in thread d valued vd,d = 1, . . . , n. Ut(st) is the expectation of U(sT−1) with re-spect to sT−1, which depends on st:

Ut(st) = Eη[E{vd}nt+ηd=nt+1

[U({nt+η, {vd}ntd=1∪{vd}nt+η

d=nt+1}]](4)

UT−1(sT−1) = U(sT−1).

where η follows a Poisson distribution Pp,T−t(·), and vd

independently follows the identical distribution Φ(·), d =nt + 1, . . . , nt + η.

To set the reservation price of a thread, the negotiatoronly needs to calculate the expected utility of the multi-threaded negotiation which does not include that thread,given the period and the state. Because the state of a dy-namic multi-threaded negotiation changes from period to pe-riod, the reservation price of a thread may also change withtime.

The computation of the expected utility follows Equa-tion 4 because the number of states is exponential with re-spect to the number of opponent’s values. If there are at mostN threads and for each opponent there are M possible val-ues, then the number of possible states will be NM . To sim-plify the computation we can approximate the result by hav-ing the opponent value instances replaced by the expectedvalue v, i.e.,

Ut(st) = Eη[U({nt + t, {vd}ntd=1 ∪ {v}nt+η

d=nt+1}]]. (5)

The compromise due to this simplification is not signifi-cant if the expected utility of a synchronized thread is or canbe approximated by a linear function of the opponents’ val-ues.

4. An example of negotiation support

The existence of uncertain dynamic outside options re-quires a negotiator to make appropriate adjustment of offersbased on the current market situation (e.g., the available out-side options) as well as speculations on the possible arrivals inthe future. The complexity of decisions requires the supportfrom a computerized autonomous system. The models pro-posed in this paper can be embeded in an intelligent agent,which can be a part of a decision support system (DSS) for ne-gotiation decisions with uncertain dynamic outside options.Thisagentobtains theuser’s request,preferences,negotiationpersonalities, negotiation strategy (e.g., thedecisionmodel tobe used), etc., from interactions with the user. It also acquiresknowledge about the “market” (e.g, the probability distribu-tion of the item values, the outside option arrival probabil-ity, the historical agreements that have been reached) fromother components of the DSS. Depending on the level of au-tonomy in making deals bestowed by the user, the agent de-cides or suggests to the user at each step whether to accept an

offer, and the counter offer to be proposed if the received of-fer is to be rejected, in each negotiation thread. In the follow-ing we give an illustrative example of the decision support fora buyer.

Let the negotiation deadline be T = 20. The preliminaryanalysis of the market shows that the reservation price of aseller follows a uniform distribution in the interval [0, 1], andthe value of a seller’s item is also uniformly distributed in[0, 1]. The probability that a new seller arrives in a period is0.2. The negotiation parameter of the buyer is β = 1.262727,which is selected by the buyer, and reflects that the buyerprefers tomakea little bit faster concessions earlier than later.Suppose the outside options arrive with the time and valueshown in Figure 3.

0 5 10 15 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9Time and values of arrivals

Time

Val

ue

Figure 3. Time and values of arrivals

The value of an outside option is an input from the user,who could be supported by a preference analysis sub-systemon the evaluation that matches the opponent’s informationwith thebuyer’s desires.Assume thedynamicmulti-threadednegotiation model where estimation by learning is used. Ta-ble 4 records a sample of the decisions for the first negotia-tion thread, in which the opponent proposes first. The col-umn “Received” is the offers that the buyer receives fromthe opponent in each step. The column ”Res. Price” liststhe reservation prices at each step. The reservation price isconstant in a single-threaded negotiation model, but changeswith the arrival of outside options in the synchronized model,and changes with time and also with the arrival of outside op-tions in the dynamic model. Based on the reservation price,the agent calculates the offer that it suggests to be proposedif the opponent’s offer is rejected. These proposed offers arelisted in the column “Propose”. The DSS suggests the buyerto reject an offer if it is worse than the offer it would pro-pose, otherwise to accept.

To effectively provide decision reference to the buyer, theagent can show the user (for each thread) the negotiation

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

7

Time Received Res. Price Propose Decision1 0.8485 0.7252 0.0676 Reject2 0.7774 0.7283 0.1176 Reject... ... ... ... ...10 0.4557 0.7446 0.4301 Reject11 0.4261 0.6675 0.4157 Reject12 0.3977 0.6423 0.4286 Accept

progress via a graph that expresses the offer curve that hasbeen realized (with the last point being the offer to be sug-gested), the counter offer curve of the opponent, the lines ofthe reservation price and the valuation. This graph changeswith time and the arrival of outside options. Figure 4 pro-vides a snap shot of the graph at Time 12 for the first thread,which arrives at Time 1.

0 2 4 6 8 10 12 14 16 18 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Time

Number of options: 5

Value=0.8214

Reservation price=0.6423

Figure 4. Visualize negotiation progress

In this figure the solid line with markers is the buyer’soffer curve, and the dashed line with markers is the oppo-nent’s offer curve.They correspond to the columns“Propose”and “Received” in Table 4 respectively. The horizontal dot-ted line shows the level of the reservation price at that step (itis also shown as the reservation price at Time 12 in Table 4),while the horizontal solid line shows how much the buyer re-ally values the item of the seller. Although the reservationprice changes with time and the market, the value is a con-stant and is givenat thebeginning of the thread (it is the valueof the first arrival shown in Figure 3), unless the user changesthe value during the process when new information is discov-eredabout the seller.Fromthegraph theuser obtains ideas onthe time and space left for concessions, the value of the threadwith outside options considered, and how close the two par-ties’ offers are.

5. Experiments

In Section 5.1 we illustrate the impact of outside optionson the negotiation strategy by showing the offer curves ad-justed by the reservation prices, compared with the originalbasic offer curve without considering outside options. In Sec-tion 5.2 we compare the average utility of a negotiator whenshe (1) does not consider outside options, (2) when she onlyconsiders concurrent outside options, i.e., the synchronizedmulti-threaded negotiation model, and (3) when she consid-ers both concurrent outside options and future arrivals, i.e.,the dynamic multi-threaded negotiation model.

In the experiments, the negotiation deadline T = 20. Thebuyer believes the reservation price of a seller follows a uni-form distribution on the interval [0, 1]. The value of a seller’sitem is also uniformly distributed on [0, 1]. The probabilitythat a new seller arrives in a period is p, and p takes the val-ues{0.05, 0.10, 0.15, 0.20, 0.25}.Theparameterβ in the time-dependent strategy of a negotiator is chosen randomly so thatwith even probability a negotiator in a thread is a Conceder(β > 1) or a Boulware (β < 1). If a negotiator is a Conceder,β−1 follows a uniform distribution on [0, 1]. If a negotiator isa Boulware, β is a random variable with a uniform distribu-tion on [0, 1]. For each arrival probability, we repeat the ex-periment 100 times and the average utility of the buyer is cal-culated.The expectedutility of adynamicmulti-threadedne-gotiation process was calculated with the approximation for-mula, Equation 5 (Section 3.3).

5.1. Reservation utilities and offer curves

We continue with the example in Section 4 to illustratethe impact of outside options on the negotiation strategy.9.The offer curves in the first thread, that are calculated basedon different estimation approaches and outside option mod-els, are shown in Figure 5. The model noted by “Single” is themodel without considering outside options. When the buyerdoes not consider outside options the offer increaseswith timeas the buyer constantly concedes (with changing pace). Butwith a synchronized or dynamic model the buyer may pro-ceed, i.e., decrease the offered price from the previous one,when a valuable new opponent arrives (e.g., at Time 11 asshown in Figure 3). This is because the reservation utility ofthe buyer increases when she sees a new seller that offers ahigh-value item. When there are no new arrivals, the buyerwill concede in all these three models. The concession pace inthe synchronized model is the same as in the single-threadedmodel, but it is greater in the dynamic model. This is becausein the dynamic model the buyer expects fewer new arrivalsand the reservation utility decreases with passing time. Theofferswithout considering outside options are higher than theoffers with considering only concurrent negotiation threads,which are again higher than the offers with additional consid-erations of outside options that may come in the future.

9 The multiple experiments with different arrival probabilitiesand β show the same pattern.

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

8

0 5 10 15 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9Offer curve with conservative estimation

SingleSynchronizedDynamic

0 5 10 15 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9Offer curve with medium estimation

SingleSynchronizedDynamic

0 5 10 15 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9Offer curve with uniform approximation

SingleSynchronizedDynamic

0 5 10 15 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9Offer curve with uniform approximation

SingleSynchronizedDynamic

0 5 10 15 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9Offer curve with estimation by learning

SingleSynchronizedDynamic

Figure 5. The offer curves

5.2. Performance results

In this section we examine and compare the average util-ities that a buyer obtains with different models. Figure 6 iscomposed of four subplots. Each subplot shows the averageutility as a function of the arrival probability based on onereservation utility estimation approach, and with differentoutside option models. The figure implies that for all esti-mation approaches and outside option models, the averageutility increases with the arrival probability. This is intuitiveand should be true for a reasonable negotiation strategy. Ahigher arrival probability implies more options on expecta-tion and should result in a better outcome for the negotia-tor. Figure 6 also shows that the average utility based on thedynamic model is higher than the one based on the synchro-nized model, which again brings higher average utility thanthe single-threaded model in which no outside option is con-sidered. This verifies the effectiveness of the solution frame-work and the heuristic approaches we have proposed.

6. Conclusion

In this paper we provide an integrative solution for the bi-lateral negotiation decision problem when negotiators faceuncertain and dynamic outside options. Outside options in-fluence negotiation strategies by their impact on the reser-vation prices. We present a solution framework that inte-grates the outside option models and negotiation strategies,andprovides reasonable heuristic approaches to set appropri-ate reservation prices considering outside options. We do notclaim that the heuristics we provide in this report are com-

plete.Rather they reflect solutions thathavebeenprovenuse-ful or plausible. Other negotiation strategies and approachesto estimate the utility from a multi-threaded negotiation canbe plugged in the solution framework, depending on the as-sumptions and requirements of the underlying application.These different models can construct a library of decisionfunctions to support the decision of negotiation agents in dif-ferent environments.

Acknowledgement

This work was supported in part by the Office of Naval Re-search and the Navy Personnel Research, Studies and Tech-nology (NPRST) under Navy contract number N6610-98-D-9501, and by a subcontract with Resource Consultants, Inc(RCI), subcontract number 03-177-7048/086.

References

[1] L. Ausubel, P. Cramton, and R. Deneckere. Bargain-ing with incomplete information, volume 3, chapter 50.Elsevier Science, 2002. Robert J. Aumann and SergiuHart, eds.

[2] K. Binmore, A. Shaked, and J. Sutton. An outsideoption experiment. Quarterly Journal of Economics,104(4):753–770, 1989.

[3] K.Chatterjee andC.C.Lee. Bargaining and searchwithincomplete information about outside options. Gamesand Economic Behavior, 22:203–237, 1998.

[4] P. Faratin, C. Sierra, and N. R. Jennings. Negotia-tion decision functions for autonomous agents. Inter-

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

9

0.05 0.1 0.15 0.2 0.250

0.05

0.1

0.15

0.2

0.25

0.3

0.35Average utility with conservative estimationDynamicSynchronizedSingle

0.05 0.1 0.15 0.2 0.250

0.05

0.1

0.15

0.2

0.25

0.3

0.35Average utility with medium estimation

DynamicSynchronizedSingle

0.05 0.1 0.15 0.2 0.250

0.05

0.1

0.15

0.2

0.25

0.3

0.35Average utility with uniform approximation

DynamicSynchronizedSingle

0.05 0.1 0.15 0.2 0.250

0.05

0.1

0.15

0.2

0.25

0.3

0.35Average utility with estimation by learning

DynamicSynchronizedSingle

Figure 6. The average utilities with varying arrival probability

national Journal of Robotics and Autonomous Systems,24(3-4):159–182, 1998.

[5] S. S. Fatima, M. Wooldridge, and N. R. Jennings. Multi-issue negotiation under time constraints. In Proceed-ings of the First International Joint Conference on Au-tonomous Agents and Multi-Agent Systems (AAMAS2002), pages 143–150, Bologna, Italy, 2002.

[6] D.Gale. Strategic foundations of general equilibrium: dy-namic matching and bargaining games. Cambridge Uni-versity Press, 2000.

[7] A. Gantner. Bargaining, search and outside options.Technical report, Department of Economics, Universityof California, Santa Barbara, 2002.

[8] E. H. Gerding, D. van Bragt, and J. L. Poutre. Scien-tific approaches and techniques for negotiation: a gametheoretic and artificial intelligence perspective. Techni-cal Report SEN-R0005, CWI, 2000.

[9] N. R. Jennings, P. Faratin, A. R. Lomuscio, S. Parsons,C. Sierra, and M. Wooldridge. Automated negotiation:prospects, methods and challenges. Int. J. of Group De-cision and Negotiation, 10(2):199–215, 2001.

[10] M. Klein, P. Faratin, H. Sayama, and Y. Bar-Yam. Ne-gotiating complex contracts. Group Decision and Nego-tiation, 12(2), 2003.

[11] V. Krishna. Auction Theory. Academic Press, 2002.

[12] C. Li, J. Giampapa, and K. Sycara. A review of researchliterature on bilateral negotiations. Technical ReportCMU-RI-TR-03-41, Carnegie Mellon University, 2003.

[13] S. Lippman and J. McCall. The economics of uncer-tainty: Selected topics and probabilistic methods, vol-ume I, chapter 6, In K.J. Arrow and M. Intriligator, edi-

tors. North-Holland Publishing Company, Amsterdam,1981.

[14] A.Muthoo. On the strategic role of outside options in bi-lateral bargaining. Operations Research, 43(2):292–297,1995.

[15] R. B. Myerson and M. A. Satterthwaite. Efficient mech-anisms for bilateral trading. Journal of Economic The-ory, 29:265–281, 1983.

[16] D. G. Pruitt. Negotiation Behavior. Academic Press,1981.

[17] H. Raiffa. The Art and Science of Negotiation. HarvardUniversity Press, 1982.

[18] J.Rosenschein andG.Zlotkin. Rules of Encounter. MITPress, 1994.

[19] S. M. Ross. Introduction to Probability Models. Aca-demic Press, 2000.

[20] A. Rubinstein. Perfect equilibrium in a bargainingmodel. Econometrica, 50(1):97–109, January 1982.

[21] K. Sycara. Utility theory in conflict resolution. Annalsof Operations Research, 12:65–84, 1988.

[22] K. Sycara. Machine learning for intelligent support ofconflict resolution. Decision Support Systems, 10:121–136, 1993.

[23] D. D. Zeng and K. Sycara. Bayesian learning in negotia-tion. International Journal of Human-Computer Stud-ies, 48:125–141, 1998.

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

10