A Survey of Research in Distributed, Continual Planning

Marie E. desJardins,� Edmund H. Durfee,y Charles L. Ortiz, Jr.,� and

Michael J. Wolverton�

Pre-publication draft. To be published, AI Magazine, Winter 2000.

Abstract

Complex, real-world domains require a rethinking oftraditional approaches to AI planning. Planning andexecuting the resulting plans in a dynamic environmentrequires a continual approach in which planning andexecution are interleaved, there may be uncertainty inthe current and projected world state, and replanningmay be required when the situation changes or plannedactions fail. Furthermore, complex planning and exe-cution problems may require multiple computationalagents and human planners to collaborate on a solu-tion.

In this article, we describe a new paradigm for plan-ning in complex, dynamic environments, which we termdistributed, continual planning (DCP). We argue thatdeveloping DCP systems will be necessary in order forplanning applications to be successful in these environ-ments. We give a historical overview of research leadingup to the current state of the art in DCP, and describeresearch in distributed and continual planning.

The increasing emphasis on real-world applicationshas led AI planning researchers to develop algorithmsand systems that more closely match realistic plan-ning environments, in which planning activity is oftendistributed, and plan generation may happen concur-rently with plan execution. These new research di-rections have increased the need for distributed sys-tems of cooperating agents for continuous planning anddecision support. Multiagent planning architectures,distributed planning, mixed-initiative planning, dis-tributed scheduling, and work ow management meth-ods are currently active areas of research.We argue that a new paradigm for planning is needed

in order to support this style of planning, and that sucha paradigm is beginning to emerge within the plan-ning research community. We characterize this new ap-

�SRI International, 333 Ravenswood Ave., MenloPark CA 94025; marie@erg.sri.com, ortiz@ai.sri.com,mjw@erg.sri.com

yEECS Department, University of Michigan, Ann ArborMI 48109; durfee@umich.edu

proach to planning as distributed, continual planning(DCP). \Distributed planning" refers to an environ-ment in which planning activity is distributed acrossmultiple agents, processes, or sites. \Continual plan-ning" refers to an ongoing, dynamic process in whichplanning and execution are interleaved.1

An agent should engage in distributed planning whenplanning knowledge or responsibility is distributedamong agents, or when the execution capabilities thatmust be employed to successfully achieve objectives areinherently distributed. An agent should plan continu-ally when aspects of the world can change dynamicallybeyond the control of the agent, when aspects of theworld are revealed incrementally, when time pressuresrequire execution to begin before a complete plan canbe generated, or when objectives can evolve over time.Thus, distributed continual planning is important foran agent whose success and e�ciency depends on howits current actions a�ect its future choices, but whichmust operate in a complex, dynamic, and multi-agentenvironment.We �rst present a historical overview of planning re-

search that emphasizes how the introduction of more re-alism into the problems faced by planning systems hasled to current approaches that �t the DCP paradigmmore closely. We then survey related research in dis-tributed and continual planning. We conclude by out-lining some important future directions that are neededfor true DCP to become a reality.

DCP: A Historical Perspective

In this section, we present a framework for categorizingplanning-and-execution systems that centers aroundthe ways in which each system handles the context inwhich its planning and execution are carried out. Weplace a range of systems within this framework, anddiscuss where continual planning, distributed planning,and distributed, continual planning arise in this frame-work.

1The term \continuous planning" is also used to de-scribe this style of planning, but since the latter term is alsosometimes used to refer to planning for continuous problemspaces (e.g., using continuous models of time and space), weuse the less ambiguous term \continual planning."

The context-centered framework gives us a structurethat we use to provide a historical perspective on re-search in this area, which we divide into overlappingstages that are characterized by increasingly sophisti-cated treatment of planning and execution context. Theearliest stage consists of systems that ignore context inorder to simplify the planning problem. In the nextstage, we begin to see systems that tolerate context byresponding to or anticipating events that occur outsideof the planning-and-execution system. The third stageincludes systems that exploit context, where a systemtakes advantage of its ability to control the world andinteract with other agents. Finally, the most sophisti-cated systems take the initiative to establish context byactively interacting with and con�guring the environ-ment.The characterizations we provide are a simpli�cation

of the issues and accomplishments being addressed ineach of the stages. In this section, we refer to sur-vey treatments and illustrative examples of work, ratherthan providing a comprehensive bibliography of the richplanning literature. Our purpose is to illustrate thechanging perspectives in the general planning commu-nity that have led up to the current interest in DCP.

Ignore Planning and Execution Context

Planning is di�cult. Even the simplest planning prob-lem, that of determining how an agent can move fromthe current world state to a world state that satis�esits preferences, is intractable in the general case. Forthat reason, traditional AI planning research (Hendleret al. 1990, Russell and Norvig 1995) has introduced as-sumptions and simpli�cations in order to make planningfeasible. An agent is typically assumed to know every-thing that is relevant to the planning problem, and toknow exactly how its available actions can change theworld from one state into another. The planning agentis assumed to be in control of the world, so that theonly changes to the state are due to the agent's delib-erate actions. The agent's preferred states of the worldare also constant throughout a planning episode (plan-ning problem or set of objectives)|it will not \changeits mind" about what goals to achieve in the midst ofplanning or while executing the plan.These simpli�cations allow the planning problem to

be serialized: a planning agent �rst formulates a planand then executes it. It is assumed that the plan-ning and execution for one episode have no bearing onthe planning and execution done for previous or futureepisodes. The context in which planning and executionare done is e�ectively ignored during the planning andexecution processes.

Tolerate Planning and Execution Context

In the real world, plans do not always proceed as ex-pected. In dynamic, uncertain environments, an agentcannot in general make accurate predictions about theoutcomes of its actions. One approach to handling un-certainty is to enumerate the possible states (contin-

gencies) that might arise at execution time and planfor each of them, constructing a (possibly large) condi-tional plan that provides alternative courses of actionfor each contingency (Schoppers 1987, Boutilier et al.1999).If the knowledge available to the agent is insu�cient

or suggests an intractably large set of contingencies, abetter approach is plan monitoring and repair (Ambros-Ingerson and Steel 1988, Doyle et al. 1986). In this ap-proach, the agent formulates a nominal plan that looksbest given what (little) the agent knows. As the planis executed, progress is monitored to ensure that thepredicted or assumed conditions actually hold. Whendeviations are detected, the agent halts execution andrevisits its planning decisions, creating a revised plan.In e�ect, the episodic nature of \plan-then-execute"is maintained, but episodes can be terminated prema-turely (when a deviation is detected) and results fromone episode can in uence another (such as when theprevious plan is repaired and reused). If no deviationsoccur, this approach is equivalent to traditional plan-ning.In complex environments, however, the planning and

execution context might change in ways that suggest achange in future plans without necessarily violating acurrently executing plan. Unexpected changes in theworld might provide opportunities to accomplish goalsmore e�ciently or e�ectively. Alternatively, an agent'sgoals might themselves change, so that although currentplan could still be carried out, the motivation for doingso is lost (Levesque et al. 1990). When goals and as-pects of the world can evolve continuously rather thanbeing �xed throughout a planning episode, an agentshould continually evaluate and revise its plans.Continual planning recognizes that plan revision

should be an ongoing process, rather than being trig-gered only by failure of current plans. It also adoptsthe perspective of not planning too much detail too farinto the future, since evolving circumstances can ren-der such details obsolete. Continual planning thereforetolerates the planning and execution context by main-taining exibility and opportunism.

Exploit Planning and Execution Context

The approach to continual planning that we have out-lined suggests an environment in which an agent isworking towards accomplishing its objectives, despitethe world dynamically changing in unpredictable waysthat interfere with its plans. Such a pessimistic mindsetneed not be adopted, however, if the agent has knowl-edge, and the ability to use it, about its relationship tothe other agents in the world. In this case, an agentshould not simply tolerate the presence of other agents,but should seek to exploit their presence by cooperatingwith them.One particularly useful source of interaction is hu-

man participants in the planning process. Humansoften have expertise and capabilities that computa-tional agents lack. Rather than formulating weak plans

based on its limited expertise and knowledge, there-fore, an agent should whenever possible work with peo-ple to formulate more e�cient and robust plans. Aplanning process that is distributed between compu-tational and human agents in this way is referred toas mixed-initiative planning (Myers 1996, Veloso 1996,Ferguson et al. 1996).A planning agent could also work with other com-

putational agents. By bringing to bear complementarycapabilities, a set of agents can collectively formulatea concerted plan of action for one or more of them tocarry out (Wilkins and Myers 1998, Kambhampati etal. 1993). Similarly, by considering the capabilities ofother agents, agents can accomplish their objectives byexploiting each other's strengths through cooperativeexecution.Even if the agents do not explicitly cooperate by shar-

ing expertise or capabilities, by exploiting the fact thatchanges to the world are often a result of other agents'planned actions, an agent can reduce its uncertainty,and increase the quality of its local plan, by in uencingthe plans that others adopt.

Establish Planning and Execution Context

The approaches to cooperative and negotiated dis-tributed planning that were alluded to in the previoussection are essentially distributed versions of traditional\plan-then-act" approaches. For a system of agentsto engage in distributed, continual planning, however,each must be a continual planning agent, and more. Aseach agent elaborates its ongoing abstract plans into ac-tions, the elaboration decisions should be compatible,and preferably mutually supporting. The agents' dis-tributed planning mechanisms should continually eval-uate the relationships between re�nement decisions andredirect decisions in cooperative (or at least compati-ble) ways.The idea of allowing agents to exploit the larger

multi-agent context for planning (working together toplan) and for execution (taking actions towards com-mon goals) opens the door to purposely establishingsuch a context in order to improve what agents can ac-complish (Tambe 1997, Shoham and Tennenholtz 1992,Stone and Veloso 1999). Agents that have formulatedabstract plans can analyze potential relationships be-tween their possible plans, commit to particular con-straints on how they will realize these plans, and thenincorporate these in uences in their elaboration deci-sions in a decentralized way (Clement and Durfee 1999).

Distributed PlanningThe problem of constructing plans in a distributed envi-ronment has been approached from two di�erent direc-tions. One approach has begun with a focus on \plan-ning," and how it can be extended into a distributedenvironment, where the process of formulating or exe-cuting a plan could involve actions and interactions ofa number of participants. The other approach has be-gun with an emphasis on the problem of controlling and

coordinating the actions of multiple agents in a sharedenvironment, and has adopted planning representationsand algorithms as a means to that end. While these twoapproaches have led to common ground in distributedplanning, and indeed many researchers embrace bothapproaches (making unequivocal partitioning into onecamp or the other impossible), we believe that the dis-tinctions still help in understanding the state of the�eld.We refer to the �rst approach as \Cooperative Dis-

tributed Planning" (CDP). Because it places the prob-lem of forming a competent (sometimes even optimal)plan as the ultimate objective, CDP is typically car-ried out by agents that have been endowed with sharedobjectives and representations, typically by a single de-signer or team of designers. While in some cases thepurpose of the agents is to form a central plan, moregenerally the purpose is that the distributed parts ofthe developing plan will jointly execute in a coherentand e�ective manner. Thus, in CDP, agents typicallyexchange information about their plans, which they it-eratively re�ne and revise until they �t together well.The second approach, which we refer to as \Negoti-

ated Distributed Planning" (NDP) has a di�erent focus.From the perspective of an individual agent engaged inNDP, the purpose of negotiating over planned activitiesis not to form good collective plans, but rather to en-sure that the agent's local objectives will be met by itsplan, when viewed in a global context. The emphasisis therefore not on cooperatively de�ning and search-ing the space of joint plans to �nd the \best" groupplan as in CDP, but rather on an agent trying to pro-vide enough information to others to convince them toaccommodate its preferences.When individual preferences are aligned in ways

where cooperation is in the self-interests of all con-cerned, CDP and NDP come together; thus, a hard-and-fast partitioning between the two approaches is im-possible, and a number of e�orts embrace both per-spectives. For example, STEAM, which we discuss inthe NDP section, draws heavily on planning research;on the other hand, both DIPART and Partial GlobalPlanning, which are described in CDP, have a strong\agent" avor. Our purpose in making the loose divi-sion between CDP and NDP is not to pigeonhole anyindividual piece of research, but instead to provide somestructure to the research landscape, ranging from ex-treme CDP (where the purpose of distribution is sim-ply to allow parallel computation of plans) to extremeNDP (where the purpose of planning is simply to �ndresource con icts) with a rich space of systems in themiddle, involving both sophisticated joint planning andthe pursuit of self-interest on the parts of agents.

Cooperative Distributed Planning

In this section, we summarize past and current workin cooperative distributed planning (as de�ned previ-ously). Most of the research we describe here involvestwo or more computer processes (\agents") cooperat-

ing to build either a single plan or multiple interact-ing plans. However, we will also mention other relatedresearch|for example, a single planner that has specialcapabilities for multi-agent execution.In building a distributed planning system, some of

the key questions to address include:

� How are plans or partial plans represented?

� What is the basic plan generation method?

� How are tasks allocated to agents?

� How do agents communicate with one another duringplanning?

� How are the agents' actions coordinated?

We discuss alternative approaches to these problemsbelow.

Plan Representation and Generation Much ofthe research in cooperative distributed planning is builtaround notions of abstract plan decomposition, such ashierarchical task network (HTN) planning (Erol et al.1994). An abstraction-based plan representation allowsa distributed planning agent to successively re�ne itsplanning decisions as it learns more about other agents'plans.Corkill's distributed version of Sacerdoti's NOAH

planner (Sacerdoti 1977) was one of the earliest e�ortsin distributing an HTN planning algorithm (Corkill1979). The basic planning procedure and plan repre-sentation used by Corkill is the same as that of NOAH:planning proceeds through a hierarchy of plan levels,where at any plan level a partial plan is a partial or-der of goals and primitive actions. Each distributedagent solves its goals in the same way as NOAH, ateach level expanding each unplanned goal by �ndingan applicable operator (called a SOUP procedure) thatsolves it. NOAH's plan representation is extended toinclude a special node type representing a placeholderfor another agent's plan, as well as special primitiveactions for synchronizing multiagent execution.Just as the SIPE-2 planner (Wilkins 1988) is con-

ceptually descended from NOAH, DSIPE (DistributedSIPE-2) (desJardins and Wolverton, this volume)is conceptually descended from Corkill's distributedNOAH. Both are distributed versions of HTN planners,and the two systems use similar approaches to some ofthe key representational problems to distributed plan-ning, especially in how the multiple agents maintaina consistent (albeit incomplete) picture of each other'splans. In DSIPE, the local view of the other agents'subplans is called a skeletal plan.Similarly, in Durfee and Lesser's Partial Global Plan-

ning (PGP) (Durfee and Lesser 1991), each agent main-tains a partial global plan, which stores its own partialpicture of the plans of all the members of the group.PGP focuses on distributed execution and run-timeplanning and uses a specialized plan representation,where a single agent's plan includes a set of objectives,a long-term strategy (ordered lists of planned-actions)

for achieving those objectives, a set of short-term details(primitive problem-solving operations), and a numberof predictions and ratings. Planning|both the ex-pansion of objectives into planned-actions and the or-dering of those actions|is based on a set of heuris-tics. PGP and its descendants, including GenericPartial Global Planning (Decker and Lesser 1992),have been applied to problems that include distributedacoustic monitoring (Durfee and Lesser 1991), cooper-ative information gathering(Decker et al. 1995), andcooperative robotics (Durfee and Montgomery 1991,Clement and Durfee 1999).

Task Allocation In cooperative planning systems,there is typically an explicit|often centralized|mechanism for assigning subgoals to the agents thatwill be responsible for planning them.

Distributed NOAH and DSIPE have no automatedmethods for allocating tasks among the planningagents; the user speci�es which agent solves which top-level goals. PGP allows agents to send potential groupplans to each other, essentially allowing agents to pro-pose (and counter-propose) an assignment of activi-ties (tasks) to each other (Durfee and Lesser 1989).Since agents can choose to participate or not in suchplans, such assignments involve mutual selection be-tween agents (Smith and Davis 1983).

DIPART (Pollack 1996) is an experimental platformfor analyzing a wide range of planning and executionmechanisms, including approaches to distributed plan-ning. One of the key areas studied in DIPART is loadbalancing (i.e., task allocation) among multiple plan-ning agents. The load balancing methods developedare high-level approaches that are independent of anyparticular planning representation or algorithm. Initialallocation of tasks to agents is speci�ed by the user,but tasks can be automatically reallocated at runtimeaccording to load balancing considerations.

Lansky's COLLAGE (Lansky 1994, Lansky andGetoor 1995) is not a multi-agent planning system perse, but it represents one promising way of approachingthe task assignment problem in distributed planning.COLLAGE uses a technique called localization to de-compose a planning problem into subproblems calledregions. Localizations in COLLAGE can be automat-ically generated based on abstraction levels or scope.A set of heuristics are used to �nd regions that mini-mize the number of interactions between regions, thuspermitting COLLAGE to solve the subproblems in eachregion (relatively) independently. Partitioning subgoalsamong distributed planning agents according to local-ization would likewise improve the overall e�ciency ofthe multi-agent planning system, by minimizing con- icts and interactions between agents (and reducing theneed to check for them).

Communication Cooperative distributed plannersuse a variety of techniques for agents to share infor-mation.

At the conclusion of each planning level, Distributed

NOAH applies an extended version of NOAH's planningcritics to coordinate the planning agents. Each agentsends the other agents a record of the important ef-fects of the newly-expanded portions of its plan (thoughCorkill does not make it clear which e�ects are consid-ered to be important). Each agent inserts a record ofall new e�ects it receives into its local plan.

DSIPE extends Distributed NOAH's approach in anumber of areas, most notably by expanding the typesof planning information that are shared among theplanning agents, and by developing irrelevance-based�ltering methods for reducing communication require-ments (Wolverton and desJardins 1998).

In PGP, communication goes through the channelsof a meta-level organization, which speci�es the rolesand responsibilities of planning agents in maintainingplan coordination (e.g., which agents have authority toreconcile con icts among agents' plans), much like aregular organization speci�es how the agents should beorganized to solve a domain-level problem. The work ofJennings (Jennings 1995) codi�es such responsibilitiesinto communication conventions that all agents followto ensure su�cient awareness about the pursuit of col-lective plans.

Communication among planning agents to supportload balancing in DIPART is generally done by multi-casting to all members of the agent's (user-assigned)subgroup. However, in heavy load situations, agentsmay trade o� complete knowledge of system load inexchange for preservation of bandwidth by selectiveunicasting (called \focused addressing" in (Smith andDavis 1983)). In this case, they must rely on estimatesof other nodes' loads based on their histories.

Coordination For a group of cooperative distributedplanners to reach a state in which the individual sub-plans jointly achieve the common objectives, the agentsmust coordinate their decision making.

In Distributed NOAH, when an agent's critics de-tect a con ict between one of its own actions and thoseof another agent, instead of adding an ordering con-straint as it would if were not distributed, it inserts aspecial execution-time coordination action: either \Sig-nal," which will inform another agent of the completionof an action, or \Wait," which waits for a signal fromanother agent.

PGP achieves coordination and task assignment bya process of negotiation. When agent A informs agentB about some portion of its local plan, B merges thenew information about A into its partial global plan.It then searches for ways to improve the global plan,e.g., by eliminating redundancy or by better utilizingthe group's computational resources. It proposes theseimprovements to some subset of the other agents. Eachother agent may accept the new global plan and incor-porate the changes made to its own local plan, it mayreject the new global plan, or it may respond with acounterproposal. Throughout the process, each agentwill continue to act on its current best assessment of

the plan it should pursue, rather than waiting to con-verge on a plan (which, due to dynamics, might be quiteephemeral anyway). Thus, this approach is designed fordomains where temporary incoherence among agents istolerated as information about unanticipated changesto plans is propagated. It is not designed for domainswith irreversible actions, where con icts among agents'plans should be resolved before any step is taken thatcould cause other agents' plans (and thus the system asa whole) to fail.Coordination between planning agents in DIPART is

accomplished in two ways: (1) Incremental merging ofindividual agent subplans (Ephrati et al. 1995a), and(2)multi-agent �ltering (Ephrati et al. 1995b), in whicheach agent commits to the goals it has already adopted,but bypasses new options that are incompatible withany other agent's known or presumed goals.

Negotiated Distributed Planning

Much of the research into the design of autonomousagents has had as its goal a formal account of mentalstate that would serve to explain the manner in whichthe attitudes of belief, desire, and intention (BDI) en-gender behavior, both at the individual agent level andat the group level. A BDI theory could be viewed asexplanatory to the extent that one felt comfortable inascribing some sense of \rationality" to the behaviorspredicted by the theory that were causally connectedto individual mental states; in this respect, the termrational balance as applied to an agent's mental stateand as coined by Nils Nilsson, is an apt one. For exam-ple, an intention can be viewed as re ecting an agent'scommitment to per form an action; such a commitmentis assumed to remain in force until satis�ed or until theagent no longer believes the action is possible (Brat-man 1987, Cohen and Levesque 1990, Shoham 1990,Wooldridge and Jennings ). By virtue of its persis-tence, an intention will thereby constrain the options anagent will be disposed to consider; a rationally balancedagent would then be one that was not continuously re-evaluating each possible option in light of new inten-tions or goals. Within such models, plans are derivativestructures built up from elements of the agent's mentalstate (Pollack 1990).The BDI decomposition strategy can also serve as

an implementation-neutral way to architect an agentand to also analyze both individual and collective be-havior. In contrast to the above research, which haslargely been formal in nature, the design of rational-agent architectures (Haddadi and Sundermeyer 1996,Bratman et al. 1988) has been guided primarily bycomputational concerns centered on an agent's inherentresource limitations (Simon 1983, Russell and Subra-manian 1995). Agent architectures augment an agent'smental world with a set of processes, each associatedwith a particular architectural module; processes caneither supply inputs for other modules or modify theagent's repository of beliefs, desires, and intentions. Asan example, an architecture might include a delibera-

tion module whose role is to weigh alternatives in lightof existing commitments (intentions) and new observa-tions (beliefs). A connection between formal theoriesof mental state and agent architectures (or implemen-tations) can be drawn by reifying mental actions, eachrepresenting the behavior of an architectural module(Ortiz 1999).

Collaboration Theories of mental state for individ-ual agents must be extended in important ways in or-der to model collaborations between agents. The prob-lem is that a group's plan to collaborate on some taskis not simply the sum of the agents' individual plans(Grosz and Kraus 1996, Bratman 1992, Searle 1990).In particular, some notion of an individual's commit-ment to (parts of) the group activity must be cap-tured. One way of accomplishing this is by introduc-ing the idea of a joint intention (Levesque et al. 1990,Sonenberg et al. 1992), which is meant to represent thegroup's commitment towards some group goal. Onecan also augment an agent architecture to include pro-cesses that capture behavioral conventions for modify-ing commitments and for supporting coordinated be-havior (Jennings 1995). One problem with the joint in-tentions model, however, is that communication pointsmust be built in and cannot be derived automatically.

The theory of SharedPlans is an alternative ap-proach that builds group plans out of ordinary beliefsand intentions, together with a new attitude called anintention-that (Grosz and Kraus 1996, Grosz et al., thisvolume). Intentions-that model commitments to statesof the world instead of actions. Such intentions can rep-resent, among other things, an individual agent's com-mitments to aspects of the world that will ensure thesuccess of the group activity; various stages of partial-ity of a shared plan can thereby be modeled (corre-sponding to, for example, partial beliefs about how toperform an action, the identity of appropriate agentsfor a group task, and the knowledge preconditions foran action). Theories based on this idea grew out ofwork in natural language understanding (Grosz andSidner 1986), and have been applied to discourse un-derstanding (Lochbaum 1998) and the development ofcollaborative user interfaces (Ortiz et al. 1999). Unlikethe irreducible joint-intentions model, communicationsare dynamically triggered as a consequence of failedintentions-that.

Some systems have been developed that combineboth approaches; for example, while the STEAM sys-tem (Tambe 1997) is based on the joint intentionsmodel, it also borrows some ideas from SharedPlans forrepresenting partial plans, and integrates ideas aboutteamwork capabilities into the design of individualagent architectures.

Decision theory provides a computational frameworkfor making decisions about coordination and communi-cation in a distributed planning context. Both STEAMand Gmytrasiewicz et al.'s Recursive Modeling Method(RMM) approach (Gmytrasiewicz et al. 1991) have

used models of the costs and bene�ts of communica-tion and applied decision-theoretic methods to deter-mine what and when to communicate. Boutilier (thisvolume) describes approaches for integrating decision-theoretic planning and multiagent systems.

Negotiation Given a representation for group plans,there remains the question of how agents should negoti-ate to reach an agreement over distribution of tasks andresources. One way to ensure coordination of agent ac-tivities is to embed them in environments in which cer-tain social laws must be adhered to (such as the tra�claws in (Shoham and Tennenholtz 1992)). This buildscoordination into the environment by focusing on pub-lic behavior rather than on private preferences. Otherapproaches make use of ideas from game theory, suchas notions of solution stability and equilibrium, and as-sume that agents are developed independently and areself-interested: that is, they maximize individual utility.Since agents are not necessarily assumed to be cooper-ative, much of the e�ort is directed to developing pro-tocols that will prevent lying. One attractive strategyis to characterize di�erent domains and identify appro-priate protocols for each type of domain (Rosenscheinand Zlotkin 1994).

Other ideas that have been adapted from the �eldof economics include negotiation strategies that taketime into account, explicitly handling the resource-constrained nature of agents (Osborne and Rubinstein1990, Kraus et al. 1992). Agents are assumed to havea common meta-goal; to minimize the amount of workthat each agent must do. In the case of agents whodo not have a common, �xed goal during negotiation,an alternative is for agents to construct arguments thatthey can use to in uence the decision making of otheragents (Sycara 1990).

Voting schemes, also borrowed from the �eld of eco-nomics, have been applied to achieving consensus. Forexample, in one multi-agent scheduling system, agents�rst express their individual preferences, and then avoting mechanism is used to determine group choice(Sen et al. ). When the assumption of cooperationis dropped, systems can become susceptible to lying.Many approaches around this problem have been pro-posed: one, for example, requires that agents pay atax (such as the Clarke tax (Ephrati and Rosenschein1991)) that, in e�ect, transfers utility outside the sys-tem. Agents are still assumed to be self-motivated andmaximize their own utility, but it can be shown thatthere is only one dominant strategy: telling the truth.

Contract nets (Smith and Davis 1983) represent amarket-based approach to negotiation. Contract netsprovide a high-level communication protocol in whichtasks are distributed among nodes in a system. Nodesare classi�ed as either contractor or manager nodes, andcontracts to perform tasks are established through abidding process. A similar approach is the \consumersand producers approach" (Wellman 1992), in whichagents are classi�ed as either consumers or producers,

and each type of agent tries to maximize its own utility.Goods have auctions associated with them, and agentscan acquire goods by submitting bids in the auction forthat commodity. A more dialogue-oriented approach isfound in the COllaborative Negotiation System basedon Argumentation (CONSA) (Tambe and Jung, thisvolume), which uses negotiation methods based on ar-gumentation structures (Toulmin 1958) to resolve con- icts.A major challenge that remains is the development of

models that encompass aspects of the partiality of bothmental state and the planning process itself. Methodsmust be developed for adapting the various approachesin a way that is consistent with the resource-constrainednature of planning agents: planning should be a con-tinuous, incremental process at both the individual andgroup level.

Continual PlanningAn agent working in a world where unexpected hin-drances or opportunities could arise should continuallybe on the lookout for changes that could render itsplanned activities obsolete. Because its plans can un-dergo continual evaluation and revision, such an agentwill be continually planning, interleaving planning withexecution.Reactive planning systems (Agre and Chapman 1987)

can be viewed as a special case of continual planning inwhich \planning" looks ahead only to the next action.Because of their near-term focus, such systems gener-ally do not handle problems that require complex or-derings of actions to accomplish tasks. The assumptionin these systems is that such orderings should not bepursued anyway because the later actions are inappro-priate by the time they are reached.Flexible plan execution systems such as PRS

(George� and Lansky 1986) and RAPS (Firby 1987)strike a compromise between looking ahead to se-quences of actions and avoiding commitments to speci�cfuture actions by exploiting hierarchical plan spaces.Rather than re�ne abstract plan operators into a de-tailed end-to-end plan, these systems interleave re�ne-ment with execution. Plan re�nement is delayed as longas possible, so that detailed decisions are made with asmuch information as possible. However, this can meanthat re�nement decisions at abstract levels are madeand acted upon before all of the detailed re�nementshave been made. If these abstract re�nements intro-duce unresolvable con icts at lower levels, the plan mayfail in the middle of execution. It is therefore criticalthat the speci�cations of abstract plan operators be richenough to summarize all (or at least most) of the rele-vant re�nements in order to anticipate and avoid suchcon icts (Clement and Durfee 1999).The Continuous Planning and Execution Framework

(CPEF|Myers, this volume) continually constructsand revises plans that evolve in response to a dynam-ically changing environment. CPEF integrates HTNplanning techniques, plan execution and monitoring,

and dynamic plan repair methods. Users can interactclosely with the system through the Advisable Plan-ner (AP). The execution component can execute plansat multiple levels of abstraction, permitting an open-ended planning process in which the level of detail ofplans can vary depending on the timing and nature ofthe particular planning task being considered.

The costs of execution monitoring and opportunisticreplanning must be considered by continual planningsystems. In many respects, execution monitoring is sim-pler: as an agent steps through its planned actions, itperiodically checks to make sure that the state of theworld that it has reached is consistent with the statethat was predicted by the plan. The trick is to keep thecosts of monitoring low enough so that they do not con-sume too much time and e�ort on the part of the agent,either by devising e�cient perceptual strategies (Doyleet al. 1986, Musliner et al. 1995) or by performingthe checks less frequently. Looking for new opportu-nities is more problematic: unlike simple monitoring,where the agent knows what to look for (which condi-tions are necessary for the rest of the plan to succeed),�nding opportunities is generally a more open-endedproblem. Researchers have explored tradeo�s betweenhaving bold agents, who seldom reconsider their plans,and cautious agents, who frequently reevaluate theirplans (Kinny and George� 1991).

Di�erent continual planning approaches implementongoing monitoring and replanning tasks di�erently.For example, UM-PRS (Lee et al. 1994) maintains thehierarchy of plans currently in progress, and before eachaction traverses this hierarchy from the top down to en-sure that the motivations for the next action are stillin force. If during this process it encounters a planabstraction that is now deemed inappropriate, it willconsider alternative ways of achieving the goal it hadbeen working on, and will look for opportunities for di-recting its attention elsewhere. In contrast, the Soararchitecture (Laird et al. 1987) can also perform con-tinual planning, but does not explicitly reason over ahierarchy of intentions. Instead, it has been extended togenerate rules that explicitly monitor the current plan'scontext (Wray and Laird 1998). Because these rules arewoven into Soar's rule network, they do not require ex-plicit periodic polling, but instead trigger interrupts toSoar when appropriate.

Future Research Directions

While distributed planning has been explored by a fewresearchers for many years, the recognition of its impor-tance in current and future applications involving net-worked agents has triggered a more broad-based, con-certed investigation into this still-maturing �eld. Rea-soning and negotiation techniques are needed that per-mit distributed planning agents to understand the as-pects of the distributed plan that are relevant for theirdecisions, identify and pursue opportunities for coor-dination, asynchronously re�ne their plans as other

agents' plans are evolving, and modify their plans inresponse to information that arrives from other agents.

In the area of continual planning, open-ended plan-ning, in which plans can be continually re�ned to vary-ing levels of abstraction as the planning horizon is ex-tended, will be a key capability. Plan repair methodsin current systems are typically either at the reactivelevel or at the generative level; methods for smoothlyintegrating plan repair techniques at multiple levels ofabstraction and varying time scales are needed.

Several papers in this volume describe ongoing re-search e�orts that are focused on true DCP|that is,methods for distributed and continual planning in dy-namic, uncertain domains. Pollack and Harty (this vol-ume) describe a plan management system that goesbeyond simply interleaving decisions about planningand execution by incorporating techniques for monitor-ing the environment, assessing alternative plans, iter-atively elaborating abstract partial plans, controllingthe meta-level planning process, and coordinating dis-tributed planning agents. Durfee (this volume) de-scribes methods for continual, cooperative elaborationand revision of distributed partial plans in a realistic ap-plication domain of semi-autonomous ground vehicles.Myers (this volume) presents a framework for continu-ous planning and execution that is partially distributed(that is, the planning and execution capabilities are dis-tributed, though the planning process itself is not).

However, most of this research is still in its earlystages, and many research challenges remain. Both dis-tributed planning and continual planning need to bebetter understood for DCP systems to be built; yetsimply combining distributed and continual planningmethods that have been independently developed maynot be su�cient. The synchronization problem in dis-tributed planning becomes even greater when the dis-tributed agents' plans are being executed concurrently,and better models of the overall planning and execu-tion process are needed. Similarly, execution of plansis complicated by the presence of other agents, requir-ing not only methods for distributed plan execution butdistributed plan repair.

References

P. E. Agre and D. Chapman. Pengi: An implementa-tion of a theory of activity. In IJCAI-87, pages 268{272, 1987.

J. A. Ambros-Ingerson and S. Steel. Integrating plan-ning, execution, and monitoring. In AAAI-88, pages81{88, 1988.

Craig Boutilier, TomDean, and Steve Hanks. Decisiontheoretic planning: Structural assumptions and com-putational leverage. Journal of AI Research, 1999. Toappear.

Michael E. Bratman, David J. Israel, and Martha E.Pollack. Plans and resource-bounded practical reason-ing. Computational Intelligence, 4:349{355, 1988.

Michael Bratman. Intentions, Plans, and PracticalReason. Harvard University Press, 1987.

Michael E. Bratman. Shared cooperative activity. ThePhilosophical Review, 101(2):327{341, 1992.

Bradley J. Clement and Edmund H. Durfee. Top-down search for coordinating the hierarchical plans ofmultiple agents. In Third Conference on AutonomousAgents, 1999.

Philip Cohen and Hector Levesque. Intention is choicewith commitment. Arti�cial Intelligence, 42:213{261,1990.

Daniel D. Corkill. Hierarchical planning in a dis-tributed environment. In IJCAI-79, 1979.

Keith S. Decker and Victor R. Lesser. Generalizing thepartial global planning algorithm. International Jour-nal of Intelligent and Cooperative Information Sys-tems, 1:319{346, 1992.

Keith Decker, Victor Lesser, M.V. Nagendra Prasad,and Thomas Wagner. MACRON: An architecturefor multi-agent cooperative information gathering. InProceedings of the CIKM Workshop on Intelligent In-formation Agents, December 1995.

R. J. Doyle, D. J. Atkinson, and R. S. Doshi. Generat-ing perception requests and expectations to verify theexecution of plans. In AAAI-86, pages 81{88, 1986.

Edmund H. Durfee and Victor R. Lesser. Negotiat-ing task decomposition and allocation using partialglobal planning. In M. Huhns and L. Gasser, editors,Distributed Arti�cial Intelligence, volume 2. MorganKaufmann, 1989.

EdmundH. Durfee and Victor R. Lesser. Partial globalplanning: A coordination framework for distributedhypothesis formation. IEEE Transactions on Systems,Man, and Cybernetics, KDE-1:63{83, 1991.

Edmund H. Durfee and Thomas A. Montgomery.Coordination as distributed search in a hierarchi-cal behavior space. IEEE Transactions on Systems,Man, and Cybernetics, SMC-21(6):1363{1378, Novem-ber 1991. Special Issue on Distributed Arti�cial Intel-ligence.

E. Ephrati and J. Rosenschein. The Clarke Tax asa consensus mechanism among automated agents. InProceedings of AAAI-91, pages 173{178, 1991.

Eithan Ephrati, Martha E. Pollack, and Je�rey S.Rosenschein. A tractable heuristic that maximizesglobal utility through local plan combination. In Pro-ceedings of the 1st International Conference on Multi-Agent Systems (ICMAS-95), 1995.

Eithan Ephrati, Martha E. Pollack, and Sigalit Ur.Deriving multi-agent coordination through �lteringstrategies. In Proceedings of the 14th Joint Interna-tional Conference on Arti�cial Intelligence (IJCAI-95), 1995.

K. Erol, D. Nau, and J. Hendler. HTN planning: Com-plexity and expressivity. In AAAI-94, 1994.

G. Ferguson, J. Allen, and B. Miller. TRAINS-95:Towards a mixed initiative planning assistant. InThird International Conference on Arti�cial Intelli-gence Planning Systems, 1996.

R. J. Firby. An investigation into reactive planning incomplex domains. In AAAI-87, pages 202{206, 1987.

Michael P. George� and Amy L. Lansky. Proceduralknowledge. Proceedings of the IEEE, 74:1383{1398,1986.

Piotr J. Gmytrasiewicz, Edmund H. Durfee, andDavid K. Wehe. The utility of communication in co-ordinating intelligent agents. In AAAI-91, pages 166{172, 1991.

Barbara J. Grosz and Sarit Kraus. Collaborativeplans for complex group action. Arti�cial Intelligence,86(1):269{357, 1996.

Barbara J. Grosz and Candace Sidner. Attention, in-tentions, and the structure of discourse. Computa-tional Linguistics, 12(3):175{204, 1986.

Afsaneh Haddadi and Kurt Sundermeyer. Belief-Desire-Intention Agent Architectures, pages 169{186.John Wiley and Sons, Inc, 1996.

James Hendler, Austin Tate, and Mark Drummond.AI planning: Systems and techniques. AI Magazine,11:61{77, 1990.

N. Jennings. Controlling cooperative problem in in-dustrial multi agent systems using joint intentions. Ar-ti�cial Intelligence, 75(2):195{240, 1995.

S. Kambhampati, M.R. Cutkosky, J.M Tenenbaum,and S.H. Lee. Integrating general purpose plannersand specialized reasoners: Cas e study of a hybridplanning architecture. IEEE Transactions on Systems,Man, and Cybernetics, 23:1503{1518, 1993.

David N. Kinny and Michael P. George�. Commit-ment and e�ectiveness of situated agents. In IJCAI-91, pages 82{88, 1991.

Sarit Kraus, Jonathan Wilkenfeld, and Gilad Zlotkin.Multiagent negotiation under time constraints. Tech-nical Report CS-TR-2975, University of Maryland,College Park, 1992.

J. E. Laird, A. Newell, and P. S. Rosenbloom. SOAR:An architecture for general intelligence. Arti�cial In-telligence, 33:1{64, 1987.

Amy L. Lansky and Lise C. Getoor. Scope and ab-straction: Two criteria for localized planning. InIJCAI-95, 1995.

Amy L. Lansky. Located planning with diverse planconstruction methods. Technical Report FIA-TR-9405, NASA Ames Research Center, 1994.

Jaeho Lee, Marcus J. Huber, Edmund H. Durfee, andPatrick G. Kenny. UM-PRS: An implementation ofthe Procedural Reasoning System for multirobot ap-plications. In AIAA/NASA Confernce on IntelligentRobotics in Field, Factory, and Space, pages 842{849,1994.

H. Levesque, P. Cohen, and J. Nunes. On acting to-gether. In Proceedings of AAAI-90, pages 94{99, 1990.

Karen E. Lochbaum. A collaborative planning modelof intentional structure. Computational Linguistics,34(4):525{572, 1998.

David J. Musliner, Edmund H. Durfee, and Kang G.Shin. World modeling for the dynamic construction ofreal-time control plans. Arti�cial Intelligence, 74:83{127, 1995.

Karen L. Myers. Advisable planning systems. InAustin Tate, editor, Advanced Planning Technology:Technological Achievements of the ARPA/Rome Lab-oratory Planning Initiative. AAAI Press, 1996.

Charles Ortiz, Barbara Grosz, and Nathan Scales. In-terpreting information requests in context: a collabo-rative web interface for distance learning. AutonomousAgents and Multi-agent Systems, 1999. Submitted.

Charles L. Ortiz. Introspective and elaborative pro-cesses in rational agents. Annals of Arti�cial Intelli-gence and Mathematics, page To appear, 1999.

M. J. Osborne and A. Rubinstein. Bargaining andMarkets. Academic Press, California, 1990.

Martha Pollack. Plans as Complex Mental Attitudes,pages 77{103. MIT Press, 1990.

Martha E. Pollack. Planning in dynamic environ-ments: The \DIPART" system. In Austin Tate, editor,Advanced Planning Technology: Technology Achieve-ments of the ARPA/Rome Laboratory Planning Ini-tiative. Morgan Kaufmann, 1996.

Je�rey S. Rosenschein and Gilad Zlotkin. Rules ofEncounter. MIT Press, 1994.

Stuart J. Russell and Peter Norvig. Arti�cial Intelli-gence: A Modern Approach. Prentice Hall, 1995.

S. J. Russell and D. Subramanian. Provably bounded-optimal agents. Journal of Arti�cial Intelligence Re-search, 2:575{609, 1995.

Earl D. Sacerdoti. A Structure for Plans and Behavior.American Elsevier, 1977.

Marcel J. Schoppers. Universal plans for reactiverobots in unpredictable environments. In IJCAI-87,pages 1039{1046, 1987.

J. Searle. Collective Intentions and Actions, pages401{ 415. MIT Press, 1990.

Sandip Sen, Thomas Haynes, and Neeraj Arora. Sat-isfying user preferences while negotiating meetings.International Journal of Human-Computer Systems,page ???, ???

Y. Shoham and M. Tennenholtz. On the synthesisof useful social laws for arti�cial agent societies. InProceedings of AAAI-92, pages 276{281, 1992.

Yoav Shoham. Agent-oriented programming. Tech-nical Report STAN-CS-90-1335, Stanford University,1990. Computer Science Department.

H. Simon. Alternative Visions of Rationality. StanfordUniversity Press, 1983.

R.G. Smith and R. Davis. Negotiation as a metaphorfor distributed problem solving. Arti�cial Intelligence,pages 63{109, 1983.

E. Sonenberg, G. Tidhar, E. Werner, D. Kinny,M. Ljungberg, and A. Rao. Planned team activity.Technical Report 26, Australian Arti�cial IntelligenceInstitute, 1992. Revised JJanuary 1994.

Peter Stone and Manuela Veloso. Task decomposition,dynamic role assignment, and low-bandwidth commu-nication for real-time strategic teamwork. Arti�cialIntelligence, 1999. To appear.

Katya Sycara. Persuasive argumentation in negotia-tion. Theory Decision, 28:203{242, 1990.

Milind Tambe. Towards exible teamwork. Journal ofArti�cial Intelligence Research, 7:83{124, 1997.

S. Toulmin. The uses of argument. Cambridge Uni-versity Press, 1958.

Manuela M. Veloso. Towards mixed-initiativerationale-supported planning. In Austin Tate, ed-itor, Advanced Planning Technology: TechnologicalAchievements of the ARPA/Rome Laboratory Plan-ning Initiative, pages 277{282. AAAI Press, 1996.

Michael Wellman. A general-equilibrium approach todistributed transportation planning. In Proceedings ofAAAI-92, pages 282{289, 1992.

David E. Wilkins and Karen L. Myers. A multiagentplanning architecture. In AIPS-98, pages 154{162,1998.

David E. Wilkins. Practical Planning: Extending theClassical AI Planning Paradigm. Morgan Kaufmann,1988.

Michael J. Wolverton and Marie desJardins. Control-ling communication in distributed planning using ir-relevance reasoning. In Proceedings of the FifteenthNational Conference on Arti�cial Intelligence (AAAI-98). AAAI Press, 1998.

M. J. Wooldridge and N. R. Jennings. Agent theories,architectures, and languages: A survey. ???, page ???,???

Robert Wray and John Laird. Maintaining consistencyin hierarchical reasoning. In AAAI-98, 1998.