Agent-Oriented Architecture forAir Combat SimulationApril, 1993Technical Note 42By:Anand RaoAndrew LucasDavid MorleyAustralian Arti�cial Intelligence InstituteMario SelvestrelGraeme MurrayDefence Science and Technology OrganisationAeronautical Research LaboratoryThis research was supported by the Defence Science and Technology Organisation{ Aeronautical Research Laboratory, a Generic Industry Research and DevelopmentGrant from the Department of Industry, Technology and Commerce, Australia and bythe Co-operative Research Centre for Intelligent Decision Systems. Abridged versionsof this paper appear in the Proceedings of the Australian Joint Conference on Arti�cialIntelligence (AI-92), Hobart, Australia, 1992 and Proceedings of Future Directions inSimulation Systems Workshop, Melbourne, Australia, 1992.

AbstractAir combat modelling using graphical simulation is a powerful means for develop-ment and evaluation of tactics. However, large models are particularly expensive andtime-consuming to maintain and modify. Multi-aircraft full mission man-in-the-loopsimulators will provide an even more complex programming environment.Agent-oriented architecture provides a suitable software environment for the devel-opment of an air combat simulation model based on the concept of rational agents. Thisapproach allows the analyst to work at a high level, formulating concepts and aims, whilekeeping the detailed computer programming hidden.

1 IntroductionComputer-based air combat modelling is a powerful tool, widely accepted for its usefulness.The extremely high cost of operating aircraft and their weapons has led to a rapid growth inthe development and use of computer simulation models as a basis for tactics development,pilot training, and operational evaluation of weapon systems. However, models having therequired level of �delity are very costly to develop, not easily modi�ed, and lack the abilityto show the tactics being used. The analyst has to be able to program in the particularprogramming language used, and the tactics routines are not readily understood by pilotsand operational planning sta�.Present simulators combine a high degree of �delity with the ability to simulate multi-aircraft combat. These typically involve tens of thousands of lines of code and are di�cultto manage and modify; combat scenarios involving multiple aircraft are especially di�cultto model.The continuing rapid increase in speed of computing and advances in visual system hard-ware are allowing further signi�cant advances to be made in simulation capability. However,these advances are dependent on further development and expansion of what is already alarge body of software. The challenge is to provide improved performance whilst avoidingthe existing shortcomings associated with large simulation codes. These are:� lack of exibility (in particular, team tactics are di�cult to represent);� the time and cost associated with maintaining and modifying the simulation code;� the di�culty of making modi�cations without introducing errors;� the need to employ analysts who are also experienced programmers, and are familiarwith the model and the code;� the air combat tactics being embedded in the model; and� the di�culty in visualizing the tactics underlying the model as there is no intuitivemeans of describing them to pilots and operational personnel.The aims of the work reported here are to assist the analyst in making use of simulatorsfor developing and evaluating new tactics, and to assist pilots and military planners toevaluate tactics prior to their operational introduction.These aims must be achieved in a way that makes use of the existing body of simulationsoftware. They can be achieved by embedding the proposed new tactics representation inthe current simulation environment, so that modi�cations required in the code representingthe physics of the model are minimal.A simulation model has a number of functional components:� a model of the aircraft and weapons physics;� a model of the aircraft sensors;� a model of the electronic warfare environment;� the air combat tactics;� user interfaces for both pilot and simulation control; and� a visual system to display the simulation.1

The approach taken here relates to the representation, display and execution of air combattactics by the simulation. The key is to separate the knowledge (tactics) from the physi-cal model and visualization. This allows modi�cation of the knowledge base without beingconcerned about the remaining code. Hence, the analyst need only work at the high level,formulating concepts and aims. This approach is eminently suited to dealing with exten-sive repertoires of pre-determined team tactics. The technology used is known as real-timeprocedural reasoning1.The current move to use of man-in-the-loop training simulations is accelerating thegrowth of computer simulation. Multi-aircraft full mission simulators will be perhaps themost demanding military application. High cost will restrict the number of pilot stationsavailable, so that there will be a need to provide computer-generated surrogate pilots forboth friendly and enemy aircraft.2 Situation Awareness and Tactics RepresentationThe tasks performed by a combat pilot can be broadly divided into two areas { situationawareness and tactics selection. Determination of the current situation is called situationawareness and the selection of appropriate actions in response to the situation is calledtactics selection. Both stages require sophisticated reasoning and are closely linked. Havingdetermined his current situation, a pilot needs to select and execute certain actions; theseactions, in part, determine the next situation.A pilot's reasoning process can be characterized as consisting of beliefs about the realworld, goals that need to be satis�ed, mental plans or procedures that satisfy certain goals,and committed partial plans or intentions that a pilot has adopted in response to externalevents or internal goals.In situation awareness, a pilot must infer the beliefs, goals and intentions of other aircraftfrom their behaviour. In tactics selection, a pilot must react to his beliefs about the currentsituation or advance his goals towards a desired situation.While the problems of situation awareness and tactics selection are di�cult for a singlepilot in combat with a single enemy, the problems become far more complex when a team (ofpilots) is in combat with an enemy team. The team as a whole needs to assess the situationby inferring not only the individual beliefs, goals, and intentions of enemy aircraft, but alsothe mutual beliefs, joint goals, and joint intentions of the entire enemy team. Similarly,tactics selection by a team is more di�cult than the selection of tactics by a single pilot,because of the co-ordination and synchronization required.3 Agent-Oriented ArchitectureA rational agent can be viewed as a system situated in the real world, continuously re-ceiving perceptual input and responding by taking actions that a�ect the world. It canbe characterized as having beliefs about the real world, goals that need to be satis�ed,compiled plans or recipes for achieving certain goals, and committed partial plans or in-tentions that have been adopted in response to external events or internal goals. Com-piled plans play an important role in such characterizations by side-stepping the classi-cal planning problem. More recently, the logical and practical design of such rational1The term real-time is used here in the context of human behaviour and speed of decision making in adynamic environment such as air combat, i.e., between one second and one minute.2

agents have received increasing attention [Bratman et al., 1988; Cohen and Levesque, 1990;Rao and George�, 1991].The Procedural Reasoning System (PRS) [George� and Lansky, 1986; Ingrand et al.,1992] is a situated system based on the BDI architecture that explicitly reasons about thebeliefs, plans, goals, and intentions of a rational agent. The functional modules of theProcedural Reasoning System are given in Figure 1. In PRS, each agent is made up of thefollowing components:Beliefs: The beliefs of an agent provide information about the state of the environment andare represented as ground �rst-order logic formulas.Goals: The goals of an agent are descriptions of desired tasks or behaviours. The goal toachieve a certain formula P is written as (! P); to test for the truth of a formula iswritten as (? P); and to wait until a formula P is true is written (^ P).Plan Library: The plan library of an agent consists of a set of plans. Each plan is anabstract speci�cation of possible sequences (and more complex orderings) of actionsthat can be used to accomplish certain goals or react to certain situations. Eachplan consists of a name; an invocation condition, which speci�es when a plan can beinvoked; a precondition, which speci�es the required belief state of an agent; and abody, which describes the possible sequences of actions as a rooted, directed, acyclicgraph in which each edge is labelled with a primitive action or a goal expression ofthe above form. The plan library can also include metalevel plans, that is, informationabout the manipulation of the beliefs, goals, and intentions of the agent itself. Theseplans encode various methods for choosing among multiple applicable plans, modifyingand manipulating intentions, and computing the amount of reasoning that can beundertaken, given the real-time constraints of the problem domain.Intention Structure: The intention structure contains a partially ordered set of tasks thatthe agent is committed to executing. These adopted tasks are called intentions. Asingle intention consists of some initial plan, together with all the sub-plans that arebeing used in attempting to successfully execute that plan. Intentions may be activeor waiting for certain conditions to hold prior to activation. Some of these intentionsmay be metalevel intentions for decisions on various alternative courses of action.Interpreter: The interpreter manipulates the above components, selecting appropriateplans based on the system's beliefs and goals, placing them on the intention struc-ture, and executing them. Unless some new belief or goal activates some new plan, thesystem will try to ful�ll any intentions it has previously decided upon. This results infocused, goal-directed reasoning in which plans are expanded in a manner analogousto the execution of subroutines in procedural programming systems. Changes in theenvironment may lead to changes in the system's goals or beliefs, which in turn mayresult in the consideration of new plans that are not means to any already intendedend. The system is therefore able to change its focus completely and pursue new goalswhen the situation warrants it.The system interacts with its environment, including other systems, through its database(which acquires new beliefs and discards old ones in response to changes in the environment)and through the actions that it performs as it carries out its intentions.3

??-6 � - 6 6�6QQQ ������ QQQ

MONITORDATABASE(Beliefs) KA LIBRARY(Plans) SENSORSEFFECTORS ENVIRONMENTINTERPRETER(Reasoner) COMMANDGENERATORINTENTIONSTRUCTUREGOALSDATAOUTPUTDATAINPUTSYSTEMINTERFACES

Figure 1: Structure of the Procedural Reasoning System4 Team-Oriented ArchitectureJust as a single agent can be considered a rational agent, a team is an organized collectionof rational agents. A team can contain other teams as well. This results in a powerfulorganizational structure capable of modelling hierarchical as well as non-hierarchical socialstructures. In addition to individual beliefs, goals, plans, and intentions, a team of rationalagents will also have: (a) mutual beliefs about the world and about each others' actions; (b)joint goals that need to be achieved collectively; (c) compiled joint plans (common to all teammembers) that specify the means of satisfying joint goals; and (d) committed joint plans orjoint intentions, adopted as a response to a joint goal or an external event. Commitment toa joint plan may entail commitment to communicate to other members of the team.The system for air combat modelling consists of two types of agents { team agents andaircraft agents. One aircraft agent is created for each aircraft in the scenario. A team agent iscreated for each group of aircraft that forms a team. Figure 2 illustrates a 4v2 scenario wherethere are four \blue" aircraft and two \red" aircraft. The four blue aircraft are organizedas two subteams, each with two aircraft. The two red aircraft also act as a team. Theroles of leader and wingman are assigned to individual aircraft as well as to teams at thepre-mission-brief stage. The team structure can be understood by looking at the team agentBLUE1234, which consists of two team agents BLUE12 and BLUE34 with the former playingthe role of the lead team and the latter playing the role of the wing team. The sub-teamBLUE12 has two aircraft agents BLUE1, playing the role of the leader, and BLUE2, playing therole of the wingman.The creation of all these agents and their interactions with the simulator are handledby the agent COORDINATOR. Team agents and aircraft agents can be dynamically created,destroyed, and modi�ed by the COORDINATOR. For example, if BLUE2 and BLUE3 are destroyed4

BLUE1234

BLUE12L

W

L

B1

B2

RED12L

R1

W

R2

BLUE34W

L

W

B3

B4Figure 2: Organization of Teams for a 4v2 Scenarioin combat, the COORDINATOR can recon�gure the agent structure so that the teams BLUE12,BLUE34, and BLUE1234 are replaced by the team agent BLUE14. Similarly, the roles of aircraftagents and team agents can be dynamically reassigned.Beliefs of each aircraft agent include: (a) heading; (b) position; (c) bearing to otheraircraft; (d) range to other aircraft; and (e) headings of other aircraft. The positionalinformation held by aircraft agents are egocentric. The beliefs of aircraft agents may varydepending on their role and their situation awareness. For example, an aircraft agent with therole of wingman might have beliefs about the position and heading of the lead aircraft agent.Also, the COORDINATOR can selectively pass range and bearing information with respect toother aircraft, thereby simulating the situation awareness of the agents. For example, theCOORDINATORmight pass information regarding the range and heading of RED1 only to BLUE1and not the others. This models situations where only BLUE1 is aware of RED1.A team agent also has beliefs of its own. These beliefs are about the team's heading, posi-tion, bearing, etc. Aircraft agents communicate their position and heading information to alltheir parents. For example, BLUE1's position and heading will be known to the team BLUE12and the team BLUE1234. Thus information or beliefs ow up the hierarchical organizationof agents.Aircraft agents tend to have low-level goals. These goals may have been adopted asa result of orders from higher level team agents or as subgoals to solve higher level goals.As a result, the plans of an aircraft agent tend to be low-level plans. The low-level plansinclude monitoring the current heading, changing the heading to acquire the desired heading,obtaining a certain lateral separation, ying at a certain heading for a given duration ordistance, ying keeping a constant bearing to a given target, �ring a missile, guiding amissile, etc. Note that none of the tactics appear as plans for the aircraft agents.The goals of team agents tend to be higher-level mission goals, like sites to defend ortargets to attack. These goals are derived from the pre- ight mission brie�ngs. Hence, teamagents tend to have high-level tactical plans. These plans include the team searching forenemy aircraft in a particular formation, the team doing a pincer or cuto� intercept, theteam deciding that a particular sub-team is under threat and doing an evasion, the teamchoosing a target, etc.The top-level team adopts goals to perform intercepts, perform evasions, and search forenemy aircraft. Depending on the current scenario the team then chooses an appropriate planto satisfy one of these goals which then becomes the intention of the team. The successfulcompletion of this intention depends on the lower-level sub-teams adopting certain lower-levelintentions. This is achieved by the team sending messages to the lower-level sub-teams to5

(a)Name: Achieve Pincer InterceptInvocation:(*GOAL (! (pincer-intercept$enemy-team)))Precondition:(and (*FACT (this-agent $us))(*FACT (team-left-right $us $left-team$right-team)))Body: k1(! (:(in-formation $us)))?k2(! (do-parallel($left-team (! (cuto�-man $enemy-team left))($right-team (! (cuto�-man $enemy-team right))))?k3(! (attack-man $enemy-team))?k4(b)Name: Achieve Cuto� Manoeuvre TeamInvocation:(*GOAL (! (cuto�-man$enemy-team $side)))Precondition:(and (*FACT (this-agent $team))(*FACT (:(single-aircraft $team)))(*FACT (:(in-formation $lead-ac))))Body: k1(! (do $lead-ac(! (cuto�-man$enemy-team $side))?k2

(c)Name: Achieve Cuto� Manoeuvre ACInvocation:(*GOAL (! (cuto�-man$enemy-team $side)))Precondition:(and (*FACT (this-agent $me))(*FACT (single-aircraft $me)))Body: k1(! (control-until(have-separation $enemy-team $side)(in-roll-range $enemy-team)))?k2(! (control-until(on-collision-course $enemy-team)(in-sort-range $enemy-team)))?k3Figure 3: Joint Plans for Pincer Intercept and Cuto� Manouevresadopt certain intentions. The top-level team intention then gets suspended until it receivesmessages from the lower-level sub-teams that they have accomplished their tasks.In addition to the goals and intentions adopted as a response to higher-level team in-tentions, the individual aircraft can also adopt goals and intentions of their own to monitortheir position, maintain bearing, �re missiles, etc.5 Sample Scenarios5.1 Pincer and Cuto� InterceptsA pincer intercept by a team involves the team subdividing into two sub-teams, with eachsub-team doing a cuto� manoeuvre followed by an attack manoeuvre, in parallel. The top-level plan to perform a pincer intercept is given by the plan Achieve Pincer Intercept(see Figure 3 (a)).A cuto� intercept by a team involves the entire team doing a cuto� manoeuvre to theleft or right, followed by an attack manoeuvre. A cuto� manoeuvre involves three phases,namely, obtaining a certain lateral separation from the enemy team's heading, ying straighttill roll range, and then ying on collision course till sort range. An attack manoeuvre fora team involves each aircraft doing an attack manoeuvre, which involves ying in collisioncourse and then �ring and guiding the missile.The top-level plan to perform a cuto� intercept is given by the plan Achieve cutoffintercept. If the target aspect is positive, the team does a cuto� manoeuvre to the right,else it does a cuto� manoeuvre to the left. There are two plans for a cuto� manoeuvre,one for a team and the other for an aircraft. The plan Achieve Cutoff Manoeuvre Team(Figure 3 (b)) involves the lead aircraft of the team doing a cuto� manoeuvre. The planAchieve Cutoff Manoeuvre AC (Figure 3 (c)) involves the aircraft obtaining separation tillroll range, followed by ying on collision course till sort range, when individual targets arechosen by each aircraft.There are three plans for an attack manoeuvre, one for a team to attack another team,the second for a team to attack a single aircraft, and the third for a single aircraft to attack6

another single aircraft. Repeated execution of the �rst and second plans will result in theteam and the enemy team being sub-divided until a single aircraft has to attack a singleenemy aircraft. This results in the third plan being executed. This plan involves the aircraft ying on collision course till �ring range, followed by �ring the missile, followed by guidingthe missile, followed by a y-through course till the missile has hit or missed.Figure 4 (a) shows a pincer intercept performed by a team of four blue agents against anopposing team of two red agents. The team agent BLUE12 is performing a cuto� manoeuvreto the left and the team agent BLUE34 is performing a cuto� manoeuvre to the right.R1

B2

maintainseparation

rollrange

sortrange

obtainseparation

R2

B1 B3 B4

R1 R2

B3 B4B2B1

....................................................................

......................................................................

.................................................................................

.................................................................................

(a) (b)

Figure 4: Pincer Intercept in a 4v2 Scenario5.2 Sample 4v2 ScenarioIn this section we consider a 4v2 scenario where there are four blue aircraft and two redaircraft. The hierarchical organization and roles of agents are as described earlier in Section 4and shown in Figure 2.Consider the scenario shown in Figure 4 (a), where the team RED12 is ying straightonto the team BLUE1234. As the size of BLUE1234 is greater than the size of RED12, theteam agent BLUE1234 decides to execute a pincer intercept with team agents BLUE12and BLUE34 executing cuto� manoeuvres. Aircraft agents BLUE1 and BLUE2 are obtainingseparation towards the left of RED12's heading and BLUE3 and BLUE4 are obtaining separationtowards the right of RED12's heading.The scenario after a few simulation cycles is depicted in Figure 4 (b). Now the team RED12alters its course and decides to do a pincer intercept. The team agent BLUE1234 notices this7

and instructs teams BLUE12 and BLUE34 to do a pincer intercept. This results in theBLUE12 team performing a pincer intercept and BLUE34 doing likewise. Thus a change insituation resulted in the team BLUE1234 changing its tactics.This change in tactics was possible because the interpreter of the team agent BLUE1234 isevaluating the situation at each and every cycle; as soon as there is any signi�cant change inthe situation it can modify its tactics and inform the sub-teams and aircraft agents to adoptalternative tactics. Similarly, as the interpreters in the sub-team agents and aircraft agentsare continuously checking if they have received any orders from their higher level teams, theycan adopt these orders as soon as they arrive.Also, the tactics, written as plans, are not written for each and every speci�c scenario.The plans operate at the level of abstraction of a team and teams can consist of an arbitrarynumber of other teams or aircraft. The plans operate by binding the teams at run-time(based on the current scenario) and then sub-dividing the teams into sub-teams to carry outparticular manoeuvres. As a result the same plans as mentioned above will work for anyother scenario, like, 2v2, 4v4, 8v4, 6v8, etc.6 ConclusionThis paper presents a multi-agent real-time system that can represent and reason aboutjoint mental attitudes such as mutual beliefs, joint goals, joint plans, and joint intentions.The approach outlined here makes it easier (and thus faster) to develop and modify tacticsin air combat modelling. The programming complexity of multi-aircraft man-in-the-loopsimulators demands such a conceptual breakthrough. Real-time procedural reasoning:� allows tactics to be constructed and displayed graphically (the analyst does not needto program the tactics in source code);� separates the tactics from the major body of the simulation code;� makes it easier to build, modify and display tactics, including team tactics;� makes simulated situations more easily understood by display of the underlying tacticsinvolved; and� can be embedded within the simulator, thus taking advantage of existing simulatorcode.The system has a number of salient features which distinguishes it from other multi-agent and distributed systems. Some of these are (a) it can represent and reason aboutabstract speci�cations of team behaviour, namely, joint plans; (b) the actions performedby an agent that is part of a team are determined not only by its own individual mentalstate, but also the joint mental state of the team; (c) the agents can be grouped togetheras teams using a number of di�erent organizational structures with speci�c roles assignedto them; (d) the agents can reorganize themselves into di�erent teams or change their rolesbased on changing situations; (e) the actions taken by the agents and the teams are basedon their current situation and changing situations can lead to changes in the intentions ofagents and teams. The above features make the system a powerful tool for reasoning aboutteam behaviour and collaboratively solving problems, as well as providing a novel means forcontrolling air combat simulation models.A prototype of the team tactics representation and execution system, implemented us-ing the Procedural Reasoning System, is running on Sun SPARCstations. PRS is written8

in Common LISP and an extension of this system, called Multi-Agent Real-time System(MARS), is currently being written in C++. It will enable direct integration with conven-tional simulation software. The prototype team tactics system will then be ported to MARS,making it accessible to programmers who are not familiar with LISP.AcknowledgmentsThe authors acknowledge the contributions of Dr. K. Anderson, Ms. S. Steuart and Mr. R.Muscat during the development of this system.References[Bratman et al., 1988] M. E. Bratman, D. Israel, and M. E. Pollack. Plans and resource-bounded practical reasoning. Computational Intelligence, 4:349{355, 1988.[Cohen and Levesque, 1990] P. R. Cohen and H. J. Levesque. Intention is choice with com-mitment. Arti�cial Intelligence, 42(3), 1990.[George� and Lansky, 1986] M. P. George� and A. L. Lansky. Procedural knowledge. InProceedings of the IEEE Special Issue on Knowledge Representation, volume 74, pages1383{1398, 1986.[Ingrand et al., 1992] F. F. Ingrand, M. P. George�, and A. S. Rao. An architecture forreal-time reasoning and system control. IEEE Expert, 7(6), 1992.[Rao and George�, 1991] A. S. Rao and M. P. George�. Modeling rational agents within aBDI-architecture. In J. Allen, R. Fikes, and E. Sandewall, editors, Proceedings of the Sec-ond International Conference on Principles of Knowledge Representation and Reasoning.Morgan Kaufmann Publishers, San Mateo, CA, 1991.

9