A Lyapunov treatment of swarmcoordination under conflict

Paul McCullough1, Mark Bacon1, Nejat Olgac1,Daniel A. Sierra2,3 and Rudy Cepeda-Gomez1

AbstractWe consider hostile conflicts between two multi-agent swarms, called pursuers and evaders. A Newtonian dynamics-based double integrator model is taken into account, as well as a control strategy using the relative positions and velocitiesof opposing swarm members. This control is introduced to achieve stability and the capture of the evaders by the pur-suers. The present document considers only swarms with equal membership strengths and equal mass for simplicity. Thiseffort begins with a set of suggested interaction force profiles, which are functions of local vectors. To formulate a robustcontrol law, a Lyapunov-based stability analysis is used. The group pursuit is conceived in two phases: the approach phase,during which the two swarms act like two individual agents, and the assigned pursuit phase, where each pursuer has anassigned evader. We show that the uncontrolled dynamics, which are marginally stable, are stabilized by the newcontroller.

KeywordsLyapunov stability, multi-agent modeling, stability of nonlinear systems

Date received: 13 July 2009; accepted: 16 November 2009

1. Introduction

This study addresses the modeling, analysis and control

of multi-agent swarms starting from Newton’s equations.

When each agent influences every other agent, the rules

of interaction and stability of group dynamics become

quite complicated. Much of the inspiration stems from

the study of biological swarms (Warburton and Lazarus,

1991; Camazine et al., 2001). Most of the earlier inves-

tigations that focus on this general theme consider homo-

genous swarms, i.e., those composed of alike members,

with a single integrator model and momenta profiles

(Chu et al., 2003, 2006; Gazi and Passino, 2003,

2004a, 2004b; Yao et al., 2007). A difference in treat-

ment in this paper is to start from Newton’s second law

instead.

Gazi and Passino (2004a) expand an earlier pioneering

work on swarm coordination by generalizing the stability

analysis for a class of attraction/repulsion functions for

homogenous swarms. As a consequence of the homoge-

nous membership and symmetric characteristics of the

momenta, stationary and stable swarm behavior is

achieved. They also propose modifications to account for

finite body size of the swarm members. In these studies, all

members are required to know (or sense) the position of all

the others.

Chu et al. (2003, 2006) address the stability analysis of

anisotropic (asymmetric behavior), but nonhostile,

swarms. They propose some aggregation rules for

swarms with reciprocal and nonreciprocal interactions

between agents. They also point out that the swarming

behavior results from interplay between long-range

attraction and short-range repulsion among individuals.

For nonreciprocal interaction, a condition of weighted

momenta is assumed (Chu et al., 2003). In the present

paper, we extend the application of asymmetric

1 Mechanical Engineering Department, University of Connecticut, Storrs,

CT, USA2 Biomedical Engineering Program at the University of Connecticut,

Storrs, CT, USA3 Electrical and Electronic Engineering School at Universidad Industrial de

Santander, Colombia

Corresponding Author:

Nejat Olgac, Mechanical Engineering Department, University of

Connecticut, Storrs, CT, USA

Email: olgac@engr.uconn.edu

Journal of Vibration and Control

ª The Author(s) 2010Reprints and permission:sagepub.co.uk/journalsPermissions.navDOI: 10.1177/1077546309360047jvc.sagepub.com

17(5) 641–650

interaction forces to hostile swarm dynamics, which is

poorly studied in the literature.

Gazi and coworkers have also incorporated sliding

mode control and artificial potential functions to ensure

swarm aggregation even under uncertainties (Gazi,

2005; Yao et al., 2007). In Yao et al. (2007), the swarm

in a formation is guided to track a target. In these investi-

gations, the principal aim is the definition of a decentra-

lized formation control to increase robustness in the

task, taking into account the limited range and angle of

vision permitted by sensors.

Kumar and coworkers investigated the dynamic coordi-

nation of multiple robots to perform cooperative tasks

(Belta and Kumar, 2004; Chaimowicz et al., 2004). They

use a hybrid systems framework to model the cooperative

tasks and dynamic role assignment among multiple robots.

The approach coordinates the cooperative execution of the

task by considering the individual characteristics of each

one of the robots in the team. The strategy is also tested

using an experimental platform. They propose an abstrac-

tion of the configuration space to define the controllers in

a lower dimensional space. This results in improved com-

putational efficiency to command a large number of robots

(i.e., agents). This same research group has used artificial

potentials to maintain connectivity and to avoid collisions

(Zavlanos and Pappas, 2007).

Sierra et al. (2009) expand the momenta-based, homo-

genous swarm dynamics to include two hostile swarms of

pursuers and evaders, where the pursuers aim to catch the

evaders and the evaders attempt to escape. The interactions

between the pursuers and evaders, along with interactions

of alike members, make the stability analysis complicated.

To compensate for this, a Lyapunov-based dissipative con-

trol momentum is deployed when needed.

Jin and Gao (2008) expand the single integrator

model of Gazi and Passino (2004a) to a double integrator

model for homogenous swarms. They investigate the sta-

bility and swarm cohesion for several attraction/repul-

sion force functions in their second-order system. It is

shown that the members of the swarm will converge to

a bounded area and their velocities converge to a com-

mon velocity.

The main contribution of this paper is the expansion of

the works done by Sierra et al. (2009) and Jin and Gao

(2008) by addressing the stability and control of two groups

of antagonistic swarms with second-order dynamics. In

particular, we consider the pursuit of a swarm of ‘‘evaders’’

by a swarm of ‘‘pursuers,’’ an operation that includes het-

erogeneous agents and hostile interactions. Owing to the

asymmetric governing dynamics, as well as the pursuer/

evader assignment policies, the controller design and

assessment of stability become quite complex. For simpli-

city, within this study, we consider only cases with an equal

number of pursuers and evaders, all with the same mass,

and a simple assignment policy.

We begin by deploying models of the individual interac-

tions similar to those described in Jin and Gao (2008). The

major novelty, however, is the introduction of heterogene-

ity (i.e., two hostile swarms), which is described in Section

2. This section treats the antagonistic scenario in two

separate phases: approach and assigned pursuit. Control

terms are added to the pursuer dynamics to ensure that cap-

ture takes place. Section 3 outlines the Lyapunov stability

of the system and Section 4 presents some simulation

results. The conclusions and discussions suggest some

future steps in this research.

It should be noted that throughout the paper, boldface

letters represent matrices or vectors, and scalars are indi-

cated by italics.

2. Multi-agent swarm modeling

The dynamic modeling of the multi-agent swarms is pre-

sented here. We consider a two-phase approach for these

interactions. Phase 1 treats the two swarms as single agents

that are conceptually lumped at the respective swarm cen-

ters. The forces on individual agents during this phase are

assumed to be uniformly distributed. This phase brings the

two swarm centers sufficiently close to transition into

phase 2. In phase 2 (the assigned pursuit) the agent-to-

agent interaction logic enables the capture of each evader

by a pursuer. The two phases of the operation are combined

under one seamless scheme as follows:

€xi ¼ gPE �yð Þ þXNp

j¼1j6¼i

gpp xi � xj

� �þ

Trans �yk kð Þ gpe zli � xið Þ þ D _zli � _xiÞ þ K zli � xið Þ;ðð1aÞ

i ¼ 1 : Np;

€zi ¼ gEP �yð Þ þXNe

j¼1j6¼i

gee zi � zj

� �þ

Trans �yk kð ÞXNp

j¼1

gepðzi � xjÞ( )

i ¼ 1 : Ne;

ð1bÞ

where xi 2 <n, zi 2 <n are the local vectors of pursuers

and evaders in n-dimensional space. li denotes the evader

assigned to pursuer i. D _zli � _xiÞð and K zli � xið Þ are the

proposed control forces applied to the pursuer to facilitate

capture. Np and Ne represent the number of pursuers and

evaders, respectively. All agents are taken as point ele-

ments for simplicity. We will use the notation

yi ¼ zli � xi 2 <n to represent the vector connecting the

pursuer (i) to its assigned evader (li).

Journal of Vibration and Control 17(5)642

The notations gpe :ð Þ and gep :ð Þ in equations 1a and 1b

refer to the forces received by a pursuer due to its assigned

evader and the forces received by the evader due to the

surrounding pursuers, respectively. The force profiles

described below and shown in Figure 1 are biologically

inspired (Warburton and Lazarus, 1991; Camazine et al.,

2001):

gpe yð Þ ¼ qpe

Ape

Bpe þ yk kCpe

!2 <n

gep yð Þ ¼ qpe

Aep

Bep þ yk kCep

!2 <n:

ð2Þ

The unit vector qpe ¼ y= yk k gives direction to the forces.

Notice that gpe :ð Þ is an attraction force that vanishes as

yk k ! 1, as does gep :ð Þ, which is a repulsion force. This

force is bounded by the peak force, gpe

�� ��max¼ Ape

�Bpe,

which occurs at ype

�� ��max¼ 0. We define a distance, dpe,

as the influence range of a force profile where the force

is higher than 2% of the maximum value. It can be shown

that dpe ¼ 49Bpe

� � 1=Cpeð Þ. Similar analysis and definitions

are valid for gep :ð Þ and dep.

We use �x and �z for the local vectors of the centers of the

pursuer and the evader swarms, respectively, and �y is the

vector connecting them:

�x ¼ 1

Np

XNp

i¼1

xi; �z ¼ 1

Ne

XNe

i¼1

zi; �y ¼ �z� �x: ð3Þ

The phase 1 forces exerted on the pursuers and evaders are

given as gPE �yð Þ and gEP �yð Þ. These forces are the same as in

equation 2, except for different parameters. Note that the

force gPE �yð Þ is common to all pursuers since the vector �ydoes not change with respect to individual pursuers. The

same can be said for gEP �yð Þ and the evaders.

Notation gpp :ð Þ in equation 1a refers to the force

received by a pursuer due to other pursuers. Likewise,

gee :ð Þ in equation 1b represents the force received by an

evader due to other evaders. These force descriptions, pre-

sented in equations 4a and 4b, are shown in Figure 2. Note

that the parameter selection for both force profiles is iden-

tical, thus the two curves overlap. Using ui;j ¼ xi � xj

� �,

vi;j ¼ zi � zj

� �, we define the force on the ith member by

the jth member as

gppðuÞ ¼ qpp

App � Dpp uk kCpp

Bpp þ uk kCpp

!¼ qpp gpp; ð4aÞ

geeðvÞ ¼ qee

Aee � Dee vk k Cee

Bee þ vk kCee

!¼ qee gee: ð4bÞ

The unit vectors qpp ¼ u= uk k and qee ¼ v= vk k give direc-

tion to the forces. Note that these forces are characterized

as functions with bounded repulsion (in short proximity)

and constant attraction (at long distance). The maximum

repulsion occurs at upp

�� ��max¼ 0 with a value of

gpp

�� ��max¼ App

�Bpp. There is a distance, dpp, at which

gpp switches from attraction to repulsion (i.e., switches sign

or gpp

�� �� ¼ 0). This distance is defined as dpp ¼ App

��DppÞ 1=Cppð Þ. At this distance, the repulsion and attraction

forces are in balance.

The transition number, Trans dð Þ in equations 1a and 1b

is defined as

Trans dð Þ ¼ d0 þ d2d0 þ d

� �1� d � d0

d � d0k k þ d

� : ð5Þ

It represents the transition from phase 1, i.e., �yk k >> d0,

when Trans �yk kð Þ ffi 0, to phase 2, i.e., �yk k << d0, when

Trans �yk kð Þ ffi 1; d0 is the preferred distance for transition

0 5 10 15 20 250

4

8

12

||y||

For

ce

||gpe||

||gep||

Figure 1. Force profile in equation (2) for parameters in theexample section.

0 5 10 15 20 25

−0.2

0.2

0.6

1

0

For

ce

||u|| and ||v||

Attraction

Repulsion

− Dpp

, − Dee

γpp

and γee

Figure 2. Force profiles between like members for parametersin the example section (overlaid).

McCullough et al. 643

from phase 1 to 2, whereas the small positive parameter dcontrols the smoothness of this transition function (see

Figure 3); d0 is the distance where Trans d0ð Þ ffi 1=2.

The earlier formulation given in equations 1a and 1b has

three critical features:

(i) It seamlessly handles the dynamic transition between

phases 1 and 2.

(ii) Each pursuer has a relative exclusive nearest evader

(RENE) assignment, as detailed below.

(iii) There is a control term D _yi þ K yi, where D 2 <þand K 2 <þ, the selection of which is discussed

later. The function of these terms is to guarantee sta-

bility and capture of the evader by the pursuer.

The term gpe yið Þ in equation 1a, in fact, represents the

force on the ith pursuer due to the llth evader, which is

identified as the RENE for the ith pursuer. No other evader

but the RENE affects this pursuer. By contrast, all pursuers

influence all evaders as gep terms are under the summation

in equation 1b. In essence, every pursuer is engaged with

an evader called the RENE, and this assignment has

some subtleties:

� The assignment is performed on the relative positions of

the pursuers and evaders. This relative position is found

by conceptually shifting the two swarms such that the

swarm centers are coincident. RENE assignment is then

done in ascending distances (i.e., the closest relative pur-

suer–evader pair is assigned first and so on). The process

continues until all pairs are identified.

� RENE assignment is performed at all times, and sudden

changes in assignment can occur. However, during the

approach phase, the transition number is approximately

zero; therefore, the effect of the RENE assignment

(excluding the control action) is minuscule.

2.1. Phase 1: For �yk k >> d0, Trans �yk kð Þ ffi 0

If the centers of the swarms are separated by a distance

much greater than d0, �yk k >> d0, the resulting dynamics

leads to the merging of the two swarms. This is achieved

by treating the two swarms as two agents who are at the

center of each swarm. The forces gPE �yð Þ and gEP �yð Þ are

evenly distributed to the pursuers and evaders, respectively.

By taking Trans �yk kð Þ ffi 0, the dynamics in equations 1a

and 1b become

€�x ¼ 1Np

XNp

i¼1

gPE �yð Þ þXNp

j¼1j6¼i

gpp xi � xj

� �þ D _yi þ Kyi

0BB@

1CCA

¼ gPE �yð Þ þ D

Np

XNp

i¼1

_yi þK

Np

XNp

i¼1

yi;

ð6aÞ

€�z ¼ 1

Ne

XNe

i¼1

gEP �yð Þ þXNe

j¼1j 6¼i

gee zi � zj

� �0BB@

1CCA ¼ gEP �yð Þ:

ð6bÞ

The merging of the two swarms (i.e., phase 1) occurs in a

bounded approach time, tapp; furthermore, tapp strongly

depends on the relative initial positioning of the two

swarms with respect to each other. If the initial distance

between the two swarms is more than the maximum

recognition distance or the range of influence of the pur-

suers, which is depicted previously as dPE, the governing

forces are predominately the control forces on the pursuers

and the like member interaction forces gpp :ð Þ and gee :ð Þ,(i.e., gPE :ð Þk k � gEP :ð Þk k � 0). In this phase, if the

approach time is larger than each of the homogeneous

swarm convergence times, the area occupied by the evader

swarm will be dictated only by the nature of the interaction

forces gee :ð Þ. This area and time are evaluated in Jin and

Gao (2008), but only for homogenous swarms.

2.2. Phase 2: For �yk k << d0, Trans �yk kð Þ ffi 1

Once the two swarm centers are closer than the distance d0,

phase 2 (assigned pursuit) becomes dominant since �yk k is

diminishing. The property �yk k � d0 is enforced via the for-

mation of the forces gPE �yð Þ, gEP �yð Þ, and the control forces.

Any departure away from this region (i.e., creating

�yk k > d0) forces Trans �yk kð Þ to be very small in equations

1a and 1b, which, in turn, invites phase 1 dynamics (equa-

tions 6a and 6b) and forces �yk k � d0. Therefore, it is clear

that �yk k � d0 is stable and attainable. In phase 2 (when

Trans �yk kð Þ is closer to 1), the attraction force on a pursuer

by its assigned evader becomes more relevant. Similarly,

the evaders are repelled by all the pursuers in equation lb.

0 2 4 6 80

0.2

0.4

0.6

0.8

1

distance(d)

Tra

ns(d

)

d0

δ=0.1

δ=0.5

Figure 3. Function Trans(d), d0 ¼ 3.

Journal of Vibration and Control 17(5)644

With this pursuer–evader pairing we study the stability of

the dynamics next.

3. Lyapunov-based stability analysis

3.1. Stability analysis

In this section we analyze the stability of the system given

in equation 1 using LaSalle’s theorem (LaSalle, 1960).

Main Lemma: Consider the governing dynamics of a

particular pursuer and assigned evader pair, from equa-

tions 1a and 1b as

€yi ¼ f iðx; zÞ � D _yi � Kyi 2 <n; ð7Þ

where f iðx; zÞ contains all the self-evident interaction

forces from equations (1a) and (1b). It can be shown that

the pursuer–evader swarms approach a bounded state of

aggregation described by εk k � c ; c > 0, where

εi ¼yi

_yi

� �2 <2n; i ¼1 : Np; and ε ¼ col εif g2<2nNp:

Proof: Equation 7 leads to the individual state equations

_εi ¼_yi

€yi

� �¼

0 1

�K �D

�� In

� �yi

_yi

� �þ

0

f iðx; zÞ

� �¼ F� Inð Þ εi þ ~f i ¼ �Fεi þ ~f i 2 <2n i ¼ 1 : Np;

ð8Þ

where �F ¼ F� In 2 <2n�2n, the F matrix is self-evident,

In is an identity matrix with dimension n, and � denotes

Kronecker multiplication. Here, ~f i is the combination of

nonlinear interaction forces between pursuers and evaders

and alike agents, ~f i ¼ 0T f iðx; zÞT�

T 2 <2n with

f i x; zð Þ 2 <n. These forces are all bounded functions as

further analyzed later.

The combined system dynamics can be written in the

following form:

_ε ¼

_y1

€y1

..

.

_yNp

€yNp

0BBBBBB@

1CCCCCCA¼ Fð Þ

y1

_y1

..

.

yNp

_yNp

0BBBBBB@

1CCCCCCAþ

0

f1ðx; zÞ...

0

fNpðx; zÞ

0BBBBBB@

1CCCCCCA

¼ Fεþ f 2 <2nNp ;

ð9Þ

where F ¼ INp � �F 2 <2nNp�2nNp and f ¼ col ~f i

� �2 <2nNp i ¼ 1 : Np .

The Kronecker multiplication of two square matrices

M1 ðn�nÞ and M2 ðm�mÞ, M12 ¼M1 � M2 2 <nm� nm has a

critical property: the mn eigenvalues of M12 are formed

as dual multiplications of those eigenvalues of M1 and

M2, or in mathematical formalism lM12;ij ¼ lM1;i lM2;j

i ¼ 1:: n; j ¼ 1::m (Theorem 13.12 of Laub, 2005).

Furthermore, if one of these matrices, say M1, is an identity

matrix, the eigenvalues of M12 will be n repetition of the m

eigenvalues of M2, i.e., n-tupled lM2;1; :::lM2;m

� �. In short,

Kronecker multiplication Iðn�nÞ �M2 would repeat the

eigenvalues of M2 n times. We utilize this property twice

for F ¼ INp� ðF� In Þ 2 <2nNp � 2nNp in order to

determine the eigenvalues of F as l Fð Þf g ¼ Np-tupled

l �Fð Þf g ¼ nNp-tupled l Fð Þf g where Np-tupled f g implies

that the set f g is repeated Np times. From equations 8

and 9, the following features of the eigenvalues can be

stated:

l Fð Þf g ¼ l : l2 þ Dlþ K ¼ 0� �

2 C2;

l �Fð Þf g 2 C2n; l Fð Þf g 2 C2nNp:

Clearly for D 2 <þ and K 2 <þ, < l Fð Þð Þ < 0, thus they

represent a stable system.

Keeping the above discussions in mind, we propose a

candidate Lyapunov function to be

V ¼XNp

i¼1

Vi ¼XNp

i¼1

εTi P� Inð Þ εi ¼

XNp

i

εTi

�Pεi¼ εTPε:

ð10ÞP and �P are positive definite symmetric (PDS) matrices,

where �P ¼ P� In and P ¼ INp � �P. P is the solution to

the Lyapunov equation

FTPþ PF ¼ �Q 2 <2�2; ð11Þ

where Q is a PDS real matrix. Q and �Q also have similar

definitions to P and P. Using the features of the Kronecker

product, we expand the F;P , and Q matrices to handle

multiple pursuers in n-dimensional space without the added

complication of very large matrices. It can be shown that

the analysis of the dynamics and stability assessment can

be reduced to studying only the P and Q matrices instead

of the higher dimensional P; �P and Q; �Q.

Taking the derivative of equation 10, and knowing that

Q and �Q are positive definite and symmetric,

_V ¼XNp

i¼1

_Vi ¼XNp

i¼1

�_εT

i�Pε

iþ εT

i�P _ε�¼ _εT

Pεþ εTP _ε

¼�

Fεþ f�T

Pεþ εTPðFεþ fÞ

¼ �εTQεþ 2εTPf :

ð12Þ

Using the features of a PDS matrix A,

xTA y�� �� � lmax Að Þ xk k yk k; ð13aÞ

lmin Að Þ xk k2� xTAx � lmax Að Þ xk k2: ð13bÞ

Equation 12 can be reduced to

_V � �lmin Qð Þ εk k2þ2 εk k fk kmax lmax Pð Þ: ð14Þ

McCullough et al. 645

To guarantee _V � 0, the following condition must be met:

εk k � 2lmax Pð Þlmin Qð Þ fk kmax¼ 2lP=Q fk kmax ð15Þ

where

lP=Q ¼lmax Pð Þlmin Qð Þ ¼

lmax Pð Þlmin Qð Þ ¼ lP=Q;

This is true because the eigenvalues for P are the

n Np-tupled eigenvalues of P, which is also true for Q and

Q. This simplification reduces equation 15 to the following

definition of c:

εk k � 2lmax Pð Þlmin Qð Þ fk kmax¼ 2lP=Q fk kmax ¼definition c 2 <;

ð16Þ

which provides the upper bound of εk k as a hypersphere.

The dynamics given by equation 9 are guaranteed to be

entrapped within this hypersphere. This bound, however,

depends on the most pessimistic value of fk kmax, which

is studied later in the paper. The obvious aim in the follow-

ing segment is to minimize this upper bound by properly

selecting the Q matrix.

3.2. Selection of Q matrix

Using P ¼ p11 p12

p12 p22

�and the F matrix defined in equa-

tions 8 and 11 yields

Q ¼2Kp12 �p11 þ Dp12 þ Kp22

�p11 þ Dp12 þ Kp22 �2p12 þ 2Dp22

�

¼q11 q12

q12 q22

�:

The Q matrix is our selection and q11; q22 2 <þ. Back sol-

ving for the elements of P, one finds

P ¼Dq11

2K� q12 þ Kq22 þ q11

2D

q11

2K

q11

2K

Kq22 þ q11

2DK

24

35: ð17Þ

To analyze the effect that the values of the Q matrix have

on the ratio of eigenvalues, lP=Q, we introduce two con-

stants, c1 and c2, in the following manner:

Q ¼ q11 q12

q12 q22

�¼ q11 q11c1

q11c1 q11c2

�¼ q11

1 c1

c1 c2

�:

ð18Þ

For Q to be positive definite and symmetric, c21 < c2 and

c2 > 0 in addition to q11 > 0. Rewriting equation 17 using

the new values from equation 18, we redefine P as

P ¼ q11

1 þ Kc2

2D� c1 þ D

2K1

2K

12K

Kc2 þ 12DK

24

35: ð19Þ

The objective is to minimize c of equation 16, which

brings us to minimize lP=Q. It is clear that lP=Q is a function

of c1, c2, K, and D. It is independent of q11 due to the lin-

earity of equation 11. Therefore, the selection of q11 is

completely arbitrary and has no influence on the dynamics.

To minimize lP=Q we utilize a MATLAB routine called

‘‘fmincon.’’ It performs a Sequential Quadratic Program-

ming (SQP) procedure as detailed in The MathWorks Inc.

(2009). This procedure results in optimum values for

c1 ¼ 0; c2 ¼ 1, and K ¼ D!1. An indefinite increase

of Kand D results in increased control efforts and diminish-

ing benefits in the minimization of c. Therefore, the user

should select Kand D to resolve the trade-off between the

control effort and the bound of c.

Using the above numerical procedure, we demonstrate

that the lowest possible bound for lP=Q is 1. The Q matrix

is defined in order to determine P in equation 19 and the

bound of εk k in equation 16.

3.3. Calculation of maximum force

From the dynamics

f i x; zð Þ ¼ gEP �yð Þ � gPE �yð Þ þXNe

j¼1j6¼li

gee zli � zj

� �

�XNp

j¼1j 6¼i

gpp xi � xj

� �

þ Trans �yk kð ÞXNp

j¼1

gep zli � xj

� �� gpe yið Þ

" #:

Notice that these forces can be categorized into three types:

forces that are dominant in phase 1, phase 2, and like-agent

interactions (friendly forces):

Phase 1 ¼ gEP �yð Þ � gPE �yð Þ;

Friendly ¼XNe

j¼1j 6¼li

gee zli � zj

� ��XNp

j¼1j 6¼i

gpp xi � xj

� �;

Phase 2 ¼ Trans �yk kð ÞXNp

j¼1

gep zli � xj

� �� gpe yið Þ

" #:

We want a pessimistic maximum value for f ik k in order to

determine our lower bound of εk k in equation 16. That is,

f ik kmax � f i x; zð Þk k i ¼ 1 : Np:

Journal of Vibration and Control 17(5)646

To do this, we consider the worst possible scenario of the

vectorial addition of forces. Keeping in mind the general

force profile depicted in Figure 1, and all other pessimistic

compositions, the following bounds are reached:

Phase 1max ¼ gPEk kmax � gEPk kmax;

Friendlymax ¼ Np � 1� �

gpp

�� ��maxþ Ne � 1ð Þ geek kmax;

Phase 2max ¼ Np � 1� �

gep

�� ��max;

which results in a conservative value for the maximum

force, f ik kmax. Note that the gpe

�� ��max

term is assisting cap-

ture, thus is not included in phase 2. Recognizing that the

transition number is also a factor of the phase 2 forces, the

upper bound of this force can be written as follows:

f ik kmax ¼ gPEk kmax� gEPk kmaxþ Np � 1� �

gpp

�� ��max

þ Ne � 1ð Þ geek kmax

þ Trans �yk kð Þ Np � 1� �

gep

�� ��max:

ð20Þ

Since the transition number varies with time, f ik kmax also

varies with time. Knowing the value of f ik kmax, we can cal-

culate fk kmax for use in equation 16 as the Euclidean norm

of f:

fk kmax¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffif 2

1maxþ f 2

2maxþ þ f 2

Npmax

q¼

ffiffiffiffiffiffiNp

pf ik kmax:

ð21ÞThe ultimate objective of the control is the capture of the

evader by the pursuers. In order to measure the success

of the pursuers, capture and the time of capture, tcapture,

must also be defined. Logically, capture is achieved when

the relative positions and velocities of a pursuer–evader

pair approach zero. In order to use a comparable quantity

among different cases when deciding if capture is com-

pleted, a fraction of the value of c in pure phase 1 is set

as the threshold for capture. e.g.,

ε tcapture

� ��� �� < 0:02 c Trans �yk kð Þ¼0

�� ¼ 0:02 c0: ð22Þ

Once εk k is below the value of 0:02 c0, capture is declared.

Another aspect to consider in this pursuit is the amount

of control energy needed to stabilize the pursuers. This

control energy is directly associated with the selection of

K and D, as these parameters will introduce control forces

on the pursuers. In order to establish a scalar comparison

basis, we evaluate the control energy:

Econtrol ¼XNp

i¼1

ðtcapture

0

D _yi þ Kyið Þ � _xið Þ dt

0@

1A ð23Þ

This control energy represents the intensity of the control

effort throughout the pursuit. It is useful in selecting the

values of K and D. Of course, there is always a balance

between performance and minimizing the control effort.

To select the parameters of K and D, several simulations

were executed and the control energy and the performance

were compared.

4. Simulation results and performancetests

In this section we present two case studies that display

interesting properties of the proposed swarm control logic.

The first case is a 2 pursuers–2 evaders (2P–2E) scenario

using the parameters given in Table 1 and random initial

conditions. The values in the table are computed based

on the desired maximum force and influence range, d,

vis-a-vis Figures 1 and 2.

The second case is a 6P–6E scenario with randomly

selected initial swarm settings and the force profiles again

using the parameters listed in Table 1. The matrix Q was

selected for both cases:

Q ¼ 2KD 0

0 2KD

�:

From equation 11, this Q matrix yields the following P

matrix:

P ¼ K2 þ D2 þ K D

D K þ 1

�;

with K ¼ D ¼ 5, lP=Q ¼ 1:11:The selection of Q is consistent with the criteria pre-

sented in Section 3.2 and K and D were selected taking the

performance of several test cases and the associated control

energy into consideration.

Figures 4a and 5a show the Lyapunov function varia-

tions for the 2P–2E scenario and the 6P–6E scenario,

respectively. The black dots in Figure 5a represent

moments when a change in the RENE assignment is made.

The vertical dotted line in Figures 4a–d indicates the time

of capture, tcapture, which was defined previously in equa-

tion 22. Capture for the 2P–2E scenario is declared as

tcapture ¼ 4:13 s and for the 6P–6E scenario as

tcapture ¼ 3:81 s.

Figures 4b and 5b present violation flags to denote sam-

ple times at which violations to the negativity of the Lyapu-

nov candidate’s derivative are found, i.e., _V > 0. Red flags

Table 1. Parametric properties of the case studies.

Aee ¼ 23:4 Ape ¼ 171:3 APE ¼ 3608Bee ¼ 20:7 Bpe ¼ 13:8 BPE ¼ 180:3Cee ¼ 3 Cpe ¼ 3 CPE ¼ 3Dee ¼ 0:03 Aep ¼ 108:7 AEP ¼ 2623App ¼ 23:4 Bep ¼ 18:8 BEP ¼ 252:2Bpp ¼ 20:7 Cep ¼ 3 CEP ¼ 3Cpp ¼ 3 d ¼ 0:75 D ¼ K ¼ 5Dpp ¼ 0:03 d0 ¼ 3

McCullough et al. 647

indicate violations before capture takes place and the blue

dots indicate violations after capture is declared. Notice

that a single time-step violation of _V < 0 occurs at one

of the changes in the RENE assignment at t ¼ 0:05 s. This

violation can be observed by the single violation flag in

Figure 5b at this time. The other changes in the RENE

assignment do not result in _V > 0, although this cannot

always be guaranteed. Figures 4c and 5c present the time

history of the transition number, Trans �yk kð Þ, signaling how

fast the swarms merge in phase 1.

Figures 4d and 5d show the time evolution of the

magnitude of the error vector εk k and the time evolution

of the stability bound c for both sample cases. Notice how

c varies with time as it is dependent on the transition num-

ber. Lyapunov stability is guaranteed as long as the error

vector is larger than that of the stability bound. Once εk kis within the bound of c, _V < 0 can no longer be assured.

It is clear from the violation flags in Figures 4b and 5b that

violations of _V < 0 do not begin until well after εk k < c.

An exception appears as a single time step violation due

to the change in RENE at t ¼ 0:05 s. The violations after

εk k < c, however, are very small in comparison to the

scale of the Lyapunov function. Both scenarios result in suc-

cessful captures of the evaders by the pursuers, as marked by

the dashed line in Figures 4 and 5. One can observe that most

of the _V < 0 violations occur after the capture is declared

and these violations of the _V < 0 condition is caused only

by residual and ignorable oscillatory motion in pursuer and

evaders, which yield only minor increases in V .

Figures 6 and 7 present the trajectories of the pursuit

(black dotted line for pursuers and red for evaders) and the

final locations for the pursuers (black squares) and evaders

(red circles) for the 2P–2E scenario and the 6P–6E sce-

nario, respectively. The solid lines represent the motion

of the centers of each swarm (blue for pursuers and red for

evaders). It is clear that the pursuers capture the evaders

successfully as is evident by their positions being coinci-

dent. What cannot be observed in the figures is that the final

−50 0 50 100

−80

−60

−40

−20

0

20

40

X

Y

Figure 6. Traces of the 2P–2E sample case (red circles; evaders,black squares: pursuers).

0 2 4 60

5

10x 10 5

V

A

0 2 4 60.8

1

Vio

latio

n F

lags

B

0 2 4 60

0.5

1

Tra

ns #

C

0 2 4 60

100

200||ε||

ψ

time (s)

|| ε

|| an

d ψ

D

Figure 5. Simulation results for 6P–6E.

0 2 4 60

1

2x 10 6

V

A

0 2 4 60.8

1

Vio

latio

n F

lags

B

0 2 4 60

0.5

1

Tra

ns #

C

0 2 4 60

100|| ε ||

ψ

time (s)

||ε ||

and

ψ

D

Figure 4. Simulation results for 2P–2E.

Journal of Vibration and Control 17(5)648

velocities of each pursuer match that of its RENE, which is

another criterion for successful capture.

To illustrate the effect of the uncontrolled dynamics

(i.e., K ¼ D ¼ 0), a 2P–2E example case is studied. The

parameters used are identical to the previous 2P–2E case.

Figure 8 shows the variation of the transition number with

time. As is obvious from the figure, the dynamics never

reach phase 2 for more than an instant, thus undermining

the effect of the RENE on the pursuer. Without a dissipa-

tive term, there is nothing to slow the pursuers swarm as

it approaches the evaders. In fact, without the control terms

D and K, the system never reaches a sustained phase 2;

instead, the system oscillates in phase 1 as the dynamics

attempt to bring together the centers of the swarm. Capture

will not occur with these dynamics and the pursuers’ swarm

center will continually oscillate about the evaders’ center.

5. Discussion and conclusions

In this study we present a control strategy involving two

swarms that are in conflict. The membership counts of the

swarms are taken as equal for simplicity. The alike and

non-alike swarm members interact under some nonlinear

forces. The strategy has two phases: phase 1, called the

approach phase, brings the center of the swarms closer, and

phase 2, the assigned pursuit phase, enables a pursuit based

on an assignment strategy.

Control rules are based on the relative position and

velocity of the pursuer and evader. Lyapunov analysis

shows that the system is guaranteed to be stable. The pro-

posed Lyapunov candidate behaves properly until the

error vector enters a bounded region. Despite small

increases in the Lyapunov candidate within this region,

capture occurs due to the control forces. Another source

of violations of the Lyapunov function, which may be

nonnegligible, are those excursions appearing due to the

reassignment decisions of the RENE. This behavior does

not disrupt the deployment of Lyapunov stability in

piece-wise manner.

Current efforts look for other strategies to establish sta-

bility until capture is reached. Another associated area for

future research is the design and deployment of control

strategies for antagonistic swarms with second-order

dynamics and unequal membership strengths.

Acknowledgements

The authors wish to thank Prof. Eldridge Adams of the University

of Connecticut for his insightful suggestions for biologically

inspired force profiles. This work was supported in part by ARO

W911NF-07-1-0557 and DHS 2008-ST-061-TS0002.

References

Belta, C. and Kumar, V., 2004, ‘‘Abstraction and control for

groups of robots,’’ IEEE Transactions on Robotics 20(5),

865–875.

Camazine, S., Deneubourg, J., and Franks, N. R., 2001, Self-

Organization in Biological Systems, Princeton University

Press, Princeton.

Chaimowicz, L., Kumar, V., and Campos, M. F. M., 2004, ‘‘A

paradigm for dynamic coordination of multiple robots,’’

Autonomous Robots 17(1), 7–21.

Chu, T., Wang, L., and Chen, T., 2003, ‘‘Self-organized motion in

anisotropic swarms,’’ Journal of Control Theory and Applica-

tions 1(1), 77–81.

Chu, T., Wang, L., and Chen, T., 2006, ‘‘Complex emergent

dynamics of anisotropic swarms: convergence vs oscillation,’’

Chaos, Solitons & Fractals 30(4), 875–885.

−40 −20 0 20 40 60

−40

−30

−20

−10

0

10

20

30

40

50

60

X

Y

Figure 7. Traces of the 6P–6E sample case (red circles; evaders,black squares: pursuers).

0 50 100 150 2000

0.2

0.4

0.6

0.8

1

Tra

ns #

time(s)

Figure 8. Transition number with K ¼ 0;D ¼ 0.

McCullough et al. 649

Gazi, V., 2005, ‘‘Swarm aggregations using artificial potentials

and sliding-mode control,’’ IEEE Transactions on Robotics

21(6), 1208–1214.

Gazi, V. and Passino, K. M., 2003, ‘‘Stability analysis of swarms,’’

IEEE Transactions on Automatic Control 48(4), 692–697.

Gazi, V. and Passino, K. M., 2004a, ‘‘A class of attractions/repul-

sion functions for stable swarm aggregations,’’ International

Journal of Control 77(18), 1567–1579.

Gazi, V. and Passino, K. M., 2004b, ‘‘Stability analysis of social

foraging swarms,’’ IEEE Transactions on Systems, Man, and

Cybernetics, Part B: Cybernetics 34(1), 539–557.

Jin, D. and Gao, L., 2008, ‘‘Stability analysis of a double integra-

tor swarm model related to position and velocity,’’ Transac-

tions of the Institute of Measurement and Control 30(3-4),

275–293.

LaSalle, J. P., 1960, ‘‘Some extensions of Liapunov’s second

method,’’ IRE Transactions on Circuit Theory 7(4), 520–527.

Laub, A. J., 2005, Matrix Analysis for Scientists and Engineers,

SIAM, Philadelphia, PA.

The MathWorks Inc. ‘‘Optimization Toolbox: fmincon’’ The

MathWorks, Inc. 1994-2009. [Online]. Available: http://

ww.mathworks.com. [Accessed: March 20, 2009].

Sierra, D., McCullough, P., Olgac, N., and Adams, E. 2009,

‘‘Swarm coordination under conflict,’’ in Proceedings of the

American Control Conference, St Louis, MO, pp. 1021–1026.

Warburton, K. and Lazarus, J., 1991, ‘‘Tendency-distance models

of social cohesion in animal groups,’’ Journal of Theoretical

Biology 150(4), 473–488.

Yao, J., Ordonez, R., and Gazi, V., 2007, ‘‘Swarm tracking using

artificial potentials and sliding mode control,’’ ASME Journal of

Dynamic Systems, Measurement and Control, 129(5), 749–754.

Zavlanos, M. M. and Pappas, G. J., 2007, ‘‘Potential fields for

maintaining connectivity of mobile networks,’’ IEEE Transac-

tions on Robotics 23(4), 812–816.

Journal of Vibration and Control 17(5)650