Copyright 2003 IEEE.

Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

SMCid03 IEEE International Workshop on Soft Computing in Industrial Applicalions Bingharnton University, Bingharnton, New York. lune 23-25,2003

Foraging in Ant Colonies applied to the Mine Detection Problem Vignesh Kumar, Member IEEE, Dr. Ferat Sahin, Member IEEE,

Multi Agent Bio-Robotics Lab, Department of Electrical engineering,

Rochester Institute of Technology, New York 14623, USA

Phone: + I 585 475 5238, E-mail: vmk7486@rit.edu

Abstract - This paper focuses on the application of swarm intelligence to the problem of mine detection. Swarm Intelligence concepts have captivated the interests of researchers mainly in collective robotics, optimization problems, and communication networks (routing)[l]. In the mine detection problem we are faced with sub problems such as searching for the mines over the minefield, defusinx them effectively and assuring that the field is clear of mines within the least possible time. In this paper, we study the ant colony system applied to the mine detection problem We highligh t certain formulation that the ants (the agents in our case involved in mine detection) are given for executing B e task effectively. Ileuristics based on cognitive maps in ants are used to aclrieve Iliis. The agents do the task effectively when they are able to assure that the minefield 13 clear of the mines within the least possible time.

I . INTRODUCTION

Heuristics arising from natural phenomenon has always captivated mankind and have been an important area of research. Many optimization problems have been characterized and solved using non-deterministic heuristics. Heuristics can be seen arising from nature in many forms. When a group of simple and similar individuals, called agenfs, interact with each other to perform a highly specialized task, heuristic methods are highlighted [I ] . Such observations are often encountered in swarms, which can a group of insects, herds and ant colonies. Adapfivify (selections of the best) and mutation (random fluctuations) are two features prominently seen in natural systems [ I , 21. Optimization is taken care by selectivity and non-determinism is a result of randomness in the system. Some of the features seen in heuristics in nature are non-determinism, multiple interactions, and evolution [I, 21.

Ant colonies are found to be a strong member of the swarm family. Strong colony level coordination and coexistence are seen in ant colonies. Some of the well- established features in ant colonies are self-organization, emergent behavior and stigmergy [ I , 21. Sev- organization of a system is the ability of the system to suit itself to the changing external and environmental conditions by interactions of positive and negative feedback [ 1 - 41. Emergenf behavior is a result of multiple interactions, which occurs when a group of agents with a set of basic, simple rules interact among themselves to produce complex patterns [ I , 21. Sfigmergy is the

phenomenon that produces actions and reactions without direct mutual interactions. The environment is generally where information is embedded and retrieved to cause actions and reactions. .

In this paper we have tried to model ant colony based heuristics to the mine detection problem. Ants being a complete member of the swarm family exhibit natural heuristics in encountering and solving complex problems. An ant colony is constantly faced with highly complex tasks such as food gathering, nest building, brood sorting, and colony level defense. In executing these tasks they resort to natural heuristics. In this paper, we model the mine detection problem with ant colony heuristics based on food gathering. Researches have shown that ants forage the terrain near to the nest for food [ I , 21. They devise their strategy based on information obtained from cognitive maps [5, 61. Cognitive maps are orientation diagrams of important landmarks on the terrain. Based on this they utilize certain heuristic methods to devise their foraging strategies. Foraging is defined to be the search of a piece of terrain to find food or other objects of interest utilizing minimal resources [ I , 5, 61. In this work, various foraging strategies adopted by the ants to collectively demine a minefield are presented. Section I I describes the ant colonies and their collective intelligence manifestations. Section Ill highlights the foraging techniques found in natural ant colonies. The later section tabulates and analyzes the results obtained during simulation of the mine detection task with various foraging strategies.

11. ANT COLONIES

Social insects are a class of insects that live in colonies and are behaviorally tuned to execute colony level activities by manifesting individual actions or fasks [ I , 21. Ant colonies have been in active research because of their highly structural organization in spite of their simplicity. Many of their routine activities such as food gathering, brood sorting, division of labor, nest building have been an area of active research. Some important features such as self-organization, emergent behavior, and stigmergy are explicit when these ant colonies are analyzed closely [ 1, 21. In self-organizing systems, behavioral patterns emerge from the interactions of the individual entities of the colony without any influence from external sources.

61

Authorized licensed use limited to: Rochester Institute of Technology. Downloaded on January 13, 2009 at 14:51 from IEEE Xplore. Restrictions apply.

The behavioral rules for the individuals in the colony are defined only by the interacting individuals and not by any central referee. A result of self-organizing individuals producing multiple interactions is emergent behavior. For analyzing emergent behaviors within a colony a clear picture of the multiple interactions in addition to the rules defined in individual interactions is simultaneously required. Examples of emergent behavior in ant colonies can be seen in colony level defense mechanisms in army ants and nest building [ I , 21.

111. FORAGING IN ANT COLONIES

Foraging is a main attribute that is seen in most of the ant colonies. Foraging is generally carried out in ant colonies out in search for food or to detect other predators straying into their terrain [ I , 4, 5, 61. The foremost goal of the foraging ants is to spend the least amount of energy to collect maximum food for the colony. Sometimes foraging is employed in ant colonies for inspecting their territory to prevent predators from entering. They are constantly under attack by predators and competitors. In both cases of food gathering and foraging for defense, the act of foraging will be successful when the maximum area is foraged with the minimum amount of time and effort. This mainly depends on the number of foragers and territory foraged. Studies conducted in aspects of division of labor, in ant colonies show that the ants are able to bring up the optimal number of workers required for a task, thus dynamically balancing the labor force for the colonies routine needs {I].

Foraging is generally followed by the act of recruiting partners or workers. In species of Novomessor (Novomessor albisetosis and Novomessor cockerelli), Holldobler shows that this takes place largely by two mechanisms [2]. They are Short-Range Recruitment (SRR) and Long-Range Recruitment (LRR) [ l , 21. In short range recruitment a scout releases a poisonous secretion over a spatial surround, which can attract the nestmates nearby. This happens when an ant senses a large probability of finding a food particle or another object of interest. If Short-Range Recruitment does not attract enough nestmates, Long-Range Recruitment is employed. In this case, the ant lays a chemical trail with a poison gland secretion from the prey to the nest; nestmates leave the nest and follow the trail toward the prey. The transition to Long-Range' Recruitment from Short-Range Recruitment is caused by factors such as traction offered by the pray or other objects of interest to motion, presence of refugee or worker ants to aid in the task or the colonies global necessities.

The use of cognitive maps in ant foraging is reported in literature [S, 61. Many animals use sophisticated maps in executing foraging tasks and navigation. Bees and wasps tend to easily locate their nests when they are close enough to it with the help of cognitive maps. Ants tend to

use cognitive maps in locating their nests back once a food particle is located. The use of land marks or visual cues for building the cognitive maps in ants for foraging activities is also reported in literature [5,6].

IV. THE MINE DETECTION PROBLEM

In the mine detection problem, we are faced with the problem of detecting and fusing randomly placed mines over a field [7, 8, 91. We assume that the agents, which are deployed to fuse the mines, are equipped with the ability to detect the mine when they approach them physically. The defusing of the mines demands the collective action of a certain number of agents. For our simulations we have assumed this to be four, though other combinations can also be solicited as per the applications demand. The main goal of the work is to eliminate ?.lie central intelligence in driving the act for the completion of the task. We have applied ant colony based techniques to the problem. The agents are considered to be individual ants performing the task of pray detection and retrieval. In the process of detection the ants move from the nest and perform a random foraging of a given area. The foragers move in the random combined heuristic fashion while detecting the mines over the field. We have modeled the problem in accordance to that of a collective transport problem found in ant colonies [ I , 21. We assume that the mines are tantamount to the prey that the ants have to detect over a field. We give the ants a foraging strategy in order to scout for the mines. When an ant reaches a mine it spreads a scent around the mine in resemblance to Short-Range Recruitment found in ant colonies. The exponential fall of the scent spread about the food particle (which has the maximum scent concentration) makes it easy for the ants to follow the increasing gradient of the scent concentration and land themselves on to the mines.

There are three distinct behaviors exhibited by the agents during the process of mine detection. The first state is foraging, During foraging the agents have the task of detecting the mines. We employ the foraging strategy discussed later for the ants during this behavior. If the ant finds a mine during its foraging it changes its behavior to waiting. Waiting is basically an idle period during which an ant may initiate a scent spread across the mine. The scent has the property that it attenuates in concentration exponentially with distance. This property is helpful for other ants to follow the increasing concentration and land themselves at the mine, This act of following is another behavior called hail folfowing. It is important that the ants do not forgo the foraging behavior even when they are in the trail following state. A serious problem that arises by the spatially additive scent concentration is that there are chances of scent spreads to overlap [4]. Overlapping of scent spreads could be catastrophic as it can destroy the increasing gradient property of the scent spread: When scent spreads overlap there are chances of overlaps, which can contain local maximas. Since the navigational properties of the ants in the trail following

62

Authorized licensed use limited to: Rochester Institute of Technology. Downloaded on January 13, 2009 at 14:51 from IEEE Xplore. Restrictions apply.

state assume that any maximum in the region is a mine location (the mine location is the global maxima with the highest scent concentration), the local maximas can be delusive. The chances of false alarms are high in the overlap regions. A study from our simulation shows that the ants could be prevented from entering any overlapped region when the exponent of the falling scent distribution is kept at 0.5 [4].

This scent decreases exponentially in a region around it. The spread of the scent is equivalent to physical stigmergy, which is basically physical changes produced on the environment so as to bring about a desired reaction, in this case attraction of other ants around the mine to the mine. The ants around the mine follow the scent's increasing gradient and finally land themselves at the mine. The following of scent is deterministic and no random movements are found inside the area covered with the scent. Thus the mine would be eventually defused when the required number of ants arrives at the mine's location. The problem is modeled such that the attraction of the required number of ants is prefixed and not at the location of the mines itself, so the aspects of deciding on the number of agents to defuse a particular mine and transportation of it, is not important. In our simulation the number of ants required to demine a single mine was fixed at four.

The foraging strategy for the ants is basically given by in accordance with the cognitive maps. Cognitive maps are mental representations of distinctive regions called cognitive regions on the minefield, which may or may not be mutually exclusive [5 ] . Figure 1 shows a field divided into distinctive cognitive regions. The regions A, B, C, and D shown in Figure 1 form the four cognitive regions. Each cognitive region can be divided into sub regions called cognitive sub-regions in the field. The ants are given the capacity of cognitive memory. The cognitive memory in the ants is the ability to retain information of mine locations with respect to the cognitive maps. Utilization of the information of the mine locations in devising foraging routes results in lesser time for the completion of the mine detection task. Navigation of the ants over the field in the case of foraging is basically roaming the cognitive regions. The processing of the information present in the cognitive maps controls the sub regions that they visit during navigation. In the case of mine detection this processing should be aimed at optimizing the route so as to visit routes where the likelihood of finding mines are larger.

For our simulations we have employed a probabilistic cost function, shown in equations 1 and 2, to maximize the chances of mine detection in the route followed by an ant in its navigational process between two random cognitive regions. In Figure the route shown is from cognitive region C to region D. We have assumed the mine distribution over the field to be uniform. Other distributions are also possible if one knows prior information about the mine concentrations over the field.

The cost function can be modeled accordingly for any prior known mine distribution.

Figure 1. A minefield divided into cognitive regions and sub regions

Let, aid, azd, ... and and qU*, u l U , ... a,,, be the knowledge of the defused and un-defused mines in regions A , , A , , ... , A,, in cognitive region A on the field shown in Figure 1, at any time instant during foraging. Similarly let qd, b2 , ... bnd and 4,. , 9,. ... b,,, be the knowledge of the defused and un-defused mines in regions B,, B,, ... , B, respectively. The number of distinctive sub regions in a cognitive region is n . An ant would tend to select a new course according to the following probability distributions in equations 1 and 2. An ant moving from point PI to point Pz in the field is shown in Figure 1. Let us assume the coordinates of points P and P, with respect to a certain boundary (in our simulations the origin was at the lower left comer) is (x,,y, ) and (yl, y , ) respectively. The probability distribution function of the ant choosing its destination cognitive sub region among various cognitive sub regions in the destination cognitive region is modeled as given in the following equations.

From Figure 1 we understand that an ant passing from point P, , which is in cognitive region C , would like to arrive at the destination point Y , in cognitive region D ,

63

Authorized licensed use limited to: Rochester Institute of Technology. Downloaded on January 13, 2009 at 14:51 from IEEE Xplore. Restrictions apply.

maximizing its chances of detecting mines in cognitive region A during its forward-path from cognitive region D to cognitive region D. During its return path from cognitive region D to cognitive region C , through cognitive region D , the chances of detecting mines should also be maximized. This is achieved by the ant choosing the coordinates of the destination point Pz , in the cognitive sub region of D , which has the maximum probability in the probability density functions given in equations 1 and 2.

The factor f i in equations 1 and 2 give the degree of relative importance given to the information gleaned during previous foraging raids. As we can see form equations 1 and 2, asp is increased the relative importance of the information regarding the undefused mines overweighs itself against the dehsed mine information in the probabilistic functions given in equations 1 and 2. Factor /3 being unity, gives egalitarian importance to both. The quantities M and N represent the total knowledge of the undefused and defused mines during foraging. They are represented as shown in equations 3 and 4.

n M = z a h

k=l (4)

We can observe that the total probability across the various cognitive sub regions is unity. This makes the function given in equations 1 and 2 a valid probability density function. An interesting feature to be observed is that the function given in equations 1 and 2 is a valid probability density function irrespective of the value of f i . This snakes F a user defined quantity and can defined independent of the probabilistic cost function.

Figure 2 is a plot of the number of iterations for runs performed on various combinations of ants and mines over a field of 96x96. Iterations obtained above 1500 were eliminated for visual clarity. The field size 96x96 was chosen because it was easier €or us to obtain symmetrical cognitive regions and sub-regions from the main field. An interesting feature that was observed during the simulations was freezCig [3]. Freezing is defined as the phenomenon where behavioral expectations of the ants becomes mutual and runs into a dependency chain. In such a situations the all the individual ants expects reactions by other ants, which would never occur. The whole system goes into a deadlock like situation and the mine detection task comes to a halt. lnstances of freezing are commonly observed when the ant-mine ratio is less. The field size also may

influence freezing. It was observed that smaller field sizes produced lesser freezing on average if the ants to mines ratio was constant.

RuLsma6eIddlMxlM

1w U FbrmberdAnfs

N W d n d n s

Figure 2. Results of runs conducted on a 96x96 field

When deploying ants for the mine detection the knowledge of freezing is essential. Long-Range Recruitment techniques can be utilized to prevent freezing fiom occurring. Normally, Long-range recruitment is employed when there are classes of ants called worker ants, which are initiated by long-range recruitment mechanisms [ I , 21. Other techniques such as finite threshold time settings for the ants in the waiting behavior can be utilized for countering freezing.

I i

10 20 30 40 50 60 70 BO 90 1W Thrsshom m e

mo--- -

Figure 3. Results of runs conducted on 48x48 field

In our simulation we have a threshold for the time ants spend during their wait around the mine. An ant resumes to the foraging state after waiting for a fixed amount of time at the mine. This state change is observed when the required number of ants does not arrive at the mine within the threshold time. The model is devised such that only ants, which arrive at a mine first, or detect the mine while foraging, take up scent spreading. Any ant in the trail- following state does not take up the task of scent spreading. Figure 2, shows the scenario where the threshold time was fixed at 50 iterations, for different combinations of ants and mines.

64

Authorized licensed use limited to: Rochester Institute of Technology. Downloaded on January 13, 2009 at 14:51 from IEEE Xplore. Restrictions apply.

Variations in threshold times were found to influence the efficiency of the ants in completing the task. Figure 3 shows this variation and suggests that there exists an optimal threshold time for a given specific mine detection task. Smaller threshold were found to be optimal when mine detection tasks with larger ant-mine ratio were simulated. Both ants with cognitive maps and those without exhibited optimal threshold times for a specific mine detection task as seen in Figure 2. Figure 2 gives the result of runs conducted on a 96x96 field with 50 ants foraging for 100 mines.

We should be able to analyze the importance of the cognitive maps in the ants in expediting the convergence of the task. During simulation it was seen that the ants employed with cognitive maps outdid the one ones without cognitive maps interms of quicker convergence. In order to attribute this fact, we bring in a term called the reduction factor R . The reduction factor R is mathematically defined as the ratio given in equation 5 .

0.8 E B

where Tmem0,,, and Tmcmo,,,~cs are the number of iterations for completing a specific mine detection task by the ants with and without memory respectively.

-

0.6

0.5

i

-

-

Ants with copmtiu mancry

1

employed with memory. The reduction factor is a good measure showing the effectiveness of have cognitive maps built in the ants. Analysis shows that the reduction factor varies with combinations of field size, number of ants deployed and number of mines present.

Vmialbn in Reduction FecW (Field 86xQ6. Miner 30) 1 I :::I

0.7

0.3

0.2[

15 20 25 30 35 N u m k d ants

Figure 5. Variation of Reduction Factor with number of ants

A study shown in Figure 5 shows that the reduction factor has an optimal lowest value among different number of ants employed for mine detection, for a given combination of field size. This was a study conducted on a 96x96 field with thirty mines randomly distributed of it. This becomes instrumental in picking up the optimal number of ants to be deployed for a specific mine detection task.

0.91 1 Mines-20

MlOS.25

Mines.30

Figure 4. Comparative performance of ants with and without cognitive memory

The comparison plot in Figure 4 shows that the importance of reduction factor is highly significant. The reduction factor tends to decrease when the mines to ants ratio is high, thus necessitating the employment of cognitive maps in the foraging ants under such situations. It is evident from Figure 4 that the rate at which the mines are defused progressively increases in the case of ants employed with cognitive maps whereas it decreases or is stagnant in the case of those without cognitive maps. The decrease in the rate of mine detection as time progresses reflects randomness and increase in the rate shows adaptation. Thus adaptivity is clearly seen in ants

1.- 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5

Beta

Figure 6. Variation of Reduction Factor with beta

A natural result that follows from the results seen in Figure 5 is that as the number of ants deployed for a specific mine detection task is increased, the reduction factor decreases. This shows that the ants given the ability of cognitive memories outperform those without cognitive memories. This efficiency is seen to converge at a particular point as observed in Figure 5 . When the number of ants is increased beyond this point the reduction factor increases. Figure 6 shows that the optimal value in R can depend on .

65

Authorized licensed use limited to: Rochester Institute of Technology. Downloaded on January 13, 2009 at 14:51 from IEEE Xplore. Restrictions apply.

Variation in the value of p affects the performance of the ants. Figure 6 illustrates the change in reduction factor against ,8 for runs on a field of 96x96 for various counts of mines. We see in our simulation results that the reduction factor was optimal for a specific value of p for different numbers of total mines. This follows the trend that the optimal reduction factor would be greater when the ant-mine ratio is greater as a result of less time for adaptivity. Generally it was seen that ants optimal reduction factor occurred for instances where the ant- mine ratio was more when a p value more than unity was employed. This shows natural adaptiveness in the whole system.

IV. FUTURE WORK

The work contains certain aspects, which could be highlighted for improvements. One major aspect that we can look at is the scenario where the mines themselves are mobile or varied. Presence of a variety of mines on the minefield requires different demining techniques for each type of mine. In such cases a prior knowledge of the type of mines and their demining mechanisms should be present. Mobile mines and sea mines present a totally different scenario and the related problems have to be addressed differently. In the present work, the foraging strategies were mainly based on deterministic natural heuristics. Random fluctuations and heuristics can be inculcated in the foraging strategies, which can lead to newer solutions and may aid faster convergence. This approach can be investigated in future. The present work does not address the event of agents failing. Such a

- scenario can be easily seen when the agents accidentally step on the mines and are lost for action. Extrapolation of the work can include this factor.

VII. CONCLUSION

The main focus of this research is to emphasize the exhibition of natural heuristics in biological systems. The mine detection is a combinatorial optimization task, which demands solutions from heuristics [lo]. Ant colonies are typical classes, which show natural heuristics. Mine detection is a task that requires tremendous technical skills and precision. Even today traditional methods employing humans for the task are preferred due to the complexity in introducing other modem techniques. This research has focused on the implementation of robots for the task. When robots are employed for the task a sense of confidence has to be built that they can perform the task effectively. This confidence stems from factors like cost of operation, effectiveness, efficiency, robustness, and simplicity of the robots. In our simulations we have captured the basis for the usage of agents for the mine detection task. Swarm intelligence was used to model the agents as a group of simple interacting robots whose common goal was to demine the minefield completely in least possible time. Swarm intelligence concepts allows us design the robots

with minimal resources. Simulations were done and the results were tabulated.

VIII. ACKNOWLEDGEMENTS

This work was done at the Multi Agent Bio-Robotics Lab at Rochester Institute of Technology. The work W i S supported by Gleason Research and Development Fund.

REFERENCES

[l] Eric Bonabeau, Marco Dorigo, Guy Theraulaz. Swarm Intelligence-From Natural to Artificial Systems, Oxford University Press, October 1999.

[2] Holldobler Bret. “Chemical ecology: The chemiay of biotic interaction.” 4 1-49 National Academy Press, 1995

[3] Tillett J, Rao TM, Sahin F, Rao R, “An ant colony applied to the ‘Mine Detection’ problem ”, Complexity International, 2002.

[4] Vignesh Kumar and Ferat Sahin, “A Swarm intelligence based approach to the mine detection problem ‘ I IEEE ConJ Systems, Man and Cybernetics, Oct 2002.

[5] J. L. Mercer and A. Lenoir, “Individual flexibility and choice of foraging strategy in Polyrhachis laboriosa F.Smith (Hymenoptera, Formicidae)” Insectes Sociau, 1999.

[6] M. Collet, T.S. Collet, S.Bisch and R. Wehnar, “Local and global vectors in desert ant navigation” Letters to Nature. 1998

[7] Scott L. Andersen, “Landmine Detection Research Pushes Forward, Despite Challenges”, IEEE Intelligent Systems, 2002.

[8] GAO, “Landmine Detection, DoD’s research program need a comprehensive evaluation strategy” United States General Accounting Ofice, Apr 200 1.

[9] J.D. Nicoud, “Vehicles and Robots for Humanitarian Demining.“ The Industrial Robot, Vol. 24, No. 2, 1997.

[ 1 O1D.V. Pynadath. “Multi-agent teamwork: Analysing the optimality and complexity of key theories and models”, Proceedings of the International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2002.

66

Authorized licensed use limited to: Rochester Institute of Technology. Downloaded on January 13, 2009 at 14:51 from IEEE Xplore. Restrictions apply.