Solving a Real World Routing Problem using MultipleEvolutionary Agents

Neil Urquhart, Peter Ross, Ben Paechter, and Ken Chisholm

Napier University, School Of Computing, 10 Colinton Rd, Edinburgh, Scotlandn.urquhart@napier.ac.uk

Abstract. This paper investigates the solving of a real world routing problemusing evolutionary algorithms embedded within a Multi-agent system (MAS).An architecture for the MAS is proposed and mechanisms for controlling theinteractions of agents are investigated. The control mechanism used in the finalsolution is based on the concept of agents submitting bids toreceive work. Theagents are also allowed to alter their bidding strategies asthe solution improves.The MAS solves the test problem is solved, which previously could not be solvedwithin the hard constraints.

1 Introduction

1.1 The Problem

Each year in the month of December, the Edinburgh Area of the Scout Associationundertakes the collection and delivery of Christmas cards within the City of Edinburgh.The delivery task is then undertaken by Scout Groups across the city. In 2001 a totalof 465,000 cards were delivered during a 2 week period. Each group undertakes todeliver everything within a particular geographical area;a group may deliver up toseveral hundred streets depending on the size of the group. The group must divide upits delivery area into a number of delivery rounds each of which should attempt tosatisfy the following constraints:

1. The maximum walking distance for each delivery round should not exceed a setlimit, typically 3km.

2. The maximum number of households requiring a delivery perround should notexceed a predefined limit, typically 250.

3. The number of delivery rounds should be minimised.4. Entire named streets should be allocated to a single round, if possible. Within each

round a street may be split if required when constructing theround. Each time astreet is split across two or more rounds, this incurs an extra sorting operation.

5. Deliveries that require entry to a building e.g. blocks offlats may not be given tochildren under 16. It is desirable to group as many of these deliveries into as fewrounds as possible.

Note that constraints 1 and 2 may be deemedhard constraints and the restsoftconstraints. An area of 1572 deliveries located in the southern suburbs of Edinburgh

currently undertaken by the 75th Blackford Scout Group has been identified to use as atrial area. This area is currently problematic as it combines a mixture of tower blocks(with over 100 dwellings per building), semi-detached villas and a street network thathas a largely random topology. It has previously been difficult coordinate deliveries tothis area and a large number of sorting errors occur (see Fig.1). The remainder of thispaper will examine the use of GAs and Agents in the construction of delivery networkssuitable for the solving of the Scoutpost problem.

1.2 Previous Work

Postal delivery based problems have previously been tackled using Street based routing(SBR) in [8] and [7]. SBR employs a representation that is derived from the geographi-cal layout of the houses and streets concerned. Houses are grouped into street sections,each street section being a section of street running between two junctions. The sectionsmay be considered either as single sides, serviced at different times within the roundor they mat be both serviced in one single operation. The EA produces a permutationof street sections; to build the final round each street section has a delivery heuristicapplied, specifying the order in which the houses containedwithin the section shouldbe serviced. This method may be contrasted with arc routing based methods as used in[5], [1] and [4]. Some research was undertaken by [2] into theuse of a GA for rout-ing garbage collection, this being a related problem, but only covering the constructionof single rounds. The arc routing methods use representations based on graphs that donot take account of street and house layouts. The arc routingmethods are not flexi-ble enough to take account of situations where it is not necessary to traverse the entirelength of an arc. In some instances the optimal delivery pattern will involve starting andfinishing at the same end of the street.

The construction of delivery networks consisting of more than one round was dis-cussed in [7]. That paper proposed the use of a method known asGroup and Build. Thusconsisted of two stages, initially a grouping EA is used to divide up the street sectionsrequiring delivery into adjacent groups (stage 1), and thena round is constructed foreach grouping (stage 2) using the SBR EA.

Multi Agent Systems (as discussed in section 3) have been previously applied toflow problems [6]. The system developed by [6] uses agents to represent the transportlinks, the customers and the goods to be transported. The agents “negotiate” to allocatecapacity within the transport links to customers allowing the goods to be transported.Much research on market-based control has been carried out by the authors of [3]. Theproblems investigated by [3] are mostly based on agents trading a commodity for whichthe supply and demand characteristics develop dynamically. In the problem being ex-amined in this paper the entire trading stock (i.e. the street sections) is a known quantityfrom the start.

2 Initial Attempts to Solve the Problem using Group and Build

The initial attempts to solve the test problem utilised the group and build method [7].For the purposes of solving the test problem the fitness function of the grouping al-

Fig. 1. The Scout Post Test Area

gorithm was modified to take account of the additional soft constraints of the ScoutAssociation problem (described in section 1.1).

The initial population of the grouping EA is seeded by randomly allocating eachstreet to a group. For instance the street sections HighSt1,HighSt2 and HighSt3 wouldall be allocated to the same group to satisfy constraint 1. Within each generational cycle50 child individuals are created by recombining two parentschosen by a tournamentof size 2, each child then has a single mutation applied to it.Three different types ofmutation are employed, they are selected randomly with a probability p:p = 0:25 A single street section may be selected at random and re-allocated to

another delivery round.p = 0:25 A whole street is selected at random and all the sections thatmake up thatstreet section are re-allocated to a randomly selected delivery round.p = 0:49 Two street sections are selected at random and exchanged between theirrespective delivery rounds.p = 0:01 A new delivery round is created, initially three complete streets are se-lected randomly and allocated to it.

The child individuals are placed in the main population, replacing the loser of atournament between two randomly selected individuals.

The modified fitness function utilises penalties that are allocated for the followingreasons:

– Each round that has more than the maximum allowed deliveries.– The standard deviation of deliveries for each round (to attempt to balance the work-

loads).– The number of different rounds converging at a single junction.– The average distance between streets within a round.– The number of times a complete street is split between rounds.– The number of rounds that may not be undertaken by children.– The number of rounds over and above the minimum required (theminimum being

total deliveries / maximum deliveries allowed pen round).

These are each suitably weighted according to their perceived significance. Thechoice of such weights is a difficult art and this is one reasonwhy this method is notideal. We do not give the weights in this paper. This group-and-build approach is usedfor comparison with the main approach described in section 3, and also to provide aninitial starting-point solution for the main approach.

Run Average RoundNo of Extra Sorts No of rounds notDist in meters.rounds Needed suitable for children

1 1464.7 8 2 42 1421.8 9 3 43 1638.1 8 0 44 1529.7 8 2 45 1469.9 8 1 56 1517.5 8 1 57 1563.5 10 2 48 1552.3 8 4 69 1497.0 8 2 410 1463.0 8 2 411 1513.2 8 2 312 1529.3 8 2 413 1496.2 8 1 414 1400.5 9 2 415 1476.7 8 3 416 1498.7 8 3 417 1540.0 8 4 418 1477.0 8 2 419 1503.2 8 5 420 1440.4 9 2 4

Average 1499.6 8.25 2.25 4.15Table 1.Summary of Initial Results

The group and build algorithm was applied to the test area, being run 20 times, re-sults may be seen in Table 1. The maximum length value was set at 3km. The maximumnumber of deliveries was set to 250 households. It may be noted that in each case theaverage distance was less than 3km. In all cases the number ofdeliveries was less than250. Within the 20 runs 165 delivery rounds were constructedgiving an average of 8rounds per solution, although the average length of each round was less than 3km, Table2 shows the distribution of lengths. Note that 7 % (12 rounds)were breaking the hard3km constraint. It may be seen from the distribution that if this constraint was loweredthe number of rounds breaking the constraint would increase. The Group and Buildapproach cannot solve the problem beacuse it cannot determine at the grouping stagewhat the actual distance of the delivery routes will be. Incorporating the SBR algorithminto fitness function so as to determine round length when individual is evaluated wasconsidered impractical by the authors because of the CPU time required to evolve therounds. The Group and Build Approach can only provide a solution where the densityof the housing is such that most groups of houses not exceeding the maximum deliver-ies constraint are tightly clustered so that the delivery round required to service them isless than the maximum length constraint.

Length L (in m)No of rounds% of totalL > 3000 12 7.33000 � L > 2500 35 21.22500 � L > 2000 24 14.5L <= 2000 94 57.0Total 165

Table 2.The Distribution of Round Lengths Produced by the Group and Build Algorithm

3 The Marketplace Algorithm

3.1 Initial MAS Experiments

The two-stage Group and Build (GAB) approach is unable to exercise any effective con-trol over the length of the delivery rounds produced. Some control over the length maybe exercised by altering the maximum deliveries constraint. To overcome this problemthe second stage may be re-implemented as a multi-agent system, with each deliveryround being constructed by an agent that incorporates a separate copy of the SBR algo-rithm. Each agent may then evolve its own round separately, to facilitate evolution of thedelivery network the agents may exchange work. The exchanging of work is carried outvia a coordinator agent to ensure that all street sections are allocated to one agent andone agent only. Each agent also has access to a central data store containing informa-tion about the size and layout of the street sections that make up the problem. The streetsections that are the basis of the SBR representation are a good unit of exchange whenswapping work between agents, rather than exchanging individual delivery points.

Promising results were obtained by allowing the agents who could not evolve around within the problem constraints to pass a street section to the coordinator, whichthen selects another agent whose round passes closest to thestreet section. To allow amore meaningful exchange of work, a flexible control mechanism is required to regulatehow the SBR agents and the coordinator communicate.

3.2 Artificial Currency

The use of an artificial “currency” was initially investigated by us as a possible controlmechanism. The currency is an abstract entity that is not directly connected with theoriginal problem and is represented by a balance value held by each agent. Street sec-tions may be bought and sold between agents and payments madeby currency. Agentsmay receive a regular payment based on the number of households served and incuroperating charges based on the length of round. Agents breaking hard constraints arefined. If an agents balance is negative the agent has to surrender a street section to thecoordinator. When the coordinator receives the street section, each agent is invited tosubmit a “bid”, this being a numerical value representing the degree to which the agentwants the work. The agent submitting the highest bid is then allocated the section, andpays the bid by having it deducted from it’s balance.

The bids are calculated as follows:

MAXDELS the maximum deliveries constraintMAXLEN maximum length constraintowner(st) returns the ID of the agent that st was previously allocated tobalance returns the agents balancecLen the length of the agents current roundcDels the no of deliveries in the agents current roundme returns the id of the current agentavgD(st) returns the average distance between each street section allocated to the

agent and st

let st = street section under offerbid = (MAXLEN - cLen) + avgD(st)if cDels + dels(st) > MAXDELS thenbid = bid /2

if owner(st) = me thenbid = bid /2

if balance < 0 thenbid = -1

In practice, the value of the balance variable has been foundto reflect the past per-formance of the agent rather than its current state. Becauseagents are identical and arenot allowed to modify their bidding strategy or parameters,there is no reason to recordthis past performance. For instance, if an agent lowers the length of its round, its bal-ance will increase, the agent may then bid for the street section allowing it to exceed

the round length constraint. Because of its high balance theagent will not be forcedto give up any street sections until the fines imposed on it have decreased the balanceto less than 0. As long as its balance is greater then 0 then an agent may continue tobid and acquire street sections whether or not it can undertake the deliveries involvedwithout breaking any constraints. Although unsuccessful,elements of this method havebeen used in the final algorithm discussed in section 3.3.

3.3 The Market Algorithm

The bidding mechanism used as part of artificial currency mechanism was found tobe an effective method of deciding which agent to allocate a surplus street section to.The bidding mechanism may be modified and incorporated in a new control systemknown as the Market Algorithm. The system is initialised as previously (using the out-put from the grouping-EA ). Each agent may now construct a delivery round based onthis initial allocation. Each agent may then evolve a delivery round based on the streetsections currently allocated to it. The co-ordinator now requests that the agent with thegreatest violation of the hard constraints surrenders a single street section back to theco-ordinator for re-allocation to another agent. Using thebidding process agents maynow submit bids for the surplus section. The agent submitting the lowest bid has thesection allocated to it.

Assume those functions and variables defined in the currencybidding logic stillhold, then bidding logic is as follows:

leastDist(st) returns the distance to the closest street tost within the currentagent

dels(st) returns the number of deliveries to stoldR(st) returns the id of the previous agent that st was allocated tordLength(st)returns the length of street (st)

Let st = street to bid for.theBid = avgD(st);del = cDels;newDels = (del + dels(st));if (newDels > MAXDELS) thentheBid = theBid + (newDels*2);

if (oldR(st) = me) thentheBid = -1;

if (cLen > MAXLEN) thentheBid = -1;

least = (leastDist + rdLength(st))*2;if ((cLen + least) > MAXLEN) then thebid = -1;bid = theBid;end;

The logic submits a bid of -1 (a -1 bid is never acceptable) if the agent estimatesthat the addition of the street section will violate any of the hard constraints. When asituation arises where no agent is bidding for a street, a newagent is allowed into theMarketplace. The new agent is formed by taking the agent withthe longest round andrandomly transferring half its street sections to the new agent.

The method used by an agent to select street sections to be returned to the coor-dinator was found to have an effect on performance. Several methods were evaluated,the final method chosen removes the street section that has the highest average distancefrom each of the other street sections allocated to that agent. When street sections havebeen removed and re-allocated in his manner the two agents affected then evolve up-dated rounds based on the changes that have taken place. In the results presented here,this cycle of transactions continues for a maximum of 100 transactions.

3.4 2-stage Bidding

The agent bidding logic described in Section 3.3 may be described as “cautious” as itproduces a bid of -1 if the agent estimates that it may break a hard constraint by accept-ing the street section on offer. In addition to the Scout Postdata set, the system was firsttested on a small and artificial data set (see Fig. 2). This data set divides into 4 logicalareas radiating from a central hub. The total length of streets in each section is 45 units,with 54 households. The maximum deliveries constraint was set to 70, and the Market-place algorithm was run with maximum length values of 50-90.The results obtainedare shown in Fig. 2. Note that the number of delivery rounds required would appear tobe excessively high. This data set should be easily split into 4 rounds, especially whenthe maximum length constraint is as high as 90 units.

An examination of the results shown in Fig. 2 suggests that the marketplace al-gorithm does not cope well with a constrained problem such asthis. If all agents arebreaking hard constraints then no exchange of street sections would be possible withoutcreating a new routing agent, even if the existing agents could cope with the workloadsimply by exchanging street sections. If the bidding logic is altered to always submita bid, regardless of whether a hard constraint is broken, then the street sections are di-vided amongst the initial set of agents regardless of whether the agents are breakingany hard constraints. Using this “relaxed” strategy the system stabilises with adjacentstreet sections being allocated to the same agent. Finally system reaches an equilibriumwith the same street section being exchanged between the same agents. Once this stagehas been reached the bidding strategy of agents may then be set back to the originalcautious strategy. This allows the rejection of surplus work to create one or more newagents, and reduce each agent’s workload to an acceptable level. This change in strategyis facilitated by allowing each agent to keep a list of those street sections last surren-dered by the agent. This list is of length 3 and initially empty, but when it contains 3identical items, the agent realising that it no longer engaged in productive transactionsand changes to a cautious strategy. Using the cautious strategy no work should be ac-cepted until some work has been surrendered allowing the agents hard constraints to befulfilled. The effect of this two stage bidding may be seen in Table 3 under “Market 2”.

Fig. 2.The Artificial Small Test Data Set

MAX LEN 90 80 70 60 50Market1Avg. length 68.360.349.037.028.1

Avg. % Rounds over 0 0 0 0 0Avg. No Rounds 5.7 6.3 7.7 9.0 10.7

Market2Avg. length 76.668.956.646.332.3Avg. % Rounds over 0 0 0 0 0.3Avg. No Rounds 4.2 4.2 5 6 8.3

Table 3.Results Obtained with Market 1 and Market 2, on the Map in Fig.2

Market 1 Market 2Maximum Length Constraint 2K 2.5K 3K 2K 2.5K 3K

Average round Length1583.21980.42175.01553.61955.92149.7Over-length Rounds 0 0 0 0 0 0

Average No of extra sorts 7.1 7.0 4.4 8.4 6.8 4.0Average No of Supervised Rounds 4.6 4.5 4.4 4.7 4.8 4.3

Average No of Rounds 12.2 9.9 9.1 12.5 10.2 9.1Table 4.Results for the Test Area, with Version 1 and Version 2 of the Marketplace Algorithm

4 Results and Conclusions

The results for the Scout Post test area (see Fig. 1) may be seen in Table 4, resultsare shown for the initial Marketplace algorithm (Market1) and the two stage algo-rithm (Market2). Both versions of the Marketplace Algorithm are capable of solvingthe ScoutPost problem without breaking any hard constraints. Although the hard con-straints are satisfied, note the increase in average round length and the increased numberof extra sorts present in the final solution, the relaxing of the soft constraints being theprice that must be paid for the solving of the hard constraints. The results demonstratethat we can solve the Scoutpost problem with the initial 3km limit, and we can also pro-duce solutions for more constrained versions of this problem with 2.5 and 2km limits.

The small contrived dataset (figure 2) shows that for a constrained area, where littlesurplus capacity exists, relaxing the constraints (by using “relaxed” bidding) allows usto construct an interim solution, which is then modified intoa final solution, by thealtering of the bidding strategy to “cautious”. The successof the Marketplace algorithmis based on the bidding strategy logic and to a lesser extent the logic used by an agentto select items for surrender to the coordinator agent.

Acknowledgements.The authors wish to thank Jonathan Tait and Mark Elliot ofthe 75th Blackford Scout Troup, Edinburgh for their assistance in this work.

References

1. Jeyakesavan V A Balaji R. A 3/2- approximation algorithm for the mixed postman problem.SIAM Journal on Discrete Mathematics, 12(4), 1999.

2. T. Bousonville. Local search and evolutionary computation for arc routing in garbage col-lection. In et al Spector L, editor,Proceedings of the Genetic and Evolutionary ComputationConference 2001.Morgan Kaufman Publishers, 2001.

3. Bruten J Cliff D. Simple bargaining agents for decentralised market-based control. TechnicalReport HPL-98-17, Hewlet Packard Laboritories., Bristol.United Kingdom., 1998.

4. Han C. Kang M. Solving the rural postman problem using a genetic algorithm with a graphtransformation. InProceedings of the 1998 ACM Symposium on Applied Computing.ACMPress, 1998.

5. Ramdane-Cherif W A Lacomme P, Prins C. A genetic algorithmfor the capacitated arc routingproblem. In Boers E J W et al., editor,Real World Applications of Evolutionary Computing.Springer-Verlag, 2001.

6. Wellman M P. A market orientated programming environmentand its application to dis-tributed multi-commodity flow problems.Journal of Artificial Intelligence Research. MorganKaufmann Publishers, 1, 1993.

7. Chisholm K Urquhart N, Paechter B. Street based routing using an evolutionary algorithm. InBoers E J W et al., editor,Real World Applications of Evolutionary Computing. Proceedingsof EvoWorkshops 2001. Springer-Verlag, 2001.

8. Paechter B Chisholm K Urquhart N, Ross P. Improving streetbased routing using buildingblock mutations. InTo Appear in: Applications of Evolutionary Computing. Proceedings ofEvoWorkshops 2002. Springer-Verlag, 2002.