DECSAI Deparment of Computer Scienceand Arti�cial Intelligence

Hierarchical Distributed GeneticAlgorithmsF. Herrera, M. Lozano, C. Moraga

Technical Report #DECSAI-97-01-18May, 1997ETS de Ingenier��a Inform�atica. Universidad de Granada.18071 Granada - Spain - Phone +34.58.244019, Fax +34.58.243317

Hierarchical Distributed GeneticAlgorithms �F. Herreraa, M. Lozanoa, C. MoragabaDept. of Computer Science and A.I.University of Granada, 18071 - Granada, Spaine-mail: herrera,lozano@decsai.ugr.esb Dept. of Computer ScienceUniversity of Dortmund, 44221 - Dortmund, Germanye-mail: moraga@cs.uni-dortmund.deAbstractGenetic algorithm behavior is determined by the exploration/exploitation balancekept throughout the run. When this balance is disproportionate, the premature conver-gence problem will probably appear, causing a drop in the genetic algorithm's e�cacy.One approach presented for dealing with this problem is the distributed genetic al-gorithm model. Its basic idea is to keep, in parallel, several subpopulations that areprocessed by genetic algorithms, with each one being independent from the others.Furthermore, a migration operator produces a chromosome exchange between the sub-populations. Making distinctions between the subpopulations of a distributed geneticby applying genetic algorithms with di�erent con�gurations, we obtain the so-calledheterogeneous distributed genetic algorithms.In this paper, we present a hierarchical model of distributed genetic algorithms inwhich a higher level distributed genetic algorithm joins di�erent simple distributed ge-netic algorithms. Furthermore, with the union of the hierarchical structure presentedand the idea of the heterogeneous distributed genetic algorithms, we propose a type ofheterogeneous hierarchical distributed genetic algorithms, the hierarchical gradual dis-tributed genetic algorithms. Experimental results show that the proposals consistentlyoutperform equivalent sequential genetic algorithms and simple distributed genetic al-gorithms.Keywords: Genetic algorithms, Distributed genetic algorithms, Gradual distributed genetic al-gorithms, Hierarchical distributed genetic algorithms.�This research has been partially supported by CICYT TIC96-0778 and DGICYT SAB95-0473.Work leading to this Research Report was done meanwhile C. Moraga was visiting the University of Granada,Granada, Spain, in 1997, under the Sponsorship of the DGICYT SAB95-0473.

1 IntroductionGenetic algorithms (GAs) are general purpose search algorithms which use principles inspired bynatural genetic populations to evolve solutions to problems ([16, 27]). The basic idea is to maintaina population of chromosomes, which represent candidate solutions to the concrete problem, thatevolves over time through a process of competition and controlled variation. Each chromosome inthe population has an associated �tness to determine which chromosomes are used to form newones in the competition process, which is called selection. The new ones are created using geneticoperators such as crossover and mutation. GAs have a great measure of success in search andoptimization problems. They are particularly adequate in large, complex, and poorly understoodsearch spaces, where classical search tools (enumerative, heuristic, etc.) are inappropriate, o�eringa valid approach to problems requiring e�cient and e�ective search techniques. The followingbibliography may be examined for a more detailed discussion about GAs and their applications[4, 12, 16, 27, 37, 38].The balance between exploration and exploitation or, in other words, between the creationof diversity and its reduction, by focusing on the individuals of higher �tness, is essential inorder to achieve a reasonable behavior for GAs in the case of complicated optimization problems([2]). Loss of critical alleles due to selection pressure, selection noise, schemata disruption due tocrossover operator, and poor parameter setting may make this exploitation/exploration relationshipdisproportionate, e.g., the proper balance between a broad search and a su�cient re�nement isnot established and produce the lack of diversity in the population ([30, 33, 42]). Under thesecircumstances a preeminent problem appears, the premature convergence problem, a prematurestagnation of the search caused by a loss of genotypical diversity ([16]). Usually the search becomestrapped in a local optimum before the global optimum is found.Some tools for monitoring the exploration/exploitation relationship have been proposed inorder to avoid the premature convergence problem and improve GA performance. These toolsinclude modi�ed selection and crossover operators ([33, 42, 22, 25]), adaptive techniques ([23]),optimization of parameter settings ([19, 46]) and diversity preservation methods based on spatialseparation ([10, 11, 34, 39, 40, 41, 51]).Distributed GAs (DGAs) are one of the most important representatives of methods based onspatial separation ([9, 41, 51]). The basic idea of DGAs lies in the partition of the population intoseveral subpopulations (whose sizes are relatively small), each one of them being processed by a GA,independently from the others. Furthermore, a operator, called migration, produces a chromosomeexchange between the subpopulations. Its principal role is to promote genetic diversity, and inessence, to allow the sharing of possible solutions. DGAs show two determinant advantages:1) the preservation of the diversity due to the semi-isolation of the subpopulations, which maystand up to the premature convergence problem, and 2) they may be easily implemented onparallel hardware, obtaining, in this way, substantial improvements on computational time. Someauthors have highlighted the interest in the distinction between the subpopulations of a DGA bymeans of the application of GAs with di�erent control parameters, genetic operators, codings, etc([14, 41, 51]). This type of DGAs are called heterogeneous DGAs. A model of these algorithmsare the gradual DGAs (GDGAs) ([26]), a class of heterogeneous DGAs based on real coding thatapplies a di�erent crossover operator to each subpopulation. These operators are di�erentiatedaccording to their associated exploration and exploitation properties and the degree thereof.Some DGA models combine the advantages of the DGAs with the ones of other spatial separa-tion based methods, such as the cellular GAs ([10, 11, 34, 39]), reporting good results. The basicidea concerns the organization of each DGA subpopulation as a cellular GA. Instances of theseproposals are the mixed parallel GAs ([21]), for neural network design and training applications,the structured DGAs ([53]), for function optimization problems, and a DGA model whose subpop-

ulations are based on the ECO framework ([11]), which was designed and used in [35] for di�erentproblems, such as quadratic assignment problem, dynamic control and parametric optimization.In this paper we present a proposal of DGAs, the hierarchical DGAs (HDGAs), DGAs whosenodes are other simple DGAs, which are connected with each other. The existence of two types ofmigrations, the local ones, in every DGA, and the global ones, between the constituent DGAs, isa key feature of HDGAs, since it establishes the proper hierarchy between the simple DGAs andthe HDGA. An important feature of the hierarchical structure to be presented is that it may beeasily joined with the idea of heterogeneous DGAs. A possible way is the design of heterogeneousHDGAs, which use di�erent basic homogeneous DGAs. In this way, we propose the hierarchicalGDGAs, GDGAs that are composed by di�erent homogeneous DGAs whose subpopulations usethe same crossover operator, following these crossover operators the underlying idea in the GDGAs.In order to develop the HDGAs, this paper is set out as follows: in Section 2, we describethe DGAs, in Section 3, we present the HDGAs, in Section 4, a type of heterogeneous HDGAsbased on homogeneous DGAs, the hierarchical GDGAs, are proposed, in Section 5, the experimentscarried out for determining the e�cacy of the algorithms presented are described, and �nally, someconcluding remarks are dealt with in Section 6.2 Distributed Genetic AlgorithmsThis section is devoted to DGAs. In Subsection 2.1, they shall be presented as a class ofparallel GAs called coarse grained parallel GAs, in Subsection 2.2, spatial separation, a basicprinciple of DGAs, is justi�ed from a biological point of view through the Shifting Balance Theoryof Evolution ([58]) and the Theory of Punctuated Equilibria ([15]), in Subsection 2.3, we describethe basic structure of DGAs, and �nally in Subsection 2.4, we review the types of DGAs presentedpreviously.2.1 Parallel Genetic AlgorithmsThe availability, over the last few years, of fast and cheap parallel hardware has favored researchinto possible ways for implementing parallel versions of GAs. GAs are good candidates for e�ectiveparallelization, since they are inspired on the principles of evolution, in parallel, for a populationof individuals ([14]). In general, three ways were followed for implementing the parallelization ofGAs ([1, 7, 14, 17, 31]):� Global ParallelizationThe evaluation of chromosome �tness and sometimes the genetic operator application arecarried out in a parallel form ([3, 18, 44]).� Coarse Grained ParallelizationThe population is divided into small subpopulations that are assigned to di�erent proces-sors. Each subpopulation evolves independently and simultaneously according to a GA.Periodically, a migration operator exchanges individuals between subpopulations, allowingnew diversity to be injected into converging subpopulations. The exchange generally takesthe form of copying individuals between the populations. GAs based on coarse grained par-allel GAs are known as distributed GAs, since they are usually implemented in distributedmemory MIMD computers. Versions of DGAs appeared in [6, 8, 9, 32, 40, 41, 50, 51, 52, 55].

� Fine Grained ParallelizationIn this model, the population is divided into a great number of small subpopulations. Usu-ally, a unique individual is assigned to each processor. The selection mechanism and thecrossover operator are applied by considering neighboring chromosomes. For example, everychromosome selects the best neighbor for recombination and the resultant individual shallreplace it. This type of GAs, known as cellular GAs, are usually implemented on massivelyparallel computers. Examples of cellular GAs are to be found in [10, 11, 34, 39].2.2 Spatial SeparationBoth distributed GAs and cellular GAs are instances of models based on spatial separation.One of the main advantages of these models is the preservation of diversity. This property causedthem to be considered as an important way to research into mechanisms for dealing with thepremature convergence problem ([1, 6, 8, 10, 11, 31, 33, 51, 52]).Many authors ([10, 11, 39, 40, 41]) have attempted to justify spatial separation models, start-ing from the Shifting Balance Theory of Evolution developed by Sewall Wright ([58]). This theoryexplains the process of evolution on the genetic composition of individuals in natural populations.According to this, large populations of organisms rarely act as a single well-mixed (panmictic)population, but rather they consist of semi-isolated subpopulations, demes, each of which is rel-atively small in size. Furthermore, the demes communicate with each other through migrationsof individuals. For Wright, the evolution process has two phases. During the �rst one, the allelefrequencies drift randomly around a local �tness peak in each deme. One of them might, by chance,drift into a set of gene frequencies that correspond to a higher peak. Then, the second phase starts;this deme produces an excess of o�spring, due to its high average �tness, which then emigrate tothe other demes, and will tend to displace them until eventually the whole population has the newfavorable gene combination. Then, the process starts again. The relatively small size of the demesallows drift to play an important role in the evolution of the population, without driving the wholepopulation towards convergence. Even if drift were to drive every local deme to �xation, each oneof them would be �xed on a di�erent genotype, thereby maintaining diversity in the population asa whole.Another biological theory adopted by people who do work on spatial separation is the theory ofPunctuated Equilibria ([15]). This theory states that evolution is characterized by long periods ofrelative stasis, punctuated by periods of rapid change associated with specialization events. In [8],it is pointed out that GAs also tends towards stasis, or premature convergence and that isolatedspecies could be formed by separating the global population into subpopulations. By injecting anindividual from a di�erent species into a subpopulation after it had converged, new building blockswould become available; furthermore, immigrants would e�ectively change the �tness landscape inthe subpopulations. In this way, premature convergence may be avoided. In this line, in [40] itwas stated that:"Subpopulations isolated for a certain time keep the diversity of the population high.After migration new promising areas can be discovered by crossing-over".which explains that ([40]):"The creative forces of evolution take place at migration and few generations afterwards.Wright's argument that better peaks are found just by chance in small subpopulationsdoes not capture the essential facts".

2.3 Basic Structure of Distributed GAsAlthough there are a lot of di�erent types of DGAs, all of them are variations on the followingbasic algorithm: Distributed Genetic Algorithm.1. Generate at random a population, P , of chromosomes.2. Divide P into SP1; :::; SPNS subpopulations.3. De�ne a neighborhood structure for SPi, i = 1; :::; NS .4. For SPi, i = 1; :::; NS , execute in parallel the next steps4.1. Apply, during fm generations, the selection mechanism and the genetic operators.4.2. Send nm chromosomes to neighboring subpopulations.4.3. Receive chromosomes from neighboring subpopulations.5. If the stop criterion is not ful�lled, return to 4.Some additional issues should be considered:1. The neighborhood structure, topology, to be used. Generally, hypercubic topologies wereconsidered for exploiting the power of DGAs ([8, 9, 41, 50, 51, 52]).2. The migration rate, nm, that controls how many chromosomes migrate.3. The migration interval, fm, the number of generations between each migration.4. The selection strategy of the genetic material to be copied. Two methods were widely used.The �rst one is to select randomly the element from current subpopulation. The advantageof this approach is the greater mix of genes that will result. A second method is to selectthe highest performing individual from each subpopulation to be copied to another subpop-ulation. This would result in more directed evolution than the �rst case as the migrantindividuals would not be tainted by genes from lower performing individuals. This is notto say that the former method is worse, for the less directed a population is, the greaterdiversity it will contain ([45]).5. The replacement strategy for including the chromosomes to be received. Some approachesare, replace the worst ones, the most similar to the incoming ones, one randomly chosen, etc.6. The choice of whether or not to replicate migrating individuals, i.e., should individuals moveto their new home or should a copy of them be sent there? If one does not copy individuals itis possible that a subpopulation could be set back several generations in evolutionary termsby the mass emigration of its best performers. On the other hand, simply copying individualsacross could lead to highly �t individuals dominating several populations ([45]).2.4 Types of Distributed GAsIn [31], the following three categorizations of DGAs are reported:� Regarding the Migration Method� Isolated DGAs. There are no migrations between subpopulations. This model is known aswell as partitioned GAs ([51, 52]).� Synchronous DGAs. Migrations between subpopulations are synchronized, i.e., they areproduced at the same time ([9, 41, 51]).

� Asynchronous DGAs. Migrations are produced when certain events appear, related with theactivity of each subpopulation. Asynchronous behavior is typically found in nature, sinceevolution is produced at di�erent states depending on the environment ([31]).� Regarding the Connection Schema� Static Connection Scheme. The connections between the subpopulations are established atthe beginning of the run, and they are not modi�ed throughout it.� Dynamic Connection Scheme. The connection topology is dynamically changed throughoutthe run. The recon�gurations in these connections may occur depending on the evolutionstate of the subpopulations. For example, in [31], a connection schema called positive-distancetopology was proposed, in which an individual is passed to another subpopulation only if theHamming distance between the best individuals in the two subpopulations is less than 24.An analogous connection schema called negative-distance topology was presented as well.Finally, we point out that some authors ([7, 28]) assumed another division, based on the con-nection schema: the island model and the stepping stone model. In the �rst model, individualscan migrate to any other subpopulation, in the second one, migration is restricted to neighboringsubpopulations.� Regarding the Subpopulation Homogeneity� Homogeneous DGAs. Every subpopulation uses the same genetic operators, control param-eter values, �tness function, coding schema, etc. Most DGAs proposed in the literature arehomogeneous ([9, 41, 51]). Their principal advantage is that they are easily implemented.� Heterogeneous DGAs. The subpopulations are processed using GAs with either di�erentcontrol parameter values, or genetic operators, or coding schema, etc. Some interestingheterogeneous DGAs are the models of adaptation by competing subpopulations ([47, 48]),the GAs based on migration and arti�cial selection (GAMAS) ([42]), the injection islandGAs (iiGAs) ([31]) and the GDGAs ([26]).3 Hierarchical Distributed Genetic AlgorithmsIn this section we present HDGAs. The main idea of these algorithms is to connect DGAs withother DGAs, building, in this way, DGAs whose nodes (subpopulations) are simple DGAs, whichshall be called basic DGAs. For this purpose we have chosen the Cube-Connected Cycles structure([43]), which has been used until now mostly for parallel processing in multiprocessor architectures;but is quite appropriate to support migrations within a hierarchical approach. Figure 1 shows anexample of HDGA, where the basic DGAs are rings with three subpopulations. It may be observedthat every subpopulation in a basic DGA is connected with another subpopulation in an adjacentbasic DGA.Two type of migrations are produced in a HDGA: 1) local migrations, produced between sub-populations belonging to the same basic DGAs and 2) global migrations, produced between subpop-ulations belonging to di�erent basic DGAs which connect these ones. The existence of these twodi�erent migrations is the key feature of HDGAs, since they establish the real hierarchy betweenthe basic DGAs and the HDGA.With the HDGA model we attempt to improve the behaviour of traditional DGAs in thefollowing ways:

GA

GA

GA

GA

GA

GA

GA

GA

GA

GA

GA

GA

GA

GAGA

GA

GA

GA

GA

GA

GA

GA

GA GA

DGA

DGA

DGA

DGA

DGA

DGA

DGADGA

Figure 1. Example of HDGA� Better performance for each node. In the HDGAs, every node is a DGA instead of a simplesubpopulation. In this way, we raise the e�ciency of these nodes.� The search may be carried out in di�erent levels. Every basic DGA develops an independentsearch and collaborates, through the global migrations, with others basic DGAs on a globalsearch of the HDGA.� The good features of the homogeneous and heterogeneous DGAs may be easily joined in twoways:- homogeneous HDGAs with heterogeneous basic DGAs, or- heterogeneous HDGAs with homogeneous basic DGAs.Finally, we should point out that the HDGAs proposed may be extended using more levels. Inthis faction, each basic DGA may be, in its turn, a DGA of DGAs, i.e., a HDGA.4 Heterogeneous HDGAs Based on Homogeneous Basic DGAsIn this section we design heterogeneous HDGA based on homogeneous basic DGAs. In order todo so, we use the GDGAs presented in [26]. In Subsection 4.1, we describe the structure of GDGAs,in Subsection 4.2, we build a particular model of them, and �nally, in Subsection 4.3, we present theheterogeneous HDGAs based on the gradual structure, which shall be called hierarchical GDGAs.4.1 Gradual Distributed Genetic AlgorithmsHeterogeneous DGAs have been considered as suitable tools for avoiding the premature con-vergence problem and for maximizing the exploration and exploitation on the search space. Forexample, in [51], Tanese suggested:"The distributed genetic algorithm enables di�erent subpopulations to run with di�erentcrossover and mutation rates, maintaining a balance between exploration and exploita-tion in a novel way".

Gradual DGAs (GDGAs) ([26]) are a class of heterogeneous DGAs based on real coding inwhich subpopulations are distinguished by applying crossover operators with di�erent degrees ofexploration or exploitation. So, a parallel multiresolution is obtained with regard to the crossoveroperator, which allows a spread search (reliability) along with an e�ective local tuning (accuracy) tobe simultaneously achieved. Furthermore, subpopulations are adequately connected for exploitingthe multiresolution in a gradual way, o�ering the re�nement or the expansion of the best zonesemerging. The migrations between subpopulations belonging to di�erent categories produce these�nal e�ects.Figure 2 outlines the basic structure of the GDGAs. They are based on a hypercube topologywith three dimensions with two important sides to be di�erentiated:� The front side is devoted to the exploration. It is made up of four subpopulations, E1, ...,E4, to which exploratory crossover operators are applied. The exploration degree increasesclockwise, starting at the lowest, E1, and ending at the highest, E4.� The rear side is for exploitation. It is composed of subpopulations e1, ..., e4, which undergoexploitative crossover operators. The exploitation degree increases clockwise, starting at thelowest, e1, and �nishing at the highest, e4.

2E

E14E

E3

e1

2e

4e

3e

+ -

+ -

Rear Side (Exploitation)

Front Side (Exploration)Figure 2. GDGA TopologyThe short diameter of the cubic topology may allow multiresolution together with migrations toproduce an e�ective exploration/exploitation balance, since genetic material is quickly exchangedbetween subpopulations with a wide spectrum of properties and exploration and exploitation de-grees, producing a spread search along with an e�ective local tuning.In order to allow the re�nement and expansion to be carried through to a suitable conclusions,and overcome two possible problems that may appear due to the crossover con�guration of GDGAs,the conquest ([31, 35, 36]) and the non-e�ect ([26]) problems, the authors proposed to use thefollowing migration schema and selection mechanism:� Migration SchemaGDGAs use emigration ([29]); individuals leave their subpopulation and migrate to exactlyone of the neighboring subpopulations. In particular, every 5 generations, the best elementof each subpopulation is sent towards the corresponding subpopulation as is shown in Figure3. The sequence of application is from left to right, i.e., �rst, the re�nement migrations,second, the re�nement/expansion migrations, third, the expansion migrations, and then, thesequence starts again. The place of an emigrant is taken by an immigrant.

2E

E14E

E3

e1

2e

4e

3e

2E

E14E

E3

e1

2e

4e

3e

2E

E14E

E3

e1

2e

4e

3e

REFINEMENT MIGRATIONS REF/EXP MIGRATIONS EXPANSION MIGRATIONS

Figure 3. The three types of migration in a GDGA� Selection MechanismThe selection mechanism of GDGAs is the linear ranking selection ([5]); the chromosomesare sorted in order of raw �tness, and then the selection probability of each chromosomeis computed according to its rank by using a non-increasing assignment function. Everyindividual receives an expected number of copies that depends on its rank, independent ofthe magnitude of its �tness. The selective pressure of this selection mechanism is determinedby a parameter, �min 2 [0; 1], which speci�es the expected number of copies for the worstchromosome; the best one has 2 � �min expected copies. If �min is low, high pressure isachieved, whereas if it is high, the pressure is low. Di�erent �min values were assigned to thesubpopulations such as is shown in Table 1.Exploitation Exploration+ � � � �! +e4 e3 e2 e1 E1 E2 E3 E40.9 0.7 0.5 0.1 0.9 0.7 0.5 0.1Table 1. �min values for each subpopulation4.2 Gradual DGA Based on FCB-crossoversIn [26], some versions of GDGAs were built based on di�erent crossover operators for real-codedGAs which allow di�erent exploration or exploitation degrees to be obtained. In this work, we useonly one of these crossover operators, the Fuzzy Connectives-Based crossovers (FCB-crossovers)([25]. Next, we comment on some of the main features for these crossover operators and describehow they were used for building a GDGA.Let us assume that X = (x1 : : : xn) and Y = (y1 : : : yn) (xi; yi 2 [ai; bi] � <, i = 1 : : : n) aretwo real-coded chromosomes that have been selected to apply the crossover operator to them. Inshort, the action interval of the genes xi and yi, [ai; bi], may be divided into three intervals, [ai; xi],[xi; yi] and [yi; bi], that bound three regions to which the resultant genes of some combination ofxi and yi may belong. These intervals could be classi�ed as exploration or exploitation zones. Theinterval with both genes being the extremes is an exploitation zone, the two remaining intervalsare exploration zones.With regards to these intervals, in [25], three monotone and non-decreasing functions are pro-posed: F , S and M , de�ned from [a; b]� [a; b] into [a; b], a; b 2 <, and which ful�ll:8c; c0 2 [a; b], F (c; c0) � minfc; c0g, S(c; c0) � maxfc; c0g and minfc; c0g �M(c; c0) � maxfc; c0g:Each one of these functions allows us to combine two genes giving results belonging to eachone of the aforementioned intervals. Now, if Q 2 fF; S;Mg, we may generate the o�spring Z =(z1 : : : zn) as

zi = Q(xi; yi); i = 1 : : : n:This crossover operator applies the same F , S or M function for all the genes in the chromo-somes to crossover. For this reason, they were called F -crossover, S-crossover and M -crossover,respectively. Moreover, they have di�erent properties: the F -crossover and S-crossover operatorsshow exploration and the M -crossover operators show exploitation. Four families of F -, S- andM -crossover operators were presented in [25]: the Logical, Hamacher, Algebraic and Einstein ones.Their e�ects along with their associated exploration or exploitation degrees may be observed inFigure 4.ixia ibiy

M-crossovers S-crossoversF-crossovers

Ein Alg LogHam EinAlgHamLogLogEinAlgHam+-+-+ - Exploitation ExplorationExplorationFigure 4. FCB-crossover OperatorsWe have considered that the maximum degree of exploitation is for the Logical M-crossover,which is based on the arithmetical mean, since it uses the maximum level of information from bothgenes, i.e., it is not biased towards either of them.Using the F -, S- and M -crossover operators, a GDGA called GDGA-FCB was implementedwith the con�guration shown in Table 2.Subp. M-crossovere1 Hamachere2 Algebraice3 Einsteine4 Logical Subp. F - and S-crossoverE1 LogicalE2 HamacherE3 AlgebraicE4 EinsteinTable 2. Crossover con�guration for GDGA-FCB4.3 Hierarchical Gradual DGAsWe may obtain heterogeneous HDGAs based on homogeneous basic DGAs by assigning to everynode of a GDGA a homogeneous DGA whose subpopulations use the same crossover operator and�min values that the corresponding node of the GDGA (see Tables 1 and 2). These HDGAs shallbe called hierarchical GDGAs. Figure 5 shows an example of these algorithms where each basicDGAs is a ring with three subpopulations. Eji with j = 1; :::; 3 and eji with j = 1; :::; 3 use thesame crossover operator and selective pressure than Ei and ei in a GDGA, respectively.With regards to the migration schema of the hierarchical GDGAs, we propose the following:1. The global migrations, produced between subpopulations belonging to di�erent basic DGAs,are similar to the migration schema of a GDGA. They occur every 10 generations in thesequence of application that is shown in Figure 3, i.e., �rst, the re�nement migrations, second,the re�nement/expansion migrations, third, the expansion migrations, and then, the sequencestarts again.2. The local migrations, produced between the subpopulations in the same basic DGA, happen5 generations after each global migration. These migrations are: E1i ! E2i , E2i ! E3i andE3i ! E1i , i = 1; :::; 4, and e1i ! e2i , e2i ! e3i and e3i ! e1i , i = 1; :::; 4.Starting from GDGA-FCB we built a hierarchical GDGA called H-GDGA-FCB.

E12

13

E

E11

E42

E41

E22

E23E2

1E3

1E32

e14

e24

e11

e21

e22

e13

e23

e33

e12

e31e3

4

E43

E33

e32

Figure 5. Hierarchical GDGA5 ExperimentsMinimization experiments on the test suite, described below, were carried out in order to studythe behavior of the hierarchical GDGA proposed. It was compared with equivalent sequentialversions and other implementations of homogeneous DGAs and homogeneous HDGAs, in whichall the subpopulations have the same con�guration.In Subsection 5.1, we describe the test functions considered and Subsection 5.2, we study theresults obtained in the experiments.5.1 The Test SuiteFor the experiments, we have considered three very complex test functions used in the GAliterature: the Generalized Rosenbrock's function ([13, 49]), the Griewangk's function ([20]) andthe Expansion of F10 ([57]). The dimension of the search space is 25.� Generalized Rosenbrock's functionfRos(~x) = n�1Xi=1 (100 � (xi+1 � x2i )2 + (xi � 1)2);with �5:12 � xi � 5:12 and f�Ros = fRos(1; : : : ; 1) = 0.fRos is a continuous and unimodal function, with the optimum located in a steep parabolicvalley with a at bottom. This feature will probably cause slow progress in many algorithmssince they must permanently change their search direction to reach the optimum. Thisfunction has been considered by some authors to be a real challenge for any continuousfunction optimization program ([47]). A great part of its di�culty lies in the fact that thereare nonlinear interactions between the variables, i.e., it is nonseparable ([56]).� Griewangk's function fGri(~x) = 1d nXi=1 x2i � nYi=1 cos� xipi�+ 1;

with d = 4000, �600:0 � xi � 600:0 and f�Gri = fGri(0; : : : ; 0) = 0.fGri is a continuous and multimodal function. This function is di�cult to optimize becauseit is nonseparable ([41]) and the search algorithm has to climb a hill to reach the nextvalley. Nevertheless, one undesirable property exhibited is that it becomes easier as thedimensionality is increased ([56]).� Expansion of F10E � F10(x1; : : : ; xn) = F10(x1; x2) + F10(x2; x3) + : : :+ F10(xn�1; xn) + F10(xn; x1);where F10(x; y) = (x2 + y2)0:25 � [sin2(50 � (x2 + y2)0:1) + 1], with x; y 2 (�100; 100] andE � F10(x�) = 0.F10 is a function that has nonlinear interactions between two variables. Its expanded version,E�F10, is built in such a way that it induces nonlinear interaction across multiple variables.It is nonseparable as well.5.2 ResultsTable 3 shows the set of algorithms executed in order to study the behaviour of the HDGAsproposed. We include hierarchical GDGAs, hierarchical homogeneous DGAs and real-coded GAs(RCGAs) ([24]). The table row has the name of the algorithm, its type, the number of subpopula-tions, the size of them, the maximum number of generations and the strategy of application of thecrossover operators. The algorithms based on the ST2 and ST4 ([25]) strategies use FCB-crossoveroperators and the ones based on the DST1 and DST2 use dynamic FCB-crossover ([22]). The val-ues of the control parameters of the algorithms whose name starts with 'R3' were suitably chosenfor requiring the same number of evaluations. In the following, we describe shortly the mentionedstrategies.� The ST2 strategyFor each pair of chromosomes from the total population that undergoes crossover, four o�-spring are generated, the result of applying two exploratory crossover operators and twoexploitative ones to them. The two most promising o�spring of the four substitute theirparents in the population.� The ST4 strategyFor each pair of chromosomes from a total of 12 � pc �N (pc is the crossover probability and Nis the population size), four o�spring are generated, the result of applying two exploratorycrossover operators, an exploitative one and an operator with "relaxed" exploitation, whichput together the two properties. All four o�spring will form part of the population, in such away that two of them substitute their parents and the other two substitute two chromosomesbelonging to the remaining 12 of the population that should undergo crossover.� The DST1 and DST2 strategiesDST1 and DST2 are similar to ST2 and ST4, respectively. They use dynamic FCB-crossoveroperators, which keep a suitable sequence between the exploration and the exploitation alongthe GA run: "to protect the exploration in the initial stages and the exploitation later".For studying the importance of the structure of the basic DGAs in the HDGA behaviour, wehave implemented a hierarchical GDGA and four hierarchical homogeneous DGAs whose basicDGAs are rings with six subpopulations, such as is shown in Figure 6. Their names start with'R6' and their features appear in Table 3. All algorithms were executed 15 times.

Figure 6. Hierarchical GDGA based on rings with six subpopulationsAlg. Type Subs. Size Gens. StrategyH-GDGA-FCB Hier. GDGA 24 15 5000H-DGA-S4-Log, ... Hier. Hom. DGAs 24 15 5000 ST4GDGA-FCB GDGA 8 45 5000DGA-S4-Log, ... Hom. DGAs 8 45 5000 ST4RGA-S2-180-Log, ... Seq. RCGAs 1 180 5000 ST2RGA-DS1-180-Dub Seq. RCGA 1 180 5000 DST1RGA-S2-60-Log, ... Seq. RCGAs 1 60 15000 ST2RGA-DS1-60-Dub Seq. RCGA 1 60 15000 DST1RGA-S4-360-Log, ... Seq. RCGAs 1 360 5000 ST4RGA-DS2-360-Dub Seq. RCGA 1 360 5000 DST2RGA-S4-120-Log, ... Seq. RCGAs 1 120 15000 ST4RGA-DS2-120-Dub Seq. RCGA 1 120 15000 DST2H-GDGA-FCB-R6 Hier. GDGA 48 8 5000H-DGA-S4-Log-R6, ... Hier. Hom. DGAs 48 8 5000 ST4GDGA-FCB-R6 GDGA 8 48 5000DGA-S4-Log-R6, ... Hom. DGAs 8 48 5000 ST4Table 3. AlgorithmsTable 5 shows the average values of the results obtained. For each function, we introduce themedium of the Best (B) from the last generation, the percentage of Success (S) with respect to thethresholds shown in Table 4, and the �nal average Online measure (O) ([13]) (the average of the�tness of all the elements appearing throughout the GA's execution). Online is considered here asa population diversity measure. Test Func. ThresholdsfRos 0.1fGri 0.005fE�F10 0.05Table 4. Thresholds for the test functions

fRos fGri fH�F10Algorithms B S O B S O B S OH-GDGA-FCB 2.8e+00 86.7 9.2e+04 1.1e-02 46.7 2.5e+02 2.6e-01 0.0 1.2e+02H-DGA-S4-Log 2.2e+01 0.0 8.0e+02 8.8e-03 53.3 3.9e+00 1.4e-01 0.0 2.2e+01H-DGA-S4-Ham 1.9e+00 6.7 6.5e+04 2.3e-01 0.0 2.9e+02 1.7e+01 0.0 2.4e+02H-DGA-S4-Alb 9.2e+01 0.0 3.5e+05 1.0e+00 0.0 6.9e+02 4.4e+01 0.0 3.0e+02H-DGA-S4-Ein 9.1e+01 0.0 5.2e+05 1.0e+00 0.0 9.2e+02 3.3e+01 0.0 3.3e+02GDGA-FCB 4.3e+00 80.0 1.4e+05 6.6e-03 53.3 3.1e+02 3.4e-03 0.0 1.3e+02DGA-S4-Log 2.2e+01 0.0 6.9e+02 1.8e-02 33.3 3.4e+00 2.1e-01 0.0 2.4e+01DGA-S4-Ham 6.7e+00 0.0 8.2e+04 2.1e-01 0.0 3.3e+02 2.4e+01 0.0 2.5e+02DGA-S4-Alb 9.2e+01 0.0 1.7e+06 1.0e+00 0.0 1.9e+03 8.9e+00 0.0 3.5e+02DGA-S4-Ein 1.1e+02 0.0 1.8e+06 5.3e-01 0.0 2.0e+03 9.9e+00 0.0 3.5e+02RGA-S2-180-Log 2.0e+01 0.0 5.1e+02 5.4e-03 40.0 1.9e+00 3.6e-03 100.0 1.0e+01RGA-S2-180-Ham 2.2e+01 0.0 7.5e+02 7.7e-03 60.0 5.7e+00 3.3e+01 0.0 7.8e+01RGA-S2-180-Alg 2.2e+01 0.0 6.8e+02 3.0e-03 73.3 4.3e+00 2.8e-01 0.0 3.6e+01RGA-S2-180-Ein 2.2e+01 0.0 7.0e+02 8.4e-03 53.3 4.2e+00 2.6e-01 0.0 3.4e+01RGA-DS1-180-Dub 2.1e+01 0.0 6.1e+02 6.7e-03 53.3 2.4e+00 1.3e-02 100.0 1.3e+01RGA-S2-60-Log 1.8e+01 0.0 3.9e+02 1.2e-02 46.7 1.6e+00 3.0e-05 100.0 6.1e+00RGA-S2-60-Ham 1.9e+01 0.0 5.6e+02 3.0e-02 6.7 4.5e+00 4.0e-02 80.0 4.7e+01RGA-S2-60-Alg 2.1e+01 0.0 5.2e+02 2.4e-02 6.7 3.1e+00 4.5e-02 73.3 3.1e+01RGA-S2-60-Ein 2.1e+01 0.0 5.4e+02 2.0e-02 20.0 3.0e+00 3.8e-02 86.7 2.8e+01RGA-DS1-60-Dub 1.6e+01 0.0 4.3e+02 1.2e-02 26.7 1.8e+00 2.4e-04 100.0 7.5e+00RGA-S4-360-Log 2.2e+01 0.0 8.6e+02 2.3e-03 73.3 3.7e+00 5.3e-01 0.0 2.5e+01RGA-S4-360-Ham 2.4e+01 0.0 9.3e+04 1.4e+00 0.0 3.6e+02 5.0e+01 0.0 2.5e+02RGA-S4-360-Alg 2.6e+02 0.0 2.2e+06 4.6e+00 0.0 2.1e+03 5.4e+01 0.0 3.6e+02RGA-S4-360-Ein 3.6e+02 0.0 2.2e+06 3.8e+00 0.0 2.2e+03 5.8e+01 0.0 3.6e+02RGA-DS2-360-Dub 2.2e+01 0.0 1.3e+03 3.3e-03 66.7 4.7e+00 5.8e-01 0.0 2.7e+01RGA-S4-120-Log 2.1e+01 0.0 6.6e+02 3.3e-02 13.3 3.2e+00 3.3e-02 93.3 2.5e+01RGA-S4-120-Ham 9.3e-01 0.0 8.7e+04 4.4e-01 0.0 3.5e+02 3.2e+01 0.0 2.5e+02RGA-S4-120-Alg 1.4e+02 0.0 2.1e+06 5.1e-01 0.0 2.1e+03 1.5e+01 0.0 3.6e+02RGA-S4-120-Ein 1.0e+02 0.0 2.2e+06 7.7e-02 6.7 2.2e+03 1.2e+01 0.0 3.6e+02RGA-DS2-120-Dub 2.0e+01 0.0 8.6e+02 1.7e-02 26.7 3.7e+00 4.2e-02 73.3 2.5e+01H-GDGA-FCB-R6 1.4e+00 93.3 7.9e+04 6.1e-03 46.7 2.3e+02 7.2e-02 13.3 1.1e+02H-DGA-S4-Log-R6 2.2e+01 0.0 9.0e+02 7.9e-03 33.3 4.2e+00 2.3e-01 0.0 2.3e+01H-DGA-S4-Ham-R6 2.2e+01 0.0 1.3e+04 1.7e-02 26.7 7.5e+01 1.8e+00 0.0 1.6e+02H-DGA-S4-Alg-R6 2.3e+01 0.0 2.4e+04 1.1e-02 33.3 1.1e+02 2.3e+00 0.0 1.7e+02H-DGA-S4-Ein-R6 2.3e+01 0.0 3.4e+04 8.6e-03 40.0 1.4e+02 1.7e+00 0.0 1.8e+02GDGA-FCB-R6 8.7e+00 60.0 1.4e+05 6.4e-03 46.7 3.2e+02 4.0e-01 0.0 1.3e+02DGA-S4-Log-R6 2.2e+01 0.0 7.2e+02 9.4e-03 40.0 3.5e+00 2.7e-01 0.0 2.5e+01DGA-S4-Ham-R6 1.9e+00 0.0 7.2e+04 1.7e-01 0.0 3.1e+02 1.9e+01 0.0 2.4e+02DGA-S4-Alg-R6 7.1e+01 0.0 1.5e+06 1.0e+00 0.0 1.7e+03 1.4e+01 0.0 3.5e+02DGA-S4-Ein-R6 8.6e+01 0.0 1.7e+06 8.7e-01 0.0 1.9e+03 8.8e+00 0.0 3.5e+02Table 5. ResultsNext, we point out some important considerations about the results obtained.? About the Hierarchical GDGAs� The B results of the hierarchical GDGA based on rings with six subpopulations, H-GDGA-FCB-R6, are better than the ones of the hierarchical GDGA based on rings with threesubpopulations, H-GDGA-FCB, for all the test functions. Furthermore, we also observe thatthe S performance of H-GDGA-FCB-R6 is better than the one of H-GDGA-FCB for fRos andfE�F10 and is equal for fGri. These results indicate that the hierarchical GDGAs achieve abetter behaviour when rings with a higher number of subpopulations are introduced.� We should highlight the good results of H-GDGA-FCB-R6 for the complex fRos, which hasbeen considered as a very complex test function ([47, 56, 26]). H-GDGA-FCB-R6 obtains the

best S value, 93.3, and the second B result, 1.4. Only, RGA-S4-120-Ham, a real-coded GAbased on the ST4 strategy, returned a better B result, 0.93, however its S performance is 0.This algorithm, RGA-S4-120-Ham, uses the ST4 strategy with the Hamacher FCB-crossoveroperators, which induces a high diversity level into the population such as is indicated in [25].Its O value shows this fact. The diversity allowed good results to be reached, however ane�cient �nal re�nement is not achieved and therefore S was not good. In H-GDGA-FCB-R6,the joined e�ects of the hierarchical structure and the gradual features produce high levelsof diversity (see its O value) and a suitable �nal local re�nement. In this way, reliabilityand accuracy were simultaneously improved and so the better result was obtained. Finally,we point out that the B and S results of the hierarchical GDGA based on rings with threesubpopulations, H-GDGA-FCB, for fRos are very good as well.� With regard to fGri and fE�F10, we observe that although H-GDGA-FCB-R6 doesn't returnthe best B results, it combines two notable qualities due to the union of the gradual featuresand the hierarchical structure:1. High O values, which indicates that it generated high diversity levels.2. Good �nal approximations, B values, which shows that the level of convergence wassuitable as well.There is not another algorithm showing simultaneously these two desirable properties. Forexample, the best algorithm for fGri, with respect to B, RGA-S4-360-Log, obtained 2.3e-03with an O value of 3.7, however, H-GDGA-FCB-R6 obtained 6.1e-03 for B with a higherO value, 2.3e+02. So, it may be considered that H-GDGA-FCB-R6 established a goodrelationship between exploration and exploitation for avoiding the premature convergenceproblem without sacri�cing the obtaining of good approximations.� In general, the hierarchical GDGA based on rings with six subpopulations, H-GDGA-FCB-R6, improves the behaviour of the GDGAs executed, GDGA-FCB and GDGA-FCB-R6. ForfRos, H-GDGA-FCB-R6 reaches better B and S results than these algorithms, for fGri, itachieves a better B result, and �nally for fE�F10, it returns better B and S results thanGDGA-FCB-R6 and a better S value than GDGA-FCB.We may conclude that the inclusion of a hierarchical structure in the GDGAs, improves thebehaviour of these algorithms in such a way that both, reliability and accuracy, are carried throughto a suitable conclusion. This fact allowed good results to be reported for complex test functions.? About the Hierarchical Homogeneous DGAs� Comparing the B results of the hierarchical homogeneous H-DGA-S4-Log, ..., H-DGA-S4-Ein, which are based on rings with three subpopulations, with the ones of their correspondinghomogeneous DGAs, DGA-S4-Log, ..., DGA-S4-Ein, we observe that, in general, no improve-ments are produced. So, the hierarchical structure based on rings with three subpopulationsdoesn't seem to introduce a suitable way for improving clearly the results for the complextest functions used.� Hierarchical homogeneous H-DGA-S4-Log-R6, ..., H-DGA-S4-Ein-R6, which are based onrings with six subpopulations, improve the B performance of their corresponding homoge-neous DGAs, DGA-S4-Log-R6,..., DGA-S4-Ein-R6, and the one of the homogeneous DGAsDGA-S4-Log, ..., DGA-S4-Ein. So, we observe again that the use of rings with six subpopu-lation introduces a good behaviour of the hierarchical structure on the test functions.

� In general, we may observe that the results of the GDGAs, GDGA-FCB and GDGA-FCB-R6,are better than the ones of homogeneous DGAs, DGA-S4-Log, ... DGA-S4-Ein and DGA-S4-Log-R6,..., DGA-S4-Ein-R6, respectively. Furthermore, the results of the hierarchicalGDGAs, H-GDGA-FCB and H-GDGA-FCB-R6, are better than the ones of the hierarchicalhomogeneous DGAs, H-DGA-S4-Log, ... DGA-S4-Ein and DGA-S4-Log-R6,..., DGA-S4-Ein-R6, respectively. These results con�rm the suitability of the multiresolution based on thecrossover operator, such as it was already claimed in [25].The two previous considerations have highlighted, separately, that the hierarchical structureand the multiresolution based on crossover operators are promising ways for empowering DGAs tohave a great measure of success on complex problems.? General Remarks About the Hierarchical StructureThe hierarchical structure proposed has a better behaviour using rings with a higher numberof subpopulations. Next, we attempt to explain this fact.In [7], Cant�u-Paz wrote:"The topology is an important factor in the performance of the parallel GA, becauseit determines how fast (or how slow) a good solution spreads to other demes. If thetopology has a dense connectivity (or a short diameter, or both), good solutions willspread fast to all the demes and may quickly take over the population. On the otherhand, if the topology is sparsely connected (or has a long diameter), solutions will spreadslower and the demes will be more isolated from each other, permitting the appearanceof di�erent solution. These solutions may come together at a later time and recombineto form potentially better individuals."With these words, Cant�u-Paz pointed out that the topology has important e�ects in the deter-mination of the exploitation versus exploration tradeo� of DGAs. If it has a dense connectivity,exploitation comes into force, whereas if it is sparsely connected, diversity is kept and so explorationtakes e�ect.The hierarchical structure based on rings with six subpopulations combines sparsely connectedtopologies, the rings of the basic DGAs, with cubic topology of the HDGA, which has a shortdiameter. In this way, the basic DGAs develop exploration, and the topology that joins themproduces exploitation.So, the hierarchical structure together with a multiresolution based on the crossover operatorand the global and local migrations produce an e�ective exploration/exploitation balance, andtherefore results are improved.6 Concluding RemarksIn this paper, we have presented HDGAs, DGAs whose nodes are simple DGAs, and in whichtwo types of migrations are produced, the local ones and the global ones. We also presented amodel that mixes the hierarchical structure and the idea of the GDGAs, the hierarchical GDGAs.The results of the experiment carried out with HDGAs and hierarchical GDGAs have shown that:� HDGAs has a better behaviour using rings with a higher number of subpopulations, improv-ing the results of traditional DGAs.� An e�ective exploitation/exploration balance may be induced by means of the combinationof the two types of topologies underlying in the hierarchical structure, the one of the basicDGAs and the one that connects them.

� The good results obtained on the test function used and in particular on the complex gen-eralized Rosenbrock's function allow us to point out that HDGAs are promising for dealingwith complex problems� The union of the gradual features and the hierarchical structure allow reliability and accuracyto be improved.So, all these conclusions indicate that the introduction of a hierarchy between DGAs is apromising way for increasing the e�cacy of DGAs. This paper is a �rst investigation on a basicimplementation of this idea, the connection of di�erent DGAs establishing a higher level DGA.Finally, we should point out that future research need to be done in the following topics relatedwith HDGAs:� Study other topologies for the basic DGAs and the ones that connect them. This studyshould consider the most appropriate multiresolution based models as basic DGAs and themost suitable types of local and global migrations associated with these topologies. A partof this study should tackle the determination of a correct balance between the behaviour ofthe basic DGAs and the global behaviour of the HDGA.� Study the impact of the control parameters that a�ect the behaviour of the basic DGAs andthe higher level DGA, such as migration rates, migration intervals, subpopulation sizes, etc.� Find e�cient models for the selection of the chromosomes that migrate, the selection of thechromosomes in the destination subpopulations that are replaced by the ones that arrive anddeal with the choice of whether or not to replicate migrating individuals.� Design homogeneous HDGAs with heterogeneous basic DGAs, such as a homogeneous hier-archical GDGA composed by the same type of basic GDGAs.References[1] P. Adamidis, Review of Parallel Genetic Algorithms Bibliography. Technical Report, AristotleUniversity of Thessaloniki, Thessaloniki, Greece, 1994.[2] T. B�ack, Selective Pressure in Evolutionary Algorithms: A Characterization of SelectionMechanisms, In: Proc. of The First IEEE Conference on Evolutionary Computation, 1994,57-62.[3] T. B�ack, T. Beielstein, B. Naujoks, J. Heistermann, Evolutionary Algorithms for the Opti-mization of Simulation Models Using PVM. In: Proc. EuroPVM'95: Second European PVMUser's Group Meeting, Paris, J. Dongarra, M. Gengler, B. Tourancheau X. Vigouroux (Eds),1995, 277-282.[4] T. B�ack, Evolutionary Algorithms in Theory and Practice, Oxford, 1996.[5] J.E. Baker, Adaptive Selection Methods for Genetic Algorithms. In: Proc. of the First Int.Conf. on Genetic Algorithms and their Applications, J.J. Grefenstette (Ed.) (L. ErlbaumAssociates, Hillsdale, MA, 1985), 101-111.[6] T.C. Belding, The Distributed Genetic Algorithm Revisited. In: Proc. of the Sixth Int. Conf.on Genetic Algorithms, L. Eshelman (Ed.) (Morgan Kaufmann Publishers, San Francisco,1995), 114-121.[7] E. Cant�u-Paz, A Summary of Research on Parallel Genetic Algorithms. IlliGAL Report95007, Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign,IL, 1995.

[8] J. Cohoon, S. Hedge, W. Martin, D. Richards, Punctuated Equilibria: A Parallel GeneticAlgorithm. In: Proc. of the Second Int. Conf. on Genetic Algorithms and their Applications,J.J. Grefenstette (Ed.) (L. Erlbraum Associates, Hillsdale, MA, 1987), 148-154.[9] J.P. Cohoon, W.N. Martin, D.S. Richards, Genetic Algorithms and Punctuated Equilibria inVLSI. In: Parallel Problem Solving from Nature 1, H.-P. Schwefel, R. M�anner (Eds.) (Berlin,Germany, Springer-Verlag, 1990), 134-144.[10] R.J. Collins, Studies in Arti�cial Evolution. Doctoral Dissertation, University of California,Los Angeles, CA, 1992.[11] Y. Davidor, A Naturally Ocurring Niche & Species Phenomenon: The Model and FirstResults. In: Proc. of the Fourth Int. Conf. on Genetic Algorithms, R. Belew, L.B. Booker(Eds.) (Morgan Kaufmmann, San Mateo), 257-263.[12] L. Davis, Handbook of Genetic Algorithms, Van Nostrand Reinhond, New York, 1991.[13] K.A. De Jong, An Analysis of the Behavior of a Class of Genetic Adaptive Systems. DoctoralDissertation, University of Michigan, 1975.[14] M. Dorigo, V. Maniezzo, Parallel Genetic Algorithms: Introduction and Overview of CurrentResearch. In: Parallel Genetic Algorithms: Theory and Applications, IOS press, 1993, 5-42.[15] N. Eldredge, S.J. Gould, Punctuated Equilibria: An Alternative to Phyletic Gradualism. In:Models of Paleobiology, T.J.M Schopf (Ed.) (San Francisco, CA: Freeman, Cooper, 1972),82-115.[16] D.E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning,Addison-Wesley, New York, 1989.[17] V.S. Gordon, D. Whitley, A.P.W. B�ohm, Data ow Parallelism in Genetic Algorithms. In:Parallel Problem Solving from Nature 2, R. M�anner, B. Manderick (Eds.) (Elsevier SciencePublishers, Amsterdam, 1992), 533-542.[18] J.J. Grefenstette, Parallel Adaptive Algorithms for Function Optimization. Technical ReportCS-81-19, Vanderbilt University, Nashville, TN, 1981.[19] J.J. Grefenstette, Optimization of Control Parameters for Genetic Algorithms. IEEE Trans.on Systems, Man, and Cybernetics 16, 1986, 122-128.[20] A.O. Griewangk, Generalized Descent of Global Optimization. JOTA 34, 1981, 11-39.[21] F. Gruau, The Mixed Genetic Algorithm. In: Parallel Computing: Trends and Application,G.R. Joubert, D. Trystram, F.J. Peters, D.J. Evans (Eds), Elsevier, 1994.[22] F. Herrera, M. Lozano, J.L. Verdegay, Dynamic and Heuristic Fuzzy Connectives BasedCrossover Operators for Controlling the Diversity and Convergence of Real-Coded GeneticAlgorithms. International Journal of Intelligent Systems 11, 1996, 1013-1041.[23] F. Herrera, M. Lozano, Adaptation of Genetic Algorithm Parameters Based on Fuzzy LogicControllers. In: Genetic Algorithms and Soft Computing, F. Herrera, J.L. Verdegay (Eds.),Physica-Verlag (Studies in Fuzziness and Soft Computing, Vol. 8), 1996, 95-125.[24] F. Herrera, M. Lozano, J.L. Verdegay, Tackling Real-Coded Genetic Algorithms: Operatorsand Tools for Behavioural Analysis. Arti�cial Intelligent Review, 1997. To appear.[25] F. Herrera, M. Lozano, J.L. Verdegay, Fuzzy Connectives Based Crossover Operators toModel Genetic Algorithms Population Diversity. Fuzzy Sets and Systems, 1997. To appear.[26] F. Herrera, M. Lozano, Gradual Distributed Genetic Algorithms. Technical Report#DECSAI-97-01-03, Dept. of Computer Science and Arti�cial Intelligence, University ofGranada, Spain, 1997.[27] J.H. Holland, Adaptation in Natural and Arti�cial Systems, The University of MichiganPress, 1975. (The MIT Press, London, 1992).

[28] U. Kohlmorgen, Overview of Parallel Genetic Algorithms. In: Proc. of the First NordicWorkshop of Genetic Algorithms, Vaasa (Finland), 1995, 135-145.[29] B. Kr�oger, P. Schwenderling, O. Vornberger, Parallel Genetic Packing on Transputers. In:Parallel Genetic Algorithms: Theory and Applications, IOS press, 1993, 151-186.[30] T-H. Li, C.B. Lucasius, G. Kateman, Optimization of Calibration Data with the DynamicGenetic Algorithm. Analytica Chimica Acta 2768, 1992, 123-134.[31] S-C Lin, W.F. Punch III, E.D Goodman, Coarse-Grain Genetic Algorithms: Categorizationand New Approach. In: Proc. Sixth IEEE Parallel and Distributed Processing, 1994, 28-37.[32] A. Loraschi, A. Tettamanzi, M. Tomassini, P. Verda, Distributed Genetic Algorithms withan Application to Portfolio Selection. Arti�cial Neural Networks and Genetic Algorithms,D.W. Pearson, N.C. Steele, R.F. Albrecht (Eds), Springer-Verlag, 1995.[33] S.W. Mahfoud, Niching Methods for Genetic Algorithms. IlliGAL Report 95001, IllinoisGenetic Algorithms Laboratory, University of Illinois at Urbana-Champaign, IL, 1995.[34] B. Manderick, P. Spiessens, Fine-Grained Parallel Genetic Algorithms. In: Proc. of the ThirdInt. Conf. on Genetic Algorithms, J. D. Scha�er (Ed.) (Morgan Kaufmann Publishers, SanMateo, 1989), 428-433.[35] J. Maresky, On E�cient Communication in Distributed Genetic Algorithms. M.S. Disserta-tion, Institute of Computer Science, The Hebrew University of Jerusalem, 1994.[36] J. Maresky, Y. Davidor, D. Gitler, G. Aharoni, A. Barak, Pro�ling Communication in Dis-tributed Genetic Algorithms. In: Proc. of the Fourteenth International Joint Conference onArti�cial Intelligence, (Morgan Kaufmann Publishers, San Mateo, California, 1995), 961-966.[37] Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs, Springer-Verlag, New York, 1992.[38] M. Mitchell, An Introduction to Genetic Algorithms, The MIT Press, 1996.[39] H. M�uhlenbein, Parallel Genetic Algorithms, Population Genetics and Combinatorial Op-timization. In: Proc. of the Third Int. Conf. on Genetic Algorithms, J.D. Scha�er (Ed.)(Morgan Kaufmann Publishers, San Mateo, 1989), 416-421.[40] H. M�uhlenbein, Evolution in Time and Space - The Parallel Genetic Algorithm. In: Founda-tions of Genetic Algorithms 1, G.J.E. Rawlin (Ed.) (Morgan Kaufmann, San Mateo, 1991),316-337.[41] H. M�uhlenbein, M. Schomisch, J. Born, The Parallel Genetic Algorithm as Function Opti-mizer. In: Fourth Int. Conf. on Genetic Algorithms, R. Belew, L.B. Booker (Eds.) (MorganKaufmmann, San Mateo, 1991), 271-278.[42] J.C Potts, T.D. Giddens, S.B. Yadav, The Development and Evaluation of an ImprovedGenetic Algorithm Based on Migration and Arti�cial Selection. IEEE Trans. on Systems,Man, and Cybernetics 24, 1994, 73-86.[43] J.F. Preparata, J.E. Vuillemin, The Cube-Connected Cycles: A Versatile Network for ParallelComputation. Communications of the ACM 24(5), 1981, 300-309.[44] W. Punch, E. Goodman, M. Pei, L. Chai-Shun, P. Hovland, R. Enbody, Further Researchon Feature Selection and Classi�cation Using Genetic Algorithms. In: Proc. of the FifthInt. Conf. on Genetic Algorithms, S. Forrest (Ed.) (Morgan Kaufmmann, San Mateo, 1993),557-564.[45] C. Ryan, Niche and Species Formation in Genetic Algorithms. In: Practical Handbook ofGenetic Algorithms: Applications, L. Chambers (Ed.) (CRC Press, 1995), 57-74.

[46] J.D. Scha�er, R.A. Caruana, L.J. Eshelman, R. Das, A Study of Control Parameters A�ectingOnline Performance of Genetic Algorithms for Function Optimization. In: Proc. of the ThirdInt. Conf. on Genetic Algorithms, J.D. Scha�er (Ed.) (Morgan Kaufmann Publishers, SanMateo, 1989), 51-60.[47] D. Schlierkamp-Voosen, H. M�uhlenbein, Strategy Adaptation by Competing Subpopulations.In: Parallel Problem Solving from Nature 3, Y. Davidor, H.-P. Schwefel, R. M�anner (Eds.)(Berlin, Germany, Springer-Verlag, 1994), 199-208.[48] V. Schnecke, O. Vornberger, An Adaptive Parallel Genetic Algorithm for VLSI-Layout Opti-mization. In: 4th International Conference on Parallel Problem Solving from Nature, Berlin(Germany), Lecture Notes on Computer Science Vol. 1141, Springer, 1996, 859-868.[49] H-P. Schwefel, Numerical Optimization of Computer Models, Wiley, Chichester, 1981.[50] R. Tanese, Parallel Genetic Algorithms for a Hypercube. In: Proc. of the Second Int. Conf. onGenetic Algorithms and their Applications, J.J. Grefenstette (Ed.) (L. Erlbraum Associates,Hillsdale, MA, 1987), 177-183[51] R. Tanese, Distributed Genetic Algorithms. In: Proc. of the Third Int. Conf. on GeneticAlgorithms, J. David Scha�er (Ed.) (Morgan Kaufmann Publishers, San Mateo, 1989), 434-439.[52] R. Tanese, Distributed Genetic Algorithms for Function Optimization. Doctoral Dissertation,University of Michigan, Ann Arbor, MI, 1989.[53] H.M. Voigt, J. Born, A Structured Distributed Genetic Algorithm for Function Optimization.In: Parallel Problem Solving from Nature 2, R. M�anner, B. Manderick (Eds.) (ElsevierScience Publishers, Amsterdam, 1992), 199-208.[54] H.M. Voigt, H. M�uhlenbein, D. Cvetkovi�c, Fuzzy Recombination for the Breeder GeneticAlgorithm. In: Proc. of the Sixth Int. Conf. on Genetic Algorithms, L. Eshelman (Ed.)(Morgan Kaufmann Publishers, San Francisco, 1995), 104-111.[55] D. Whitley, D. Starkweather, Genitor-II: A Distributed Genetic Algorithm. J. Expt. Theor.Artif Int. 2, 1990, 189-214.[56] D. Whitley, S. Rana, J. Dzubera, E. Mathias, Evaluating Evolutionary Algorithms. Arti�cialIntelligence 85, 1996, 245-276.[57] D. Whitley, R. Beveridge, C. Graves, K. Mathias, Test Driving Three 1995 Genetic Algo-rithms: New Test Functions and Geometric Matching. Journal of Heuristics 1, 1995, 77-104.[58] S. Wright, The Roles of Mutation, Inbreeding, Crossbreeding and Selection in Evolution. In:Proc. of the Sixth International Congress of Genetics, Vol. 1, 1934, 356-366.