A Hybrid Gravitational Search with Levy Flight for Global Numerical Optimization

Inf. Sci. Lett.4, No. 2, 71-83 (2015) 71

Information Sciences LettersAn International Journal

http://dx.doi.org/10.12785/isl/040204

A Hybrid Gravitational Search with L evy Flight for GlobalNumerical OptimizationAhmed Fouad Ali∗

Suez Canal University, Dept. of Computer Science, Faculty of Computers and Information, Ismailia, Egypt

Received: 26 Feb. 2105, Revised: 25 Mar. 2015, Accepted: 26 Mar. 2015Published online: 1 May 2015

Abstract: Gravitational search algorithm (GSA) is a recent population based metaheuristics algorithm. It has a good ability to performa wide exploration and a deep exploitation, however it becomes inactive when the premature convergence happens and loses its abilityto explore new solutions in the search space. In order to avoid this problem, we propose in this paper a new hybrid gravitational searchalgorithm with a Levy flight operator. The proposed algorithm is called Hybrid Gravitational Search with Levy Flight (HGSLF). Whenthe distance between two masses become very close and both ofthem are not a best solution in the population, the Levy flight operatoris applied on one of them to increase the diversity of the algorithm and avoid trapping in local minima. The general performanceof the proposed HGSLF algorithm is tested on 13 unconstrained (7 uni-model problems and 6 multi-model problems), 8 constrainedoptimization problems and compared against 8 different algorithms. The numerical results show that the proposed HGSLFalgorithmcan solve unconstrained, constrained optimization problems in reasonable time and faster than standard GSA and other comparativealgorithms.

Keywords: Gravitational search algorithm, Levy flight, unconstrained optimization problems, constrained optimization problems,numerical function optimization

1 Introduction

Gravitational search algorithm (GSA) is a nature inspiredalgorithm which is based on the law of gravity and massinteractions [1]. GSA was proposed by Rashedi et al. [1]in order to solve global optimization problems. GSA is apopulation based metaheuristics algorithm, the solution inthe population is called agent or searcher agent whichinteract with each other through the gravity force [1]. Theperformance of each agent (solution) in the population ismeasured by their masses.

Although GSA and other metaheuristics algorithmssuch as Ant Colony Optimization (ACO) [2], ParticleSwarm Optimization (PSO) [3], Artificial Bee Colony [4],Firefly algorithm [5], Bacterial foraging [6], Batalgorithm [7], Wolf search [8], Bee Colony Optimization(BCO) [9], Cat swarm [10], Fish swarm/school [11], etc,have been applied to solve global optimization problems,they suffer from slow convergence. Due to the powerfulperformance of the GSA and its ability to balancebetween exploration and exploitation, many researchershave applied it in their works such as Yazdani et al. [12]proposed Niche GSA (NGSA) algorithm to find multiple

solutions in multimodal problems. In NGSA, the mainswarm of masses is divided into smaller sub-swarms andthree strategies (K-nearest neighbors (K-NN), an elitismstrategy and modification of active gravitational massformulation) are applied to preserve sub-swarms.Doraghinejad et al [13] improved the performance of thestandard GSA to solve unimodel optimization problemsby inserting a black hole operator in GSA and assumingsome of the heavy objects are stars in a gravitationalsystem. In Soleimanpour et al. work [14], the state of amass is presented by wave function instead of positionand velocity to find the optimum result for unimodel andmultimodel functions. Zhang et al. [15] improved theconvergence speed and antibody diversity to raisediversity of agent to avoid falling into local optimumsolution. Wang and Li [16] improved the performance ofthe standard gravitational search algorithm by introducingthree boundary conditions for solving unconstrainedoptimization. Sombra et al [17] applied change in alphaparameter throughout the iterations to achieve betterconvergence than the standard GSA. Hatamloue et al.[18] incorporated a k-mean algorithm in generating the

∗ Corresponding author e-mail:[email protected]

c© 2015 NSPNatural Sciences Publishing Cor.

http://dx.doi.org/10.12785/isl/040204

72 A. F. Ali: A Hybrid Gravitational Search with Levy Flight for...

initial population for GSA to increase the convergencespeed of the GSA algorithm.

The aim of this paper is to propose a new hybridgravitational search algorithm with Levy flight operator toincrease the exploration ability of the standardgravitational search algorithm and avoid the prematureconvergence and the stagnation in order to solve globaloptimization problems. The proposed algorithm is calledHybrid Gravitational Search with Levy Flight (HGSLF).The Levy flight is applied when the distance between twomasses become very close and non of them is a bestsolution in the population. Invoking the Levy flightoperator can accelerate the search and increase thediversity of the algorithm and avoid trapping in localminima.

The rest of the paper is organized as follow. Thedefinition of the unconstrained and constrainedoptimization problems is presented in Section2, InSection3, we describe in details the standard gravitationalsearch algorithm. The proposed HGSLF algorithm ispresented in details in Section4, In Section 5, wereported the experimental results and finally, theconclusion makes up Section6.

2 Definition of the problems

In this section and its subsections, we present thedefinitions of the unconstrained and constrainedoptimizations problems as follow.

2.1 Unconstrained optimization problems

Mathematically, the optimization is the minimization ormaximization of a function of one or more variablessubject to constrains on its variables. By using thefollowing Equation:

minl≤x≤u

f (x) (1)

–x = (x1,x2, ...,xn) - a vector ofvariablesor functionparameters;

–f - the objective functionthat is to be minimized ormaximized; a function ofx;

–l = (l1, l2, ..., ln) andu= (u1,u2, ...,un) - thelower andupper boundsof the definition domain forx;

–c - a set of functions ofx that represent theconstraints;

2.2 Constrained optimization problems

The constrained optimization problems and constrainthandling is one of the most challenging in manyapplications. A general form of a constrainedoptimization is defined as follows:

Minimize f (x), x= (x1,x2, · · · ,xn)T , (2)

Subject to

gi(x)≤ 0, i = 1, · · · ,mh j(x) = 0, j = 1, · · · , lxl ≤ xi ≤ xu

Where f (x) is the objective function,x is the vector ofnvariables,gi(x) ≤ 0 are inequality constraints,h j(x) = 0are equality constraints,xl ,xu are variables bounds. Thereare differen techniques to handel constraints in manyoptimization algorithms, these techniques are classifiedby Michalewicz [19] as follows:

–Penalty function technique.–Rejection of infeasible solutions technique.–Repair algorithms technique.–Specialized operators technique.–Behavior memory technique.

In this paper, we used the penalty function technique tosolve constrained optimization problems.

3 Overview of gravitational search algorithm

In the following steps, we will give an overview of themain concepts and structure of the gravitational searchalgorithm as follow.

–Main conceptsGravitational search algorithm (GSA) is a populationsearch algorithm proposed by Rashedi et al. in 2009[1]. The GSA is based on the low of gravity and massinteractions. The solutions in the GSA population arecalled agents, these agents interact with each otherthrough the gravity force. The performance of eachagent in the population is measured by its mass. Eachagent is considered as object and all objects movetowards other objects with heavier mass due to thegravity force. This step represents a globalmovements (exploration step) of the object, while theagent with a heavy mass moves slowly, whichrepresents the exploitation step of the algorithm. Thebest solution is the solution with the heavier mass.

–Gravitational constant GThe gravitational constantG at iterationt is computedas follows.

G(t) = G0e−αt/T (3)

WhereG0 andα are initialized in the beginning of thesearch, and their values will be reduced during thesearch.T is the total number of iterations.

–The gravity lowThe objects masses are obeying the low of gravity asshown in Equation4 and the low of motion as shownin Equation5

F = GM1M2

R2 (4)


Inf. Sci. Lett.4, No. 2, 71-83 (2015) /www.naturalspublishing.com/Journals.asp 73

Equation 4 represents the Newton law of gravity,whereF is a magnitude of the gravitational force,G isgravitational constant,M1 is the mass of the firstobject,M2 is the mass of the second object andR isthe distance between the two objectsM1, M2.According to the Newton’s second low, when a forceF is applied to an object, the object moves withaccelerationa depending on the applied force and theobject massM as shown in Equation5.

a=FM

(5)

–Acceleration of agentsThere are three kind of masses, active gravitationalmassMa, passive gravitational massMp and inertialmassMi . The gravitational forceFi j that acts on massi by massj is defined by:

Fi j = GMa j ×Mpi

R2 (6)

WhereMa j, Mpi are the active and passive masses ofobjects j, i, respectively. The acceleration of object(agent)i is computed as follows.

ai =Fi j

Mii(7)

WhereMii is inertia mass of agenti.–Agent velocity and positionsDuring the search, theagents update their velocities and positions as shownin Equations8, 9, respectively.

Vi(t +1) = randi ×Vi(t)+ai(t). (8)

Xi(t +1) = randi ×Vi(t)+ai(t). (9)

3.1 Gravitational search algorithm

In this subsection, we present the main structure of thestandard gravitational search algorithm as shown inAlgorithm 1.

The main steps of the GSA can be summarized asfollows.

–Step 1. The algorithm starts by setting the initialvalues of gravitational constantG0, α, ε and theiteration countert.

–Step 2.The initial population is generated randomlyand consists ofN agents, the position of each agent isdefined by:

Xi(t) = (x1i (t),x

2i (t), . . . ,x

di (t), . . . ,x

ni (t)), i = 1,2, . . . ,N,

Wherexdi presents the position of the agenti in thedth

dimension.–Step 3. The following steps are repeated untiltermination criteria satisfied

–Step 3.1.All agents in the population are evaluated andthe best, worst agents are assigned.

Algorithm 1 The standard gravitational search algorithm1: Set the initial values of gravitational constantG0, α andε.2: Set the initial iterationt = 0.3: for i = 1;i ≤ N do4: Generate an initial populationXi(t) randomly, where

Xi(t) = (x1i (t),x

2i (t), . . . ,x

di (t), . . . ,x

ni (t)).

5: end for6: repeat7: Evaluate the fitness functionf (Xi(t)) for each agent in the

populationX(t).8: Assign the best, worst agent in the populationX(t).9: Update the gravitational constantG as shown in Equation

3.10: for i = 1;i ≤ N do11: for j = i+1; j < N do12: Calculate the force acting on agenti from agentj as

shown in Equation10.13: Calculate the total force that act on agenti, as shown

in Equation11.14: Calculate inertial massMi as shown in Equations12,

13 .15: Calculate the acceleration of the agenti as shown in

Equation1416: Update the velocity of agenti as shown in Equation

8.17: Update the position of agenti as shown in Equation

9.18: end for19: end for20: Sett = t +1.21: until Termination criteria satisfied22: Return the best solution

–Step 3.2.The gravitational constant is updated asshown in Equation3

–Step 3.3.When agentj acts on agenti with force, at aspecific time (t) the force is calculated as following:

Fdi j (t) = G(t)

Mpi(t)×Ma j(t)

Ri j (t)+ ε(xd

j (t)− xdi (t)) (10)

WhereMa j is the active gravitational mass of agentj,Mpi is the passive gravitational mass of agenti, G(t) isgravitational constant at timet

–Step 3.4.At iterationt, calculate the total force actingon agenti as following:

Fdi (t) = ∑

j∈Kbest, j 6=i

randjFdi j (t) (11)

Where Kbest is the set of firstK agents with the bestfitness value and biggest mass

–Step 3.5.Calculate the inertial mass as following:

mi(t) =f it i −worst(t)

best(t)−worst(t)(12)

Mi(t) =mi(t)

∑Nj=1mj(t)

(13)


www.naturalspublishing.com/Journals.asp


–Step 3.6.The acceleration of agenti is calculated asfollowing:

ai(t) =Fi(t)Mii (t)

. (14)

–Step 3.7.The velocity and the position of agenti arecomputed as shown in Equations8, 9

–Step 3.8. The iteration counter is increased untiltermination criteria satisfied

–Step 4.The best optimal solution is produced.

3.2 Levy flights

Recent studies show that the behavior of many animalswhen searching for foods have the typical characteristicsof Levy Flights. [20], [23], [22] and [21]. Levy flight [20]is a random walk in which the step-lengths are distributedaccording to a heavy-tailed probability distribution. Aftera large number of steps, the distance from the origin ofthe random walk tends to a stable distribution.

A new solution is generated randomly using a L ´evyflight as follow.

xt+1i = xt

i +α ⊕Levy(λ ), (15)

Where⊕ denotes entry-wise multiplication,α is thestep size, and Levy (λ ) is the Levy distribution.

4 The proposed HGSLF algorithm

The main steps of the proposed HGSLF algorithm arepresented in Algorithm2. The HGSLF algorithm uses theLevy flight operator when the distance between twoneighboring solutions are too close to each other and bothof them are not the best global solution. We cansummarize the main steps of the proposed algorithm asfollow.

–Step 1.The proposed algorithm applies the standardGSA algorithmline (1-19)

–Step 2.If the Euclidian distanceR between solutioniand solution j is less thanξ , where ξ is a smallconstant,i 6= j 6= g∗, g∗ is the global best solution,then the Levy flight operator is applied in one of themin order to increase the exploration process in thesearch space and avoid trapping in local minima.line(20-23)

–Step 3. The overall processes are repeated tilltermination criteria satisfied and the best obtainedsolution is producedline (24-26)

5 Numerical experiments

In this section, we investigate the general performance ofthe proposed HGSLF by testing it on 13 unconstrained

Algorithm 2 The proposed HGSLF algorithm1: Set the initial values of gravitational constantG0, α andξ .2: Set the initial iterationt = 0.3: for i = 1;i ≤ N do4: Generate an initial populationXi(t) randomly, where

Xi(t) = (x1i (t),x

2i (t), . . . ,x

di (t), . . . ,x

ni (t)).

5: end for6: repeat7: Evaluate the fitness functionf (Xi(t)) for each agent in the

populationX(t).8: Assign the best, worst agent in the populationX(t).9: Update the gravitational constantG as shown in Equation

3.10: for i = 1;i ≤ N do11: for j = i+1; j < N do12: Calculate the force acting on agenti from agentj as

shown in Equation10.13: Calculate the total force that act on agenti, as shown

in Equation11.14: Calculate inertial massMi as shown in Equations12,

13 .15: Calculate the acceleration of the agenti as shown in

Equation1416: Update the velocity of agenti as shown in Equation

8.17: Update the position of agenti as shown in Equation

9.18: Calculate the Euclidean distanceRi j between agent

i and agentj19: if Ri j < ξ then20: Update the position of agenti using Levy flight

operator as shown in Equation15.21: end if22: end for23: end for24: Sett = t +1.25: until Termination criteria satisfied26: Return the best solution

and 8 constrained optimization problems and comparingit against variant algorithms. HGSLF was programmed inMATLAB, the results of the comparative algorithms aretaken from their original papers. In the followingsubsections, the parameter setting of the proposedalgorithm with more details and the properties of theapplied test functions have been reported as follow. Theparameters of the HGSLF algorithm are reported withtheir assigned values as shown in Table1. These values

Table 1: Parameter setting.Parameters Definitions ValuesN Population Size 50G0 Gravitational constant 100α Gravitational constant 20ξ Threshold constant 10−3

Maxitr Maximum number of iterations 1000



are based on the common setting in the literature ofdetermined through our preliminary numericalexperiments.

–Population size N. The population size is set toN = 50, increasing this number will increase theevaluation function values without any improvementin the obtained results.

–Standard gravitational search algorithmparameters G0, α. In the proposed algorithm, wehave applied the same standard gravitational searchalgorithm parameters with their values, which arereported in [1] where the gravitational constantG0 = 100 andα = 20.

–Threshold constantξ The experimental tests showthat the best threshold constantξ value is set to 10−3.

–Maximum number of iterations Maxitr In order tomake a fair comparison between the proposedalgorithm and the other algorithms, We have appliedthe same termination criterion which is the maximumnumber of iterations is set to 1000.

5.1 Unconstrained test problems

The proposed algorithm is tested on 13 unconstrainedoptimization functions (7 unimodel functions, 6multimodel functions), which are listed in Tables2, 3,respectively.

5.2 The efficiency of the proposed HGSLFalgorithm with unconstrained problems

In order to investigate the idea of combining the Levyflight operator in the proposed algorithm with thestandard gravitational search algorithm, we present thegeneral performance of the proposed algorithm and thegeneral performance of the standard gravitational searchalgorithm by plotting the number of iterations (functionevaluations) versus the function values (mean errors) forFunctionsF7, F8, F9 andF10 as shown in Figure1. InFigure 1, the solid line represents the standardgravitational search algorithm (SGSA) results, while thedotted line represents the proposed HGSLF algorithmresults. Figure1 show that, the function values of theproposed algorithm are rapidly decreases faster than thefunction values of the standard gravitational searchalgorithm. We can conclude from Figure1, that thecombination of the Levy flight operator with he standardgravitational search algorithm can accelerate the searchand avoid stagnation and trapping in local minima.

5.3 The general performance of the HGSLFalgorithm with unconstrained problems

The general performance of the proposed HGSLFalgorithm is shown in Figure2, by plotting the function

values (mean errors) versus the number of iterations(function evaluations) for functionsF1, F2, F3, F5, F6 andF12 (piked randomly). Figure2 shows that the functionvalues are rapidly decreases while the number ofiterations are slightly increases (few number ofiterations).

5.4 HGSLF and other algorithms forunconstrained optimization problems

The proposed HGSLF algorithm is compared against 3algorithms in order to investigate its performance withunconstrained optimization problems. These algorithmsare reported as follow.

–SGSA. SGSA is a standard gravitational searchalgorithm [1]. The main parameters of SGSA is set asfollow. G0 is set to 100,α is set to 20

–PSO. PSO is a particle swarm optimization algorithm,which is proposed by Kennedy and Eberhart in 1995[3]. The parametersr1, r2 ∈ [0,1], c1 = c2 are positiveconstants and equal to 2 and the inertia weightω isdecreasing linearly from 0.9 to 0.2.

–RGA. RGA is a real genetic algorithm, RGA uses aroulette wheel selection and arithmetical crossover,Gaussian mutation with crossover and mutationprobabilitiesPc, Pm equal to 0.3 and 0.1, respectivelyas presented in [25].

5.4.1 Comparison between HGSLF and other algorithmsfor unconstrained optimization problems.

The proposed HGSLF algorithm is applied on 13unconstrained (7 uni-model and 6 multi-model)optimization problems with 30 dimension and comparedagainst 3 algorithms. The average, median, best andstandard deviation (Std) are reported over 30 runs asshown in Table4 for the uni-model problems and in Table5 for the multi-model problems. In order to make a faircomparison, the proposed algorithm applies the sametermination criterion in the other algorithms which is themaximum number of iterationstmax is equal to 1000. InTables4, 5, the best results are reported inbold text. Theresults in Tables4, 5 show that the proposed HGSLFalgorithm is better than the other algorithms in mostcases.

5.5 Constrained optimization problems

The second investigation of the proposed HGSLFalgorithm is to test it on 8 constrained optimizationproblems and compare it against 6 algorithms. The testedconstrained functions are reported in Table6, while theoptimal value for each function in Table6 is reported inTable7. The optimal values for functionsG1 andG3 arenot reported because they are unknown.




0 10 20 30 40 50 60 70 80 90 1000

20

40

60

80

100

120

140

160

180

Iterations

Fun

ctio

n va

lues

F7

SGSAHGSLF

0 10 20 30 40 50 60 70 80 90 100−12000

−10000

−8000

−6000

−4000

−2000

Iterations

Fun

ctio

n va

lues

F8

SGSAHGSLF

0 10 20 30 40 50 60 70 80 90 1000

100

200

300

400

500

Iterations

Fun

ctio

n va

lues

F9

SGSAHGSLF

0 10 20 30 40 50 60 70 80 90 1000

5

10

15

20

25

Iterations

Fun

ctio

n va

lues

F10

SGSAHGSLF

Fig. 1: The efficiency of the proposed HGSLF algorithm with unconstrained problems

0 10 20 30 40 50 60 70 80 90 1000

1

2

3

4

5

6

7

8x 10

4

Iterations

Fun

ctio

n va

lues

F1

0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

2

2.5

3

3.5

4

4.5x 10

12

Iterations

Fun

ctio

n va

lues

F2

0 10 20 30 40 50 60 70 80 90 1000

2

4

6

8

10

12

14x 10

4

Iterations

Fun

ctio

n va

lues

F3

0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

2

2.5x 10

8

Iterations

Fun

ctio

n va

lues

F5

0 10 20 30 40 50 60 70 80 90 1000

1

2

3

4

5

6

7x 10

4

Iterations

Fun

ctio

n va

lues

F6

0 10 20 30 40 50 60 70 80 90 1000

1

2

3

4

5x 10

8

Iterations

Fun

ctio

n va

lues

F12

Fig. 2: The general performance of the HGSLF algorithm with unconstrained problems



Table 2: Unimodal test functionsTest function S fopt

F1(X) = ∑ni=1 x2

i [−100,100]n 0

F2(X) = ∑ni=1 | xi |+∏n

i=1 | xi | [10,10]n 0

F3(X) = ∑ni=1

(

∑ij=1 x j

)2[−100,100]n 0

F4(X) = maxi | xi |, 1≤ i ≤ n [−100,100]n 0

F5(X) = ∑n−1i=1 [100(xi+1−x2

i )2+(xi −1)2] [−30,30]n 0

F6(X) = ∑ni=1([xi +0.5])2 [−100,100]n 0

F7(X) = ∑ni=1 ix4

i + random[0,1) [−1.28,1.28]n 0

Table 3: Multimodal test functionsTest function S fopt

F8(X) = ∑ni=1−xisin(

√

| xi |) [−500,500]n −418.9829×n

F9(X) = ∑ni=1[x

2i −10cos(2πxi)+10] [−5.12,5.12]n 0

F10(X) =−20exp(

−0.2√

1n ∑n

i=1 x2i

)

−exp( 1

n ∑ni=1 cos(2πxi)

)

+20+e [−32,32]n 0

F11(X) = 14000∑n

i=1x2i −∏n

i=1 cos(

xi√(i)

)

+1 [−600,600]n 0

F12(X) = πn 10sin2(πy1)+∑m−1

i=1 (yi −1)2[1+10sin2(πyi+1)]+(yn−1)2 [−50,50]n 0+∑m

i=1 u(xi ,10,100,4)

+yi = 1+ xi+14 ,u(xi ,a,k,m) =

k(xi −a)m xi > a

0 −a< xi < a

k(−xi −a)m xi <−a

F13(X) = {0.1sin2(3πx1)+∑ni=1(xi −1)2[1+sin2(3πxi +1)] [−50,50]n 0

+(xn−1)2[1+sin2(2πxn)]}+∑ni=1u(xi ,5,100,4)

5.5.1 The Penalty function technique

The penalty function technique is used to transform theconstrained optimization problems to unconstrainedoptimization problem by penalizing the constraints andforming a new objective function as follow:

f (x) =

{

f (x) if x∈ feasible regionf (x)+penalty(x) x 6∈ feasible region.

(16)

Where,

penalty(x) =

{

0 if no constraint is violated1 otherwise.

There are two kind of points in the search space of theconstrained optimization problems (COP), feasible pointswhich satisfy all constraints and unfeasible points whichviolate at least one of the constraints. At the feasible

points, the penalty function value is equal to the value ofobjective function, but at the infeasible points the penaltyfunction value is equal to a high value as shown inEquation 16. In this paper, a non stationary penaltyfunction has been used, which the values of the penaltyfunction are dynamically changed during the searchprocess. A general form of the penalty function as definedin [31] as follows:F(x) = f (x)+h(k)H(x), x∈ S⊂ R

n, (17)Where f (x) is the objective function,h(k) is a nonstationary (dynamically modified) penalty function,k isthe current iteration number andH(x) is a penalty factor,which is calculated as follows:

H(x) =m

∑i=1

θ (qi(x))qi(x)γ(qi (x)) (18)

Where qi(x) = max(0,gi(x)), i = 1, . . . ,m, gi are theconstrains of the problem,qi is a relative violated




Table 4: Minimization results of benchmark functions in Table2 with n= 30 tmax= 1000Test function SGSA PSO RGA HGSLFF1 Average 2.12 E-17 0.0018 23.1310 1.35E-51

Median 2.08 E-17 0.0012 21.87 2.34E-72Best 9.39 E-18 1.07 E-4 9.49 5.48E-75Std 6.10 E-18 0.0020 12.14 2.24E-53

F2 Average 2.23 E-8 2.0016 1.0725 3.85E-45Median 2.22 E-8 0.0019 1.1371 2.48E-48Best 1.51 E-8 6.69 E-4 0.6557 8.79E-50Std 3.65 E-9 4.2162 0.2666 5.48E-45

F3 Average 238.17 411.27 561.71 7.45E-49Median 222.14 222.82 569.01 6.48E-56Best 98.42 139.62 395.88 2.49E-63Std 101.76 322.96 125.60 4.36E-62

F4 Average 3.42 E-9 8.1607 11.7803 7.25E-23Median 3.12 E-9 7.4464 11.9402 1.58E-30Best 2.15 E-9 5.5391 9.3608 7.94E-36Std 9.33 E-10 2.4104 1.5762 1.86E-22

F5 Average 29.76 3.64 E+4 1.18 E+3 26.24Median 26.06 1.79 E+3 1.04 E+3 28.45Best 25.76 82.2979 544.9827 25.27Std 18.89 4.61 E+4 548.0843 3.48

F6 Average 2.07E-17 0.0010 24.0129 5.78E-18Median 2.08 E-17 6.63 E-4 24.5594 2.48E-18Best 9.71E-18 6.05E-5 4.0495 2.75E-19Std 6.54E-18 0.0011 10.1747 4.96E-18

F7 Average 0.0165 0.0433 0.0675 5.48E-7Median 0.0146 0.0432 0.0635 1.78E-6Best 0.0012 0.0331 0.0333 3.48E-9Std 0.0103 0.0064 0.0287 5.78E-6

function of the constraints,θ (qi(x)) is the power of thepenalty function, the values of the functionsh(.), θ (.) andγ(.) are problem dependant. We applied thesame values, which are reported in [31].

The following values are used for the penalty function:

γ(qi(x)) =

{

1 if qi(x)< 1,2 otherwise.

Where the assignment function was

θ (qi(x))) =

10 if qi(x)< 0.001,20 if 0.001≤ qi(x)< 0.1,100 if 0.1≤ qi(x)< 1,300 otherwise.

and the penalty valueh(t) = t ∗√

t.

5.6 The general performance of the HGSLFalgorithm with constrained optimizationproblems

The general performance of the proposed HGSLFalgorithm with constrained optimization problems isshown in Figure3 by plotting the number of iterationsversus the function values for functionsG2, G4, G5, G6,G7 and G8 (picked randomly). Figure3 show thatproposed HGSLF algorithm can obtain the global or nearglobal minimum in a reasonable time (a few number ofiterations).

5.7 HGSLF and other algorithms

The proposed HGSLF algorithm is compared against 6algorithms as follow.



Table 5: Minimization results of benchmark functions in Table3 with n= 30 tmax= 1000Test function SGSA PSO RGA HGSLFF8 Average -2.82E+3 -9.88E+3 -1.2483E+4 -1.2569E4

Median -2.83E+3 -2.83E+3 -1.2496E+4 -1.2569E4Best -3.67E+3 -1.0665E+4 -1.2523E+4 -1.2569E4Std 404.55 512.22 53.2640 1.1E-4

F9 Average 15.52 55.1429 5.9020 5.48E-15Median 15.91 55.6035 5.7165 1.48E-15Best 8.95 35.3898 3.7858 0.00Std 3.60 15.4611 1.1710 2.47E-15

F10 Average 3.55E-9 0.0090 2.1395 0.00Median 3.53E-9 0.0066 2.1680 0.00Best 2.74E-9 0.0031 1.3778 0.00Std 3.75E-10 0.0076 0.4014 0.00

F11 Average 3.99 0.0101 1.1683 1.48E-16Median 3.89 0.0081 1.1411 2.45E-16Best 1.24 6.16E-4 1.0470 0.00Std 1.42 0.0093 0.0795 2.43E-17

F12 Average 0.0524 0.2926 0.0510 6.48E-13Median 1.86E-19 0.1140 0.0399 6.489E-15Best 6.55E-20 6.87E-4 0.0110 6.78E-17Std 0.1144 0.3164 0.0352 3.58E-17

F13 Average 2.90E-32 3.19E-18 0.0817 7.15E-32Median 2.39E-32 2.24E-23 0.0325 9.14E-32Best 5.99E-33 1.21E-31 2.52E-8 8.75E-33Std 1.78E-32 8.33E-18 0.1074 5.48E-32

–SASP[24]. Simulated Annealing SimulationPerturbation method is a hybridization of thesimulated annealing (SA) with the descent method, byestimating the gradient using simultaneousperturbation. The descent method is used to find alocal minimum and the simulated annealing isexecuted in order to escape from the currentlydiscovered local minimum to a better one

–P. Shen et al [30]. proposed a new method in order tosolve a global optimization of signomial geometricprogramming by employing an exponential variabletransform to the initial nonconvex problem (SGP),then a linear relaxation is obtained based on the linearlower bounding of the objective function andconstraints.

– M.J. Rijckaert et al [ 27]. proposed a new algorithmto solve generalized geometric programming problems(GGP).

–K. Ritter et al [ 28] proposed a stochastic method forsolving constrained global optimization problems byusing a penalty approach. The interesting propertiesand a parallel implementation of the proposedalgorithm enable the treatment of problems with alarge number of variables.

–T.P. Runarsson et al [29]. present a new view onpenalty function methods in terms of the dominanceof penalty and objective functions.

–S. QU et al [26]. proposed an algorithm to solve theglobal minimum of (GGP) problems by utilizing anexponential variable transformation and the inherentproperty of the exponential function and they appliedsome techniques to reduce the initial nonlinear andnonconvex (GGP) problem to a sequence of linearprogramming problems.

5.7.1 Comparison between HGSLF and other algorithmsfor constrained optimization problems

In order to investigate the efficiency of the proposedalgorithm, we compare it against 6 algorithms as shownin Table 8. In Table 8, the name of the comparativealgorithms, optimal solution and the obtained value of thecomparative algorithms. The best obtained values arereported inbold text. The proposed HGSLF algorithmresults are reported over 30 runs after 1000 iterations. Thereported results in Table8, show that the proposedalgorithm can obtain the optimal or near optimal solutionsbetter than the other algorithms in most cases.




Table 6: Constrained optimization problemsTest functionG1 min G1(X) = 5x1+50000x−1

1 +46.2x2+72000x−11 +144000x−1

3 ,

S.t g(X) = 4x−11 +32x−1

2 +120x−13 ≤ 1,

1≤ x1,x2,x3 ≤ 220

G2 min G2(X) = 0.5x1x−12 −x1−5x−1

2 ,

S.t. g(X) = 0.01x2x−13 +0.01x2+0.0005x1x3 ≤ 1,

70≤ x1 ≤ 150, 1≤ x2 ≤ 30, 0.5≤ x3 ≤ 21.

G3 min G3(X) = 168x1x2+3651.2x1x2x−13 −40000x−1

4 ,

S.t. g1(X) = 1.0425x1x−12 ≤ 1,

g2(X) = 0.00035x1x2 ≤ 1,g3(X) = 1.25x−1

1 x4+41.63x−11 ,

40≤ x1 ≤ 44, 40≤ x2 ≤ 45, 60≤ x3 ≤ 70,0.1≤ x4 ≤ 1.4.

G4 max G4(X) =

(√10

)10

∏10i=10xi ,

S.t. h1(X) = ∑10i=1x2

i −1= 0,0≤ x≤ 1.

G5 min G5(X) = 5.3578547x23 +0.8356891x1x5+37.293239x1 −40792.141,

S.t.g1(X) = 85.334407+0.0056858x2x5+0.0006262x1x4−0.0022053x3x5−92≤ 0,g2(X) =−85.334407−0.0056858x2x5−0.0006262x1x4+0.0022053x3x5 ≤ 0,g3(X) = 80.51249+0.0071317x2x5+0.0029955x1x2+0.0021813x2

3 −110≤ 0,g4(X) =−80.51249−0.0071317x2x5−0.0029955x1x2−0.0021813x2

3 −110≤ 0,g5(X) = 9.300961+0.0047026x3x5+0.0012547x1x3+0.0019085x3x4−25≤ 0,g6(X) =−9.300961−0.0047026x3x5−0.0012547x1x3−0.0019085x3x4+20≤ 0,78≤ x1 ≤ 102, 33≤ x2 ≤ 45 and 27≤ xi ≤ 45 (i = 3;4;5).

G6 min G6(X) = 3x1+0.000001x31 +2x2+(0.000002/3)x3

2,S.t. g1(X) =−x4+x3−0.55≤ 0,g2(X) =−x3+x4−0.55≤ 0,h3(X) = 1000sin(−x3−0.25)+1000sin(−x4 −0.25)+894.8−x1 = 0,h4(X) = 1000sin(x3−0.25)+1000sin(x3 −x4−0.25)+894.8−x2 = 0,h5(X) = 1000sin(x4−0.25)+1000sin(x4 −x3−0.25)+1294.8 = 0,0≤ x1 ≤ 1200, 0≤ x2 ≤ 1200 −0.55≤ x3 ≤ 0.55and−0.55≤ x4 ≤ 0.55.

G7 min G7(X) = (x1−10)3+(x2−20)3,S.t. g1(X) =−(x1−5)2− (x2−5)2+100≤ 0,g2(X) = (x1−6)2+(x2−5)2−82.81≤ 0,13≤ x1 ≤ 100, and 0≤ x2 ≤ 100.

G8 min G8(X) =sin3(2πx1)sin(2πx2)

x31(x1+x2)

,

S.t. g1(X) = x21−x2+1≤ 0,

g2(X) = 1−x1+(x2−4)2 ≤ 0,0≤ x1 ≤ 10, and0≤ x2 ≤ 10.

6 Conclusion

GSA has a powerful ability to balance betweenexploration and exploitation operations during the search,however it suffers from the premature convergence whenall solutions trapped in local minima and the algorithmbecomes unable to escape from stagnation. In this paper,a new hybrid gravitational search algorithm and a Levy

flight operator has peen proposed in order to overcomethe premature convergence problem in the standard GSA.The proposed algorithm is called Hybrid GravitationalSearch with Levy Flight algorithm (HGSLF). When twomasses (solutions) become very close to each other andnon of them is the best global solution in the population,the Levy flight is applied on one of them in order to



Table 7: Function optimal valuesTest function Optimal valuesG1 -

G2 -147.6669538

G3 -

G4 -1.000

G5 -30665.539

G6 5126.498

G7 -6961.8138755

G8 -0.09582504

0 10 20 30 40 50 60 70 80 90 100−147.7

−147.6

−147.5

−147.4

−147.3

−147.2

−147.1

−147

Iterations

Fun

ctio

n va

lues

G2

0 10 20 30 40 50 60 70 80 90 100−120

−100

−80

−60

−40

−20

0

Iterations

Fun

ctio

n va

lues

G4

0 10 20 30 40 50 60 70 80 90 100−3.08

−3.06

−3.04

−3.02

−3

−2.98

−2.96x 10

4

Iterations

Fun

ctio

n va

lues

G5

0 10 20 30 40 50 60 70 80 90 1000

1000

2000

3000

4000

5000

6000

Iterations

Fun

ctio

n va

lues

G6

0 10 20 30 40 50 60 70 80 90 100−7000

−6000

−5000

−4000

−3000

−2000

−1000

0

Iterations

Fun

ctio

n va

lues

G7

0 10 20 30 40 50 60 70 80 90 100−90

−80

−70

−60

−50

−40

−30

−20

−10

0

Iterations

Fun

ctio

n va

lues

G8

Fig. 3: The general performance of the HGSLF algorithm with constrained optimization problems

increase the diversity of the algorithm and avoid trappingin local minima. The proposed HGSLF algorithm is testedon 13 unconstrained and 8 constrained optimizationproblems and compared against 6 different algorithms.The numerical results show that the proposed HGSLFalgorithm is a promising algorithm and more precise thanthe standard GSA and the comparative algorithms.

References

[1] E. Rashedi, H. Nezamabadi-pour, and S. Saryazdi, GSA: AGravitational Search Algorithm, Information Science 17,922322248(2009).

[2] M. Dorigo, Optimization, Learning and Natural Algorithms,Ph.D. Thesis, Politecnico di Milano, Italy, (1992).




Table 8: Computational results of optimal solution and obtained value test problemsTest function Algorithms Optimal solution Obtained valueG1 SASP 174.0274,74.1981,220.0000 5653.7

[30] 108.734706796,85.126214158,204.32459429 6299.842427922[27] 107.4,84.9,204.5 6300[26] 109.32546781,84.04821454,214.32459429 6217.46548921HGSLF 175.3433,74.1198, 220.0000 5651.378048

G2 SASP 150.0000,30.0000,8.7810 -147.6667[30] 88.724706796,7.672652781,1317862596 -83.249728406[27] 88.310,7.454,1311 -82.21[26] 88.875643887,7.5637589,1.3124563877 -83.661573642HGSLF 150.0000,30.0000,0.5414 -147.6666667

G3 SASP 43.0267,44.8918,69.8129,1.1023 461806[30] 43.013755728,44.71484034,66.423933664,1.07004583 623249.8761181[27] 43.02,44.85,66.39,1.11 623370.0754[26] 43,0899785,44.9997852,66.419945664,1.1069987564 468479.996875421HGSLF 41.75500, 43.52958,69.9999, 0.1 157.9562

G4 SASP xi =1√10, i = 1, . . . ,10 1

[29] xi =1√10, i = 1, . . . ,10 1

HGSLF 0.31624, 0.31624,0.31624,0.31624,0.31624 -1.00050.31624, 0.31624,0.31624,0.31624,0.31624

G5 SASP 78,33,27,27,27 -32217.431[29] 78,33,29.995256025682,45,36.7758 -30665.539HGSLF 78,33,29.9952,44.9999,36.7758 -30665.539

G6 SASP 679.94526,1026.067,0.1188,-0.3962335291 5126.49793[29] 679.9453, 1026.067,0.1188764,-0.3962336 5126.4981HGSLF 679.9451,1026.0669,0.1188764,-0.39623345 5126.496714

G7 SASP 14.095,0.84296812 -6961.813848[29] 14.095,0.84296 -6961.81388HGSLF 14.095,0.84296078 -6961.81388

G8 SASP 1.22781649, 3.74490786 -0.10545950[29] 1.2279713,4.2453733 -0.095825HGSLF 1.22797135,4.245373366 -0.095825041

[3] J. Kennedy, RC. Eberhart, Particle Swarm Optimization,Proceedings of the IEEE International Conference on NeuralNetworks 4, 19421948(1995).

[4] D. Karaboga and B. Basturk. A powerful and efficientalgorithm for numerical function optimization: artificialbeecolony (abc) algorithm. Journal of global optimization, 39,459-471 (2007).

[5] X.S. Yang. Firefly algorithm, stochastic test functionsanddesign optimisation. International Journal of Bio-InspiredComputation, 2, 78-84 (2010).

[6] M.K. Passino, Biomimicry of bacterial foraging fordistributed optimization and control. Control Systems, IEEE22, 52-67 (2002).

[7] X.S. Yang. A new metaheuristic bat-inspired algorithm.Nature Inspired Cooperative Strategies for Optimization(NICSO 2010), pages 6574, 2010.

[8] R. Tang, S. Fong, X.S. Yang, and S. Deb. Wolf searchalgorithm with ephemeral memory. In Digital InformationManagement (ICDIM), 2012 Seventh InternationalConference on Digital Information Management, pages165-172, (2012).

[9] D. Teodorovic and M. DellOrco. Bee colony optimizationacooperative learning approach to complex tranportationproblems. In Advanced OR and AI Methods inTransportation: Proceedings of 16th MiniEURO Conferenceand 10th Meeting of EWGT (13-16 September 2005).Poznan:Publishing House of the Polish Operational and SystemResearch, pages 51-60, 2005.

[10] S.A. Chu, P.-W. Tsai, and J.-S. Pan. Cat swarm optimization.Lecture Notes in Computer Science (including subseriesLecture Notes in Artificial Intelligence and Lecture Notes inBioinformatics), 4099 LNAI:854-858, (2006).



[11] X.L. Li, Z.J. Shao, and J.X. Qian. Optimizing methodbased on autonomous animats: Fish-swarm algorithm. XitongGongcheng Lilun yu Shijian/System Engineer- ing Theoryand Practice, 22, 32 (2002).

[12] S. Yazdani, H. Nezamabadi-pour, and S. Kamyab,A Gravitational Search Algorithm for MultimodalOptimization, Swarm and Evolutionary Computation,pp. 114, (2013).

[13] M. Doraghinejad, H. Nezamabadi-pour, A.Hashempoursadeghian, and M. Maghfoori, A HybridAlgorithm Based on Gravitational Search Algorithmfor Unimodal Optimization, in 2012 2nd InternationaleConference on Computer and Knowledge Engineering(ICCKE), pp. 129132, (2012).

[14] M. Soleimanpour-moghadam and H. Nezamabadi-pour, Animproved quantum behaved gravitational search algorithm,in 20th Iranian Conference on Electrical Engineering,(ICEE2012), pp. 711-714, (2012).

[15] Z. Zhang, Y. , Li, Y., Xia, F., Luo, Immunity-based gravitational search algorithm, in 3rd InternationalConference on Information Computing and Applications(ICICA 2012), pp. 754761, (2012).

[16] W. JiaNan and L. XiangTao, An Improved GravitationSearch Algorithm for Unconstrained Optimization,Advanced Materials Research, 143, 409413(2010).

[17] A. Sombra, F. Valdez, P. Melin, and O. Castillo, ANew Gravitational Search Algorithm using Fuzzy Logicto Parameter Adaptation, in 2013 IEEE Congress onEvolutionary Computation, no. 3, pp. 10681074, (2013).

[18] A. Hatamlou, S. Abdullah, and H. Nezamabadi-pour, Acombined approach for clustering based on K-means andgravitational search algorithms, Swarm and EvolutionaryComputation. 6, 4752(2012).

[19] Z. Michalewicz, A Survey of Constraint HandlingTechniques in Evolutionary Computation Methods,Evolutionary Programming, 4, 135 (1995).

[20] N. Bacanin, M. Tuba, Artificial Bee Colony (ABC)Algorithm for Constrained Optimization Improved withGenetic Operators, Studies in Informatics and Control, 21,137-146 (2012).

[21] A. M. Reynolds and M. A. Frye, Free-flight odor tracking inDrosophila is consistent with an optimal intermittent scale-free search, PLoS One, 2, e354 (2007).

[22] I. Pavlyukevich, Levy fights, non-local search and simulatedannealing, J. Computational Physics, 226, 1830-1844 (2007).

[23] C.Brown, LS. Liebovitch, R. Glendon, Levy fights inDobe Ju/hoansi foraging patterns, Human Ecol. 35, 129-138(2007).

[24] S. El Moumen, R. Ellaia and R. Aboulaich, AppliedMathematics and Computation 218, 32653276(2011).

[25] R. L. Haupt and E. Haupt, Practical Genetic Algorithms,2nded., John Wiley& Sons (2004).

[26] S. Qu, K. Zhang, F. Wang, A global optimizationusing linear relaxation for generalized geometricprogramming, European Journal of Operational Research190, 345356(2008).

[27] M.J. Rijckaert, X.M. Martens, Comparison of generalizedgeometric programming algorithms, Journal of OptimizationTheory and Application 26, 20524(1978).

[28] K. Ritter, S. Schffer, A stochastic method for constrainedglobal optimization, SIAM Journal of Optimization 4,894904(1994).

[29] T.P. Runarsson, X. Yao, Stochastic ranking for constrainedevolutionary optimization, IEEE Transactions onEvolutionary Computation 4, 284294(2000).

[30] P. Shen, K. Zhang, Global optimization of signomialgeometric programming using linear relaxation, AppliedMathematics and Computation 150, 99114(2004).

[31] J.M. Yang, Y.P. Chen, J.T. Horng and C.Y. Kao. Applyingfamily competition to evolution strategies for constrainedoptimization. In Lecture Notes in Mathematics Vol. 1213, pp.201-211, New York, Springer, (1997).

Ahmed Fouad Alireceived the B.Sc., M.Sc. andPh.D. degrees in computerscience from the AssiutUniversity in 1998, 2006 and2011, respectively. Currently,he is a Postdoctoral Fellow atThompson Rivers University,Kamloops, BC Canada.In addition, he is an Assistant

Professor at the Faculty of Computers and Informatics,Suez Canal University, Ismailia, Egypt. He served as amember of computer science department Council from2014-2015.He worked as director of digital library unit atSuez Canal University; he is a member in SRGE(Scientific Research Group in Egypt). He also served as atechnical program committee member and reviewer inworldwide conferences. Dr. Ali research has been focusedon meta-heuristics and their applications, globaloptimization, machine learning, data mining, web mining,bioinformatics and parallel programming. He haspublished many papers in international journals andconferences and he has uploaded some meta-heuristicslectures in slidshare website.



A Hybrid Gravitational Search with Levy Flight for Global Numerical Optimization

Documents

Transcript of A Hybrid Gravitational Search with Levy Flight for Global Numerical Optimization