GHS+LEM: Global-best Harmony Search using Learnable Evolution
Models
Carlos Cobos a,b
, Dario Estupiñán a, José Pérez
a
a Information Technology Research Group (GTI) members, Electronic and Telecommunications Engineering Faculty, University of Cauca, Colombia
b Full time professor, Computer Science Department, Electronic and Telecommunications Engineering Faculty, University of Cauca, Colombia
Abstract
This paper presents a new optimization algorithm called GHS+LEM, which is based on the Global-best Harmony Search algorithm (GHS)
and techniques from the Learnable Evolution Models (LEM) to improve convergence and accuracy of the algorithm. The performance of
the algorithm is evaluated with fifteen optimization functions commonly used by the optimization community. In addition, the results
obtained are compared against the original Harmony Search algorithm, the Improved Harmony Search algorithm and the Global-best
Harmony Search algorithm. The assessment shows that the proposed algorithm (GHS+LEM) improves the accuracy of the results obtained
in relation to the other options, producing better results in most situations, but more specifically in problems with high dimensionality,
where it offers a faster convergence with fewer iterations.
© 2014 Elsevier Ltd. All rights reserved.
Keywords: Harmony search; Meta-heuristics; Evolutionary algorithms; Optimization, Learnable evolution models; Machine Learning; Prism
1. Introduction
Meta-heuristics are defined as high-level strategies that guide other heuristics in the search for feasible solutions and are
generally used in problems for which it is not possible to obtain an optimal solution using complex mathematical methods [1,
2]. Within the realm of meta-heuristic we find Harmony Search algorithms, which are based on the musical improvisation
process [3, 4]. Harmony Search (HS) has been successfully applied in many optimization problems [5-27] and has undergone
several changes in combination with other optimization techniques, among which we highlight the Improved Harmony
Search algorithm (IHS) [28] and the Global-best Harmony Search (GHS) [29].
In 2000, Learnable Evolution Model (LEM) was presented as a new optimization technique. In the Darwinian method of
evolutionary computation, the populations evolve based on processes of selection, combination, and mutation. In LEM,
machine learning techniques are also employed to generate new populations. On using the machine learning mode, LEM can
determine which individuals in a population (or a group of individuals from previous populations) are superior to others in
performing certain tasks. This reasoning, expressed as inductive hypotheses are used to generate new populations. Then,
when the algorithm runs in the Darwinian evolution mode, it uses random or semi-random operations for the generation of
new individuals (using traditional combination and/or mutations techniques) [30].
This paper proposes a new version of GHS, called GHS+LEM, which makes use of the concepts of LEM in order to
improve the accuracy of the original GHS algorithm. LEM was adapted to function in a simple way, to locate promising areas
where the global optimum is to be found and work with discrete and continuous variables. The performance of GHS+LEM
was compared with ten well-known optimization functions and which were originally used in testing the GHS algorithm [29].
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 2
The performance of the algorithm was then studied by changing the number of dimensions of the problems proposed and the
variation of the algorithm parameters.
This paper is organized as follows: Section 2, 3 and 4 are a summary of the algorithms HS, IHS and GHS. Section 5
presents the new algorithm, GHS+LEM. Section 6 presents the results of the algorithm against the optimization functions
used and the impact of changes in the values of the algorithm parameters. Finally section 7 presents conclusions and future
work that the research group hopes to undertake.
2. Harmony Search
HS is a meta heuristic algorithm, i.e. a general purpose algorithm that consists of iterative procedures that guide a
heuristic, combining in an intelligent way different concepts to explore and properly exploit the search space [1]. The
discrete-variable version of this algorithm was originally proposed by Zong Woo Geem et al [1] in 2001, then in 2005 Geem
and Lee [31] proposed the continuous-variable version of the algorithm. HS simulates the process of musical improvisation in
which musicians attempt to produce an agreeable harmony determined by the aural aesthetic standard [4]. Table 1 shows the
actions performed by a musician when improvising and its corresponding representation (formalized in [1]) in the HS
algorithm.
Table 1 Relationship of the components of the HS algorithm to the actions in musical improvisation
Actions Components
Plays a tune learned previously Use of harmonic memory
Play something similar to the above tune, gradually adjusting to reach the desired pitch Pitch adjustment
Composes a new melody based on his musical knowledge from randomly selected notes Randomness
"In musical improvisation, each musician plays a note within a possible range, forming an array of harmonies. If all the
notes played by musicians are considered a good harmony, this is stored in the memory of each musician, increasing the
possibility of producing a good harmony next time. Similarly, in the process of optimization in engineering, each decision
variable takes values initially randomized within the possible range, forming a solution vector. If this set of values that make
up the vector are a good solution, this is stored in the memory of each variable, enhancing the chances of finding better
solutions in the next iteration" [31]. These three operations are summarized using the following formulas ([32]):
Pitch
Memory
Random
i
HMSiiii
iiiii
Newi
Ppw
Ppw
Ppw
mkx
xxxkx
Kxxxkx
x
.
.
..
)(
,,,)(
)(,),2(),1()(21
(1)
The steps that indicate the operation of the HS algorithm (for continuous-variables) and its description are described
below:
2.1. Initialize the problem and HS parameters
The optimization problem is defined as . Where f (x) is the
objective function, x is a candidate solution consisting of N decision variables ( ), and and are the lowest and
highest decision limit for each decision variable, respectively. The HS parameters are specified in this step and these are:
harmonic memory size (HMS), the harmony memory consideration rate (HMCR), the pitch adjustment rate (PAR), the pitch
adjustment bandwidth (bw) and the number of improvisations (NI).
2.2. Initialize the harmonic memory
The initial harmonic memory is generated from a uniform distribution in the ranges [ , ], where .This is
done as follows: The variable r refers to a random
number and to the function generating the uniform random number between 0 and 1.
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 3
2.3. Improvise a new harmony
Generating a new harmony is called improvisation. The new harmony vector, , is generated using
the following rules: memory consideration, pitch adjustment and random selection. The value of BW is a bandwidth
arbitrary distance for continuous variables and r is a uniform random number between 0 and 1 [31]. This procedure is shown
below in Figure 1.
for each do
if then /*memory consideration*/
begin
, where
if then /*pitch adjustment*/
where end_if
end
else /*random selection*/
end_if
done Fig.1. Improvisation of a new harmony in HS
2.4. Update harmony memory
The new harmony vector generated, , replaces the worst harmony stored in the harmony memory
(HM), only if its fitness (or fitness value of the new harmony, measured in terms of the objective function) is better than that
of the worst harmony.
2.5. Check the stopping criterion
The execution of the algorithm ends when the maximum number of improvisations (NI) is reached; otherwise steps 2.3
and 2.4 are repeated.
HS is generally less sensitive to the parameter values [3, 4], therefore, the algorithm does not require an exhaustive tuning
of these to obtain good results. Despite this, it should be noted that the parameters HMCR and PAR help the method in
searching for globally and locally improved solutions, respectively. PAR and bw have a profound effect on the performance
of the algorithm and that is why the adjustment of these two parameters is very important.
3. Improved Harmony Search
Improved Harmony Search is a new harmony algorithm proposed in 2007 by Mahdavi et al [28]. It uses a method for
generating new solution vectors based on the dynamic adjustment of the PAR (pitch adjustment rate) and bw (pitch
adjustment bandwidth) parameters, thus achieving improved accuracy and convergence speed. In this variant only the step
that creates a new harmony is adjusted. PAR and bw change dynamically with the number of generations and are calculated
using the following formulas:
Where,
Pitch adjustment rate for each improvisation (iteration).
Minimum pitch adjustment rate.
Maximum pitch adjustment rate.
Total number of improvisations (Maximum iteration).
Current iteration
(2)
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 4
,
Where,
Bandwidth for each iteration
Minimum bandwidth
Maximun bandwidth
(3)
The PAR parameter increases linearly with the number of generations (although some papers claim otherwise with
numerical simulation results [33]), while bw decreases exponentially (for better bw, Das et al. [34] provided a theoretical
background of the exploratory power of HS). Given this change in the parameters, IHS does improve the performance of HS,
since it finds better solutions both globally and locally. “A major drawback of the IHS is that the user needs to specify the
values for bwmin and bwmax which are difficult to guess and problem dependent” [29].
4. Global-best Harmony Search
Global-best Harmony Search (GHS) is a stochastic optimization algorithm proposed in 2008 by Mahamed G.H. Omran
and Mehrdad Mahdavi [29], which hybridizes the original Harmony Search with the concept of swarm intelligence proposed
in PSO (Particle Swarm Optimization) [29], in which a swarm of individuals (called particles) fly through the search space.
Each particle represents a candidate solution to the optimization problem. The position of a particle is influenced by the best
position visited by itself (own experience) and the position of the best particles in the swarm (swarm experience). GHS
modifies the pitch adjustment step in HS in such a way that the newly-produced harmony can mimic the best one in the
harmony memory. This allows GHS to work efficiently in continuous and discrete problems. GHS is generally better than
IHS and HS when applied to problems of high dimensionality and when noise is present [29], but there are different
simulation results [35]. For the water network design, while GHS is better than HS in small (n=8) and medium (n=34) sized
problems, GHS is worse than HS in large (n=454) sized problems.
GHS has exactly the same steps as the IHS and the HS algorithms with the exception of the modification of step 3, which
corresponds to the improvisation of a new harmony, modified according to Figure 2.
for each do
if then /*memory consideration*/
begin
, where
If then /*pitch adjustment with PSO*/
, where is the index of the best harmony in and
end_if
end
else /*random selection*/
end_if
done Fig.2. Improvisation in the Global-best Harmony Search algorithm (GHS)
5. Proposed algorithm: GHS+LEM
Inspired by the concept of the Learnable Evolution Model (LEM) proposed by Michalsky [30] , this paper proposes a new
variation of the GHS algorithm. In LEM, machine learning techniques are used to generate new populations along with the
Darwinian method, applied in evolutionary computation and based on mutation and natural selection. This method can
determine which individuals in a population (or set of individuals from previous populations) are better than others in
performing certain tasks. This reasoning, expressed as an inductive hypotheses, is used to generate new populations. Then,
when the algorithm is run in Darwinian evolution mode, it uses random or semi-random operations for the generation of new
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 5
individuals (using traditional mutation and/or recombination techniques). The LEM process can be summarized in the
following steps:
1. Generate a population
2. Run the machine learning mode
3. Run the Darwinian learning mode
4. Alternate between the two modes until the stop criterion is reached.
The machine learning mode (item 2 listed above) also comprises the following four steps [30].
A. Derive extreme: select from the current population of two groups: High performance group, (H-group), and Low
performance group (L-group), based on the values of the fitness function.
B. Create a hypothesis: apply a machine learning method to create a description of the H-group that differentiates it from
the L-group. Consideration of previous populations is also an option.
C. Generate a new population: Generate new individuals by the rules learned from the description of the H-group.
D. Go to Step A and repeat until the stop criterion of the machine learning process is reached.
In our research, the machine learning process in the new algorithm uses a variation of the PRISM algorithm proposed by
Cendrowska [36, 37]. PRISM takes as input a training set given as a file of ordered sets of attribute values, each one
determined by a classification. Information on attributes and classifications (name, number of possible values, list of possible
values, etc.) is the input of a file separated at the beginning of the program, and the results are issued as individual rules for
each of the classifications that figure in terms of the attributes described.
The approximation that is used from the PRISM algorithm in the proposed algorithm is designed to mimic the simple
handled by the Harmonic family and can work with both continuous and discrete variables. To this end, it has a set of
conjunctive rules ( ), which delineate the regions about which there is a greater chance of finding a
better value for each (for example , where LV and HV are the lower and upper limits of the rules for the
value ). Given the combination of rules (R) for each dimension the search space is limited to regions most likely to generate
a global optimum. The rule inference algorithm is run for the first time immediately after the creation of the initial harmony
memory. The steps of the rule inference procedure and related routines are summarized in Figure 3.
/// The resulting rule is of type , where P is a conjunction of the rules that have the highest probability for
each attribute, and Q corresponds to class 1 (high-performance group). ///
Rule_Inference_Procedure (Harmony_Memory HM, integer HLGS)
begin
Use the Derive_Extreme_Procedure to generate E instances based on HM and HLGS
Initialize R as an empty rules set
while E contains instances do
Use the Single_Rule_Procedure to generate the best perfect rule r based on E
Add the rule r to R
Remove instances and attributes covered by r from E
end_while
return R (one rule by each attribute in HM)
end
/// From the current harmony memory, the high performance group and low performance group are chosen using the
following formula: where i = HLGS, HMS is the
size of the harmony memory (HM) and HLGS is the size of high and low performance groups. A matrix is returned
for this routine. This matrix stores the attributes values and the corresponding fitness as follows, if it is Hgroup it is
assigned 1, otherwise 0 is assigned. ///
Derive_Extreme_Procedure (Harmony_Memory HM, integer HLGS)
begin
Sort (from highest to lowest) HM based on fitness values
Copy first HLGS instances (rows) from HM into E and assign fitness value equal to 1 to each instance
Copy last HLGS instances from HM into E and assign fitness value equal to 0 to each instance
return E
end
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 6
/// This is a rule learner routine based on a covering approach. Accuracy in this routine is the probability of
occurrence of each value in a specific range (number of ones in fitness function -high-performance instances- over
total number of instances). ///
Single_Rule_Procedure (Instances E)
begin
Create an empty list of rules (PR)
for each attribute A in E do
Sort E based on values of A from lowest to highest
Create continuous Ranges of highest values and store accuracy of each range
Select Range RE which maximize the accuracy
Add RE to PR
done
Select the best rule generated from PR (the one with the highest accuracy)
return the best rule
end Fig.3. Rule inference procedure and related routines
Next in Figure 4 a sample of the rule inference procedure is shown. In this sample, an Initial Harmony Memory of five (5)
vector solutions is used. The HLGS parameter is fixed to 2, so, Hgroup and Lgroup have two vector solutions each of them at
the end of the derive extreme procedure. These groups are joined in the matrix E. Matrix E is reduced when finished the first
execution of the Single Rule Procedure.
Initial Harmony Memory (HM)
x1 x2 Fitness
9.009928237185779 5.0302307098313381 208.45984614156086
5.1580518834097546 3.8518099458198112 83.752645840308276
4.4178110335105139 -4.0427427711164308 85.590708763858316
-2.2681191620734147 -0.81956197080181958 5.6948494127811582
-5.7056648543596573 9.2243086822444162 817.4313955621659
Derive Extreme Procedure:
Sort (from highest to lowest) HM based on fitness values
x1 x2 Fitness
-2.2681191620734147 -0.81956197080181958 5.6948494127811582
5.1580518834097546 3.8518099458198112 83.752645840308276
4.4178110335105139 -4.0427427711164308 85.590708763858316
9.009928237185779 5.0302307098313381 208.45984614156086
-5.7056648543596573 9.2243086822444162 817.4313955621659
Matrix (E) resulting of the procedure: 1 for Hgroup and 0 for Lgroup.
x1 x2 Fitness
-2.2681191620734147 -0.81956197080181958 1
5.1580518834097546 3.8518099458198112 1
9.009928237185779 5.0302307098313381 0
-5.7056648543596573 9.2243086822444162 0
First Execution of the Single Rule Procedure:
First attribute:
Sort E based on values of x1
x1 x2 Fitness
-5.7056648543596573 9.2243086822444162 0
-2.2681191620734147 -0.81956197080181958 1
5.1580518834097546 3.8518099458198112 1
9.009928237185779 5.0302307098313381 0
Possible rules (PR) for x1 based on continuous ranges
Min x1 Max x1 Accuracy
-2.2781191620734145 5.1680518834097544 0.5 (2/4)
Second attribute:
Sort E based on values of x2
x1 x2 Fitness
-2.2681191620734147 -0.81956197080181958 1
5.1580518834097546 3.8518099458198112 1
9.009928237185779 5.0302307098313381 0
-5.7056648543596573 9.2243086822444162 0
Possible rules (PR) for x2 based on continuous ranges
Min x2 Max x2 Accuracy
-0.82956197080181959 3.861809945819811 0.5 (2/4)
Select the best rule generated from PR (When rules have same accuracy take decision based on minimum range):
Attribute Min Max
x2 -0.82956197080181959 3.861809945819811
Remove instances and attributes covered by r from E:
x1 Fitness
-2.2681191620734147 1
5.1580518834097546 1
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 7
Second Execution of the Single Rule Procedure:
First (and unique) attribute for current matrix E:
Matrix E sorted by x1
x1 Fitness
-2.2681191620734147 1
5.1580518834097546 1
Possible rules (PR) for x1 based on continuous ranges
Min x1 Max x1 Accuracy
-2.2781191620734145 5.1680518834097544 1
Select the best rule generated from PR (When rules have same accuracy take decision based on minimum range):
Attribute Min Max
x1 -2.2781191620734145 5.1680518834097544
Remove instances and attributes covered by r from E: Empty
All generated rules are:
Attribute Min Max
x2 -0.82956197080181959 3.861809945819811
x1 -2.2781191620734145 5.1680518834097544
Fig.4. Sample of the rule inference procedure and related routines
The new proposal is named Global-best Harmony Search using Learnable Evolution Models (GHS+LEM) and the steps of
the algorithm are presented below:
5.1. Initialize the problem and the GHS+LEM parameters
This step is similar to that proposed in GHS and adds three parameters that are explained later, namely the rate of rule
update (RRU), the size of high performance and low performance groups (HLGS) and the rate of consideration of rules
(RCR).
5.2. Initialize the harmonic memory
The process of initialization proposed in HS is carried out without any changes.
5.3. Run the rule inference procedure for the first time
The rule inference procedure is executed for the first time based on the initial harmony memory and the HLGS parameter.
This parameter (HLGS) indicates the size of high performance and low performance groups, this value must be .
5.4. Improvise a new harmony
This step presents the use of the rules generated in the previous step for defining the values of the dimension of the new
improvise (see Figure 5). This is executed based on the parameter called rate of consideration of rules (RCR). This parameter
(RCR) decides what percentage of the time the rules are used. It will otherwise run the traditional method based on random
generation (based in the general search space, original in HS).
for each do
if then /*memory consideration*/
begin
, where
if then /*pitch adjustment with PSO*/
, where is the index of the best harmony in and
end_if
end
else
if then /*rule consideration rate*/
,where best is the best set of rules for
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 8
else /* random selection */
end_if
end_if
done Fig.5. Improvisation in the algorithm GHS+LEM
5.5. Update harmonic memory
The new harmony vector generated, replaces the worst harmony stored in the harmony memory
(HM), only if its fitness (or fitness value of the new harmony, measured in terms of the objective function) is better than that
of the worst harmony.
5.6. Check the rule update criteria
This is done through the RRU parameter, which specifies in what percentage of occasions the rules need to be updated. If
a random number generated uniformly between 0 and 1 is less than the value of RRU, the rule inference procedure is
executed again (see Figure 6).
if then /*rule update*/
Run the rule inference procedure
end_if Fig. 6. Rule update procedure in GHS+LEM
5.7. Check the stopping criterion
The execution of the algorithm ends when the maximum number of improvisations (NI) is reached; otherwise steps 5.4,
5.5 and 5.6 are repeated.
6. Experimental Results
This section shows the performance of the GHS+LEM algorithm compared to the original harmony search (HS), the
Improved harmony search (IHS) and the Global-best harmony search (GHS). The parameters used to execute the algorithms
in each experiment are presented before each table of results, but Table 2 shows the parameter settings generally used.
Table 2 General settings for the tests
Variable HS IHS GHS GHSLEM
HMS 5 5 5 5
HCMR 0.9 0.9 0.9 0.9
PAR 0.3 N.A. N.A. N.A.
PARmin N.A. 0.01 0.01 0.01
PARmax N.A. 0.99 0.99 0.99
Bw 0.01 N.A. N.A. N.A.
bwmin N.A. 0.0001 N.A. N.A.
bwmax N.A. N.A. N.A.
HLGS N.A. N.A. N.A. RCR N.A. N.A. N.A. 0.9
RRU N.A. N.A. N.A. 0.2
All functions, except the Six-Hump Camel-Back, which is bi-dimensional, were implemented for 2, 3, 5, 10, 15, 20, 30, 40
and 50 dimensions. For each of the dimensions we used 50, 500, 5,000 and 50,000 iterations. In each case, we specify how
many dimensions were used in this specific test and the number of iterations. The initial harmony memory is generated
randomly within ranges specified for each function.
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 9
6.1. Test functions used in the evaluation
For comparison we used the functions listed in Table 3 of unimodal and multimodal functions. These are based on the
functions proposed in the article on GHS [29] that provides an adequate balance between unimodal and multimodal
functions. For each of the functions, the global minimum is searched, defined as:
Given
Find such that , where is the number of dimensions
Table 3 Test Functions
1. Sphere (DeJong´s first function) [38]
Where and
for
F1 in CEC 2005 competition [39].
2. Schwefel’s Problem 2.22 [40]
Where and
for
3. Step
Where and
for
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 10
4. Rosenbrock [38]
Where and
for
Similar to F6 in CEC 2005 competition [39].
5. Rotated Hyper-Ellipsoid [29]
Where and
for
F2 in CEC 2005 competition [39].
6. Generalized Schwefel 2.26 [40]
Where and
for
7. Rastrigin [38]
Where and
for
F9 in CEC 2005 competition [39].
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 11
8. Ackley [38]
Where and
for
F8 in CEC 2005 competition [39].
9. Griewank [38]
Where and
for
F7 in CEC 2005 competition [39].
10. Six-Hump Camel-Back [29]
Low-dimensional function with few local minima.
Where y
for
11. Shifted Rotated High Conditioned Elliptic (SRHCE)
[39]
Where and for
F3 in CEC 2005 competition [39].
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 12
12. Shifted Schwefel’s Problem 1.2 with Noise in
Fitness(Schwefel’s with Noise) [39]
Where and for
F4 in CEC 2005 competition [39].
13. Shifted Rotated Expanded Scaffer’s F6 (SRESF6)[39]
Where and for
F14 in CEC 2005 competition [39].
14. Shifted Rotated Weierstrass [39]
Where a=0.5, b=3, kmax=20, and for
F11 in CEC 2005 competition [39].
15. Sum of Different Power [40]
Where and
for
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 13
6.2. Results of the comparison
Table 4 presents the results of comparative tests applied to the GHS+LEM algorithm in order to measure its accuracy
against other algorithms in the harmony family. The number of iterations for the test was set at 50,000 and the number of
dimensions was set at 30 for all functions except for the Six-Hump Camel-Back which is defined in two dimensions. The
algorithms were run 100 times to ensure a reliable average deviation.
Table 4 Mean and standard deviation ( of the optimization tests (Nd = 30, NI=50.000).
HS IHS GHS GHS+LEM
Sphere Media 0.000684005 0.017838978 4.0457E-05 2.09647E-10
(±SD) 9.67781E-05 0.00710319 7.29366E-05 3.60168E-10
Schwefel’s Problem 2.22 Media 0.143656975 0.997096357 0.040860755 5.70121E-05
(±SD) 0.047911784 0.200329207 0.037067055 4.49754E-05
Rosenbrock Media 312.2431152 423.9427774 72.47196696 15.77537882
(±SD) 486.5124844 330.6943507 103.3253058 22.43722786
Step Media 11.56 11.22 0 0
(±SD) 4.608555943 3.945538332 0 0
Rotated Hyper-Ellipsoid Media 4200.19364 4444.37734 6636.76182 162.758505
(±SD) 1319.27706 1338.22348 7763.48957 294.309443
Schwefel’s Problem 2.26 Media -12545.01282 -12540.34846 -12569.46257 -12569.48662
(±SD) 9.274118296 10.54344883 0.03971048 3.40336E-11
Rastrigin Media 1.266797341 2.722732645 0.009457309 3.81653E-08
(±SD) 1.023021844 1.130249802 0.014012005 6.06791E-08
Ackley Media 0.981392208 1.584674315 0.024746761 5.83147E-06
(±SD) 0.485630315 0.331393069 0.026603311 4.86194E-06
Griewank Media 1.085396028 1.087082117 0.091022469 2.1722E-11
(±SD) 0.035098647 0.031926489 0.192952247 4.5967E-11
Six-Hump Camel- Back Media -1.031600318 -1.031628428 -1.031568182 -1.031628452
(±SD) 3.48248E-05 5.53445E-09 8.34751E-05 1.45999E-09
SRHCE Media 2641376.45 2736608.87 2639715.92 2638638.74
(±SD) 2123.26009 84608.4261 2365.63557 0.00134496
Schwefel’s with Noise Media 10734.2988 10822.5744 9606.13828 2900.82884
(±SD) 4207.32949 3978.9036 9516.94576 3964.44543
SRESF6 Media 1.96830452 2.48652905 3.20171518 0.71929327
(±SD) 0.58544715 0.52748454 1.57984035 0.62934122
Shifted Rotated Weierstrass Media 4.6535775 1.96625309 0.34992759 0.27805826
(±SD) 0.38824192 0.4172199 0.24911012 0.03101279
Sum of Different Power Media 8.0773E-06 0.00104216 6.6663E-05 3.5302E-11
(±SD) 7.3632E-06 0.00258109 0.00014128 7.426E-11
The results of the applied tests suggest that GHS+LEM exceed the accuracy obtained by HS, IHS and GHS in all
optimization functions performed. The standard deviation of the tests is also lower for all functions in which the algorithm
was applied, except SRESF6 in which IHS rates lower. But even in the worst case, the result obtained with GHS+LEM
exceed the best result obtained by IHS.
Table 5 presents the results of scalability tests to which the algorithm GHS+LEM was subjected. The number of iterations
was defined as 50000, the number of dimensions set at 50 and the number of executions of each algorithm was set at 100.
The results for the Six-Hump Camel-Back are not included but presented in Table 4. GHS+LEM improves accuracy in each
of the optimization functions used, proving to be better than HS, IHS and GHS in conditions of high dimensionality.
Moreover, in discontinuous functions such as Step, GHS+LEM maintains its optimum performance unlike other options,
which suffer under conditions of higher dimensionality.
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 14
Table 5 Mean and standard deviation ( of the optimization tests (Nd = 50, NI=50.000).
HS IHS GHS GHS+LEM
Sphere Media 1.231713634 1.36689254 0.005550663 2.58528E-08
(±SD) 0.287760963 0.308892735 0.00776301 5.5786E-08
Schwefel’s Problem 2.22 Media 9.594968369 10.03102019 0.411417885 0.000441435
(±SD) 1.080214642 1.361876185 0.397066313 0.000376589
Rosenbrock Media 28119.05942 27416.72832 357.7255365 39.49848422
(±SD) 10535.26342 9607.338888 726.1644056 59.9930843
Step Media 513.92 535.07 0.09 0
(±SD) 101.4288505 112.1655752 0.637149587 0
Rotated Hyper-Ellipsoid Media 29509.5846 28901.7791 66423.751 9698.04483
(±SD) 5773.08368 5338.41442 22022.3762 7340.12668
Schwefel’s Problem 2.26 Media -20065.84929 -20055.73906 -20944.1766 -20949.14436
(±SD) 183.3728874 187.2603852 8.953405915 3.8441E-09
Rastrigin Media 35.35722669 45.97323267 0.407654111 4.13742E-06
(±SD) 4.955395824 5.414365656 0.622184354 8.94894E-06
Ackley Media 5.259876609 5.382419569 0.324365569 6.09717E-05
(±SD) 0.384154298 0.38808497 0.444545366 5.45021E-05
Griewank Media 5.701261605 5.887341775 0.700857354 5.35254E-09
(±SD) 1.080435525 1.108359613 0.368310151 1.198E-08
Six-Hump Camel- Back Media -1.031600318 -1.031628428 -1.031568182 -1.031628452
(±SD) 3.48248E-05 5.53445E-09 8.34751E-05 1.45999E-09
SRHCE Media 4849016.47 5112265.38 4323424.84 4070199.91
(±SD) 321380.054 477042.762 382439.522 0.00843488
Schwefel’s with Noise Media 44173.8891 44978.9758 77200.6744 23470.0405
(±SD) 9303.92094 9664.43851 19279.3161 11482.3296
SRESF6 Media 7.53326153 7.56787837 8.2040702 1.27372272
(±SD) 0.91845325 0.73623798 4.45743971 1.20134079
Shifted Rotated Weierstrass Media 13.1610808 12.7139825 1.73347675 1.25572551
(±SD) 1.03368193 1.16549521 1.02389123 0.1074032
Sum of Different Power Media 11.8118627 9.55819903 0.03464226 7.1917E-11
(±SD) 14.739244 36.363862 0.07923361 1.0876E-10
6.3. Effects of varying the parameters HCMR, HMS, PAR and RCR
To study the variation of the parameters HCMR, HMS, PAR and RCR, the values specified in Table 2 were used. The
number of dimensions is 30, the number of iterations is 50000 and the number of executions for each test is 30. Table 6
shows the effects of varying the HCMR parameter in the proposed algorithm. The accuracy of the algorithm is seen to
improve with higher values of HCMR. A high HCMR (≥ 0.9) favors convergence. For Six-Hump Camel-Back, Rotated
Hyper-Ellipsoid, and Schwefel’s with Noise functions a lower value of HCMR is required to increase the capacity of the
algorithm for exploration. In other words, a lower HCMR (0.7 for example) favors optimal search in those functions in which
greater exploration is required.
Table 6
Mean and standard deviation ( with varying HCMR (Nd = 30, NI=50000)
HCMR 0.5 0.7 0.9 0.95
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 15
Sphere Media 0.000197381 2.16632E-05 2.46402E-10 3.33277E-11
(±SD) 3.46488E-05 8.90181E-06 4.85473E-10 6.94329E-11
Schwefel’s Problem 2.22 Media 0.046732628 0.008995544 4.78127E-05 1.67511E-05
(±SD) 0.00547272 0.003419191 3.78919E-05 1.41968E-05
Rosenbrock Media 23.02466444 16.02993218 26.47111615 32.26301449
(±SD) 32.03192831 14.24846561 32.66147183 108.4442821
Step Media 0 0 0 0
(±SD) 0 0 0 0
Rotated Hyper-Ellipsoid Media 0.00127694 0.00117081 149.42678 624.776002
(±SD) 0.00065481 0.00246363 205.003201 732.55286
Schwefel’s Problem 2.26 Media -12569.48659 -12569.48662 -12569.48662 -12569.48662
(±SD) 5.24351E-06 1.07255E-06 4.07439E-11 1.62063E-11
Rastrigin Media 0.677007539 0.00421244 3.39398E-08 4.2067E-09
(±SD) 3.47513999 0.001785797 7.24178E-08 7.68495E-09
Ackley Media 0.010563105 0.003265899 8.99544E-06 3.70719E-06
(±SD) 0.001000305 0.000749123 9.99327E-06 3.20833E-06
Griewank Media 9.62872E-06 9.92445E-07 2.79283E-11 2.95602E-12
(±SD) 2.24672E-06 4.86263E-07 3.46684E-11 4.3954E-12
Six-Hump Camel- Back Media -1.031628453 -1.031628453 -1.031628453 -1.031628448
(±SD) 3.15616E-10 2.0307E-10 1.36426E-09 1.88262E-08
SRHCE Media 2638639.61 2638638.81 2638638.74 2638638.74
(±SD) 0.23388803 0.03133231 0.0011759 0.00068605
Schwefel’s with Noise Media 0.00183615 1.93310742 3514.66702 7329.6367
(±SD) 0.00291021 7.89363197 3937.72542 5220.25806
SRESF6 Media 1.15749862 0.47597451 0.81255243 0.49298221
(±SD) 1.38547041 0.48806422 0.72467993 0.43153488
Shifted Rotated Weierstrass Media 2.76053748 1.45427489 0.28669758 0.17334889
(±SD) 0.18666743 0.13396875 0.03304307 0.02110725
Sum of Different Power Media 7.8452E-11 5.2276E-11 2.9351E-11 1.0869E-10
(±SD) 1.4035E-10 8.9653E-11 6.208E-11 1.5017E-10
Table 7 presents the results of HMS parameter variation on the proposed algorithm. The best results for the proposed
algorithm are seen to be located between the values 5 and 10 (60%). The next best results are values equal to 20 (27 %) and
50 (33%). The proposed algorithm obtains better results with small harmony memory sizes as recommended in the original
HS algorithm, but some functions -Rotated Hyper-Ellipsoid, Griewank, Six-Hump Camel- Back, Schwefel’s with Noise,
SRESF6, and Sum of Different Power- need larger values of harmony memory sizes in order to favors a greater exploration
of search space. The Step function, being discontinuous, seems not to be affected by this parameter. A future work should try
to determine whether or not by using a history of the rules the accuracy of the algorithm can be improved even if using a
smaller harmonic memory size.
Table 7
Mean and standard deviation ( with varying HMS (Nd = 30, NI=50000)
HMS 5 10 20 50
Sphere Media 1.09049E-08 9.42334E-09 4.56347E-08 6.10931E-08
(±SD) 1.97455E-08 1.21108E-08 8.98288E-08 1.00668E-07
Schwefel’s Problem 2.22 Media 0.00058248 0.000613678 0.000721842 0.000857497
(±SD) 0.000568687 0.000648931 0.00074417 0.000758539
Rosenbrock Media 31.0074059 48.32958186 39.0007948 31.84389228
(±SD) 51.12983659 57.16531525 47.41503805 35.13859472
Step Media 0 0 0 0
(±SD) 0 0 0 0
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 16
Rotated Hyper-Ellipsoid Media 4482.80339 4028.15389 3177.56436 840.301105
(±SD) 205.003201 285.078399 165.933003 58.0258614
Schwefel’s Problem 2.26 Media -12569.48662 -12569.48662 -12569.48662 -12569.48596
(±SD) 4.53706E-08 1.32036E-07 7.77143E-08 0.00362494
Rastrigin Media 2.95253E-06 5.09339E-06 2.9867E-06 1.61558E-05
(±SD) 3.63668E-06 1.01325E-05 4.88117E-06 2.98689E-05
Ackley Media 7.03661E-05 8.6753E-05 7.74663E-05 0.000163219
(±SD) 5.66978E-05 0.000139645 6.64779E-05 0.000179311
Griewank Media 0.030470641 0.010369382 0.002829622 0.021107642
(±SD) 0.157625199 0.037402693 0.014132011 0.050745481
Six-Hump Camel- Back Media -1.031628349 -1.031628331 -1.031628381 -1.031628382
(±SD) 1.22241E-07 1.37004E-07 7.88387E-08 1.01315E-07
SRHCE Media 79159162.3 79159162.3 79159162.3 79159162.3
(±SD) 0.0011759 0.00095803 0.00086158 0.00133169
Schwefel’s with Noise Media 3715.24708 1597.31424 464.521089 64.6658585
(±SD) 4967.42551 2352.99565 850.102275 116.587692
SRESF6 Media 0.81255243 0.61550188 0.61462859 0.60094146
(±SD) 0.72467993 0.56970373 0.49020067 0.36547509
Shifted Rotated Weierstrass Media 0.28669758 0.29544614 0.29098197 0.29226029
(±SD) 0.03304307 0.03894089 0.03032137 0.03896236
Sum of Different Power Media 2.9351E-11 6.5673E-11 5.9275E-11 2.8704E-11
(±SD) 6.208E-11 1.7835E-10 8.3191E-11 4.1108E-11
Table 8 deals with the results of the PAR parameter variation in the proposed algorithm. In the original proposal of GHS
[29] the PAR is dynamically adjusted with respect to the number of iterations [28, 29]. The HS algorithm proposed by Geem
[1] establishes that the PAR parameter has to be fixed and a value of 0.3 is recommended. In this group of tests a constant
PAR value of 0.1, 0.3, 0.5, 0.7, 0.9 is set along with a dynamic one, as in IHS. It is noted that the best performances of the
proposed algorithm are obtained when the PAR is dynamic for all the optimization functions. In non-continuous functions the
PAR parameter seems not to affect the accuracy of the algorithm. Even in functions whose convergence is slow (Rotated
Hyper-Ellipsoid) the best choice is a dynamic PAR. The difference between the results of Schwefel’s Problem 2.26 for the
selected PAR and the dynamic PAR is 0.003%, making it statistically irrelevant.
Table 8
Mean and standard deviation ( with varying PAR (Nd = 30, NI=50000)
PAR 0.1 0.3 0.5 0.7 0.9 Dynamic
Sphere Media 3.16E-09 2.43E-09 1.06E-08 1.55E-08 2.05E-08 2.46402E-10
(±SD) 7.21E-09 6.07E-09 2.73E-08 2.94E-08 3.23E-08 4.85473E-10
Schwefel’s Problem
2.22 Media 1.45E-04 2.19E-04 2.41E-04 4.29E-04 5.84E-04 4.78127E-05
(±SD) 1.05E-04 1.94E-04 2.80E-04 3.92E-04 5.55E-04 3.78919E-05
Rosenbrock Media 2.67E+01 4.97E+01 5.35E+01 5.82E+01 2.55E+01 2.65 E+01
(±SD) 4.16E+01 7.67E+01 7.03E+01 6.29E+01 4.82E+01 3.27 E+01
Step Media 0.0 0.0 0.0 0.0 0.0 0.0
(±SD) 0.0 0.0 0.0 0.0 0.0 0.0
Rotated Hyper-Ellipsoid
Media 1.49E+02 1.49E+02 1.49E+02 1.49E+02 1.49E+02 1.49E+02
(±SD) 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02
Schwefel’s Problem 2.26
Media -1.26E+04 -1.26E+04 -1.26E+04 -1.26E+04 -1.26E+04 -1.26E+04
(±SD) 1.16E-09 1.08E-09 8.46E-10 5.13E-09 6.67E-09 4.07439E-11
Rastrigin Media 5.75E-07 7.07E-07 1.23E-06 1.41E-06 2.71E-06 3.39E-08
(±SD) 8.04E-07 1.20E-06 2.06E-06 2.43E-06 5.04E-06 7.24E-08
Ackley Media 2.37E-05 2.87E-05 3.51E-05 5.17E-05 9.03E-05 9.00E-06
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 17
(±SD) 2.40E-05 2.43E-05 3.35E-05 5.68E-05 7.93E-05 9.99E-06
Griewank Media 4.79E-02 3.85E-10 9.14E-10 4.92E-09 3.46E-09 2.79E-11
(±SD) 1.62E-01 5.39E-10 1.80E-09 1.94E-08 5.65E-09 3.47E-11
Six-Hump Camel-
Back Media -1.03E+00 -1.03E+00 -1.03E+00 -1.03E+00 -1.031E+00 -1.03E+00
(±SD) 1.30E-06 1.76E-06 3.99E-07 4.35E-07 1.90E-07 1.36E-09
SRHCE Media 2.64E+06 2.64E+06 2.64E+06 2.64E+06 2.64E+06 2.64E+06
(±SD) 1.18E-03 1.18E-03 1.18E-03 1.18E-03 1.18E-03 1.18E-03
Schwefel’s with Noise Media 2.97E+03 3.71E+03 2.58E+03 3.55E+03 2.82E+03 3.51E+03
(±SD) 3.74E+03 6.09E+03 2.38E+03 5.19E+03 3.20E+03 3.94E+03
SRESF6 Media 8.13E-01 8.13E-01 8.13E-01 8.13E-01 8.13E-01 8.13E-01
(±SD) 7.25E-01 7.25E-01 7.25E-01 7.25E-01 7.25E-01 7.25E-01
Shifted Rotated
Weierstrass Media 2.87E-01 2.87E-01 2.87E-01 2.87E-01 2.87E-01 2.87E-01
(±SD) 3.30E-02 3.30E-02 3.30E-02 3.30E-02 3.30E-02 3.30E-02
Sum of Different Power
Media 2.94E-11 2.94E-11 2.94E-11 2.94E-11 2.94E-11 2.94E-11
(±SD) 6.21E-11 6.21E-11 6.21E-11 6.21E-11 6.21E-11 6.21E-11
The results of varying the RCR parameter are shown in Table 9. The RCR parameter determines in what percentage of the
times the inferred rules (result of the PRISM adapted algorithm) are used in generating a new harmony. A clear tendency to
use a large value for RCR is observed ( ). The recommended setting for RCR by default in the proposed
algorithm is 0.9, at which the highest level of effectiveness is shown. The possibility is left open for a study with a RCR
factor varying between 0.7 and 1.0 (it is similar to the PAR parameter).
Table 9 Mean and standard deviation ( with varying RCR (Nd = 30, NI=50000)
RCR 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1
Sphere 2.161E-10 2.352E-10 2.453E-10 3.132E-10 2.729E-10 4.625E-10 1.399E-09 2.230E-09 4.838E-09 1.011E-08
5.040E-10 4.509E-10 4.745E-10 5.985E-10 4.207E-10 7.560E-10 2.929E-09 4.419E-09 9.146E-09 1.561E-08
Schwefel’s
Problem
2.22
3.607E-05 4.470E-05 5.970E-05 6.989E-05 7.995E-05 9.629E-05 1.086E-04 1.732E-04 2.777E-04 5.493E-04
4.350E-05 3.848E-05 7.632E-05 7.082E-05 8.713E-05 7.975E-05 1.053E-04 1.367E-04 3.002E-04 4.912E-04
Rosenbrock 2.101E+01 1.861E+01 1.476E+01 3.032E+01 1.994E+01 2.617E+01 2.719E+01 7.098E+01 4.208E+01 6.856E+01
3.121E+01 3.042E+01 2.344E+01 3.910E+01 3.451E+01 3.375E+01 4.439E+01 1.682E+02 8.332E+01 1.333E+02
Step 1.20 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
5.94 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Rotated
Hyper-
Ellipsoid
635.279583 149.42678 166.249878 235.400082 426.184667 572.885247 1036.88623 1819.2857 2980.20403 3225.82524
943.517838 205.003201 232.271253 267.291022 511.991995 591.857973 695.852484 879.099952 973.000542 1156.0523
Schwefel’s Problem
2.26
-1.250E+04 -1.257E+04 -1.257E+04 -1.257E+04 -1.257E+04 -1.257E+04 -1.257E+04 -1.257E+04 -1.257E+04 -1.257E+04
5.025E+02 3.855E-11 5.829E-11 5.530E-11 6.038E-11 2.142E-10 7.002E-10 3.531E-10 5.424E-10 5.673E-04
Rastrigin 1.194E+00 6.038E-08 5.753E-08 5.499E-08 8.028E-08 1.356E-07 2.565E-07 2.105E-07 7.543E-07 5.055E-06
5.909E+00 1.366E-07 1.132E-07 9.368E-08 1.141E-07 3.275E-07 3.396E-07 4.158E-07 1.073E-06 1.152E-05
Ackley 1.430E-01 7.612E-06 7.243E-06 9.316E-06 1.307E-05 9.818E-06 1.325E-05 2.711E-05 3.759E-05 5.917E-05
7.076E-01 6.617E-06 6.937E-06 1.132E-05 1.028E-05 1.205E-05 1.479E-05 2.417E-05 3.438E-05 6.534E-05
Griewank 2.272E-11 1.208E-11 1.400E-11 2.154E-11 8.018E-11 5.236E-11 1.672E-10 1.272E-02 2.786E-03 5.095E-02
3.142E-11 1.542E-11 2.240E-11 3.105E-11 2.327E-10 8.748E-11 4.402E-10 8.994E-02 1.957E-02 1.551E-01
Six-Hump
Camel- Back
-9.500E-01 -1.032E+00 -1.032E+00 -1.032E+00 -1.032E+00 -1.032E+00 -1.032E+00 -1.032E+00 -1.032E+00 -1.032E+00
2.473E-01 1.645E-09 2.202E-09 3.530E-09 3.106E-09 6.871E-09 1.025E-08 8.118E-09 4.938E-08 9.653E-08
SRHCE 2638638.74 2638638.74 2638638.74 2638638.74 2638638.74 2638638.75 2638638.76 2638638.79 2638638.92 2638642.6
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 18
0.00064421 0.0011759 0.00226699 0.00199823 0.00223534 0.01068915 0.01420785 0.04666398 0.22487547 8.10863555
Schwefel’s with Noise
1711.30168 2458.47163 3037.39566 4055.39407 5332.82446 7432.53838 7040.69871 9545.10601 8802.23788 11530.1239
2607.64288 3243.75066 3543.72205 3941.58669 3765.86402 4362.01585 4431.63942 4254.99433 3531.34188 4924.39438
SRESF6 1.01717905 0.81255243 0.61587394 0.82358984 0.83525756 0.83516347 0.92880128 0.89577229 1.36191868 1.59949269
1.14063446 0.72467993 0.55317585 0.61607082 0.59695425 0.69573287 0.67341881 0.55251217 0.78662756 0.66011548
Shifted
Rotated Weierstrass
0.25688648 0.28669758 0.29982573 0.32922784 0.362599 0.42054823 0.46084905 0.57014011 0.70368141 1.06326949
0.02942249 0.03304307 0.04379959 0.03585728 0.03978034 0.05615239 0.05870615 0.04996005 0.07453427 0.14235103
Sum of
Different
Power
4.0421E-11 2.9351E-11 5.7058E-11 5.5714E-11 1.6194E-10 3.0699E-09 9.4552E-09 3.2809E-08 9.1628E-08 1.3382E-06
1.0258E-10 6.208E-11 1.1399E-10 9.6809E-11 2.275E-10 5.7561E-09 1.8955E-08 5.4911E-08 1.579E-07 2.3263E-06
Other tests were performed by varying the number of iterations as shown in Table 10. The number of iterations for this test
is 5000. It is noted that the GHS+LEM algorithm outperforms other proposals when the number of iterations is low. The
proposed algorithm maintains an acceptable level of approximation to the optimal solution, despite the low number of
iterations proposed. The differences in the function of Six-Hump Camel-Back are not statistically significant.
Table 10 Mean and standard deviation ( with varying number of iterations (Nd = 30, NI=5000)
Functions HMS IHS GHS GHS+LEM
Sphere Media 1.391113338 1.492336778 0.008490508 5.93279E-08
(±SD) 0.442516349 0.429375922 0.015014868 9.80204E-08
Schwefel’s Problem 2.22 Media 7.343708198 7.942415139 0.38849092 0.000681506
(±SD) 1.298620467 1.376467735 0.392227894 0.000587781
Rosenbrock Media 44801.42361 44335.30705 365.8780006 35.91286497
(±SD) 27087.38846 25827.36583 988.1136926 68.65861969
Step Media 577.17 599.34 3.25 0
(±SD) 178.0053373 181.6569761 8.62738768 0
Rotated Hyper-Ellipsoid Media 18648.2804 19114.2613 22845.7019 9742.75094
(±SD) 4664.11977 5576.63382 12812.2556 5923.66993
Schwefel’s Problem 2.26 Media -11723.5473 -11734.68475 -12561.42156 -12569.4597
(±SD) 194.1725169 182.3846445 16.70991705 0.248116333
Rastrigin Media 29.46075697 36.89200558 0.881408659 1.17383E-05
(±SD) 5.461732931 6.233664508 1.594135534 1.89959E-05
Ackley Media 6.335464888 6.480145971 0.535233079 0.000125684
(±SD) 0.620693868 0.686288345 0.701961543 0.000103427
Griewank Media 6.277931588 6.370436585 0.795212512 0.006467333
(±SD) 1.494900647 1.816294084 0.350711873 0.054165186
Six-Hump Camel- Back Media -1.03155243 -1.031628431 -1.026283458 -1.030628257
(±SD) 5.65151E-05 7.41828E-09 0.008075092 0.004327667
SRHCE Media 4949527.45 5272015.41 3257133.71 2638638.87
(±SD) 1549998.56 2003255.08 1000322.75 0.17113924
Schwefel’s with Noise Media 25859.5942 25706.8318 33126.0056 16884.5567
(±SD) 6127.04081 6656.33665 14920.4074 8512.95715
SRESF6 Media 5.40555247 5.66821959 3.73505659 1.38260315
(±SD) 0.74702013 0.57290456 2.6546381 1.1580235
Shifted Rotated Weierstrass Media 8.1876236 9.14968576 1.62251403 0.9166651
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 19
(±SD) 1.08444438 1.03659385 1.09541355 0.08870444
Sum of Different Power Media 56.7816376 70.2377894 0.04011324 8.4062E-09
(±SD) 84.9067592 135.452682 0.06083117 2.4547E-08
The results of GHS+LEM with 5000 iterations were compared with the results obtained by other methods (HS, IHS and
GHS) with 50000 iterations (see Table 11). It is noted that even with a low number of iterations (5000) the proposed
algorithm improves the results in almost all the optimization functions used. In cases such as Schwefel’s Problem 2.26 the
difference is not statistically significant (0.000023%). For functions with slow convergence, as Rotated Hyper-Ellipsoid, a
greater number of iterations are required to improve the accuracy of the proposed algorithm.
Table 11 Mean and standard deviation ( comparing accuracy with different numbers of iterations (Nd = 30; GHS+LEM with
NI=5000; HS, IHS y GHS with NI=50000)
Functions HMS IHS GHS GHS+LEM (5000 NI)
Sphere Media 0.000684005 0.017838978 4.0457E-05 5.93279E-08
(±SD) 9.67781E-05 0.00710319 7.29366E-05 9.80204E-08
Schwefel’s Problem 2.22 Media 0.143656975 0.997096357 0.040860755 0.000681506
(±SD) 0.047911784 0.200329207 0.037067055 0.000587781
Rosenbrock Media 312.2431152 423.9427774 72.47196696 35.91286497
(±SD) 486.5124844 330.6943507 103.3253058 68.65861969
Step Media 11.56 11.22 0 0
(±SD) 4.608555943 3.945538332 0 0
Rotated Hyper-Ellipsoid Media 4234.47788 4183.56875 6880.91826 9742.75094
(±SD) 1140.43614 1040.33435 7812.23325 5923.66993
Schwefel’s Problem 2.26 Media -12545.01282 -12540.34846 -12569.46257 -12569.4597
(±SD) 9.274118296 10.54344883 0.03971048 0.248116333
Rastrigin Media 1.266797341 2.722732645 0.009457309 1.17383E-05
(±SD) 1.023021844 1.130249802 0.014012005 1.89959E-05
Ackley Media 0.981392208 1.584674315 0.024746761 0.000125684
(±SD) 0.485630315 0.331393069 0.026603311 0.000103427
Griewank Media 1.085396028 1.087082117 0.091022469 0.006467333
(±SD) 0.035098647 0.031926489 0.192952247 0.054165186
Six-Hump Camel- Back Media -1.031600318 -1.031628428 -1.031568182 -1.030628257
(±SD) 3.48248E-05 5.53445E-09 8.34751E-05 0.004327667
SRHCE Media 2641799.17 2741995.37 2639726.35 2638638.87
(±SD) 2878.93967 89375.4964 2506.22217 0.17113924
Schwefel’s with Noise Media 10045.7637 11298.1914 10638.4058 16884.5567
(±SD) 2605.17481 3410.50321 10996.5284 8512.95715
SRESF6 Media 1.82019811 2.55920064 3.45257774 1.38260315
(±SD) 0.61530057 0.54004037 1.40491917 1.1580235
Shifted Rotated Weierstrass Media 4.65605359 1.89005574 0.28266306 0.9166651
(±SD) 0.36447473 0.39416867 0.19571374 0.08870444
Sum of Different Power Media 8.3165E-06 0.0015714 8.6639E-05 8.4062E-09
(±SD) 6.9039E-06 0.00344916 0.00022289 2.4547E-08
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 20
In addition tests were performed to evaluate the performance of the algorithm in integer programming problems. The tests
implemented were defined in the article on GHS [29] and correspond to five problems identified as F1, F2, F3, F4 and F5.
The results can be seen in Table 12. In these results it can also be seen that GHS+LEM improves the accuracy for high-
dimensional function F1 with respect to the other proposals. For all other functions implemented it is seen that GHS+LEM
performs in a similar way against the other algorithms, improving the performance shown by GHS in F2, F3 and F5.
Table 12 Integer programming problems
HS IHS GHS GHS+LEM
F1 (N=5) Media 0 0 0 0
(±SD) 0 0 0 0
F1 (N=15) Media 0 0.833333333 0 0
(±SD) 0 0.461133037 0 0
F1 (N=30) Media 6.266666667 11.26666667 0.433333333 0
(±SD) 1.387961376 1.964044619 0.504006933 0
F2 Media 0 0 0.3 0
(±SD) 0 0 0.70221325 0
F3 Media 0 9 0.133333333 0
(±SD) 0 16.29258346 0.434172485 0
F4 Media -7 -7 -6.933333333 -6.933333333
(±SD) 0 0 0.253708132 0.253708132
F5 Media -3880 -3880 -3879.633333 -3880
(±SD) 0 0 0.556053417 0
6.4. Convergence vs. iterations
The results of the algorithms were compared in relation to the convergence speed to the optimal solution and the number
of iterations to reach such solutions. As an example, Table 13 shows a graphic of the convergence on the test functions taking
30 dimensions into account. It can be seen that the convergence to the global optimum in the GHS+LEM algorithm is
achieved with a smaller number of iterations. The process of inference rules and their application in the generation of new
harmonies allows GHS+LEM to execute qualitative jump towards the global optimum, so that optimal results are achieved in
an average of 500 iterations over all test functions, while other algorithms need over 3000 iterations, and even 5000
iterations.
Table 13 Convergence curves for all algorithms in test functions
Sphere with 500 iterations
Schwefel’s Problem 2.22 with 500 iterations
0
20
40
60
80
100
120
140
160
180
200
1
28
55
82
10
9
13
6
16
3
19
0
21
7
24
4
27
1
29
8
32
5
35
2
37
9
40
6
43
3
46
0
48
7
GHS+LEM
GHS
IHS
HS
0
20
40
60
80
100
120
1
28
55
82
10
9
13
6
16
3
19
0
21
7
24
4
27
1
29
8
32
5
35
2
37
9
40
6
43
3
46
0
48
7
GHS+LEM
GHS
IHS
HS
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 21
Rosenbrock with 1000 iterations (logarithmic scale)
Step with 600 iterations (logarithmic scale)
Rotated Hyper-Ellipsoid with 500 iterations
Schwefel’s Problem 2.26 with 500 iterations
Rastrigin with 500 iterations
Ackley with 1000 iterations
Griewank with 500 iterations
Shifted Rotated High Conditioned Elliptic with 500
iterations
1
10
100
1000
10000
100000
1000000
10000000
100000000
1E+091
64
12
7
19
0
25
3
31
6
37
9
44
2
50
5
56
8
63
1
69
4
75
7
82
0
88
3
94
6
GHS+LEM
GHS
IHS
HS
1
10
100
1000
10000
100000
1
35
69
10
3
13
7
17
1
20
5
23
9
27
3
30
7
34
1
37
5
40
9
44
3
47
7
51
1
54
5
57
9
GHS+LEM
GHS
IHS
HS
0
200000
400000
600000
800000
1000000
1200000
1400000
1600000
1800000
1
31
61
91
12
1
15
1
18
1
21
1
24
1
27
1
30
1
33
1
36
1
39
1
42
1
45
1
48
1
GHS+LEM
GHS
IHS
HS
-14000
-12000
-10000
-8000
-6000
-4000
-2000
0
1
29
57
85
11
3
14
1
16
9
19
7
22
5
25
3
28
1
30
9
33
7
36
5
39
3
42
1
44
9
47
7
GHS+LEM
GHS
IHS
HS
0
100
200
300
400
500
600
1
28
55
82
10
9
13
6
16
3
19
0
21
7
24
4
27
1
29
8
32
5
35
2
37
9
40
6
43
3
46
0
48
7
GHS+LEM
GHS
IHS
HS
0
5
10
15
20
25
1
54
10
7
16
0
21
3
26
6
31
9
37
2
42
5
47
8
53
1
58
4
63
7
69
0
74
3
79
6
84
9
90
2
95
5GHS+LEM
GHS
IHS
HS
0
100
200
300
400
500
600
700
800
900
1000
1
28
55
82
10
9
13
6
16
3
19
0
21
7
24
4
27
1
29
8
32
5
35
2
37
9
40
6
43
3
46
0
48
7
GHS+LEM
GHS
IHS
HS
0
500000000
1E+09
1,5E+09
2E+09
2,5E+09
3E+09
1
33
65
97
12
9
16
1
19
3
22
5
25
7
28
9
32
1
35
3
38
5
41
7
44
9
48
1
GHS+LEM
GHS
IHS
HS
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 22
Shifted Schwefel’s Problem 1.2 with Noise in Fitness
with 500 iterations
Shifted Rotated Expanded Scaffer’s F6 with 500
iterations
Shifted Rotated Weierstrass with 500 iterations
Sum of Different Power with 250 iterations (logarithmic
scale)
7. Conclusions and future work
This paper presents a new version of the GHS algorithm called GHS+LEM. The proposed algorithm uses LEM techniques
to create a set of rules that allows the inferring of new candidates in the population that emerge not only from the random
scan. The amendment allows the new algorithm to perform efficiently in both discrete and continuous functions. The
algorithm was subjected to ten classic optimization features and in most cases improved the results against other methods
(HS, IHS, and GHS). It was also concluded following a scalability test that the algorithm maintains its accuracy even in high
dimensions ( . We investigated the effects of the HCMR, HMS, PAR and RCR parameters on the performance of the
proposed algorithm. It resulted that in HCMR ≥ 0.9 the algorithm generally improves its efficiency. Moreover, with regard to
the size of the harmony memory, tests show that the proposed algorithm generally performs better when the size is between 5
and 10, which is consistent with the recommendations of the original HS algorithm. It can also be seen too that better results
are obtained when the PAR value is dynamic, as proposed in IHS. With respect to variation in the RCR parameter, a better
overall performance of the algorithm was achieved when the rule procedure application is carried out using a probability of
between 0.7 and 1. The algorithm also was shown to maintain a higher accuracy than the other algorithms even when the
number of iterations is 10 times lower than used by those other harmony algorithms.
As future work, the research group proposes a study of the performance of the algorithm with other functions [39] and
real-world problems; introducing variations in the inference rules procedure that allow manage harmony memory history to
be taken into account and modify the inference rules procedure to update each time a change in the harmonic memory is
made in order to evaluate the algorithm’s performance under these new conditions. A study for improving the parameter
setting process in GHS+LEM or a study to bypass this process should be conducted. Generalize the algorithm to work with
attributes with different range of values (with different upper and lower bounds on each dimension). Use non-parametric
statistical tests to validate results of the present paper [41].
0
100000
200000
300000
400000
500000
6000001
29
57
85
11
3
14
1
16
9
19
7
22
5
25
3
28
1
30
9
33
7
36
5
39
3
42
1
44
9
47
7
GHS+LEM
GHS
IHS
HS
0
2
4
6
8
10
12
14
16
1
28
55
82
10
9
13
6
16
3
19
0
21
7
24
4
27
1
29
8
32
5
35
2
37
9
40
6
43
3
46
0
48
7
GHS+LEM
GHS
IHS
HS
0
10
20
30
40
50
60
1
28
55
82
10
9
13
6
16
3
19
0
21
7
24
4
27
1
29
8
32
5
35
2
37
9
40
6
43
3
46
0
48
7
GHS+LEM
GHS
IHS
HS
1,00E+00
1,00E+02
1,00E+04
1,00E+06
1,00E+08
1,00E+10
1,00E+12
1,00E+14
1,00E+16
1,00E+18
1,00E+20
1,00E+22
1,00E+24
1
16
31
46
61
76
91
10
6
12
1
13
6
15
1
16
6
18
1
19
6
21
1
22
6
24
1
GHS+LEM
GHS
IHS
HS
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 23
Acknowledgements
This research work was supported by a Research Grant from the University of Cauca under project VRI-2560. The authors
would like to thank the anonymous reviewers for helpful comments and suggestions.
References
1. Geem, Z., J. Kim, and G.V. Loganathan, A New Heuristic Optimization Algorithm: Harmony Search. Simulation, 2001.
76(2): p. 60-68.
2. Garcia, E. RSJ-PM Tutorial: A Tutorial on the Robertson-Sparck Jones Probabilistic Model for Information Retrieval.
2009; Available from: http://www.miislita.com/information-retrieval-tutorial/information-retrieval-probabilistic-model-
tutorial.pdf.
3. Yang, X.-S., Harmony Search as a Metaheuristic Algorithm, in Music-Inspired Harmony Search Algorithm. 2009,
Springer Berlin / Heidelberg. p. 1-14.
4. Geem, Z.W., Music-Inspired Harmony Search Algorithm: Theory and Applications. Studies in Computational
Intelligence. Vol. 191. 2009, Rockville, Maryland: Springer Publishing Company, Incorporated. 206.
5. Geem, Z.W., Recent Advances In Harmony Search Algorithm. Studies in Computational Intelligence. Vol. 270. 2010,
Annandale,Virginia: Springer
6. Geem, Z.W., et al., Recent Advances in Harmony Search. Advances in Evolutionary Algorithms, 2008: p. 16.
7. Tangpattanakul, P., A. Meesomboon, and P. Artrit, Optimal Trajectory of Robot Manipulator Using Harmony Search
Algorithms, in Recent Advances In Harmony Search Algorithm. 2010, Springer Berlin / Heidelberg. p. 23-36.
8. Panigrahi, B., et al., Population Variance Harmony Search Algorithm to Solve Optimal Power Flow with Non-Smooth
Cost Function, in Recent Advances In Harmony Search Algorithm. 2010, Springer Berlin / Heidelberg. p. 65-75.
9. Mohsen, A., A. Khader, and D. Ramachandram, An Optimization Algorithm Based on Harmony Search for RNA
Secondary Structure Prediction, in Recent Advances In Harmony Search Algorithm. 2010, Springer Berlin / Heidelberg.
p. 163-174.
10. Fourie, J., S. Mills, and R. Green, Visual Tracking Using Harmony Search, in Recent Advances In Harmony Search
Algorithm. 2010, Springer Berlin / Heidelberg. p. 37-50.
11. Forsati, R. and M. Mahdavi, Web Text Mining Using Harmony Search, in Recent Advances In Harmony Search
Algorithm. 2010, Springer Berlin / Heidelberg. p. 51-64.
12. dos Santos Coelho, L. and D. de A. Bernert, A Harmony Search Approach Using Exponential Probability Distribution
Applied to Fuzzy Logic Control Optimization, in Recent Advances In Harmony Search Algorithm. 2010, Springer Berlin
/ Heidelberg. p. 77-88.
13. Ayvaz, M., Solution of Groundwater Management Problems Using Harmony Search Algorithm, in Recent Advances In
Harmony Search Algorithm. 2010, Springer Berlin / Heidelberg. p. 111-122.
14. Geem, Z., State-of-the-Art in the Structure of Harmony Search Algorithm, in Recent Advances In Harmony Search
Algorithm. 2010, Springer Berlin / Heidelberg. p. 1-10.
15. Jaberipour, M. and E. Khorram, Solving the sum-of-ratios problems by a harmony search algorithm. Journal of
Computational and Applied Mathematics, 2010. 234(3): p. 733-742.
16. Zou, D., et al., A novel global harmony search algorithm for reliability problems. Computers & Industrial Engineering,
2009. 58(2): p. 307-316.
17. Geem, Z.W., Optimal Design of Water Distribution Networks Using Harmony Search. 2009: p. 112.
18. Saka, M., Optimum Design of Steel Skeleton Structures, in Music-Inspired Harmony Search Algorithm. 2009, Springer
Berlin / Heidelberg. p. 87-112.
19. Panchal, A., Harmony Search in Therapeutic Medical Physics, in Music-Inspired Harmony Search Algorithm. 2009,
Springer Berlin / Heidelberg. p. 189-203.
20. Mahdavi, M., Solving NP-Complete Problems by Harmony Search, in Music-Inspired Harmony Search Algorithm. 2009,
Springer Berlin / Heidelberg. p. 53-70.
21. Fesanghary, M., Harmony Search Applications in Mechanical, Chemical and Electrical Engineering, in Music-Inspired
Harmony Search Algorithm. 2009, Springer Berlin / Heidelberg. p. 71-86.
22. Geem, Z.W. and J.C. Williams, Ecological optimization using harmony search, in Proceedings of the American
Conference on Applied Mathematics. 2008, World Scientific and Engineering Academy and Society (WSEAS):
Cambridge, Massachusetts.
23. Prasad, B. and Z. Geem, Harmony Search Applications in Industry, in Soft Computing Applications in Industry. 2008,
Springer Berlin / Heidelberg. p. 117-134.
C. Cobos, D. Estupiñán, J.Perez/ Applied Mathemathics and Computation 24
24. Geem, Z.W. and J.-Y. Choi, Music Composition Using Harmony Search Algorithm. Applications of Evolutionary
Computing, 2007. 4448/2007: p. 7.
25. Geem, Z.W., Optimal cost design of water distribution networks using harmony search. Engineering Optimization,
2006. 38.
26. Geem, Z.W., Harmony search algorithms for structural design optimization, in Studies in computational intelligence,
v.239. 2009, Springer Berlin Heidelberg: Berlin, Heidelberg. p. 228.
27. Cobos, C., et al. Web document clustering based on Global-Best Harmony Search, K-means, Frequent Term Sets and
Bayesian Information Criterion. in 2010 IEEE Congress on Evolutionary Computation (CEC). 2010. Barcelona, Spain:
IEEE.
28. Mahdavi, M., M. Fesanghary, and E. Damangir, An improved harmony search algorithm for solving optimization
problems. Applied Mathematics and Computation, 2007. 188(2): p. 1567-1579.
29. Omran, M.G.H. and M. Mahdavi, Global-best harmony search. Applied Mathematics and Computation, 2008. 198(2): p.
643-656.
30. Michalski, R.S., LEARNABLE EVOLUTION MODEL: Evolutionary Processes Guided by Machine Learning. Machine
Learning, 2000-01-01. 38(1): p. 9-40.
31. Lee, K. and Z. Geem, A new meta-heuristic algorithm for continuous engineering optimization: harmony search theory
and practice. Computer Methods in Applied Mechanics and Engineering, 2005. 194(36-38): p. 3902-3933.
32. Geem, Z.W., Novel derivative of harmony search algorithm for discrete design variables. Applied Mathematics and
Computation, 2008. 199(1): p. 223-230.
33. Geem, Z.W. and K.-B. Sim, Parameter-setting-free harmony search algorithm. Applied Mathematics and Computation,
2010. 217(8): p. 3881-3889.
34. Das, S., et al., Exploratory Power of the Harmony Search Algorithm: Analysis and Improvements for Global Numerical
Optimization. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on. 41(1): p. 89-106.
35. Geem, Z.W., Particle-swarm harmony search for water network design. Engineering Optimization, 2009. 41: p. 297-
311.
36. Cendrowska, J., PRISM: An algorithm for inducing modular rules. International Journal of Man-Machine Studies, 1987.
27(4): p. 349-370.
37. Witten, I.H. and E. Frank, Data mining: practical machine learning tools and techniques with Java implementations.
SIGMOD Rec., 2002. 31(1): p. 76-77.
38. M. Molga, C.S., Test functions for optimization needs. available at www.zsd.ict.pwr.wroc.pl/files/docs/functions.pdf
2005.
39. Suganthan, P.N., et al., Problem definitions and evaluation criteria for the CEC 2005 Special Session on Real Parameter
Optimization. 2005.
40. Xin, Y., L. Yong, and L. Guangming, Evolutionary programming made faster. Evolutionary Computation, IEEE
Transactions on, 1999. 3(2): p. 82-102.
41. Garcia, S., et al., A study on the use of non-parametric tests for analyzing the evolutionary algorithms' behaviour: a case
study on the CEC'2005 Special Session on Real Parameter Optimization. Journal of Heuristics, 2009. 15(6): p. 617-644.
Top Related