3328 Journal of Applied Sciences Research, 9(5): 3328-3337, 2013 ISSN 1819-544X This is a refereed journal and all articles are professionally screened and reviewed

ORIGINAL ARTICLES

Corresponding Author: Habib Shah, Faculty of Computer Science and Information Technology Universiti Tun Hussein Onn Malaysia (UTHM) Parit Raja, 86400 Batu Pahat. Johor, Malaysia

E-mail: habibshah.uthm@gmail.com

Hybrid Global Artificial Bee Colony Algorithm for Classification and Prediction Tasks

Habib Shah, Rozaida Ghazali, and Nazri Mohd Nawi Faculty of Computer Science and Information Technology Universiti Tun Hussein Onn Malaysia (UTHM) Parit Raja, 86400 Batu Pahat. Johor, Malaysia ABSTRACT

From Last ten years, computer scientists show the interest in the study of social insect’s behavior algorithms like, Artificial Bee Colony (ABC), Ant Colony Optimization. Chief among of them, Standard ABC is well-known and new swarm optimization technique used for solving different combinatorial problems; however, it is often trapped in local optima in global optimization. This study investigates the new hybrid technique called Global Artificial Bee Colony-Backpropagation (GABC-BP) algorithm. This new technique shows great advantages of convergence property and excellent solution. To hybrid technique, GABC-BP algorithm used to this work for classification and prediction task. The performance of GABC-BP is benchmarked against BP, ABC and GABC. The experimental result shows that GABC-BP performs better than that standard BP, ABC and GABC for Boolean function classification and heat wave's temperature time series prediction. Key words: Global Artificial Bee Colony, Backpropagation, Hybrid Artificial Bee Colony. Introduction

Artificial Neural Network (ANN) is inspired by the functioning of the human biological neural networks has provided an exciting alternative method for solving a variety of problems in different areas of science and engineering technology. “A neural network is a massively parallel distributed processor who has a natural propensity for storing experimental knowledge and making it more available for use." It resembles the brain in two respects 1) knowledge is required for network through a learning process 2) interneuron connection strength known as synaptic weights are used to store knowledge” (Haykin, 1999). eural network is a system composed of many simple processing elements operating in parallel whose function is determined by network structure, connection strengths, and the processing performed at computing elements or nodes"(Darpa, 1988).

ANN can often provide a suitable solution for problems that generally are characterized by nonlinear, high dimensionality, noisy, complex, imprecise, imperfect and/or error-prone sensor data, poorly understood physical and statistical models, and lack of clearly stated mathematical solution or algorithm (Zaknich, 2003). Mostly, ANN approaches are capable of scientific, electrical engineering, earth knowledge, mathematics and of course for the statistical task (Jurado et al., 2001; Karaki & Chedid, 1994; Shuai et al., 2009). The well known approaches of ANN showed the best performance in different problems such as image processing (Dunstone, 1994; Leung et al., 1990; Naka et al., 1993) pattern and speech recognition (Filippi et al., 1994; Kumar et al., 2009; Min & Jianhui, 2002; Tsenov & Mladenov, 2010; Yonghong et al., 1997), smooth function approximation (Ferrari & Stengel, 2005), weather forecasting (Niska et al., 2005),volcanoes classification (Falsaperla et al., 1996), earthquake magnitude prediction, clustering (K.-L, 2010), scheduling (Chang-Shui et al., 1991; Changshui et al., 1992; Yong et al., 2009), feature selection techniques (Hofmann et al., 2004; Kavzoglu & Mather, 2000), constrained (Youshen et al., 2002) and unconstrained problems (Seong-Whan, 1995).

Learning in the context of ANN is defined as a process by which the free parameters of ANN are adapted through a process of presenting signals from the environment in which the network is embedded. The type of learning is determined by the way the parameter changes take place (Haykin, 1999). The notion of learning in ANN, is the process of guiding the network to provide a particular output or response for a specific given input. Learning is necessary when information about the input-output relationship is unknown or incomplete a-priori.

If the ANN finds the behavior of data, optimal weight values with suitable activation function, then performance will be appropriate for the different task such as prediction, classification, pattern recognition, clustering and scheduling. With the absence of the above four categories, the ANN can be trapped in local minima; effect on convergence speed and sometimes networks might be failing. The learning object is to

3329 J. Appl. Sci. Res., 9(5): 3328-3337, 2013

minimize the cost function which is the difference between desired output and neural network output. The network will train for finding optimal weights, which reduce the error until the convergence.

From the last decade, there have been several types of computational, hybrid, local search, global search, mathematical, biological-inspired meta heuristic algorithms developed for classification and time series prediction, these are, ant colony optimization (ACO) inspired by the foraging behavior of ant colonies (Dorigo & Stützle, 2003) , particle swarm optimization (PSO) inspired by the social behavior of bird flocking or fish schooling (Kennedy & Eberhart, 1995), and Hybrid Ant Bee Colony Algorithm inspired by the foraging behavior of ant and bee colonies (Shah et al., 2012) Hybrid Artificial Bee Colony inspired by the foraging behavior of bees and Levenberg–Marquardt algorithm, Global Artificial Bee Colony, Gbest_Guided Artificial bee Colony, Multiple Gbest_Guided Artificial Bee Colony, Improved Artificial Bee Colony, and so many other bio inspired population based algorithms (Karaboga et al., 2007; Karaboga & Ozturk, 2011; Shah et al., 2012, 2013; Shah, et al., 2012; Shah et al., 2012; Zhu & Kwong, 2010).

Recently, Artificial Bee Colony Algorithm has been very famous and attracted to solve many kinds of problems besides classification and prediction like clustering, numerical function optimization, scheduling, software testing, training Artificial Neural Networks and Higher Order Neural Networks for earthquake magnitude prediction, heat waves temperature, forecasting, modeling and so many other tasks (Ghazali et al., 2007). The performance of ABC shows that compared with GA, ACO, BP, and PSO, ABC algorithm can obtain better quality solutions.

For getting high efficiency, researchers have used the hybridization method of standard and improved ABC algorithms with BP, PSO, ACO, Deferential Evolution (DE), GA, etc, for different tasks (Ozturk & Karaboga, 2011). Global Artificial Bee Colony (GABC), is population-based algorithms that can provide the best possible solutions for different mathematical problems by using inspiration techniques from honey bees (Shah, et al., 2013). Backpropagation (BP) is a well known supervised form learning algorithm of obtaining the many weight's values in ANN applications, devolved by (Rumelhart et al., 1986.). BP algorithm is widely used to solve many engineering modeling problems (Najafzadeh et al., 2013; Zweiri et al., 2002). The basic BP algorithm is based on minimizing the error of the network using the derivatives of the error function. The BP used to adjust the network’s weight and threshold so to minimize the error for the different task such as classification, clustering and prediction on the training set (Abunawass et al., 1998; Bin Mohd Azmi & Cob, 2010). The hybridization of BP with swarm intelligence algorithms makes it more attractive for further improvement.

In this research article, two famous learning algorithms, Global Artificial Bee Colony and Backpropagation are hybrid called HGABC algorithm. It is hoped that the proposed learning algorithm will cover the drawbacks of standard ABC and BP. The hybrid technique is proposed here for Boolean classification problems and heat waves temperature time series prediction. The performance of the algorithm is compared with standard BP, ABC, and GABC algorithms. Related Works:

In this research paper two solution presented by proposed and standard learning algorithms, are Boolean

Function Classification and Time Series Prediction. Each task discussed in the following subsections.

a) Boolean Function Classification: There are many types of Boolean operations; however, commonly used are XOR, 3-Bit Parity and Encoder

/ Decoder operators. XOR is not linearly separable, consequently it cannot be implemented using single layer network; a three layered network is required to solve the problem. The 3-bit parity or even parity problem can be considered as a generalized XOR problem but it is more difficult for classification (Stork & Allen, 1992). This problem has been addressed because they are nonlinearly separable, and hence can not be solved by Single Layer Perceptron (SLP), such as the perceptron (Minsky & Papert, 1969). In other words, if the number of binary inputs is odd, the output is 1, otherwise it is 0 (V.P. Plagianakos, 1998) . The third classification task is 4-Bit Encoder/Decoder, which is well-known in computer science. The network is presented with four distinct input patterns, each having only one bit turned on. A decoder is a logic circuit which accepts a set of inputs that represents a binary number and activates only the output that corresponds to the input number. 4-Bit Encoder/Decoder is quite close to the real-world pattern classification task, where small changes in the input pattern cause small changes in the output pattern (Fahlman, 1988). b) Heat Waves Temperature Prediction:

The Oklahoma City, UK, daily heat wave's temperatures for up to four months of May, June, July, August

and September of the year of 2000, 2006, 2011 and 2012 will be used for prediction. The prediction is based on

3330 J. Appl. Sci. Res., 9(5): 3328-3337, 2013

its pattern which is heat wave temperatures in Fahrenheit. The heat waves dataset of temperature derived from the National Oceanic and Atmosphere Administration (NOAA) (NOAA, 2012) . In ANN learning, the network has one input pattern with one output pattern is the heat wave temperature. Global Artificial Bee Colony (GABC) algorithm:

Global Artificial Bee Colony algorithm is the advance version of standard ABC, has been successfully

applied for training ANN and numerical function optimization and so on. GABC has improve the exploitation procedure of standard ABC with global best neighbour information updates rules. There are three steps involved in GABC algorithm in standard employed, onlookers and scout bees of ABC. They become global employed, onlookers and scout bees through GABC strategy. The modified steps are:

Step 1: It modifies the employed section as

yxxxv kjijijijij (1)

ijjijj xycxxcy best2

best1 0,1rand0,1rand (2)

Step 2: Repeat the above formula with onlookers section.

Where y shows Best_Food_Source, 1c and 2c are two constant values, bestjx is the j-th element of the

global best solution found so far, bestjy is the j-th element of the best solution in the current iteration, ij is a

uniformly distributed real random number in the range [-1, 1]. Step 3: Modified the scout section as:

minmaxminrand 0,1rand ijijijij xxxx (3)

If 50.0,1rand , then

ijbestj

b

ijij xxiter

iterxx

max

mutation 11,0rand (4)

Else

ijbestj

b

ijij xyiter

iterxx

max

mutation 11,0rand (5)

Then comparing the fitness value of random generated solution randijx and mutation solution

mutationijx the

better one is chosen as a new food source, where b is a scaling parameter which is a positive integer within the range of [2,5]. Back Propagation Learning Algorithm:

Back-Propagation is one of the most attractive and novel supervised-learning algorithm proposed by

Rumelhart, Hinton, & Williams (1986) for Multilayer Perceptron training. Due to its high rate of flexibility and learning capabilities, it has been successfully used in wide range of science and engineering applications. It can be proved that BP can reach the extreme within a limited number of epochs for a given task. The merits of BP are that the adjustment of weights is always towards the descending direction of the error function and that only some local information is needed. The forward phase where the activations propagate from the input layer to the output layer. The backward phase, where then the observed actual value and the requested nominal value in the output layer are propagated backwards so it can modify the weights and bias values. Each node is composed of two sections. The first section generates a sum of the products of the weights multipliers and input signals. The second takes the result of the first section and puts it through its activation function, with scales the its input to a value between 0 and 1. Signal e is the output of the first section, and y = f(e) is the output of the second section. Signal y is also the output signal of an artificial neuron.

3331 J. Appl. Sci. Res., 9(5): 3328-3337, 2013

In the feed-forward phase the input signals are propagated through the input and hidden layers of processing elements, generating an output pattern in response to the input pattern presented.

Fig. 1: Feed-Forward phase of BP learning algorithm

Fig. 2: Back-Forward phase of BP learning algorithm

To teach the ANN, need a training data set. The training data set consists of input signals (x1 and x2 )

assigned with a corresponding target (desired output) z. The above figures 1and 2, illustrate how the signal is propagated through the network, Symbols w(xm)n represent the weights of connections between network input xm and neuron n in the input layer. Symbols yn represents output signal of neuron n.

nn)x(n)x(nn xwxw(fymm

1 (6)

In the next algorithm step the output signal of the network y is compared with the desired output value (the target), which is found in the training data set. The difference is called error signal of output layer neuron. This training will be continuing until the output error value, for all patterns in the training set. The BP learning technique will update the weight values for minimizing the node transfer function E (Chien-Cheng & Bin-Da, 2002). The training will be continued until convergence. Usually, the node transfer function is a nonlinear function such as a sigmoid function, a Gaussian function, and etc. The network error function E will be minimized as:

N

jkjkj )Ot(

N))t(w(E

1

1 (7)

E = error for the p th presentation vector tkj is the desired value from output node (k) to hidden node (j) Okj is the network value from output node (k) to hidden node (j) The back propagation is a gradient descent method in which gradient of the error is calculated with respect

to the weight's values for a given input by propagating the error backwards from the output layer to hidden layer and further to input layer continuously. The local gradient δj is calculated as:

)OT)(net(f jjjj (8)

3332 J. Appl. Sci. Res., 9(5): 3328-3337, 2013

Where Tj is the target output at node j and Oj is the actual output at node j. In Back-forward phase as shown in Figure 2.10 , the error calculated in the output layer is then propagated

back through the hidden layer where the synapse weights are updated to reduce the training error.

nn

nnmn

m/ x

de

)e(df)x(w)x(w (9)

Hybrid Global Artificial Bee Colony Algorithm:

The Global Artificial Bee Colony (GABC ) algorithm has a global ability to find global optimistic result,

and the BP algorithm (Shah, et al., 2012; Shah et al., 2012) is a standard method, relatively with simple implementation and work very well. Combining the step of GABC with BP, a new hybrid (HGABC) algorithm is proposed in this article for training MLP. The key point of this hybrid GABC-BP algorithm is that the GABC is used at the initial stage of searching for the optimum using global best methods, Then, the training process is continued with the BP learning algorithm (Peng et al., 2011). The flow-diagram of the ABC-BP model is shown in Figure 3. In the initial stage, the ABC algorithm finishes its training procedures, then, the BP algorithm starts training with the optimal weights of GABC algorithm and then BP trains the network for 100 epochs more. The Figure 3, shows the flow chart for proposed HGABC algorithm as:

Fig. 3: GABC-BP algorithm flow diagram Simulation results and discussion:

In this work, swarms intelligent-based combine technique based on HGABC algorithm is used to train feed-

forward artificial neural networks. In order to calculate the performance of the HGABC with ABC and BP algorithms in terms of Mean Square Error (MSE), Normalized Mean Square Error, Signal to Noise Ratio and

3333 J. Appl. Sci. Res., 9(5): 3328-3337, 2013

success rate using the boolean function for classification, where simulation experiments performed by Matlab 2010a software.

The stopping criteria minimum error is set to 0.0001 BP while ABC, GABC and HGABC stopped on MCN. During the experimentation, 10 trials were performed for training. The sigmoid function is used as activation function for network output. During the simulation, when the number of inputs, hidden and output nodes of the NNs and running time varies, the performance of training algorithms were stable, which is important for the designation of NNs in the current state. The value of c1 and c2 were selected 2.5. From the simulation experiment, the HGABC performance can be affected by c1 and c2. So the best values selected for these two constant values. Parameters setting is given in Table 1 and 2, respectively. Table 1: Parameters of the problems for Boolean Function Classification

Problem Colony Range NN structure D MCN XOR6 [100,-100] 2-2-1 (No Bias) 6 7500 XOR9 [10,-10] 2-2-1+ Bias(3) 9 100 XOR13 [-10,10] 2-3-1+ Bias(4) 13 75 3-Bit [-10,10] 3-3-1+bias(4) 16 1000 Enc/Dec [10,-10] 4-2-4+Bias(6) 22 1000

Table 2: Parameters of the problems for Heat Waves Temperature

Problem Colony Range Hidden Nodes Dimension (D) MCN ABC [10, 10] [2,9] [9, 49] 2000 GABC [10, 10] [2,9] [9, 49] 2000 BP 2000 HGABC [10, 10] [2,9] [9, 49] 1900+100

The stopping criterion for ABC and GABC, GGABC and 3G ABC is 1000 Maximum Cycle Number

(MCN) for boolean function classification and 2000 MCN for heat waves temperature time series prediction. During the experimentation, 10 trials performed in training. The values of c1 and c2 was selected 2.5. The sigmoid function used as activation function for network production for classification and prediction. During the simulation, when the number of input signals, hidden nodes, output node and running time varies, performing training algorithms were stable, which is important for delegation NNs in the current state. The average results for boolean function classification using standard and proposed learning algorithm given in Table 3. The success rate given in table 5. Table 3: Average of Mean Square Error using ABC,BP,GABC, HGABC algorithms.

Problems\methods ABC BP GABC HGABC XOR6 0.007051 0.1107 0.12076 0.000541 XOR9 0.006956 0.0491 0.000106 1.12E-12 XOR13 0.006079 0.0078 0.000692 2.21E-11 3-Bit Parity 0.006679 0.0209 0.00062 5.12E-09 4-bit End/Dec 0.008191 0.0243 0.000294 1.10E-09

Table 4: Success rate og ABC, BP,GABC and HGABC algorithm

Problems\Methods ABC BP GABC HGABC XOR6 100 6 100 100 XOR9 100 66.66 100 100 XOR13 100 96.66 100 100 3-Bit Parity 100 73.33 100 100 4-bit Enc/Dec 100 73.33 100 100

The testing Mean Square Error for XOR6, XOR9, XOR13, 3-bit Parity and 4-bit encoder/decoder are

0.000221, 1.02E-09, 2.11E-09, 4.12E-05 and 1.10E-07 respectfully using HGABC algorithm. The simulation results of Heat Waves Temperature for Prediction are given in table 5 and 6. Table 5: MSE Best Average results by for Heat Waves Temperature Prediction

NN Structure ABC BP GABC HGABC 2-2-1 0.009323 0.007185 0.009984 0.000122 2-3-1 0.002486 0.006887 0.000311 0.004810 3-3-1 0.002221 0.007283 0.000128 0.001023 3-5-1 0.002797 0.005643 0.001001 0.001081 4-4-1 0.008318 0.003872 0.000391 0.000110 4-5-1 0.005065 0.005652 0.000669 0.000368 5-6-1 0.001062 0.006121 0.001031 0.000711 5-9-1 0.000417 0.006291 0.000287 0.000118

3334 J. Appl. Sci. Res., 9(5): 3328-3337, 2013

Table 6: Best Simulation Results for Heat Waves temperature Prediction Time Series MSE NMSE SNR ABC 0.000352 0.295393 34.4177 BP 0.002853 0.372521 25.0961 GABC 0.000479 0.345013 35.0848 HGABC 0.000115 0.273215 36.3023

Fig. 4: Heat Waves Time Series Prediction by Proposed HGABC during Training

Fig. 5: Heat Waves Time Series Prediction by Proposed HGABC during Testing

Fig. 6: Learning Curves of Heat Waves Temperature Time Series

3335 J. Appl. Sci. Res., 9(5): 3328-3337, 2013

The Proposed HGABC success in getting maximum SNR for heat wave's temperature time-series data as given in Table 5. The SNR reached to 36.3023 by HGABC, while the ABC arises to 34.8159 and GABC-MLP 36.8972. The NMSE of the proposed HGABC learning algorithm is given in table 5, shows the minimum NMSE compared to other's algorithms. For evaluating the performance of proposed learning technique, the optimal weight values test for a prediction job with the 25% data of heat wave's temperature time series. The testing MSE and NMSE of HGABC give very less prediction error from standard ABC, BP and GABC, given in Figure 4 and 5. The learning The BP, ABC and GABC training algorithm did not converge faster than other proposed HGABC for heat wave's temperature time-series data. Comparative simulation studies have been presented to show the effectiveness of our new algorithms. The simulation results in Figure 6, demonstrate the effectiveness of the new HGABC learning algorithms. Conclusion:

The HGABC algorithm collects the exploration and exploitation processes successfully, which proves the

high performance of training MLP. It has the powerful ability of searching global optimal solution. So, the proper weights of the MLP may speed up the initialization and improve the classification and prediction accuracy of the trained NNs. The simulation results show that the proposed HGABC algorithm can successfully train boolean data for classification purpose and heat wave's time series for prediction, which further extends the quality of the given approach. The performance of HGABC is compared with the traditional BP, GABC and ABC algorithms. HGABC shows significantly higher results than BP, GABC and ABC algorithms during experiment. Acknowledgement

The authors would like to thank University Tun Hussein Onn Malaysia (UTHM) for supporting this

research under the Postgraduate Incentive Research Grant Vote No 0739. References

Abunawass, A.M. et al., 1998. Fuzzy clustering improves convergence of the backpropagation algorithm. Paper

presented at the Proceedings of the 1998 ACM symposium on Applied Computing. Bin Mohd Azmi, M.S., & Z.C. Cob, 2010. Breast Cancer prediction based on Backpropagation Algorithm.

Paper presented at the Research and Development (SCOReD), 2010 IEEE Student Conference on. Chang-Shui, Z., Y. Ping-Fan, & C. Tong, 1991. Solving job-shop scheduling problem with priority using neural

network. Paper presented at the Neural Networks, 1991. 1991 IEEE International Joint Conference on. Changshui, Z., Y. Pingfan, & C. Tong, 1992. Solving Job-Shop Scheduling Problem with Amv Using Neural

Network. Paper presented at the Intelligent Control and Instrumentation, 1992. SICICI '92. Proceedings., Singapore International Conference on.

Chien-Cheng, Y., & L. Bin-Da, 2002. A backpropagation algorithm with adaptive learning rate and momentum coefficient. Paper presented at the Neural Networks, 2002. IJCNN '02. Proceedings of the 2002 International Joint Conference on.

Darpa (Ed.)., 1988. Darpa Neural Network Study: October 1987-February 1988 Afcea Intl Pr. Dorigo, M., & T. Stützle, 2003. The Ant Colony Optimization Metaheuristic: Algorithms, Applications, and

Advances. In F. Glover & G. Kochenberger (Eds.), Handbook of Metaheuristics, 57: 250-285. Dunstone, E.S., 1994. Image processing using an image approximation neural network. Paper presented at the

Image Processing, 1994. Proceedings. ICIP-94., IEEE International Conference. Fahlman, S., 1988. An empirical study of learning speed in backpropagation networks,. Paper presented at the

Technical Report CMU-CS-88-162. Falsaperla, S. et al., 1996. Automatic classification of volcanic earthquakes by using Multi-Layered neural

networks. Natural Hazards, 13(3); 205-228. Ferrari, S., & R.F. Stengel, 2005. Smooth function approximation using neural networks. Neural Networks,

IEEE Transactions on, 16(1): 24-38. Filippi, E., M. Costa, & E. Pasero, 1994. Multi-layer perceptron ensembles for increased performance and fault-

tolerance in pattern recognition tasks. Paper presented at the Neural Networks, 1994. IEEE World Congress on Computational Intelligence., 1994 IEEE International Conference on.

Ghazali, R. et al., 2007. Dynamic Ridge Polynomial Neural Networks in Exchange Rates Time Series Forecasting. In B. Beliczynski, A. Dzielinski, M. Iwanowski & B. Ribeiro (Eds.), Adaptive and Natural Computing Algorithms, 4432: 123-132. Springer Berlin Heidelberg.

Haykin, S., 1999. Neural Networks: A Comprehensive Foundation (second edition ed.). Prentice-Hall, Upper Saddle River, NJ.

3336 J. Appl. Sci. Res., 9(5): 3328-3337, 2013

Hofmann, A., T. Horeis, & B. Sick, 2004. Feature selection for intrusion detection: an evolutionary wrapper approach. Paper presented at the Neural Networks, 2004. Proceedings. 2004 IEEE International Joint Conference on.

Jurado, F. et al. 2001. Neural networks and fuzzy logic in electrical engineering. Paper presented at the Electrical and Computer Engineering, 2001. Canadian Conference on.

K.-L, D., 2010. Clustering: A neural network approach. Neural Networks, 23(1): 89-107. Karaboga, D., B. Akay, & C. Ozturk, 2007. Artificial Bee Colony (ABC) Optimization Algorithm for Training

Feed-Forward Neural Networks. In V. Torra, Y. Narukawa & Y. Yoshida (Eds.), Modeling Decisions for Artificial Intelligence., 4617: 318-329.

Karaboga, D., & C. Ozturk, 2011. A novel clustering approach: Artificial Bee Colony (ABC) algorithm. Applied Soft Computing, 11(1): 652-657.

Karaki, S., & R. Chedid, 1994. Artificial neural networks: a new approach for treating electrical engineering problems. Paper presented at the Frontiers in Education Conference, 1994. Twenty-fourth Annual Conference. Proceedings.

Kavzoglu, T., & P.M. Mather, 2000. Using feature selection techniques to produce smaller neural networks with better generalisation capabilities. Paper presented at the Geoscience and Remote Sensing Symposium, 2000. Proceedings. IGARSS 2000. IEEE 2000 International.

Kennedy, J., & R. Eberhart, 1995. Particle Swarm Optimization",. Paper presented at the Proceeding of IEEE International Conference on Neural Network 4, Australia,.

Kumar, T.L., T.K. Kumar, & K.S. Rajan, 2009. Speech Recognition Using Neural Networks. Paper presented at the 2009 International Conference on Signal Processing Systems.

Leung, M.T., W.E. Engeler, & P. Frank, 1990. Fingerprint image processing using neural networks. Paper presented at the TENCON 90. 1990 IEEE Region 10 Conference on Computer and Communication Systems.

Min, H., & X. Jianhui, 2002. Radial basis perceptron network and its applications for pattern recognition. Paper presented at the Neural Networks, 2002. IJCNN '02. Proceedings of the 2002 International Joint Conference on.

Minsky, M., & S.A. Papert, 1969. Perceptrons: An Introduction to Computational Geometry, . MA: Cambridge: MIT Press.

Najafzadeh, M., G.-A. Barani, & M.R. Hessami Kermani, 2013. GMDH based back propagation algorithm to predict abutment scour in cohesive soils. Ocean Engineering, 59(0): 100-106.

Naka, M. et al., 1993. Thermal image processing using neural network. Paper presented at the Neural Networks, 1993. IJCNN '93-Nagoya. Proceedings of 1993 International Joint Conference on.

Niska, H.et al., 2005. Evaluation of an integrated modelling system containing a multi-layer perceptron model and the numerical weather prediction model HIRLAM for the forecasting of urban airborne pollutant concentrations. Atmospheric Environment, 39(35): 6524-6536.

NOAA., 2012. National Ocean and Atmosphere Adminstartion National Climate Data Center, 2012, from http://www.srh.noaa.gov/oun/?n=climate-okc-heatwave

Ozturk, C., & D. Karaboga, 2011. Hybrid Artificial Bee Colony algorithm for neural network training. Paper presented at the Evolutionary Computation (CEC), 2011 IEEE Congress on.

Peng, G., C. Wenming, & L. Jian, 2011. Global artificial bee colony search algorithm for numerical function optimization. Paper presented at the Natural Computation (ICNC), 2011 Seventh International Conference on.

Rumelhart, D.E., H.G., & J, W.R. 1986. Learning internal representations by back-propagation errors. Seong-Whan, L., 1995. Multilayer cluster neural network for totally unconstrained handwritten numeral

recognition. [doi: 10.1016/0893-6080(95)00020-Z]. Neural Networks, 8(5): 783-792. Shah, H., R. Ghazali, & N. Nawi, 2012. Hybrid Ant Bee Colony Algorithm for Volcano Temperature

Prediction. In B. Chowdhry, F. Shaikh, D. Hussain & M. Uqaili (Eds.), Emerging Trends and Applications in Information Communication Technologies, 281: 453-465): Springer Berlin Heidelberg.

Shah, H., R. Ghazali, & N. Nawi, 2013. Global Artificial Bee Colony Algorithm for Boolean Function Classification. In A. Selamat, N. Nguyen & H. Haron (Eds.), Intelligent Information and Database Systems, 7802: 12-20): Springer Berlin Heidelberg.

Shah, H. et al., 2012. Global Hybrid Ant Bee Colony Algorithm for Training Artificial Neural Networks Computational Science and Its Applications – ICCSA 2012. In B. Murgante, O. Gervasi, S. Misra, N. Nedjah,

A. Rocha, D. Taniar & B. Apduhan (Eds.), 7333: 87-100): Springer Berlin / Heidelberg. Shah, H., R. Ghazali, & N.M. Nawi, 2012. Hybrid Ant Bee Colony Algorithm for Volcano Temperature

Prediction Emerging Trends and Applications in Information Communication Technologies. In B. S. Chowdhry, F. K.

Shaikh, D. M. A. Hussain & M. A. Uqaili (Eds.), 281: 453-465): Springer Berlin Heidelberg.

3337 J. Appl. Sci. Res., 9(5): 3328-3337, 2013

Shah, H. et al., 2012. G-HABC Algorithm for Training Artificial Neural Networks International Journal of Applied Metaheuristic Computing, 3(3): 20. Shuai, L. et al., 2009. A Hybrid Conversion of GPS Height Approach Based on Neural Networks and

EGM96 Gravity Model. Paper presented at the Information Engineering and Computer Science, 2009. ICIECS 2009. International Conference on.

Stork, D.G., & J.D. Allen, 1992. How to solve the N-bit parity problem with two hidden units. [doi: 10.1016/S0893-6080(05)80088-7]. Neural Networks, 5(6): 923-926.

Tsenov, G. T., & Mladenov, V. M. (2010, 23-25 Sept. 2010). Speech recognition using neural networks. Paper presented at the Neural Network Applications in Electrical Engineering (NEUREL), 2010 10th Symposium on.

Plagianakos, V.P., D.G.S., and M.N. Vrahatis, 1998. An Improved Backpropagation Method with Adaptive Learning Rate (No. TR98-02). Greece: University of Patras

Yong, S. et al., 2009. Design of Neural Network Gain Scheduling Flight Control Law Using a Modified PSO Algorithm Based on Immune Clone Principle. Paper presented at the Intelligent Computation Technology and Automation, 2009. ICICTA '09. Second International Conference on.

Yonghong, Y., M. Fanty, & R. Cole, 1997. Speech recognition using neural networks with forward-backward probability generated targets. Paper presented at the Acoustics, Speech, and Signal Processing, 1997. ICASSP-97., 1997 IEEE International Conference on.

Youshen, X., H. Leung, & W. Jun, 2002. A projection neural network and its application to constrained optimization problems. Circuits and Systems I: Fundamental Theory and Applications, IEEE Transactions on, 49(4): 447-458.

Zaknich, A., 2003. Neural networks for intelligent signal processing. World Scientific Pub.: River Edge, NJ.

Zhu, G., & S. Kwong, 2010. Gbest-guided artificial bee colony algorithm for numerical function optimization. Applied Mathematics and Computation, 217(7): 3166-3173.

Zweiri, Y.H. et al., 2002. A new three-term backpropagation algorithm with convergence analysis. Paper presented at the Robotics and Automation, 2002. Proceedings. ICRA '02. IEEE International Conference on.