Backpropagation Algorithm for Neurofuzzy Filter

INTERNATIONAL JOURNAL OF COMPUTATIONAL COGNITION (HTTP://WWW.YANGSKY.COM/IJCC/), VOL. 9, NO. 3, SEPTEMBER 2011 41

Backpropagation Algorithm for Neurofuzzy FilterA.Q. Ansari and Neeraj Gupta

Abstract— In this paper, a neurofuzzy filter (NFF) is presented,which is based on fuzzy if-then rules (structure learning) and thetuning of the parameters of membership function (parameterlearning). In the structure learning, fuzzy rules are found basedon the matching of input-output cluster and in parameterlearning, fine tuning is obtained for output accuracy of a network.To minimize the output error, a new kind of backpropagationalgorithm is developed. In this algorithm we propose a variationalmethod for determining globally optimal learning parametersand learning rules for on-line gradient descent training ofmultilayer neural network. Both structure and parameter learn-ing are performed simultaneously as the adaptation proceeds.Simulations in hand free mobile & echo cancellation on atransmission line are performed. Noise cancellation of hand freemobile by the proposed NFF is illustrated and good performanceis achieved with root mean square error (RMSE) of 0.0440 andmaximum instant error of 0.02.Also good performance is achievedby applying the NFF to echo cancellation on a transmissionline. Copyright c© 2011 Yang’s Scientific Research Institute, LLC.All rights reserved.

Index Terms— Structure learning, parameter learning, back-propagation algorithm, neurofuzzy filters, noise cancellation, echocancellation.

I. INTRODUCTION

ADAPTIVEfiltering has achieved widespread applicationsand success in many areas such as image processing,

control and communications [1]. In classical filtering theory,signal estimation based on a priori knowledge of both desiredsignal and the noise may largely depend on linearity of modeland stationary mathematical methods. But in case of high non-linearity the estimation of performance may become unaccept-able. For such cases, neurofuzzy filtering may provide bettersolution to noise filtering [2]-[7] problem. Neural networksare composed of a large number of highly interconnectedprocessing elements (nodes), which are connected throughthe weights. When looking into the structure and parameterlearning of neural networks, many common points of themethods used in adaptive signal processing can be found. Thebackpropagation algorithm used to train the neural networkis a generalized Widrow’s least mean square algorithm andcan be contrasted to the least mean squares (LMS) algorithmusually used in adaptive filtering.

Many kinds of nonlinear filters designed using neural net-works have been proposed [8]-[12]. One of them is the neuralfilter, whose learning algorithm is shown to be more efficient

Manuscript received May 4, 2010; revised September 4, 2010.A.Q. Ansari, Department of Electrical Engg., Jamia Millia Islamia,New

Delhi – 110025 Email:[email protected] Gupta, Department of Electrical Engg., Krishna Institute of Engg.

& Technology, Ghaziabad -201206. Email:neeraj [email protected] Item Identifier S 1940-5863(10)10608-9/$20.00

Copyright c©2011 Yang’s Scientific Research Institute,LLC. Al-l rights reserved. The online version posted on June 20, 2012 athttp://www.YangSky.com/ijcc/ijcc93.htm

than Lin’s adaptive stack filtering algorithm [8]. This classof neural filters is based on the threshold decomposition andneural networks, and is divided into hard neural filters (whoseactivation functions are unit steps) and soft neural filters(whose activation functions are sigmoid functions). Anotherkind of neural filter is the recursive filter obtained by traininga recurrent multilayer perceptron (RMLP) [13]. Other applica-tions of neural networks in adaptive filtering include nonlinearchannel equalizers [14],[15] and noisy speech recognition[16],[17] where neural networks are used to map the noisyinput features into clean output features for recognition. Aproblem encountered in the design of neural filters is that theinternal layers of neural networks are always opaque to theuser, so it is not easy to determine the structure and size of anetwork.

To enable a neural network to learn from numerical dataas well as expert knowledge expressed as fuzzy if-then rules,several approaches have been proposed [18]-[22]. To overcomethe shortcomings encountered in neural filters, while stillkeeping their advantages, a neurofuzzy filter is presented inthis paper. Neurofuzzy filter learns system behavior by usingsystem input-output data and so does not use any mathematicalmodeling. After learning the system’s behavior, neurofuzzyfilter automatically generates fuzzy rules and membershipfunctions and thus solves the key problem of fuzzy logicand reduces the design cycle very significantly. Neurofuzzyfilter then verifies the solution (generated rules and member-ship functions). It also optimizes the number of rules andmembership functions. The fuzzy logic solution developedby neurofuzzy filter [23] solves the implementation problemassociated with neural nets. Unlike conventional fuzzy logic,neurofuzzy filter uses new defuzzification, rule inferencingand antecedent processing algorithms, which provide morereliable and accurate solutions. These new algorithms aredeveloped based on neural net structure. Finally, automaticcode converter converts the optimized solution (rules andmembership functions) into embedded controller’s assemblycode.

In order to facilitate an easy and systematic understandingof the proposed work, section II includes a description of thestructure of the neurofuzzy filter. Neurofuzzy data processingsystem along with a new kind of backpropagation algorithm ispresented in section III. Section IV illustrates the implemen-tation of several typical neurofuzzy filter applications and atthe end, conclusion is summarized.

II. STRUCTURE OF NEUROFUZZY FILTER

Figure 1. shows the function diagram of the proposedneurofuzzy filter (NFF). The neurofuzzy filter possesses theadvantages of both the neural and fuzzy systems. It brings the

42 INTERNATIONAL JOURNAL OF COMPUTATIONAL COGNITION (HTTP://WWW.YANGSKY.COM/IJCC/), VOL. 9, NO. 3, SEPTEMBER 2011

low-level learning and computational power of neural network-s into fuzzy systems and provides the high-level human-likethinking and reasoning of fuzzy systems into neural networks.

Fig. 1: Functional diagram of the proposed NFF

Both structure and parameter learning are used concur-rently for the construction of an adaptive neurofuzzy filter.The structure learning includes the pre-condition and con-sequent structure identification. The pre-condition structurecorresponds to the input space partitioning and is formulatedas a combinational optimization problem with objective tominimize the number of rules generated and number of fuzzysets. The main task of consequent structure identification isto decide when to generate a new membership function andwhich significant terms (input variables) should be added tothe consequent parts. Least mean squares (LMS) or recursivemean squares (RLS) algorithms adjust the parameters of theconsequent parts and the pre-condition part parameters areadjusted by using backpropagation algorithm to minimize thecost function. During the learning process the system can beused anytime for normal operation without repeated trainingon the input output patterns. Rules are created dynamicallyon receiving online training data by performing followingprocesses:• Input / Output space partitioning;• Fuzzy rules and membership function generator;• Fuzzy rules verifier and optimizer;• Parameter learning.In this learning process the first three steps belong to

structure learning and the last step belong to the parameterlearning. The various blocks of an adaptive neurofuzzy filtercan be explained as follows:

A. Input / Output Space Partitioning

This block determines the number of rules extracted fromthe training data as well as the number of fuzzy sets on theuniverse of discourse disclosure of each input variable. A rulecorresponds to a cluster in the input space with mi and pirepresenting the center and the variance of the cluster. For

the incoming pattern x the strength a rule is fired can beinterpreted as the degree of incoming pattern belonging to thecorresponding cluster. For computational efficiency, we canuse the firing strength as follows:

F i(x) =∏i

ui = e−[pi(x−mi)]T [pi(x−mi)] (1)

where F i ∈ [0, 1] and the argument of the exponential is thedistance between x and the cluster i. The above criteria canbe used for the generation of new fuzzy rule. Let x(t) be thenewly coming pattern then

J = arg max1≤j≤r(t)

F j(x) (2)

where r(t) is the number of existing rules at time t. If F j ≤F (t), then a new rule is generated where F (t) ∈ [0, 1].

Once a new rule is generated the next step is to assigninitial centers and widths of the corresponding membershipfunctions, which can be set as follows

mr(t)+1 = x (3)

pr(t)+1 = − 1

βdiag[

1

lnF J· · · · · · · · · 1

lnF J] (4)

In the system the width is taken into account in degreemeasure so that a cluster with larger width, fewer rules willbe generated in its vicinity than a cluster with smaller width.

B. Fuzzy Rules and Membership Function Generator

The generation of a new input cluster corresponds to thegeneration of a new fuzzy rule, with its pre-condition partconstructed by the learning algorithm. At the same time, theconsequent part of the generated rule is decided. The algorithmis based on the fact that different pre-conditions of differentrules may be mapped to the same consequent fuzzy set. Sinceonly the center of each output membership function is used fordefuzzification, the consequent part of each rule may simplybe regarded as a singleton.

C. Fuzzy Rule Verifier and Optimizer

The generated fuzzy rules and membership functions canbe verified by using neurofuzzy rule verifier. Both trainingset as well as a test set should be used for the verificationprocess. If the generated rules and membership functions donot produce satisfactory result, one can easily manipulateappropriate parameters (e.g. more data, smaller error criterion,learning rate etc) so that the neural net learns more aboutthe system behavior and finally produce satisfactory solution.The number of rules and membership function can also beoptimized using the neurofuzzy rule verifier, which is anothervery important feature. This reduces memory requirementand execution speeds – the two very desirable features formany applications. The optimization process might lose someaccuracy and one can make some trade-offs between accuracyand cost.

A.Q.ANSARI AND NEERAJ GUPTA, BACKPROPAGATION ALGORITHM FOR NEUROFUZZY FILTER 43

D. Parameter Learning

After the network structure is adjusted according to thecurrent training pattern, the network then enters the parameterlearning phase to adjust the parameters of the network opti-mally based on the same training pattern. Parameter learningis performed on the whole network after structure learning,irrespective of whether the nodes (links) are newly added orare existent originally. Using backpropagation for supervisedlearning the error function as follow.

D =1

2[O(t)− T (t)]2 (5)

where T (t) is the desired output and O(t) is the current output.For each training data set, starting at the input nodes, a

forward pass is used to compute the activity levels of all thenodes in the network to obtain the current output O(t).Thenstarting at the output nodes, a backward pass is used tocompute ∂D�∂w for all the hidden nodes. Noting that wis the adjustable parameter in a node, the update rule used is

∆w ∝ −∂D∂w

(6)

w(t+ 1) = w(t) + η(−∂D∂w

) (7)

where η is the learning rate, ∂D∂w = ∂D

∂a ·∂a∂w and a is the

activation function.

E. Automatic Code Converter

After a satisfactory solution is obtained, automatic codeconverter converts the solution to an embedded processorsassembly code. Various options can be provided to have thecode optimized for accuracy, speed, and/or memory. As a casestudy we show the definition and evaluation of instructionset extension for neurofuzzy processing [24]. To supportneurofuzzy computations, we have considered the followinginstructions:

mula Multiply & Accumulatorlbtf Compute transfer functionslw Loads fuzzified values from the memoryrev Unweighted rule evalutionrevw Weighted rule evalutionwav Defuzzification

III. NEUROFUZZY INTEGRATED DATAPROCESSING SYSTEM

Neurofuzzy data processing system contains six layer struc-tures as shown in figure 2. The input layer represents in-put variables. Each node of input layer corresponds to oneinput variable, which only transmits input values to nextlayer directly. The second layer represents fuzzy quantizationof each input variable space. Each neuron of second layerrepresent a term of an input linguistic variable and acts asa membership function. The Gaussian membership functionis used in this layer. The third layer contains rule nodes thatevolve through supervised and/or unsupervised learning. Thenumber of neurons in layer four is equal to that in layer threeand this layer, with only the error signal need to be computed.

The fifth layer of neurons represents fuzzy quantization of theoutput variables similar to the input fuzzy representation. Thesixth layer represents the values of the output variables.

Fig. 2: Neurofuzzy integrated data processing system

A. Network Training

The training process involves two rounds of propagation.First, the input weights to each hidden unit carry differentinformation within a common layer; the input weights to eachhidden unit are simplified by deleting small weights. Then, thehidden units within the same layer are clustered on the basisof the similarity between their input weight vectors. After thefirst round of propagation, the trained network then receivesthe following treatment in order to make each hidden unitcarry different pattern information within a common layer. Thetraining algorithm is described below:

1)Weight initialization: Weights are initialized according tothe given domain knowledge formulated as rules.

2)Calculation of activation(a)The activation level of an input unit is determined by

the instance presented to the network.(b)The activation level Oj of a hidden and output unit is

determined by

Oj = F (Wj1O1,Wj2O2 · · · − θj) (8)

Where Wji ’s and Oi’s are input weights and input to unit j,respectively,θj is the threshold on the unit, and F is a sigmoidfunction. We can also represent in another form as below

Oj = F (∑

WjiOi − θj) (9)

3)Weight training in backpropagation: (a)Start at the out-put units and work backward to the hidden layers ecursively.Adjust weights by

Wji(t+ 1) = Wji(t) + ∆Wji (10)

Where Wji(t) is the weight from unit i to unit j at iterationt and ∆Wjiis the weight adjustment.


(b)The weight change is computed by

∆Wji = ηDj(∂Oj�∂Wji)− ζWji(t) (11)

where η is a trial independent learning rate, ζ is a weightdecay parameter, Dj is the discrepancy error between theactual and the desired output activation at unit j , and theterm ∂Oj�∂Wji is the partial derivative of Oj with respectto Wji.

(c)The discrepancy is given by(1)For output units:

Dj = Tj −Oj (12)

Where Tj is the desired (target) output activation and Oj isthe actual output activation at output unit j.

(2)For hidden units:

Dj =∑k

(∂Ok�∂Oj)Dk (13)

Where Dkis the discrepancy at unit k to which a connectionpoints from hidden unit j. In the summation, each discrepancyDkis weighted by the partial derivative of Okwith respect toOj .

(d)Repeat the above iterations until stopping conditionsare met.

B. Layer-wise Backpropagation Description

Layer Six: Only the error signal needs to be computed andpropagated which is calculated as

δ(6)i = − ∂D

∂a(6)= T (t)−O(t) (14)

where δ(6)i is the error signal.Layer Five: The update rule for aji is

− ∂D

∂aji= − ∂D

∂a(6)∂a(6)

∂a(5)∂a(5)

∂aji(15)

and∂a(6)

∂a(5)= 1 (16)

∂a(5)

∂aji= xj

∑i

p(5)i (17)

where the summation is over the number of links from layerfive for the i th node.Hence the parameter aji is updated by

aji(t+ 1) = aji(t) + η[T (t)−Ot]xj∑i

p(5)i (18)

where x0 = 1 .We propose a variational method for determin-ing globally optimal learning parameters and learning rules foron-line gradient decent training of multilayer neural network.The proposed approach is based on maximizing the total de-crease in generalization error over a fixed time window, using astatistical mechanics description of the learning process. Themethod is employed for obtaining optimal learning rates inboth realizable and noise-corrupted tasks, for determining therelation between optimal learning rates of different weightsand for examining the efficacy of regularizers in noisy andover-realizable training scenarios.

We find the locally optimal learning rate by determiningthe value of η that minimizes ∂D�∂a, using the followingequations

dRijda

= ηφij ,dQikda

= ηψik + η2vik (19)

where φij = 〈δiOj〉, ψik = 〈δixk + δkxi〉 and vik = δiδkand the fact that the change in generalization error over timedepends exclusively on the overlaps. The expression obtainedfor the locally optimal learning rate is then

η = −Σij

∂D∂Rij

φij + Σik∂D∂Qik

ψik

Σij∂D∂Qik

vik(20)

Although the value of η obtained in this manner may beuseful for some phases of the learning process it is likelyto be useless for others. For example, the lowest general-ization error for the symmetric phase characterized by lackof differentiation between the nodes is achieved by graduallyreducing the learning rate towards zero. However, decaying thelearning rate in the symmetric phase will prevent the systemfrom escaping the symmetric fixed point, thus resulting in sub-optimal solution.

Layer Four: Only the error is need to be calculated in hislayer:

δ(4)i = − ∂D

∂a(4)= − ∂D

∂a(5)∂a(5)

∂a(4)(21)

where∂a(5)

∂a(4)=

∑j

ajixi (22)

if there are multiple outputs, then the error signal becomes

δ(4)i = Σkδ

(4)k (23)

where the summation is performed over the consequents of arule node.

Layer Three: As in layer four , only the error signal needto be computed in this layer

δ(3)i = − ∂D

∂a(3)= −

∑j

∂D

∂a(4)j

∂a(4)j

∂a(3)j

(24)

∂a(4)j

∂a(3)i

=

∑i a

(3)i − a3j

[c∑i=1

a(3)i ]2

, j = i (25)

∂a(4)j

∂a(3)i

=−a3j

[c∑i=1

a(3)i ]2

, j 6= i (26)

so, we have

δ(3)i =

∑j

δ(4)j

∂a(4)j

∂a(3)i

(27)

Layer Two: With the choice of the Gaussian membershipfunction, the operation performed in this layer is using.f [(uij)

2] = − [∝(2)i −cij ]

2

σ2ij

where cij and σij are center andwidth of the Gaussian membership function.


The update rule of c2ij is derived as in the following:

− ∂D

∂c(2)ij

= − ∂D

∂a(3)

∑k

∂a(3)

∂a(2)k

∂a(2)k

∂c2ij(28)

where ∂a(3)

∂a(2)k

= a(3)

a(2)k

∂a(2)k

∂c2ij=

{a(2)k

2(xi−Cij)

σ2ij

if term node jis connected to rule node k,

0 otherwise.(29)

So, the update rule of C2ij is

C2ij(t+ 1) = C2

ij(t)− η∂a(3)

∂C2ij

(30)

The update rule of σ2ij is derived as

− ∂D

∂σ2ij

= − ∂D

∂a(3)

∑k

∂a(3)

∂a2k

∂a2k

∂σ(k)ij

(31)

∂a(2)k

∂σ2ij

=

{a(2)k

2(xi−Cij)2

σ3ij

if term node jis connected to rule node k,

0 otherwise.(32)

The update rule of σ2ij is :

σ2ij(t+ 1) = σ2

ij(t)− η∂D

∂σ2ij

(33)

IV. APPLICATIONS AND SIMULATIONS

The adaptive noise cancelling using NFF is shown in figure3. An input signal contaminated by the noise is transmitted tothe receiver.

Fig. 3: Adaptive noise cancelling using NFF

The principle of the adaptive noise cancellation techniques[25] is to adaptively process (by adjusting the filter’s weights)the reference noise c1(t) to generate a replica of c(t) , andthen subtract the replica of c(t) from the primary input x(t)to recover the desired signal h(t) . We denote the replica ofc(t), i.e., the adaptive filter output, as process c(t).

With the diagram, the received signal x(t) can be describedas:

x(t) = h(t)+c(t) = h(t)+f(n(t), n(t−1), · · · · · · · · · (t−k))(34)

Where the f(·) represents the passage channel of the noisesource n(t). The basic principle of the adaptive noise cancel-lation is to estimate the desired signal form the corrupted. Thenoise cancelling is based on the following assumptions[1]:

� h, c, and the noise signal n, are zero-mean process(statistical stationary and zero means);

� h is uncorrelated with nand c;� nand c are correlated by the function f(·)The recovered signal r(t), which is served as the error signal

in the adaptive process, can be given as

r(t) = h(t) + c(t)− c(t) (35)

Where the c(t) is the estimated output from the filter. Bysquaring, the expectation of (35) is given as

E[r(t)2] = E[h(t)2] +E[(c(t)− c(t))2] +E[h(t)(c(t)− c(t))](36)

Based on the second assumption, the third item of (36) canbe removed and given as

E[r(t)2] = E[h(t)2] + E[(c(t)− c(t))2] (37)

In the nature of application, the signal power ofh(t),E[h(t)2], remains unaffected while the adaptive filter isused to minimize the power of the recovered signal. In otherwords, in adaptation of the error signal, the power differenceis minimized between the contaminated signal and the filteroutput, i.e.

Emin[r(t)2] = E[h(t)2] + Emin[(c(t)− c(t))2] (38)

From (38), the conclusion is deducted that the powerminimization of the signal r(t) is equivalent to minimizingthe power of noise. In this simulation, the neurofuzzy filter istrained to remove the noise in the adaptive noise cancellingprocess.

Traditionally, the design of the adaptive filters for the afore-mentioned noise cancelling problem is based upon a linearfilter adapted by the LMS or RLS algorithm. In real situations,the environment between n(t) and c(t) or n(t) and c1(t) is socomplex that c(t) or c1(t) is in fact a nonlinear function of n(t)[26]-[28]. Higher performance of noise cancellation by using anonlinear filter can thus be expected. In order to demonstratethe capability and effectiveness of the NFF, simulations inhand free mobile and echo cancellation on a transmission lineare performed.

A. APPLICATION TO HAND FREE MOBILE

The important application of NFF includes a mobile hungonto the vehicle control panel. The noise created by theengine affects the ongoing conversation, since it adds to thevoice signal and enters into the microphone. Thus the netoutput electrical signal of the microphone would contain twocomponents noted above.


Consider the human input source

h(t) = sin (sin (20 · t) · t · 200) (39)

as shown in figure 4. and noise generated by engine are givenas

n(t) = sin (20 · pi · t)

2+ (0.2 · sin (100 · pi · t− 100)) (40)

The input signal is a sum of usual voice signal h(t) andengine noise n(t). A neurofuzzy filter has been used asthe adaptive noise filter for getting extracting signal. TheNFF achieves good performance with RMSE of 0.0440 andmaximum instant of 0.02. The extracted signal through the useof trained NFF is much closer to the useful voice signal asshown in figure 4.

Fig. 4: Illustrations of hand free mobile using NFF (a) Usefulvoice signal h. (b) The engine noise n (c) Noisy signal x = h+ 0.833 n (d) Extracted signal. (e) Extraction error.

B. Application to Echo Cancellation on Transmission Line

Another important application of NFF includes eliminatingthe echo produced on a transmission line, for example atelephone line. Computer simulation assesses the performanceof the developed adaptive system for echo cancellations. Echophenomenon on a transmission line is shown in figure 5.

The line characteristics, and therefore its shortcoming, aredifferent for every link created. The use of the adaptiveneurofuzzy filtering to limit the effect of these shortcomingstherefore seems perfectly justified as follows:

1)Echo cancellation is the key to deliver superior voicequality in long distance wire line telephone calls.

2)Because of the line length and its possible loss of adapt-ability, a given out signal can be sent back towards its emitter,thus producing an unpleasant echo phenomenon.

3)Application of adaptive filtering is employed to eliminatethe echo produced on a transmission line.

Fig. 5: Echo Phenomenon On a Transmission Line

4)Echo cancellation involves first recognizing the originallytransmitted signal that re-appears, with some delay, in thetransmitted or received signal.

5)Once the echo is recognized, it can be removed bysubtracting it from the transmitted or received signal.

For transmission line modeling we limit our self to atransmission delay due to the signal propagation duration. Theecho signal is produced from the received signal, multiplied bya attenuation factor after having undergone a light distortion.

The transmission time, controlled at 10ms, is weak inrelation to delay, which can occur on long distance links. Thischoice is made simply because the transport delay block dueto its inherent function, will make the output signal appearonly after time has elapsed.

The adaptive filter is achieved with help of an S- function, towhich we will pass as parameters the Ts sampling period, theδ filter adaptation coefficient and the n filter order. Simulinkmodel of using NFF for solving echo cancellation is shownin figure 6. The output signal through the use of the trainedNFF is much closer to the original signal is shown figure 7.

Fig. 6: Simulink Model of using NFF for solving echo cancel-lation


Fig. 7: (a) A and B transmitters signals (b) Transmitted signalwith echo (c) Filter output signal (d) Error signal

V. CONCLUSION

In this paper a neurofuzzy filter is presented. The NFFhas been trained by numerical data and linguistic informationexpressed by fuzzy if-then rules. This feature makes theincorporation of apriori knowledge into the design of filterspossible. Another important feature of the NFF is that, withoutany given initial structure, the NFF has been constructedautomatically from numerical training data. Simulations inhand free mobile and echo cancellation on a transmission linehave been demonstrated and good performance is achieved byapplying the NFF. Hand free mobile achieves good perfor-mance with RMSE of 0.0440 and maximum instant error of0.02. The proposed research has wide application for manyfields, like noisy speech recognitions, noisy image filtering,nonlinear adaptive control, intelligent agents and performanceanalysis of dynamical systems. Future implementations invarious engineering problems are under consideration.

REFERENCES

[1] B. Widrow and S. D. Stearns, Adaptive Signal Processing. EnglewoodCliffs, NJ: Prentice-Hall, 1985.

[2] C. F. Juang and C. T. Lin, “Noisy speech processing by recurrentlyadaptive fuzzy filters,” IEEE Transactions on Fuzzy Systems, vol. 9,issue 1, pp.139-152, Feb.2001.

[3] A. I. Hanna, D. P. Mandic, “Nonlinear FIR adaptive filters with agradient adaptive amplitude in the nonliearity,” IEEE Signal ProcesingLetters, vol. 9, issue 8, pp. 253-255, August 2002.

[4] K. P. Seng, Z. Man. H. R. Wu, “Lyapunov-theory-based radial basisfunction networks for adaptive filtering,” IEEE Transactions on circuitsand systems-I: Fundamental Theroy and Applications, vol. 49, issue 8,pp. 1215-1220, August 2002.

[5] Z. M. Cojbasic, V. D. Nikoli? “An approach to neuro-fuzzy filtering forcommunications and control,” 5th International conference on telecom-munications in modern satellite, cable and broadcasting service, vol. 2,pp. 719-722, Sept. 2001.

[6] Abdullah Toprak, Mehmet Sirac Ozerdem, nan Guler “Suppression ofimpulse noise in MR images using artificial intelligent based neuro-fuzzy adaptive median filter ” Digital Signal Processing, vol 18, issue3, pp 391-405, May 2008.

[7] Hao Qin, Simon X. Yang “Adaptive neuro-fuzzy inference systems basedapproach to nonlinear noise cancellation for images ” Fuzzy Sets andSystems, Volume 158, issue 10, pp. 1036-1063, May 2007.

[8] L. Yin, J. Astola, and Y. Neuvo, “A new class of nonlinear filters—neural filters,” IEEE Trans. Signal Processing, vol. 41, pp.1201–1222,Mar. 1993.

[9] A.N. Birkett, R.A. Goubran, “Fast nonlinear adaptive filtering using apartial window conjugate gradient algorithm” International Conferenceon Acoustics, Speech, and Signal Processing, 1996. ICASSP-96. Con-ference Proceedings. vol 6, pp. 3541 - 3544, May 1996 .

[10] Riyanto T. Bambang “Adjoint EKF learning in recurrent neural networksfor nonlinear active noise control”Applied Soft Computing, vol 8, Issue4, pp 1498-1504,Sept. 2008.

[11] Haiquan Zhao, Jiashu Zhang “ Nonlinear dynamic system identifi-cation using pipelined functional link artificial recurrent neural net-work”Neurocomputing, vol 72, issues 13-15, pp 3046-3054,Aug. 2009.

[12] Pramod Gupta, Naresh K. Sinha “Neural Networks for Identificationof Nonlinear Systems: An Overview” Soft Computing and IntelligentSystems, pp 337-356,2000.

[13] T. H. J. Lo, “Synthetic approach to optimal filtering,” IEEE Trans.Neural Networks, vol. 5, pp. 803–811, Sept. 1994.

[14] G. J. Gibson, S. Siu, and C. F. N. Cowan, “The application of nonlinearstructures to the reconstruction of binary signals,” IEEE Trans. SignalProcessing, vol. 39, pp. 1877–1884, Aug. 1991.

[15] S. Chen, G. J. Gibson, C. F. N. Cowan, and P. M. Grant, “Reconstructionof binary signals using an adaptive radial-basis-function equalizer,”Signal Process., vol. 22, pp. 77–93, 1991.

[16] S. Moon and T. N. Hwang, “Coordinated training of noise removingnetworks,” Proc. Int. Conf. Acoust., Speech, Signal Processing, pp. 573–576, 1993.

[17] H. B. D. Sorensen, “A cepstral noise reduction multilayer neuralnetwork,” in Proc. Int. Conf. Acoust., Speech, Signal Processing, pp.933–936, 1991.

[18] R. C. Lacher, S. I. Hruska, and D. C. Kuncicky, “Back-propagationlearning in expert networks,” IEEE Trans. Neural Networks, vol. 3,pp.62–72, Jan. 1992.

[19] C. T. Lin and C. S. G. Lee, “Neural-network-based fuzzy logic controland and decision system,” IEEE Trans. Comput., vol. 40, issue 12,pp.1320–1336, 1991.

[20] J. S. Jang, “ANFIS: Adaptive-network-based fuzzy inference system,”IEEE Trans. Syst., Man, Cybern., vol. 23, pp. 665–685, May 1993.

[21] L. X. Wang and J. M. Mendel, “Back propagation fuzzy systemsas nonlinear dynamic system identifiers,” in Proc. IEEE Int. Conf.FuzzySystems, San Diego, CA, pp. 1409–1418,1992.

[22] H. Ishibuchi, R. Fujioka, and H. Tanaka, “Neural networks that learnfrom fuzzy if-then rules,” IEEE Trans. Fuzzy Syst., vol. 1, pp. 85–87,May 1993.

[23] A. Q. Ansari, Neeraj Gupta “Noise Cancellation in Hand Free CellphoneUsing Neurofuzzy Filter ” 4th International Conference On ComputerApplications in Electrical Engineering Recent Advances (CERA-09)Indian Institute of Technology (IIT) Roorkee, pp.280-283, Feb. 19-21,2010

[24] A. Q. Ansari, Neeraj Gupta “MIPS-NF Instructions for fourth GenerationProcessor” 2nd IEEE International Conference on Computer, Control &Communication, pp.1-5, Feb. 2009.

[25] J.B. Theocharis “A high-order recurrent neuro-fuzzy system with internaldynamics: Application to the adaptive noise cancellation ” Fuzzy Setsand Systems, vol. 157, issue 4, pp 471-500,16 Feb. 2006.

[26] M. J. Coker and D. J. Simkins, “A nonlinear adaptive noise canceller,”IEEE Int. Conf. Acoust., Speech, Signal Processing, pp. 470–473, 1980.

[27] J. C. Stapleton and G. Ramponi, “Adaptive noise cancellation for a classof nonlinear dynamic reference channels,” in Int. Symp. Circuits Systems,pp. 268–271,1984.

[28] S. A. Billings and F. A. Alturki, “Performance monitoring in nonlinearadaptive noise cancellation” J. Sound Vibration, vol. 157, issue 1,pp.161–175, 1992.

Backpropagation Algorithm for Neurofuzzy Filter

Documents

Transcript of Backpropagation Algorithm for Neurofuzzy Filter