On Constructing Parsimonious Type-2 Fuzzy Logic Systems via Influential Rule Selection

654 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 17, NO. 3, JUNE 2009

On Constructing Parsimonious Type-2 Fuzzy LogicSystems via Influential Rule Selection

Shang-Ming Zhou, Member, IEEE, Jonathan M. Garibaldi, Robert I. John, Member, IEEE, and Francisco Chiclana

Abstract— Type-2 fuzzy systems are increasing in popularity,and there are many examples of successful applications. Whilemany techniques have been proposed for creating parsimonioustype-1 fuzzy systems, there is a lack of such techniques for type-2systems. The essential problem is to reduce the number of rules,while maintaining the system’s approximation performance. Inthis paper, four novel indexes for ranking the relative contributionof type-2 fuzzy rules are proposed, which are termed R-values,c-values, ω1 -values, and ω2 -values. The R-values of type-2 fuzzyrules are obtained by applying a QR decomposition pivoting algo-rithm to the firing strength matrices of the trained fuzzy model.The c-values rank rules based on the effects of rule consequents,while the ω1 -values and ω2 -values consider both the rule-basestructure (via firing strength matrices) and the output contri-bution of fuzzy rule consequents. Two procedures for utilizingthese indexes in fuzzy rule selection (termed “forward selection”and “backward elimination”) are described. Experiments are pre-sented which demonstrate that by using the proposed methodology,the most influential type-2 fuzzy rules can be effectively retainedin order to construct parsimonious type-2 fuzzy models.

Index Terms—Parsimony, QR, ranking index, rule ranking, ruleselection, singular-value decomposition—QR (SVD-QR), type-2fuzzy sets.

I. INTRODUCTION

TYPE-2 fuzzy sets were initially proposed by Zadeh in1975 [1]. Unlike type-1 fuzzy sets, whose membership

values are precise numbers in [0, 1], membership grades of atype-2 fuzzy set are themselves type-1 fuzzy sets, and therefore,type-2 fuzzy sets offer an opportunity to model higher leveluncertainty in the human decision-making process than type-1fuzzy sets [2]–[5]. In a type-2 fuzzy inference system (T2FIS),some fuzzy sets used in the antecedent and/or consequent partsand each rule inference output are type-2 fuzzy sets. T2FISshave been used in many successful applications in various areaswhere uncertainties occur, such as in decision making [6]–[8],diagnostic medicine [9], [10], signal processing [11], [12], trafficforecasting [13], mobile robot control [14], pattern recognition

Manuscript received October 10, 2007; revised February 15, 2008; acceptedApril 14, 2008. First published July 16, 2008; current version publishedJune 11, 2009. This work was supported by the Engineering and PhysicalSciences Research Council Research under Grant EP/C542207/1 and GrantEP/C542215/1.

S.-M. Zhou, R. I. John, and F. Chiclana are with the Centre for ComputationalIntelligence, Department of Informatics, De Montfort University, LeicesterLE1 9BH, U.K. (e-mail: [email protected]; [email protected]; [email protected]).

J. M. Garibaldi is with the Automated Scheduling, Optimization, andPlanning Group, School of Computer Science and Information Technology,University of Nottingham, Nottingham NG8 1BB, U.K. (e-mail: [email protected]).

Color versions of one or more of the figures in this paper available online athttp://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TFUZZ.2008.928597

[15]–[17], intelligent control [18], [19], and ambient intelligentenvironments [20].

However, one challenge in type-1 fuzzy systems also remainsin T2FISs, i.e., the curse of dimensionality: The number of fuzzyrules required increases exponentially with the dimensionalityof the input space. An additional challenge in T2FIS modeling isthat it involves higher computational overhead than type-1 FISmodeling [2]. Importantly, in data-driven type-2 fuzzy model-ing, the type-2 fuzzy rule base generated by the backpropagationmethod [2], [21] will almost certainly suffer from rule redun-dancy [26]. This is because, as an accuracy-oriented method, thebackpropagation training process is concerned only with pro-ducing an accurate system model, without any consideration ofparsimony, consistency, and transparency of generated rule base.Hence, accurate and parsimonious modeling techniques are ur-gently needed for T2FIS modeling. As a matter of fact, even intype-1 FIS modeling, the problem of developing parsimoniousfuzzy modeling technique with as few fuzzy rules as possi-ble is a very important research topic [22]–[25]. Interestingly,Liang and Mendel have suggested using the singular-valuedecomposition—QR (SVD-QR) method [27], [28] to performrule reduction for the sake of designing parsimonious intervalT2FIS (IT2FIS) [26] in which QR decomposition with columnpivoting is further applied to the unitary matrix V after the SVDof the firing strength matrix. However, one issue arising in ap-plying their method is the estimation of an effective rank. Someresearch on type-1 fuzzy models indicates that rule reduction us-ing the SVD-QR with column pivoting algorithm is very sensitiveto the chosen rank in that different estimates of the rank oftenproduce dramatically different rule-reduction results [29], [30].

In order to avoid the estimation of the rank for the SVD-QRwith column pivoting algorithm, this paper applies the pivotedQR decomposition algorithm to type-2 fuzzy rule reduction.The absolute values of diagonal elements of matrix R in QRdecomposition (termed the R-values) of fuzzy rules tend totrack the singular values and can be used for rule ranking.However, both the pivoted QR decomposition algorithm andthe SVD-QR with column pivoting algorithm only considerthe rule-base structure, without paying attention to the effectof rule consequents during rule selection. In other words, bothalgorithms only employ the information from the premises ofthe fuzzy rules when carrying out rule reduction but ignore theinformation from the rule consequents. In type-1 fuzzy systemmodeling, one way of considering the effects of fuzzy ruleconsequents on rule selection is via the orthogonal-least squares(OLS) technique [31]. Wang and Mendel first applied the OLSmethod to fuzzy rule selection [32] in which it was used to selectthe most important fuzzy basis functions, each of which was

1063-6706/$25.00 © 2009 IEEE

Authorized licensed use limited to: UNIVERSIDAD DE GRANADA. Downloaded on January 21, 2010 at 07:55 from IEEE Xplore. Restrictions apply.

ZHOU et al.: ON CONSTRUCTING PARSIMONIOUS TYPE-2 FUZZY LOGIC SYSTEMS VIA INFLUENTIAL RULE SELECTION 655

associated with a fuzzy rule. Later, Wang and Langari employedthe OLS method to remove less important consequent terms fora first-order Tagaki–Sugeno fuzzy model [33]. Mastorocostaset al. [34] proposed a constrained OLS approach as an improve-ment to the basic OLS approach for producing compact fuzzyrule bases. The OLS-based methods select the most importanttype-1 fuzzy rules based on their contributions of variance to thevariance of the output. Another way of considering the outputcontribution of the fuzzy rules has been proposed by Zhouand Gan in fuzzy modeling based on support vector machines(SVMs) [25]. The Lagrangian multipliers of an SVM (termedthe α-values of fuzzy rules) are used to rank the contributionsof associated rule consequents. In this paper, we propose anovel method to consider the effects of type-2 rule consequentsin order to select the most influential rules, which we termc-values. It can be seen that the existing methods of selectinginfluential fuzzy rules by considering the contributions of ruleconsequents [25], [32]–[34] are type-1 fuzzy system oriented,which cannot be applied in type-2 fuzzy models. Anotheradvantage of the proposed c-values of fuzzy rules is that thesevalues can be derived directly from given type-2 fuzzy rules: noadditional computational techniques such as OLS or SVM areneeded. In type-1 fuzzy system modeling, other techniques ofconstructing parsimonious rules, such as the one in [35] focus onmerging similar fuzzy sets, do not take into account the schemeof selecting the influential rules in terms of rule-ranking results.

Ideally, a rule-ranking method should take into account boththe structure of the rule base and the effects of rule consequentsin order to generate a parsimonious fuzzy model with goodgeneralization performance. However, currently, this kind ofscheme is very rare, even in type-1 fuzzy system modeling.In this paper, we propose ω1-values and ω2-values in order totake into account both the type-2 rule-base structure and thecontributions of type-2 rule consequents.

The organization of this paper is as follows. Section II re-views T2FIS with emphasis on IT2FIS whose secondary mem-bership grades are unity. In Section III, we propose some newrule-ranking indexes for T2FISS. Two procedures for utilizingthese indexes in type-2 fuzzy rule selection are described inSection IV. Section V includes the experimental results of eval-uating the proposed methods in three examples. Finally, discus-sion of the various indexes is presented in Section VI togetherwith conclusions.

II. OVERVIEW OF TYPE-2 FUZZY LOGIC SYSTEMS

A T2FIS is a fuzzy logic system in which at least one ofthe fuzzy sets used in the antecedent and/or consequent partsand each rule inference output is a type-2 fuzzy set. Considera type-2 Mamdani FIS having n inputs x1 ∈ X1 , . . . , xn ∈ Xn

and one output y ∈ Y . The rule base contains K type-2 fuzzyrules expressed in the following form:

Rk : if x1 is F k1 and · · · and xn is F k

n , then y is Gk

(1)where k = 1, . . . ,K, F k

i and Gk are type-2 fuzzy sets.These rules represent fuzzy relations between the multiple-

dimensional input space X∆= X1 × · · · × Xn and the output

space Y . However, the computing load involved in deriving thesystem output from a general T2FIS model is high in prac-tice, and this has become a major factor curtailing applicationsof general T2FIS. Advances have recently been made in com-puting general T2FIS inference by using geometrical approxi-mations [36], [37]. For an IT2FIS in which the fuzzy sets F k

i

and Gk are interval fuzzy sets, the computing of T2FISs canbe greatly simplified. The membership grades of interval fuzzysets can be fully characterized by their lower and upper member-ship grades of the footprint of uncertainty (FOU) separately [2].Without the loss of generality, let µF k

i(x) = [µ

F ki

(x), µF ki(x)]

and µGk (y) = [µGk (y), µGk (y)] for each sample (x, y). The

firing strength of an IT2FIS µF k (x) = ni=1µF k

i(x) is an inter-

val [2], i.e.,

µF k (x) =[fk (x), f

k(x)

](2)

where

fk (x) = µF k

1(x) ∗ · · · ∗ µ

F kn(x) (3)

and

fk(x) = µF k

1(x) ∗ · · · ∗ µF k

n(x) (4)

where (*) is a t-norm operator. In this paper, the singleton fuzzi-fier is used in the type-2 fuzzy inference process. The centroidof the type-2 interval consequent set Gk is an interval calculatedas follows:

CGk

∆= [yk , yk ] =

∑Ni=1 yiθi∑Ni=1 θi

∣∣∣∣ θ1 ∈[µ

Gk (y1), µGk (y1)]

. . . , θN ∈[µ

Gk (yN ), µGk (yN )]

(5)

for the discretised y-domain y1 , . . . , yN . The IT2FIS outputset via type reduction Yc(x) is also an interval set having thefollowing structure:

Yc∆= [yl , yr ] =

∑Ki=1 yif i∑Ki=1 fi

∣∣∣∣ y1 ∈ [y1 , y1 ], . . . , yK

∈ [yK , yK ], f 1 ∈ [f 1 , f1], . . . , fK ∈ [fK , f

K]

. (6)

Then, the defuzzified output of the IT2FIS is

y =yl + yr

2. (7)

However, special attention should be paid to the calculationsof the end points of Yc(x), yl , and yr . From (6), we see that yl

and yr can be expressed separately as follows [2]:

yl =

∑Ki=1 yif i

l∑Ki=1 fi

l

=∑K

i=1yipi

l (8)

yr =∑K

i=1 yif ir∑K

i=1 fir

=∑K

i=1yipi

r (9)

where fil = fi or f

icontributing to yl , fi

r = fi or fi

contribut-

ing to yr , pil = fi

l /∑K

i=1 fil , and pi

r = fir /

∑Ki=1 fi

r . Hence, it



is necessary to determine which of fiKi=1 and fiK

i=1 con-

tribute to yl and which of fiKi=1 and fiK

i=1 contribute to yr .This can be done by the Karnik–Mendel procedure developedin [2].

Given the datasetx(i) , y(i)

S

i=1 for designing an IT2FIS, thebackpropagation algorithm can be used to train an IT2FIS suchthat the following mean square error (MSE) is minimized:

e =12

S∑t=1

(y(x(t)) − y(t)

)2. (10)

This differentiable mse measure is a common error minimizationheuristic used in system modeling. The validity of using the mseas an objective function to minimize error relies on the assump-tion that system outputs are offset by inherent Gaussian noise.For regression tasks in which the objective is to approximatethe function of an arbitrary signal, this presumption often holds.However, this assumption may be invalid for some classifica-tion tasks where other error metrics such as cross-entropy [38]or maximal class margin [39] may be more suited. Our paperfocuses on regression-type problems.

In IT2FIS modeling, currently, the primary membership func-tions are usually defined by correspondingly blurring type-1 fuzzy membership functions, such as triangular functions,trapezoid functions, and Gaussian functions, in different ways.Among these, Gaussian primary membership functions are themost widely used ones [2]. This is because it is necessary tocompute the derivatives in the backpropagation algorithm fortuning IT2FIS parameters. Computing such derivatives is muchmore challenging than in a type-1 fuzzy logic system. Obviously,the continuity and differentiability of Gaussian primary mem-bership functions make the computation involved much moreaccessible to fuzzy logic system designers [21], [40]. However,the proposed methods in the next sections of this paper are ap-plicable to any kind of type-2 membership functions, becausethe proposed rule ranking and rule selection in this paper areperformed with no need to specify the nature of the type-2membership functions.

For the sake of constructing an initial IT2FIS model in thispaper, we use Gaussians with uncertain means as the primarymembership functions of the type-2 fuzzy sets F k

i and Gk [2],i.e.,

µF ki(xi) = exp

(−1

2

(xi − mk

i

σki

)2)

, mki ∈

[mk

i1 ,mki2

](11)

and

µGk (y) = exp

(−1

2

(y − mk

σk

)2)

, mk ∈[mk

1 ,mk2].

(12)The backpropagation method can then be used to tune the an-tecedent and consequent parameters in (11) and (12) to minimizethe MSE (10). Details of this training method can be found in [2]and [21].

One issue arising is that the type-2 fuzzy rule base generatedby the backpropagation method will almost certainly suffer from

rule redundancy; therefore, we need to select the most importantfuzzy rules and remove the redundant ones from the generatedrule base. Liang and Mendel applied the SVD-QR with col-umn pivoting algorithm to an IT2FIS for generating a compacttype-2 rule base by removing the redundant rules [26], but it isnecessary to estimate an efficient rank rn in this algorithm. Inour experiments using the SVD-QR algorithm for rule ranking,it was found that different rn produced different rule-rankingresults. Hence, in order to avoid the estimation of rank rn , wepropose to apply the pivoted QR decomposition method to ruleranking, further suggest the indexes of c-values, ω1-values, andω2-values of fuzzy rules for rule ranking from different aspectsof IT2FIS with the goal of constructing a parsimonious model.

III. NEW RULE-RANKING INDEXES

A. R-Values of Fuzzy Rules

The R-values of fuzzy rules are obtained by applying the piv-oted QR decomposition method to the firing strength matrices.The idea behind this method is to assign a rule significance in-dex to each fuzzy rule and then rank and select the influentialfuzzy rules in terms of this index.

However, one issue arising in designing an IT2FIS is thatthe fuzzy rule sequence in the rule base is changed after cal-culating yr and yl . This is because in the Karnik–Mendel pro-cedure [2], it is necessary to arrange the

yi

K

i=1 in ascending

order for calculating yr (and similarly, to arrange theyi

K

i=1in ascending order for calculating yl). Hence, in applying thepivoted QR decomposition method, it is necessary to restorethe fuzzy rule sequence when calculating the firing strengthmatrices [2]. Specifically, for yr , let the original rule order beI = [1, 2, . . . ,K]T . After reordering

yi

K

i=1 in ascending or-der, the rule order becomes I ′ = ΦI (where Φ is a permutationmatrix). Next, the rule order in I ′ is renumbered as 1, 2, . . . , Kfor calculating yr by the Karnik–Mendel procedure. Then, thefiring strength matrices in the original order I are calculated asfollows.

First, the number K determined in the Karnik–Mendel pro-cedure is very important, because for i ≤ K, fi

r = fi , and for

i > K, fir = f

i. Thus

yr =

∑Ki=1 yif i +

∑Ki=K +1 yif

i∑Ki=1 fi +

∑Ki=K +1 f

i. (13)

Therefore

pir =

f i(∑K

i = 1f i +

∑K

i = K + 1f

i) , i ≤ K

f i(∑K

i = 1f i +

∑K

i = K + 1f

i) , i > K

(14)

Then, a firing strength vector given an input x is obtained byrestoring the original rule order

p(x) = Φ−1[p1

r , . . . , pKr , pK +1

r , . . . , pKr

]T

(15)



Therefore, the S training samplesx(i) , y(i)

S

i=1 lead to S firingstrength vectors composing a firing strength matrix Pr of sizeK × S

Pr =[p(x(1)), . . . , p(

x(S ))]T

. (16)

Finally, the QR with column pivoting algorithm addressed in thefollowing steps is applied to Pr , in which each rule is assignedan R-value as its significance index value.Step 1: Calculate the QR decomposition of Pr , and get the

permutation matrix Π via PrΠ = QR, where Q is aunitary matrix, and R is an upper triangular matrix.The absolute values of the diagonal elements of R,denoted by |Rii |, decrease as i increases and are termedR-values.

Step 2: Rank the fuzzy rules and find their positions in therule base in terms of the R-values and the permutationmatrix Π. Each column of Π has one element taking thevalue one and all the other elements taking the valuezero. Each column of Π corresponds to a fuzzy rule.The numbering of the jth most important rule in theoriginal rule base is indicated by the numbering of therow where the element of the jth column containingone is located. For example, if the value one in the firstcolumn is in the fourth row, then the fourth rule is themost important one and its importance is measured as|R11 |. The rule corresponding to the first column is themost important one, and in descending order, the rulecorresponding to the last column is the least importantone.

Because the R-values of Pr tend to approach to singular val-ues of Pr , they can be used as a rule-ranking index in designinga T2FIS with a compact rule base. In practice, we usually usethe normalized R-values defined as follows to perform the ruleranking

Rin =

|Rii |max

i|Rii |

. (17)

In terms of the normalized R-values obtained by applying QRwith column pivoting to Pr , the available fuzzy rules in rulebase are ranked. Let Ωr denote the set with these ranked rules.

Similarly, the fuzzy rules are ranked in terms of the normal-ized R-values obtained by applying QR pivoted decompositionto the firing strength matrix Pl for yl . Let Ωl denote the set withthese ranked rules. Hence, two sets of ranked fuzzy rules, i.e.,Ωr and Ωl , are obtained.

B. C-Values of Fuzzy Rules

Both the SVD-QR with column pivoting method and pivotedQR method only take into account the rule-base structure, focus-ing on the rule antecedent parts when applied to rule reductionof IT2FIS. In the following, we further propose an approach toranking type-2 fuzzy rules based on the effects of rule conse-quents Gi .

As a matter of fact, it can be seen from the procedure fordesigning IT2FIS described in the earlier section that for each

type-2 fuzzy rule, the magnitude of the left and right end pointsof the centroid of the consequent set Gi ,

∣∣yi∣∣, and

∣∣yi∣∣ separately

determines the strength of the effects of the rule consequent onthe output end points yl and yr . Hence,

∣∣yi∣∣ and

∣∣yi∣∣ are very

useful indexes for measuring the output contributions of type-2fuzzy rules. These

∣∣yi∣∣ and

∣∣yi∣∣ are called the c-values of type-2

fuzzy rules in this paper.In practice, these c-values of fuzzy rules C(i) =

∣∣yi∣∣ or

∣∣yi∣∣

are usually normalized as well for rule ranking

Cin =

C(i)

maxi

C(i) . (18)

In the rule-ranking process, first, the normalized c-values∣∣yi∣∣ are used as rule-ranking indexes for calculating yr . Let Ωr

denote the set with ranked rules in terms of∣∣yi

∣∣. Second, thefuzzy rules are ranked in terms of the normalized c-values

∣∣yi∣∣

for calculating yl , and let Ωl denote the set with ranked rules interms of

∣∣yi∣∣.

C. ω1-Values and ω2-Values of Fuzzy Rules

Although type-2 fuzzy rule ranking by c-values takes into ac-count the output contribution of the rule consequents, it ignoresthe rule-base structure. In order to consider both the rule-basestructure and the consequent contribution of fuzzy rules for ruleranking, another two new types of rule-ranking indexes, termedthe ω1-values and ω2-values, are separately suggested as fol-lows. First

ωi1 = Ci

nRin (19)

where C(i) =∣∣yi

∣∣ (or∣∣yi

∣∣) for yl (or yr ), and |Rii | are theR-values of Pl (or Pr ). Second

ωi2 = min

(Ci

n ,Rin

)(20)

where C(i) =∣∣yi

∣∣ (or∣∣yi

∣∣) for yl (or yr ), and |Rii | are theR-values of Pl (or Pr ).

The choice behind the definition of the ω1 in (19) lies in thata fuzzy rule-ranking result considering both rule-base structureand contributions of rule consequents is expected to be a mono-tonically increasing function of ranking results from individualaspects (i.e., from rule-base structure or contributions of ruleconsequents), a higher (lower) ranking result from one aspectleads to a higher (lower) ω1-value. However, a higher rankingresult in terms of ω2-value defined in (20) is obtained only ifboth its two operands are higher.

To obtain the rule ranking, first, the ω1- (or ω2-) values arecalculated by choosing C(i) as

∣∣yi∣∣ for yr and |Rii | of Pr in

(19) [or (20)]. Let Ωr denotes the set of rules ranked in termsof these calculated ω1- (or ω2-) values. Second, the ω1- (or ω2-)values are calculated by choosing C(i) as

∣∣yi∣∣ for yl and |Rii |

of Pl in (19) [or (20)]. Let Ωl denotes the set of rules ranked interms of these calculated ω1- (or ω2-) values.



IV. TYPE-2 FUZZY RULE SELECTION AND IMPLEMENTATION

A. Type-2 Fuzzy Rule Selection Procedures

Let DT be the test dataset and DV be the validation dataset.Given a type-2 fuzzy model T2FIS∗ constructed from data bythe backpropagation algorithm, the generalization performanceof the T2FIS∗ is measured in terms of the Err

(∗)t obtained by

applying the model to the testing samples in the dataset DT ,while the Err

(∗)v denotes the validation performance obtained

by applying to the validation samples in the dataset DV . TheR-values, c-values, ω1-values, and ω2-values can be used toidentify the most influential type-2 fuzzy rules in the T2FIS∗.Assume that Ωl and Ωr are the rule-ranking results obtainedfor calculating yl and yr separately in terms of the chosen rule-ranking index

Ωl =

Ωl(1),Ωl(2), . . . ,Ωl(K)

Ωr =

Ωr (1),Ωr (2), . . . ,Ωr (K)

where K denotes the number of fuzzy rules in rule base ofthe initial fuzzy model T2FIS∗. The rule importance denotedby Ωl(i) or Ωr (i) decreases as i = 1, 2, . . . ,K; correspond-ingly, the rule redundancy denoted by Ωl(i) or Ωr (i) decreasesas i = K,K − 1, . . . , 1. In this paper, two type-2 fuzzy rule-selection procedures, which are termed forward selection (FS)and backward elimination (BE), are described, based on theproposed rule-ranking indexes. Given an error tolerance thresh-old eh , the two procedures are used to determine the smallestpossible model that explains the available data well from therule base of the model T2FIS∗. The threshold eh lies betweenErr

(∗)v and Errv , i.e., eh ∈ [Err

(∗)v , Errv ], where Errv de-

notes the maximal validation root mean square error (RMSE) oftype-2 fuzzy models with one rule in rule bases. The choice ofeh > Err

(∗)v implies that one is ready to sacrifice some system

approximation ability for the sake of obtaining a more compactrule base for a fuzzy model. The FS procedure acts to directlyselect the influential fuzzy rules from Ωl and Ωr , while the BEprocedure acts to remove the redundant rules from Ωl and Ωr

(correspondingly retaining the important fuzzy rules). Let Σi bethe rule subset selected recursively.

1) FS Procedure: The FS procedure is a heuristics for ruleselection that starts with an empty set of type-2 fuzzy rules (i.e.,Σ0 = ∅). One at a time, the most important type-2 fuzzy rulesfrom Ωl and Ωr are added (separately) to Σi (the set of selectedrules), the validation RMSE Err

(i)v of the fuzzy model con-

structed by Σi tends to be smaller. This rule-selection processcontinues until the chosen criterion (the validation RMSE of thefuzzy model) is below a model error tolerance threshold eh . Asindicated in Fig. 1, the FS procedure consists of the followingsteps.Step 1: Set Σ0 = ∅, i = 1, and assign a model error tolerance

threshold eh .Step 2: Select the most important type-2 fuzzy rules from Ωl

and Ωr as follows:

Σi = Σi−1 ∪ Ωl(i) ∪ Ωr (i)

Fig. 1. FS procedure for selecting important fuzzy rules.

where Ωl(i) and Ωr (i) are the ith most important rulesin Ωl and Ωr , respectively.

Step 3: Construct a type-2 fuzzy model T2FISi by using therules in Σi .

Step 4: Apply T2FISi to the validation dataset DV and thetest dataset DT to obtain new RMSEs: Err

(i)v and

Err(i)t .

Step 5: If Err(i)v ≤ eh , stop the selection process, and use

T2FISi as the final compact fuzzy model with se-lected rule set Σi , and with Err

(i)t as the measure of

generalization performance for T2FISi ; otherwise,increase i by 1, and go to step 2.

It should be noted that, because eh ≥ Err(∗)v , the termination

of the FS procedure is guaranteed, in the sense that the FSprocedure always finds a nonempty set of influential rules Σi .At least, the initial model T2FIS∗ satisfies the terminationcondition.

2) BE Procedure: The BE procedure is a heuristic for re-moving redundant rules while retaining influential rules. Itworks by starting from the full set of ranked rules, i.e., Σ0 = Ωl

(or Ωr ), for the initial fuzzy model T2FIS∗. One at a time, themost redundant type-2 fuzzy rules from Ωl and Ωr are deleted(separately) from Σi (the set of selected rules), the validationRMSE Err

(i)v of the fuzzy model constructed by the selected

rules in Σi tends to be greater. This rule-reduction process con-tinues until the chosen criterion (the validation RMSE of thefuzzy model) is above a model error tolerance threshold eh . Asindicated in Fig. 2, the BE procedure consists of the followingsteps.Step 1: Set Σ0 = Ωl (or Ωr ), i = 1, and assign a model error

tolerance threshold eh .



Fig. 2. BE procedure for removing redundant fuzzy rules.

Step 2: Remove the most redundant type-2 fuzzy rules by per-forming a difference of rule subsets as follows:

Σi = Σi−1 −

Ωl(K + 1 − i)∪Ωr (K + 1 − i)

.

Step 3: Construct a type-2 fuzzy model T2FISi by using therules in Σi .

Step 4: Apply T2FISi to the validation dataset DV and thetest dataset DT to obtain new RMSEs: Err

(i)v and

Err(i)t .

Step 5: If Err(i)v > eh or i = K (the full number of rules),

stop the reduction process and use T2FISi−1 as thefinal compact fuzzy model with selected rule set Σi ,and with Err

(i−1)t as the measure of generalization

performance for T2FISi−1 ; otherwise, increase i by 1,and go to step 2.

Because Err(∗)v ≤ eh ≤ Errv , the termination of the BE cri-

teria is enforced when the rule set has been reduced to justΩl(1) ∪ Ωr (1). In practice, if this occurred, then the BE pro-cedure could be repeated with a lower eh , while still ensuringeh ≥ Err

(∗)v .

As a further comment, it should be noted that an idiosyncrasyof type-2 rule ranking is that the two ranked sets, Ωl and Ωr ,which is obtained by calculating yl and yr separately, may bedifferent. Consequently, the final fuzzy rule-selection resultsobtained through the FS and BE procedures described earliermay be different.

B. Implementation of the Proposed Procedures

Given a training datasetx(i) , y(i)

S

i=1 , the process of apply-ing these proposed procedures is as follows.

Step 1: Set the initial mean and width parameters for the Gaus-sian primary functions with uncertain means in an ini-tial input space partition.

Step 2: Train the IT2FIS model by applying the backpropaga-tion algorithm.

Step 3: Rank the fuzzy rules of the trained IT2FIS model interms of the selected rule-ranking index such as one ofthe proposed indexes introduced in Section III.

Step 4: Conduct one or both of the FS and BE proceduresdescribed in Section IV-A to obtain the reduced type-2fuzzy model(s).

It should be noted that not only the proposed indexes inSection III, but in addition, the rule-ranking index obtained bythe SVD-QR method [26] can be used in the earlier procedures.Indeed, the rule-selection process used in the SVD-QR methodis the same as the FS procedure using the rule-ranking resultsobtained by SVD-QR decomposition.

C. Applications to Type-1 Mamdani Fuzzy Systems

It is known that an interval type-2 Mamdani fuzzy logic sys-tem is a generalization of a type-1 Mamdani system (T1MFIS).If the lower and upper membership grades of the FOU in anIT2FIS are equal, then the IT2FIS reduces to the T1MFIS.Hence, the proposed methods can be applied to T1MFIS aswell by treating the T1MFIS as a special case of the IT2FIS.

Specifically speaking, for a T1MFIS, the K × S firingstrength matrices Pl and Pr are the same (i.e., Pl = Pr ), so onlyone group of R-values used as rule-ranking index is obtained forconstructing the T1MFIS with a compact rule base. Similarly,the left end points of the centroid of the consequent set Gi , yi willbe equal to the right end points yi (i.e., yi = yi). Consequently,they both reduce to the traditional centroid of the consequentset obtained by defuzzification of the T1MFIS. Hence, the tra-ditional centroids of the consequent sets in a T1MFIS are thec-values of type-1 fuzzy rules to measure the strength of the ef-fects of the rule consequents on the output. Accordingly, thereis only one group of ω1-values (ω2-values) for considering theoutput contribution of rule consequents as well as the rule basestructure. As the sets of ranked rules Ωl and Ωr are the same,the final rule-selection results obtained through the FS and BEprocedures are the same. That is to say, either the FS or BE pro-cedures can be used in identifying the influential type-1 fuzzyrules for the T1MFIS with the same results.

V. EXPERIMENTS

In this section, we use three examples to evaluate the proposedrule-reduction methods for constructing parsimonious IT2FISs,and compare with the established SVD-QR with column pivotingmethod for rule ranking. Furthermore, type-1 fuzzy logic mod-els are compared with the constructed IT2FIS models in orderto examine the benefits of the type-2 approach. The first exam-ple recovers an original signal from data highly contaminatedby noise. Although the modeled system in this example seemssimple, it is known in the signal-processing community that itis rather challenging to recover the original signal from datahighly contaminated by noise, without prior knowledge. The



second example is a real-world problem in which we wish topredict automobile fuel consumption in miles per gallon (MPG)based on several attributes of an automobile’s profile. Automo-bile MPG prediction is a typical nonlinear regression problem.The third example considers a liquid-saturated steam heat ex-changer [45]. The main motivation for the choice of the heatexchange process is that this plant is a significant benchmarkfor nonlinear control design purposes, because it is characterizedby a nonminimum phase behavior that makes the design of suit-able controllers particularly challenging even in a linear designcontext [45]. Hence, it is highly suitable as a context in whichan IT2FIS approach is used to predict the system behaviors.

A. Signal Recovery Problem

In the experiments for this example, the noisy signal is gen-erated by

y(t) = v(t) + θ(t) (21)

where v is the original signal and θ is an interference signal.The original signal is generated by

v(t) = sin(

40(x(t) + 0.03)

)+

x(t − 1)10

. (22)

The interference signal is generated from another Gaussian noisesource n, with a mean of zero and a standard deviation of one,via an unknown nonlinear process

θ(t) =4 sin (n(t)) n(t − 1)

(1 + n(t − 1)2). (23)

The measured signal y is the sum of the original informationsignal v and the interference θ. However, we do not know theinterference signal θ. The only signal available to us is themeasured signal y. The task is to learn the characteristics of theoriginal information signal v from the measured signal y andthen recover the original signal. A T2FIS may be suitable forsuch a signal-processing problem due to its strong capability ofcharacterizing higher uncertainty exhibited within such noisydata [12].

1) Initial Type-2 Fuzzy Model: In the following, we con-structed an initial IT2FIS model with two inputs x(t) andx(t − 1) selected in terms of regularity criteria [43] and one out-put y(t). The antecedent and consequent parameters in (11) and(12) were optimized by the backpropagation algorithm [2], [21].In order to train the interval type-2 fuzzy model to represent thenonlinearity and higher uncertainty of the system, the data-generation process (21)–(23) was run ten times. In each of theruns, 100 samples x(h) , y

(h)k S

h=1 (S = 100, k = 1, . . . , 10)were generated with x(h) ∈ [2, 5] and y

(h)k obtained by (21).

Then, the dataset x(h) ,mink

y(h)k 100

h=1 was used to generate

the antecedent means mi1 in (11) by the fuzzy c-means(FCM) unsupervised clustering algorithm [41], while the datasetx(h) ,max

ky

(h)k 100

h=1 was used to generate the means mi2.

The consequent means m1 and m2 in (12) were randomlyselected from the output data samples. The width parametersin (11) and (12) were determined using the nearest-neighbor

Fig. 3. Signal recovering by IT2FIS model on training samples: the dotsrepresent the measured signal, solid line represents the original signal v, anddashed line represents the recovered signal.

heuristic suggested by Moody and Daken [42], based on thecorresponding datasets. Hence, all the initial antecedent andconsequent parameters were determined from the given datasets,rather than manually derived.

Four initial type-2 fuzzy sets were generated for each inputvariable, which led to 16 rules in the initial interval type-2fuzzy model. After the training process, this interval type-2fuzzy model had the ability to recover the original signal wellwith RMSE 0.2739, as illustrated in Fig. 3. Further evaluationof the generalization performance of this trained model wasundertaken in order to examine whether, given system inputsthat were different from the training samples, the outputs of thetrained model could follow the characteristics of the originalsignal rather than act as the noisy signal. A test dataset with40 testing inputs that were different from the earlier availabletraining inputs was generated. The generalization performanceof the trained fuzzy model was measured by the RMSE of theoriginal signal values calculated by (22) and the trained modeloutputs given these testing inputs.

In our experiments, this type-2 fuzzy model achieved a gen-eralization performance with the RMSE 0.1178 when beingapplied to the testing samples.

2) Type-2 Fuzzy Rule Ranking: Next, we applied the pro-posed rule-reduction methods to the trained interval type-2 fuzzymodel, and evaluated them in comparison with the SVD-QRrule-reduction method. First, the QR with column pivoting al-gorithm was applied to the firing strength matrices Pl and Pr inwhich the R-values of fuzzy rules were generated for selectinginfluential rules in calculating yl and yr separately. Fig. 4 de-picts the corresponding R-values and singular values of fuzzyrules in descending order obtained from the firing strength ma-trices Pl and Pr individually. These indicate that the R-valuestrack the singular values well, and therefore, the R-values of Pl

and Pr can be used to rank the fuzzy rules. Fig. 5 illustrates theω1-values, normalized R-values and c-values of fuzzy rules withthe rule orders in rule base, while Fig. 6 depicts the ω2-values,



Fig. 4. R-values and singular values of firing strength matrices Pl (bottom)and Pr (top) in descending order in the signal recovering problem.

Fig. 5. ω1 -values, normalized c-values and R-values of type-2 fuzzy ruleswith the rules orders in rule base: ω1 -values with normalized

∣∣yi∣∣ and R-values

of Pr (top) and ω1 -values with normalized∣∣yi

∣∣ and R-values of Pl (bottom) insignal-recovering problem.

normalized R-values, and c-values of fuzzy rules with the ruleorders in rule base. These two figures show that there existcertain correlations among the proposed indexes. However, itcan be seen that each rule has different index values, and thata rule with higher R-value does not necessarily mean that ithas higher c-value, ω1-value, or ω2-value, etc. Hence, these in-dexes evaluate the importance of fuzzy rules in their own ways.Table I illustrates the rule-ranking results obtained in terms ofthe proposed indexes: the R-values, c-values, ω1-values, andω2-values of fuzzy rules obtained when applied to the firingstrength matrices Pl and Pr . As a comparison, the rule-rankingresults obtained by the SVD-QR with column pivoting algo-rithm are summarized in Table II. It can be seen that the ruleranking obtained by SVD-QR with column pivoting algorithmindeed depends on the rank rn .

Fig. 6. ω2 -values, normalized c-values and R-values of type-2 fuzzy ruleswith the rules orders in rule base: ω2 -values with normalized

∣∣yi∣∣ and R-values

of Pr (top) and ω2 -values with normalized∣∣yi

∣∣ and R-values of Pl (bottom)in signal-recovering problem.

TABLE IRULE-RANKING RESULTS BY THE PROPOSED RULE INDEXES IN THE

SIGNAL-RECOVERING PROBLEM

TABLE IIRULE-RANKING RESULTS BY SVD-QR WITH COLUMN PIVOTING IN THE


3) Type-2 Fuzzy Rule Selection by the FS Procedure: In thissignal-recovery problem, because the original signal remainedunknown and the only available data were the measured sig-nal, the noisy data were used as the validation dataset during therule-selection process. The testing dataset was the earlier datasetwith 40 testing samples. The model performance was measuredby the RMSEs of a constructed fuzzy model on the two datasets,Errv and Errt . The model parsimony was evaluated in termsof the number of fuzzy rules in the rule base of the constructedmodel. The RMSE tolerance threshold eh was set to be 0.3.Table III summarizes the rule-selection results obtained by ap-plying the FS procedure using each of the proposed indexes(R-values, c-values, ω1-values, and ω2-values). As a compari-son, Table IV summarizes the rule-selection results obtained by



TABLE IIIRULE-SELECTION RESULTS BY FS PROCEDURE WITH THE PROPOSED RULE-RANKING INDEXES IN THE SIGNAL-RECOVERING PROBLEM

TABLE IVRULE-SELECTION RESULTS VIA FS PROCEDURE WITH THE RULE RANKING

BY SVD-QR WITH COLUMN PIVOTING METHOD IN THE


applying the FS procedure using the SVD-QR with column piv-oting algorithm with rn = 4, 5, and 6. In this signal-recoveryproblem, with the consideration of the consequent effects oftrained fuzzy rules on the overall system output, the normalizedc-values

∣∣yi∣∣ and

∣∣yi∣∣ of fuzzy rules depicted in Fig. 5 or Fig. 6

led to a parsimonious interval type-2 fuzzy model featuring 11rules selected from 16 rules. For the normalized R-values shownin Fig. 5, ω1-values shown in Fig. 5, and ω2-values shown inFig. 6, 12 important rules were identified for the final model.As a comparison, 15 fuzzy rules were chosen by the SVD-QRpivoting algorithm with rn = 4 and 5, while for rn = 6, theSVD-QR algorithm did not indicate any rules to be removed.

4) Type-2 Fuzzy Rule Selection by the BE Procedure: Theproposed rule-ranking indexes were then used in the BE pro-cedure to select the most important fuzzy rules, and the resultswere compared to those obtained by SVD-QR pivoting algo-rithm. Table V summarizes the rule-selection results obtainedby applying the BE procedure using each of the proposed in-dexes (R-values, c-values, ω1-values, and ω2-values). As a com-parison, Table VI summaries the rule-selection results obtainedby applying the BE procedure using the SVD-QR with columnpivoting algorithm with rn = 4, 5, and 6. It can be seen that forthe R-values and ω2-values, 13 important rules were identified,while the BE procedure with c-values and ω1-values identified14 rules. As a comparison, in this procedure, 15 fuzzy ruleswere selected by the SVD-QR pivoting algorithm with rn = 4and 5. Again, the SVD-QR algorithm with rn = 6 indicated norules to remove.

5) Comparison With a Type-1 Mamdani Approach: As a fur-ther comparison, we constructed a type-1 Mamdani fuzzy sys-tem model for this signal-recovery problem in which the initialantecedent means of Gaussian membership functions were theaverages of the mi1 and mi2 used to construct aforemen-tioned initial IT2FIS model, the consequent means were ran-domly selected from the output samples, and the width param-eters were determined using the nearest-neighbor heuristic [42]

based on the given datasets. The trained type-1 Mamdani fuzzysystem model recovered the original signal with RMSE 0.3790on the training samples and produced an RMSE of 0.2551 foroutputs on the 40 testing inputs with the corresponding origi-nal signal values. By setting the RMSE tolerance threshold eh

for rule selection to be 0.4, the best result of rule selection forconstructing a parsimonious type-1 Mamdani fuzzy model wasobtained in terms of the c-values of fuzzy rules. In this bestmodel, 11 influential type-1 fuzzy rules were selected from the16 rules, producing an RMSE of 0.3951 when recovering thesignal from the training inputs and an RMSE of 0.2474 fortesting samples.

6) Effects of Noise Changes: In this signal-recovery prob-lem, it is natural to want to know what happens if the noise char-acteristics change, after a type-2 fuzzy model has been trainedaccording the available measured data. As stated earlier, givena measured signal y, for example, generated by (21), the taskis to construct a type-2 fuzzy model to learn the characteristicsof the original information signal v from the measured signal y,and then, recover the original signal. After the training process,the trained type-2 fuzzy model has learned the characteristicsof the original signal (22). That is to say, this model possessesthe ability to approximate the original signal, given the inputs x.The model output is the recovered signal. In rule selection, rulereduction or other such procedures, no matter how the noiselevel θ in (23) is changed again, the signal-recovery perfor-mance achieved by the trained model will not be changed. Thisis because the original signal source (22) is not impacted [i.e.,the characteristics of the original signal are not changed, onlythe measured signal (21) is influenced by the changes of thenoise θ]. Hence, there is no need to retrain the model. However,in an application of the trained model, the RMSE between themodel output and the measured signal given an input will varyalong with the change of the noise θ. In our experiments, we didnot use the RMSE between the model output and the originalsignal given an input as the training performance measure forthe trained model, because the training data hail from the mea-sured signal (21). However, if the original signal source (22) ischanged, no matter whether or not the noise level θ in (23) ischanged, the type-2 fuzzy model must be retrained according tothe new measured data. This is because the characteristics of thefundamental signal in the measured data have been changed.

B. MPG Prediction Problem

In this example of automobile MPG prediction, the datacollected from automobiles of various makers and models



TABLE VRULE-REDUCTION RESULTS BY BE PROCEDURE WITH THE PROPOSED RULE-RANKING INDEXES IN THE SIGNAL-RECOVERING PROBLEM

TABLE VIRULE-REDUCTION RESULTS VIA BE PROCEDURE WITH THE RULE RANKING

BY SVD-QR WITH COLUMN-PIVOTING METHOD IN THE


are available in the University of California at Irvine (UCI)Machine Learning Repository. In the available dataset with 392samples, there are six input attributes (number of cylinders,displacement, horsepower, weight, acceleration, and modelyear) and one output attribute (the fuel consumption in MPG).However, only three input variables (weight, acceleration, andmodel year) were considered here, based on the regularitycriterion [43]. The 392 samples were randomly partitioned intoa training set (196 samples), a testing set (120 samples), and avalidation set (76 samples) for building and evaluating an initialIT2FIS model. The validation set was used in the rule-selectionand reduction process, while the performance of the IT2FISmodels constructed was evaluated in terms of the RMSEs onthe testing samples.

1) Initial Type-2 Fuzzy Model: Given the input–output datasamples, the FCM algorithm generated three clusters mx ac-cording to a partition entropy measure [35]. Then, the means ofthree initial antecedent interval type-2 fuzzy sets for each inputvariable were produced in a manner similar to that suggestedby Mendel [2]: [mx − 0.5σx − 5.5σn , mx − 0.5σx + 5.5σn ]for the weight attribute and [mx − 0.5σx − 0.25σn , mx −0.5σx + 0.25σn ] for the acceleration and year attributes, whereσx and σn are the standard deviation of the training samplesand additive noise, respectively. The means of consequent in-terval type-2 fuzzy sets were randomly selected from the outputsamples, while the width parameters in (11) and (12) were cho-sen as 0.5σx . The aforementioned initial input–output partitionled to 27 rules in an initial interval type-2 fuzzy model. Af-ter training by the backpropagation algorithm with a trainingRMSE 2.37, the interval type-2 fuzzy model predicted the fuelconsumption of testing samples reasonably well with a testingof RMSE 2.46.

2) Type-2 Fuzzy Rule Ranking: The proposed rule-reductionmethods were then applied to the trained interval type-2 fuzzymodel. Table VII shows the rule-ranking results obtained bythe proposed indexes. For comparison, the rule-ranking resultsobtained by the SVD-QR algorithm with rn = 4, 5, and 6 areillustrated in Table VIII. Once again, it can be seen that therule-ranking results vary with the choice of rn .

3) Type-2 Fuzzy Rule Selection by the FS Procedure: Themost influential type-2 fuzzy rules were selected based on theabove rule-ranking results via the FS procedure to constructparsimonious type-2 fuzzy models, and the results were com-pared with those obtained by the SVD-QR pivoting algorithm.The RMSE tolerance threshold eh was set to 3.0. Table IXsummarizes the rule-selection results obtained using the FSprocedure with the proposed indexes. Table X summarizes therule-selection results using the FS procedure with the SVD-QRpivoting algorithm for rn = 4, 5, and 6. In this real-world prob-lem, the R-values and ω1-values led to a parsimonious intervaltype-2 fuzzy model constructed with only six rules selectedfrom the original 27 rules. For the c-values and ω2-values, 20rules were retained. As a comparison, 23–25 fuzzy rules wereseparately identified by the SVD-QR algorithm with rn = 4, 5,and 6, respectively.

4) Type-2 Fuzzy Rule Selection by the BE Procedure: Fi-nally, the BE procedure was applied and the results comparedwith those obtained by the SVD-QR algorithm. The RMSE tol-erance threshold eh was again set as 3.0. Table XI summarizesthe results obtained using the BE procedure with the proposedindexes. Table XII summarizes the results obtained using the BEprocedure with the SVD-QR algorithm for rn = 4, 5, and 6. Inthis problem, the ω1-values resulted in 18 important rules beingselected, while the R-values, c-values, and ω2-values produced20, 21, and, 23 rules, respectively. As a comparison, 23–25rules were selected by the SVD-QR pivoting algorithm withrn = 4, 5, and 6, respectively. It can be seen from these resultsthat the proposed rule-ranking indexes outperform the existingSVD-QR with column pivoting algorithm, and are very effec-tive in identifying the significant rules and removing redundantones.

5) Comparison With a Type-1 Mamdani Approach and OtherApproaches: For further comparison, we used the same trainingsamples, testing samples, and validation samples to construct aparsimonious type-1 Mamdani fuzzy system model in this MPGprediction problem. The initial antecedent means of Gaussianmembership functions were the averages of the means used forconstructing initial IT2FIS model mentioned before, the conse-quent means were randomly selected from the output samples,and the width parameters were chosen as 0.5σx . The trainedtype-1 Mamdani fuzzy system model predicted the fuel con-sumptions with an RMSE of 2.4762 on training samples andan RMSE of 2.6348 on testing samples. By setting the RMSEtolerance threshold eh for rule selection as 3.0, the best result ofrule selection for constructing a parsimonious type-1 Mamdanifuzzy model was obtained in terms of the c-values of fuzzy rulesin which 22 influential type-1 fuzzy rules were selected from



TABLE VIIRULE-RANKING RESULTS BY THE PROPOSED RULE INDEXES IN THE MPG-PREDICTION PROBLEM

TABLE VIIIRULE-RANKING RESULTS BY SVD-QR WITH COLUMN PIVOTING IN THE MPG-PREDICTION PROBLEM

TABLE IXRULE-SELECTION RESULTS BY FS PROCEDURE WITH THE PROPOSED RULE-RANKING INDEXES IN THE MPG-PREDICTION PROBLEM

TABLE XRULE-REDUCTION RESULTS VIA FS PROCEDURE WITH THE RULE RANKING BY SVD-QR WITH COLUMN PIVOTING METHOD IN THE MPG-PREDICTION PROBLEM

the 27 rules. This type-1 Mamdani model with the 22 selectedrules predicted the testing samples with an RMSE of 3.5954.Many other researchers have also built system modeling algo-rithm to perform MPG prediction for this problem. Competitiveresults include those of Kilic et al. [44] in which they proposeda modeling method with a generalization performance (RMSE)of 2.61. It can be seen that the results obtained in our par-simonious type-2 approach are superior to a type-1 approachand comparable, if not a little better than other approaches(although we stress that this cannot be stated as a definitiveconclusion).

C. Steam Heat Exchanger

In the last example, water is heated by pressurized saturatedsteam through a copper tube. Saturated steam is used to provideprimary heat to a process fluid in a heat exchanger. The process

plant is illustrated in Fig. 7. The plant output is the outlet liquidtemperature, and the inputs are the liquid flow rate, the steamtemperature, and the inlet liquid temperature. In this experiment,the steam temperature and the inlet liquid temperature are keptconstant to their nominal values; therefore, we only consideredthe liquid flow rate as the plant input variable.

In our experiment, 500 heat exchanging samples were usedto construct an IT2FIS model with four inputs selected in termsof the regularity criterion [43], i.e., vt = f(vt−1 , vt−2 , vt−4 , ut),where vt is the outlet liquid temperature, and ut the liquid flowrate, at time t. These 500 samples were randomly partitioned into300 training samples, 100 testing samples, and 100 validationsamples. The training samples were used to build an initialIT2FIS model, the model performance was evaluated in terms ofthe RMSEs on the testing samples, while the validation sampleswere used for rule selection and reduction.



TABLE XIRULE-SELECTION RESULTS BY BE PROCEDURE WITH THE PROPOSED RULE-RANKING INDEXES IN THE MPG-PREDICTION PROBLEM

TABLE XIIRULE-REDUCTION RESULTS VIA BE PROCEDURE WITH THE RULE RANKING BY SVD-QR COLUMN PIVOTING METHOD IN THE MPG-PREDICTION PROBLEM

Fig. 7. Structure of the steam heat exchanger.

1) Initial Type-2 Fuzzy Model: Given the training samples,the FCM algorithm generated two clusters mx based on thegiven input–output samples.

The means of three initial antecedent interval type-2 fuzzysets for each input variable were again produced in a man-ner similar to that suggested by Mendel [2], as [mx − 0.3σx −0.1σn , mx − 0.3σx + 0.1σn ] for the liquid flow rate attributeand [mx − 0.3σx − 0.25σn , mx − 0.3σx + 0.25σn ] for otherattributes, where σx and σn are the standard deviations of thetraining samples and additive noise, respectively. The means ofconsequent interval type-2 fuzzy sets were randomly selectedfrom the output samples, while the width parameters in (11)and (12) were chosen as 0.3σx . The earlier initial input–outputpartition resulted in 16 rules in the initial interval type-2 fuzzymodel. After training by the backpropagation algorithm with atraining rmse of 0.2642, the initial type-2 fuzzy model predictedthe outlet liquid temperature well for the testing samples withan RMSE of 0.2436.

2) Type-2 Fuzzy Rule Ranking and Rule Selection: The pro-posed rule-ranking indexes were then calculated for the trainedinterval type-2 fuzzy model. The most influential type-2 fuzzyrules were selected based on the rule-ranking results. The bestresult achieved for constructing a parsimonious type-2 fuzzyrule base was obtained by using the BE procedure with c-value

index and eh = 1.0. In this case, 11 rules were selected with avalidation RMSE of 0.6276 and a testing RMSE of 0.5698.

3) Comparison With Type-1 Mamdani Approach: For fur-ther comparison, we used the same training, testing, and valida-tion samples to construct a parsimonious type-1 Mamdani fuzzysystem model for this problem. The initial antecedent means ofGaussian membership functions were the averages of the meansused for constructing the initial IT2FIS model described earlier,the consequent means were randomly selected from the outputsamples, and the width parameters were chosen as 0.3σx . Thetrained type-1 Mamdani fuzzy system model predicted the outletliquid temperature with an RMSE of 0.2784 on training samplesand an rmse of 0.2847 on testing samples. By setting the rmsetolerance threshold eh for rule selection as 1.0, the best result ofrule selection for constructing a parsimonious type-1 Mamdanifuzzy model was obtained with c-value index. In this case, 12type-1 fuzzy rules were selected with an RMSE of 0.8052 on thetesting samples. Again, it can be seen that the type-1 approachwas inferior to the type-2 approach.

VI. DISCUSSION AND CONCLUSION

We have proposed some novel indexes of type-2 fuzzy rulesthat focus on different aspects of type-2 fuzzy logic systems inorder to determine the relative importance of the various rules.These indexes are termed R-values, c-values, ω1 -values, and ω2 -values. The R-values of type-2 fuzzy rules obtained by QR de-composition pay attention to the rule-base structure, the c-valuesfocus on contributions of rule consequents, while the ω1-valuesand ω2-values take into account both the rule-base structure andcontributions of rule consequents. Moreover, two procedures,i.e., the FS procedure and BE procedure, have been described toutilize these indexes in determining a parsimonious rule base.The experimental results have demonstrated that parsimonioustype-2 fuzzy system models can be effectively constructed interms of the fuzzy rules selected by these proposed indexesand that the proposed methods outperform the existing SVD-QR with column pivoting method for rule reduction. Indeed, in



the second example (MPG prediction), the proposed methodsselected only six rules to be used, while maintaining a reason-able rmse of 2.699. In contrast, the best SVD-QR performanceresulted in retaining 23 rules with an RMSE of 2.571.

One possible issue arising in the proposed methodology thathas not been addressed in this paper is which fuzzy rule indexis best among the proposed four indexes. In practice, one cansimply pick any one of these rule-ranking indexes to constructa parsimonious type-2 fuzzy model; alternatively, one can runboth the FS and BE procedures on each of these indexes, inturn, and then pick the best. This issue certainly merits furtherresearch.

ACKNOWLEDGMENT

The authors would like to thank the anonymous reviewers fortheir excellent suggestions that have been incorporated into thispaper.

REFERENCES

[1] L. A. Zadeh, “The concept of a linguistic variable and its application toapproximate reasoning-1,” Inf. Sci., vol. 8, pp. 199–249, 1975.

[2] J. M. Mendel, Uncertainty Rule-based Fuzzy Logic Systems: Intro-duction and New Directions. Englewood Cliffs, NJ: Prentice–Hall,2001.

[3] J. M. Mendel and R. I. John, “Type-2 fuzzy sets made simple,” IEEETrans. Fuzzy Syst., vol. 10, no. 2, pp. 117–127, Apr. 2002.

[4] J. M. Mendel, R. I. John, and F. Liu, “Interval type-2 fuzzy logic systemsmade simple,” IEEE Trans. Fuzzy Syst., vol. 14, no. 6, pp. 808–821, Dec.2006.

[5] R. John and S. Coupland, “Type-2 fuzzy logic: A historical view,” IEEEComput. Intell. Mag., vol. 2, no. 1, pp. 57–62, Feb. 2007.

[6] J. L. Chaneau, M. Gunaratne, and A. G. Altschaeffl, “An applicationof type-2 sets to decision making in engineering,” in Analysis ofFuzzy Information-Vol. II: Artificial Intelligence and Decision Systems,J. C. Bezdek, Ed. Boca Raton, FL: CRC, 1987.

[7] R. R. Yager, “Fuzzy subsets of type II in decisions,” J. Cybern., vol. 10,pp. 137–159, 1980.

[8] J. M. Garibaldi and T. Ozen, “Uncertain fuzzy reasoning: A case studyin modelling expert decision making,” IEEE Trans. Fuzzy Syst., vol. 15,no. 1, pp. 16–30, Feb. 2007.

[9] R. I. John and P. R. Innocent, “Modeling uncertainty in clinical diagnosisusing fuzzy logic,” IEEE Trans. Syst., Man Cybern. B, Cybern., vol. 35,no. 6, pp. 1340–1350, Dec. 2005.

[10] L. Di Lascio, A. Gisolfi, and A. Nappi, “Medical differential diagnosisthrough type-2 fuzzy sets,” in Proc. FUZZ-IEEE, Reno, NV, May 2005,pp. 371–376.

[11] N. N. Karnik, J. M. Mendel, and Q. Liang, “Type-2 fuzzy logic systems,”IEEE Trans. Fuzzy Syst., vol. 7, no. 6, pp. 643–658, Dec. 1999.

[12] J. M. Mendel, “Uncertainty, fuzzy logic, and signal processing,” SignalProcess., vol. 80, no. 6, pp. 913–933, 2000.

[13] L. Li, W.-H. Lin, and H. Liu, “Type-2 fuzzy logic approach for short-termtraffic forecasting,” Proc. Inst. Elect. Eng. Intell. Transp. Syst., vol. 153,no. 1, pp. 33–40, 2006.

[14] H. A. Hagras, “A hierarchical type-2 fuzzy logic control architecture forautonomous mobile robots,” IEEE Trans. Fuzzy Syst., vol. 12, no. 4,pp. 524–539, Aug. 2004.

[15] H. B. Mitchell, “Pattern recognition using type-II fuzzy sets,” Inf. Sci.,vol. 170, pp. 409–418, 2005.

[16] J. Zeng and Z. Q. Liu, “Type-2 fuzzy hidden Markov models and theirapplication to speech recognition,” IEEE Trans. Fuzzy Syst., vol. 14, no. 3,pp. 454–467, Jun. 2006.

[17] H. Wu and J. M. Mendel, “Classification of battlefield ground vehiclesusing acoustic features and fuzzy logic rule-based classifiers,” IEEETrans. Fuzzy Syst., vol. 15, no. 1, pp. 56–72, Feb. 2007.

[18] O. Castillo, P. Melin, O. Montiel, A. Rodriguez-Diaz, and R. Sepulveda,“Handling uncertainty in controllers using type-2 fuzzy logic,” J. Intell.Syst., vol. 14, no. 3, pp. 237–262, Aug. 2005.

[19] R. Sepulveda, O. Castillo, P. Melin, A. Rodriguez-Diaz, and O. Montiel,“Experimental study of intelligent controllers under uncertainty using

type-1 and type-2 fuzzy logic,” Inf. Sci., vol. 177, no. 10, pp. 2023–2048,2007.

[20] H. Hagras, F. Doctor, A. Lopez, and V. Callaghan, “An incremental adap-tive life long learning approach for type-2 fuzzy embedded agents in am-bient intelligent environments,” IEEE Trans. Fuzzy Syst., vol. 15, no. 1,pp. 41–55, Feb. 2007.

[21] H. Hagras, “Comments on, ‘Dynamical optimal training for interval type-2fuzzy neural network (T2FNN),’” IEEE Trans. Syst., Man, Cybern. B,Cybern, vol. 36, no. 5, pp. 1206–1209, Oct. 2006.

[22] K. M. Bossley, D. J. Mills, M. Brown, and C. J. Harris, “Construction anddesign of parsimonious neurofuzzy systems,” in Proc. Neural Netw. Eng.Control Syst., K. J. Hunt, G. R. Irwin, and K. Warwick, Eds. New York:Springer-Verlag, 1995, pp. 153–177.

[23] M. J. Er and S. Wu, “A fast learning algorithm for parsimonious fuzzyneural systems,” Fuzzy Sets Syst., vol. 126, no. 3, pp. 337–351, 2002.

[24] H. Roubos and M. Setnes, “Compact fuzzy models through complex-ity reduction and evolutionary optimization,” in Proc. FUZZ-IEEE, SanAntonio, TX, May 2000, pp. 762–767.

[25] S.-M. Zhou and J. Q. Gan, “Constructing parsimonious fuzzy classifiersbased on L2-SVM in high-dimensional space with automatic model se-lection and fuzzy rule ranking,” IEEE Trans. Fuzzy Syst., vol. 15, no. 3,pp. 398–409, Jun. 2007.

[26] Q. Liang and J. M. Mendel, “Designing interval type-2 fuzzy logic systemsusing an SVD-QR method: Rule reduction,” Int. J. Intell. Syst., vol. 15,pp. 939–957, 2000.

[27] G. C. Mouzouris and J. M. Mendel, “A singular-value-QR decompositionbased method for training fuzzy logic systems in uncertain environments,”J. Intell. Fuzzy Syst., vol. 5, pp. 367–374, 1997.

[28] G. C. Mouzouris and J. M. Mendel, “Designing fuzzy logic systems for un-certain environments using a singular-value-QR decomposition method,”in Proc. FUZZ-IEEE, New Orleans, LA, 1996, pp. 295–301.

[29] M. Setnes and R. Babuska, “Rule base reduction: Some comments on theuse of orthogonal transforms,” IEEE Trans. Syst., Man, Cybern. C, Appl.Rev., vol. 31, no. 2, pp. 199–206, May 2001.

[30] M. Setnes and H. Hellendoorn, “Orthogonal transforms for ordering andreduction of fuzzy rules,” in Proc. 9th IEEE Int. Conf. Fuzzy Syst., SanAntonio, TX, 2000, pp. 700–705.

[31] S. Chen, C. F. N. Cowan, and P. M. Grant, “Orthogonal least squares learn-ing algorithm for radial basis function networks,” IEEE Trans. NeuralNetw., vol. 2, no. 2, pp. 302–309, Mar. 1991.

[32] L. X. Wang and J. M. Mendel, “Fuzzy basis functions, universal approxi-mation, and orthogonal least squares learning,” IEEE Trans. Neural Netw.,vol. 3, no. 5, pp. 807–814, Sep. 1992.

[33] L. Wang and R. Langari, “Building Sugeno-type models using fuzzydiscretization and orthogonal parameter estimation techniques,” IEEETrans. Fuzzy Syst., vol. 3, no. 4, pp. 454–458, Nov. 1995.

[34] P. A. Mastorocostas, J. B. Theocharis, and V. S. Petridis, “A constrainedorthogonal least squares method for generating TSK fuzzy models: Ap-plication to short-term load forecasting,” Fuzzy Sets Syst., vol. 118, no. 2,pp. 215–233, 2001.

[35] S.-M. Zhou and J. Q. Gan, “Constructing accurate and parsimonious fuzzymodels with distinguishable fuzzy sets based on an entropy measure,”Fuzzy Sets Syst., vol. 157, no. 8, pp. 1057–1074, 2006.

[36] S. Coupland and R. I. John, “Geometric type-1 and type-2 fuzzy logicsystems,” IEEE Trans. Fuzzy Syst., vol. 15, no. 1, pp. 3–15, Feb. 2007.

[37] S. Coupland and R. John, “A fast geometric method for defuzzification oftype-2 fuzzy sets,” IEEE Trans. Fuzzy Syst., vol. 16, no. 6, pp. 929–941,Nov. 2008.

[38] M. Rimer and T. Martinez, “Classification-based objective functions,”Mach. Learn., vol. 63, no. 2, pp. 183–205, 2006.

[39] V. Vapnik, The Nature of Statistical Learning. New York: Springer-Verlag, 1995.

[40] J. M. Mendel, “Computing derivatives in interval type-2 fuzzy logic sys-tems,” IEEE Trans. Fuzzy Syst., vol. 12, no. 1, pp. 84–98, Feb. 2004.

[41] J. C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algo-rithms. New York: Plenum, 1981.

[42] M. J. Moody and C. J. Darken, “Fast learning in networks of locally-tunedprocessing units,” Neural Comput., vol. 1, no. 2, pp. 281–294, 1989.

[43] M. Sugeno and T. Yasukawa, “A fuzzy-logic-based approach to qualitativemodelling,” IEEE Trans. Fuzzy Syst., vol. 1, no. 1, pp. 7–31, Feb. 1993.

[44] K. Kilic, B. A. Sproulec, I. B. Turksen, and C. A. Naranjo, “A fuzzysystem modeling algorithm for data analysis and approximate reasoning,”Robot. Auton. Syst., vol. 49, no. 3/4, pp. 173–180, 2004.

[45] S. Bittanti and L. Piroddi, “Nonlinear identification and control of a heatexchanger: A neural network approach,” J. Franklin Inst., vol. 334, no. 1,pp. 135–153, 1997.



Shang-Ming Zhou (M’01) received the B.Sc. de-gree in mathematics from Liaocheng University,Liaocheng, China, the M.Sc. degree in applied math-ematics from Beijing Normal University, Beijing,China, and the Ph.D. degree in computer science fromthe University of Essex, Colchester, U.K.

He is currently with the Centre for Computa-tional Intelligence, Department of Informatics, DeMontfort University, Leicester, U.K. His research in-terests include fuzzy logic systems (type-1 and type-2) and applications, decision support systems under

uncertainty environments, model interpretability and transparency in data-drivenneurofuzzy systems, kernel machine learning (in support vector machines) withfuzzy computing, artificial neural networks and pattern recognition, and intelli-gent signal and image processing. He has published extensively on these topics.

Jonathan M. Garibaldi received the B.Sc (Hons.)degree in physics from Bristol University, Bristol,U.K., in 1984 and the M.Sc. degree in intelligent sys-tems and the Ph.D. degree in uncertainty handling inimmediate neonatal assessment from the Universityof Plymouth, Plymouth, U.K., in 1990 and 1997, re-spectively.

He is currently an Associate Professor and aReader with the Intelligent Modelling and Analysis(IMA) Research Group, School of Computer Science,University of Nottingham, Nottingham, U.K., where

he is engaged in intelligent modeling, utilizing data analysis and transformationtechniques to enable deeper and clearer understanding of complex problems.He has authored or coauthored over 40 papers on fuzzy expert systems andfuzzy modeling, including three book chapters, and has edited two books. Hiscurrent research interests include modeling uncertainty in human reasoning, es-pecially modeling the variation in normal human decision making, particularlyin medical domains. He has created and implemented fuzzy expert systems anddeveloped methods for fuzzy model optimization.

Robert I. John (M’01) received the B.Sc. (Hons.)degree in mathematics from Leicester Polytechnic,Leicester, U.K., in 1979, the M.Sc. degree in statisticsfrom the University of Manchester Institute of Sci-ence and Technology (UMIST), Manchester, U.K.,in 1981, and the Ph.D. degree in type-2 fuzzy logicfrom De Montfort University, Leicester, in 2000.

He is currently a Professor and the Director of theCentre for Computational Intelligence, De MontfortUniversity. He is a Member of the Editorial Boards ofthe International Journal of Cognitive Neurodynam-

ics, the International Journal of Computational Intelligence, the InternationalJournal for Computational Intelligence, and Information and Systems Sciences.He has authored or coauthored extensively in the area of type-2 fuzzy logic. Hiscurrent research interests include the general field of modeling human decisionmaking using type-2 fuzzy logic.

Prof. John is the Vice Chairman of the Fuzzy Technical Committee (FTC)of the IEEE Neural Networks Society and a Member of the Engineering andPhysical Sciences Research Council’s (EPSRC) College, U.K. He was the Co-General Chair of the IEEE International Conference on Fuzzy Systems, LondonU.K., July 24–26, 2007.

Francisco Chiclana received the B.Sc. and Ph.D. de-grees in mathematics from the University of Granada,Granada, Spain, in 1989 and 2000, respectively.

In August 2003, he joined the Centre for Computa-tional Intelligence (CCI) Research Group, De Mont-fort University, Leicester, U.K., where he is currentlya Reader in computational intelligence. His currentresearch interests include fuzzy-logic-based systems,fuzzy preference modeling, decision-making prob-lems with heterogeneous fuzzy information, decisionsupport systems, the consensus-reaching process, ag-

gregation and consistency of information, and modeling situations with missinginformation.


On Constructing Parsimonious Type-2 Fuzzy Logic Systems via Influential Rule Selection

Documents

Transcript of On Constructing Parsimonious Type-2 Fuzzy Logic Systems via Influential Rule Selection