A Numerical Approach For Performance Analysis of Higher-Order Statistics Based Identification Methods

K. Abderrahim*, R. Ben Abdennour*, F. Msahli*, M. Ksouri, **, G. Favier***

(*) : National School of Engineers of Gabes, Route de Medenine, 6029, Gabes, Tunisia. Tel 00 216 5 282 100, Fax 00 216 5 280 190, e-mail : Ridha.BenAbdennour@enig.mu.tn (**) : National School of Engineers of Tunis, B.P. 47, Le BelvCdtre, 1002, Tunis, Tunisia.

e-mail : Mekki.Ksouri@isetr.rnu.tn (** *) : 13s Laboratory, UNSA/CNRS, 2000 Route des Lucioles, Batiment Algorithmes/Euclide, Sophia-Antipolis, F-

06410, Biot, France. e-mail : Gerard.FAVIER@i3s.unice.fr

ABSTRACT This paper addresses the problem of performance analysis of linear algebra methods for Higher-Order Statistics based identification. In fact, we propose another approach which can be used to evaluate the performance of these methods. This approach is based on the use of the condition number of system of equations. To illustrate the effectiveness of our approach, several simulation examples are presented.

1. INTRODUCTION Recently, several linear algebra methods for Higher-Order Statistics (HOS) based identification of FIR systems have been proposed in the literature [ l , 2 , 3 4 ,6 ,7 ,8,11,16,18 19, 20, 21, 23, 241. These methods consist to obtain a linear system of equations which are solved using least squares approaches. All these methods are obtained assuming the exact cumulants of the observed process are known and consequently they give exact results. But in practice situations, the cumulants are estimated from the sample average of the given data. Because of this, even without the presence of noise, the exact cumulants are unknown and so the system can not be estimated exactly. In fact, it is important to analyse the performance of these algorithms in order to choose the best algorithms. To achieve this analyse, we can use either numerical study or the Porat & Friedlander approach [17]. The numerical study consists to compare these methods through the results obtained from simulations of some examples. However, The Porat & Friedlander approach consists to determine the expressions covariance of the estimates of the FIR system parameters [17]. Unfortunately, the obtained expressions are very cumbersome because the nonlinearities involved in the expressions of cumulants expressions estimates and it is very difficult to compare the performance of the algorithms analytically [20, 211. Hence, it is necessary that this comparison is done by considering some numerical examples [21].

In this paper, we propose another approach which is based on the use of the condition number of the system of equations [12]. This condition is generally used to quantify the robustness of the solution of linear system of equations to the perturbations [12].

2. MODEL AND ASSUMPTIONS

We consider the following FIR system : 9

x(n) = 2 h(i)e(n - i)

where {e(n)} is the input sequence, {h(i)} is the impulse response, {x(n)} is the non measurable output sequence, {v(n)} is the noise sequence and {y(n)} is the true sequence.

The following assumptions are assumed to be hold :

Al. The input e(n) is zero mean, independent and identically distributed (i.i.d.), non Gaussian and its kth order cumulant is given by:

(3) Yk,e # 0 for at least two different k’s.

A2. The additive noise v(n) is assumed to be an i.i.d Gaussian sequence (white or colored), independent of e(n) and with unknown variance.

A3. The order q of the model is known.

A4. The transfer function is stable, causal and has h(0)=1.

0-7803-5731-0/99$10.00 01999 IEEE V -92

3. PERFORMANCE ANALYSIS

The linear algebra methods for HOS based identification use the same approach in order to determine the FIR system coefficients which can be summarised as follows: - Constructing of an overdetermined system of

equations: AB-b (4)

where 8 is the unknown parameters vector, A is a matrix and b is a vector. The contents of A and b are determined from the cumulants of the system output. - Solving the obtained system (4): The matrix A has full column rank and consequently there is a unique least squares (LS) solution 8, and it solves the symmetric positive definite system:

ATAB, = ATb So to can grant the confidence in the solution BL-, the latter must be robust with regard the errors presented in A and b because the exact cumulants used to construct A and b are unknown and they are replaced by estimates obtained from noisy observations. In fact, it is important

to study the quality of the computed solution 6 ~ . This quality is assessed by the following two criteria [12]: - HOW to close 6, to eLS?

How to minimise iLs = b - Ah, when compared to rLs = b-ABLS?

The relative importance of these two criteria varies from application to another. In any case it is important to understand how 0, and r s are affected by errors in A and b. These quantities may be quite sensitive if the columns of A are nearly dependent [12]. In fact, we must use the notion of condition which give an idea of the dependence of the columns of a given matrix [12].

-

Definition. For rectangular matrix with full column rank, the condition number is defined [12]:

where omax(A) and omin(A) are the largest and the smallest singular values respectively.

Remark. Matrices with small condition numbers are said to be well conditioned. However, if K(A) is large then A is said an ill-conditioned matrix and the columns of A are nearly dependent [12]. Consequently, a given method performs better if its condition number is lower.

4. SIMULATION STUDY

The purpose of these simulations is to illustrate the role of the condition number on the performance analysis of linear algebra methods.

4.1. Considered Methods we have simulated only the methods of Srinivas & Hari (SRH95) [19, 211 and the ours (ABM99) [3] (see appendix 6.1 and 6.2 for details). These methods can be considered among the best algorithms because they use the third and fourth order cumulant and not autocorrelation. Consequently, they give consistent estimates in the presence of colored Gaussian noise. Moreover, they exploit nearly all the available statistics information which is expected to improve the quality of estimation [21].

4.2. Relation between condition number and FIR system coefficients In this part, we show that the condition number of a given method depends on the coefficients of the system to be identified. In fact, we have considered the following system:

x(k)= e(k)+ h(l)e(k -1)-1.87e(k -2)+3.02e(k-3) - 1.435e(k - 4) + 1.49e(k - 5 ) with - 3 s h(1) 5 1

For each h(1)’s value, we have calculated the exact cumulants using the Brillinger & Rosenblatt relation [5 ] :

9 C m , ~ ( z ~ , * . . ~ ~ m - 1 ) = Ym,e zh(i)h(i + q)..h(i + zm-1)

i-0 The input signal e(n) is assumed to be zero mean, exponentially distributed and i.i.d noise sequence with ~ 2 , e = 1 9 ~ 3 , e 2 and Y 4,e = 6 .

The simulation results are depicted in figure 1 which presents the condition number (CN)of the two methods as functions of h(1).

Figure 1. The theoretical Condition number as a function of h(1)

-2,l -1,l -0,l 0,9

W)

v -93

Conclusion From figure 1, we can conclude that coefficients of the system to be identified determine the condition number which is used to quantify the sensitivity of a given method to the perturbations. Consequently, the choose of the best method depends on these coefficients.

4.3. Performance Analysis In this section, we illustrate the previous conclusion using the SRH95 & ABM99 methods. In fact, we have considered from figure 1 two values of h(1) which insure the following situation: - The condition number of SRH95method is lower

than of ABM99 method: h(l)=0.1. - The condition number of SRH95 method is higher

than of ABM99 method: h(l)=-2.205. In fact, we have the following systems:

System 1

x(k)= e(k)-O.l.e(k-1)-1.87e(k -2)+3.02e(k-3)

with zeros -2.0236,0.9331 * 0.71583 and 0.0287 20.7291i - 1.435e(k - 4) + 1.49e(k - 5 )

System 2

x(k)= e(k)-2.205.e(k -1)-1.87e(k -2)+3.02e(k-3) -1.435e(k -4)+1.49e(k -5)

with zeros 2.5244, -1.4294, 1 and. 0.0550 * 0.6402i

The simulations are performed under the following conditions: - The input signal e(n) is zero mean, exponentially

distributed and i.i.d noise sequence with ~ 2 , ~ = 1 ,

- The additive colored noise v(n) is created as the Gaussian

~ 3 , e 2 and Y 4,e = 6

output of MA model driven by an i.i.d sequence{w(k)}:

v(k) = 0.5w(k) - 0.25.w(k - 1) - 0.5w(k - 3) + 0.25w(k - 3) - 0.25w(k - 4)

- The Signal-to-Noise Ratio (SNR) is defined as:

- The parameters were obtained from 20 Monte Carlo runs, where 10240 data were used to estimate the third and fourth order cumulants. The system is solved using Standard Least Squares method [12]. The mean (p), the standard deviation (U) and the mean square error (p) values are considered to study the performance of each methods.

-

-

The results are presented in tables 1 and 2.

rable 1. System 1; Solution ot

- SNR

3y least squares

Observations relative Table 1 - The SRH95 method performs better than ABM99

method at SNR=lOdB, but at SNR=OdB, the o and p values become large even though the mean p values are acceptable. This could be because for lower SNR values the fourth order cumulants used in SRH95 method need a larger data so as to have a low variance. The ABM99 method has very large 0 and p values even with SNR=lOdB and not able to estimate the system.

-

Observations relative Table 2 - The ABM99 has lower U and p values, but this bias is

higher because the p values are away to the exact values.

- For the two SNR values considered, the SRH95 method has large values of U and p.

v -94

h(1)

h(2)

h(3)

h(4)

-2.205

-1.87

3.02

-1.435

ho

h(1)

0.49

- -2.205

ho

ho

ho

ho

-1.87

3.02

-1.435

0.49

Table 2.

1 2.1976 1 1.575611

0.8542 0.2724 -2.1012 -1.8599

-- System 2; Solution obtained

0

0

p

0

0.8849 0.2726 3.9479 1.9753 2.1142 0.6383 2.3089 1.2242

-1.8058 -1.1172 1.0298 0.2346 1.0946 0.3950 1.9478 0.8273 1.1635 0.3633 1.2503 0.7557

The previous observations show that the condition number plays an important role on the performance analysis. This role is due essentially to the ' estimation errors of the cumulants. In fact, we can overcome this problem using one of the following solutions:

- Minimise the estimation errors by in by increasing the number of output samples used in the estimation of cumulants. Use the preconditioning techniques which allow to minimise the condition number. Use another approach to solve the linear system of equations.

- -

All these solutions increase the computational cost. In the next section, we exploit only the last one.

4.4. Total Least Squares In this section, we propose to use the total least squares method [12] to solve the linear system of equations in

order to improve the quality of solution. This method assumes that there are errors in the elements of A and b. This situation is identical to our case because the contents of A and b are obtained from estimation (for details see appendix 6.3). The simulations are performed at the same conditions. The results are presented on the tables 3 and 4.

h(1)

h(2)

h(3)

h(4)

h(5)

0.1

-1.87

3.02

-1.435

0.49

P

h(1)

h(2)

ho

ho

ho

0.1

-1.87

3.02

-1.435

0.49

- Table 3.

SRH95 ABM99 0.4759 0.3085 0.4199 0.6638 0.5636 0.6958

1.2172 1.4358 1.6936 1.4906 4.1548 3.4572 1.2423 1.6107 1.6826 1.6690

0.8659 1.0418 1.3256 1.0711 1.9488 1.6894 0.4917 0.5372 0.6725 0.5731 0.1622 0.0597 0.2169 0.2182 0.2256 0.2219

0.5614 0.4370 0.6114 0.4427 3.2734 2.9923 0.6018 0.4814 0.6530 0.4822

0.4147 0.2693 0.4986 0.2694 1.5365 1.4988 0.2788 0.1787 0.2827 0.1789

-3.0476 -2.2704

-2.4386 -1.6842

-2.1121 -1.7998

-1.7119 -1.4416

- SNR 0

- 10

total least squares

Observations relative to Tables 3 & 4 - The use of the Total Least squares shows better

improvement to the quality of the identification. The ABM99 performs better than SRH95 method for the two systems. This could because the SRH95 uses cumualnt slices which are away from the origin and hence the estimates of the cumulants used in the method have a larger estimate variance than the cumulants used by ABM99 method as this method uses cumulants whose arguments are close to the origin.

-

v -95

0.9158 0.4186

h(2) ~1 -1.6155 -1.9364

0.4011

0.2580 0.2654 1 S386 0.3239 0.3275

0.1980 0.2025

0.1613

-1.3725

-2.2475

-1.9321

h(3) 1 10.3834 10.1728

2.5926 3.0958 1 3.02 0 0.2853 0.2234 h(4) 1 10.5139 10.2359 11

-1.3940 -1.4761 -1.435 (T 0.2250 0.1261

0 0.2323 0.2763

Table 4. System 2; Sc total least sauares

0.1452 0.1524

ution obtained by

5. CONCLUSION

In this paper, we have proposed an approach allowing to analyse the performance of the linear algebra methods. This approach is based on the use of the condition number which is generally used to quantify the sensitivity of system of equations to the perturbations. Simulation results showed the role of this condition. Moreover, we have suggested several propositions which can be used in order to improve the quality of the estimation. For example, we have used, in this paper, only the total least squares. The simulations illustrated the effectiveness of this solution.

6. APPENDIX

6.1. The method of Srinivas & Hari [19,21] This method is based on the following relation:

Y4 where E =A Y 3,e

This equation is concatenated for {- q 5 al s 2q, a1 5 a2 s min[al + q,2qlto a obtain

system of [ 5q2 + 7 q + 2 ) equations in ( q 2 + 5 q + 2 )

6.2. Our method [3] This method is based on the following relation:

h(kb(k P1 k 4 , x (O,O, + (A-2) k=0

Y4 where E =A Y 3,e

This equation is concatenated for (0 5 p1 s q, - 2q + P1 s al 5 q} to obtain a system of

6.3. The total least squares [12] Given A is a matrix of size (m,n) and b is a vector of size m. The goal is to compute 8 T U solution of:

(A.3) (A + AA)~TLs = b + Ab where AAand Ab are the estimate errors of A and b respectively.

Golub & Van Loan propose the following algorithm:

1. Construct C=[A b];

2. Compute the Singular Value Decomposition of C: C=UDVT

where U is a matrix of size( (m,m), V is matrix of size(n+l,n+l) and D is a diagonal matrix of size of (m,n+l) such that:

diag(D)- (01 0 2 ... 0, o , , ~ ) ~

3. If on then there is unique least squares solution defined by:

For i = l t o m

V(i, n + I) G4.4) eTu(i) = - V(n +I, n +I)

7. REFERENCES

[ l ] K. Abderrahim, R. Ben Abdennour, F. Msahli, M. Ksouri, G. Favier, Identification of NMP FIR systems using the fourth order cumulants, Third European Robotics, Intelligent Systems & Control Conference (EURISCONP8), Athens, Greece, 22- 25 June 1998.

[2] K. Abderrahim, F. Msahli, R. Ben Abdennour, M. Ksouri, G. Favier, A modified method for the identification of NMP FIR systems based on the third and the fourth order cumulants, IMACS- CSC'98, Piraeus, Greece, 26-28 October 1998.

[3] K. Abderrahim, R. Ben Abdennour, F. Msahli, M. Ksouri, G. Favier, A New General Relationship For a Higher-Order Statistics Based Identification, European Control Conference (ECC'99), Kalshrue, Germany, 31 August-03 September, 1999.

[4] S. Alshebeili, A. Venetsanopoulos, and E. Cetin, Cumulant based identification approaches for nonminimum phase FIR systems, IEEE Trans. Signal Processing, vol. 41, no. 4, pp. 1576-1588, 1993.

[5] D. Brillinger, M. Rosenblatt, Computation and interpretation of kth order spectra, Spectral Analysis of time signals, pp : 907-938, 1967.

[6] P. Comon, MA identification using fourth order cumulants, Signal Processing, vol. 26, no.3, pp.

[7] D. Dembtlt, Identification de mod61esARMA linkaires a l'aide de statistiques d'ordre klevk: application a l'kgalisation aveugle, Thbse de Doctorat, Universitt de Nice Sophia Antipolis, 1995.

[8] G. Favier, D. Dembtlt and J. L. Peyre, ARMA . identification using high-order statistics based

linear methods: an unified presentation, EUSIPCO, pp: 203-207, Edinburg, September 1994.

[9] J. A. R. Fonollosa and J. Vidal, System identification

381-388, 1992.

[12] G. H. Golub, C.F. Van Loan, Matrix computations, second edition, The Johns Hopkins University press, Baltimore and London, 1989.

[13] J. L. Lacoume, P. 0. Amblard, P. Comon, Statistiques d'ordre supkrieur pour le traitement du signal, Masson, 1997.

[ 141 J. M. Mendel, Tutorial on higher-order statistics (spectra) in signal processing and system theory: Theoretical results and some applications, Proc.

[15] C. L. Nikias, A. P. Petropulu, Higher-Order spectra analysis, PTR Prentice Hall, n Englewood Cliffs, New Jersey, 07632, 1993.

[16] Y. J. Na, K. S. Kim, I. Song, T. Kim, Identification of nonminimum phase Systems using the third and fourth order cumulants, IEEE Trans. Signal Processing, vol. 43, no. 8, pp. 2018-2022, 1995.

[17] B. Porat, B. Friedlander, Performance analysis of parameter estimation algorithms based on higher- order moments, Int. J . Adaptive Control and Signal Processing, vo1.3, pp. 191-229, February, 1989.

[18] A. G. Stogioglou, S. McLaughlin, MA parameter estimation and cumulant enhancement, IEEE Trans. Signal Processing, vol. 44, no. 7, pp. 1704-1718, 1996.

[19] L. Srinivas, K. V. S. Hari, FIR system identification using higher order cumulants - A generalised approach, IEEE Trans. Signal Processing, vol. 43, no. 12, pp. 3061-3065, 1995.

[20] L. Srinivas, K. V. S. Hari, FIR system identification based on subspaces of a higher-order cumulant matrix, IEEE SP 44(6) pp:1485-1491, June 1996.

[21] L. Srinivas, FIR system identification using higher order cumulants - A generalised approach, Master Thesis, Indian Institute of Science, Bangalore, 560012, India, 1994.

[22] A. Swami, J. M. Mendel, Closed-Form recursive estimation of M A coeficients using autocorrelations and third-order cumulants, IEEE Trans. Acoust. Speech, Signal Processing, vol. 37, no. 11, pp. 1794-1795, 1989.

[23] J. K. Tugnait, Approaches to FIR system identification with noisy data using higher order statistics, IEEE Trans. Acoust. Speech, Signal Processing, vol. 38 no. 7, pp. 1307-1317, 1990.

[24] J. K. Tugnait, New results on FIR system identification using higher order statistics, IEEE Trans. Signal Processing, vol. 39, no. 10, pp. 2216-

IEEE, vol. 79, pp. 278-305, 1991.

- .. I <

using a linear combination of cumulant slices, IEEE Trans. Signal Processing, vol. 41, no. 7, pp. 2405- [25] F.C. Zheng, Cumulant based deconvolution and 2412, 1993. identification: several new families of linear

G. B. Giannakis, Cumulants: A powerful tool in equations, Signal Processing, vol. 27, no.1, pp.

2221,1991.

signal processing, Proc. IEEE, vol. 75, pp. 1333- 199-219 1993. 1334, 1987.

G. B. Giannakis, J.M. Mendel, Identification of nonminimum phase systems using higher order statistics, IEEE Trans. Acoust. Speech, Signal Processing, vol. 37, no. 3, pp. 360-377, 1989.

v -97