Quarterly Journal of the Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

Inversion for atmospheric thermodynamical parameters of IASIdata in the principal components space

G. Masiello,a C. Serioa* and P. Antonellib

aCNISM, Unita di Ricerca di Potenza, University of Basilicata, Potenza, ItalybSSEC, University of Wisconsin, Madison, Wisconsin, USA

*Correspondence to: C. Serio, University of Basilicta, DIFA, via Ateneo Lucano 10, 85100 Potenza, Italy.E-mail: serio@unibas.it

The problem of reducing the dimensionality of infrared atmospheric soundinginterferometer (IASI) data space through a suitable transform and performingthe retrieval process for thermodynamical parameters within the transformed dataspace is addressed in this paper. The reduction of dimensionality is performed withthe principal components transform, which allows us to represent the full IASIspectrum with a few coefficients of the expansion. This truncated expansion couldhave a twofold beneficial effect: (i) it could improve the present exploitation andperformance of IASI data for the retrieval of temperature and moisture; and (ii)it could save transmission bandwidth, data rate and costs for the disseminationto users of IASI data. A suitable form of the inverse/forward model completelyembedded in the transformed space has been derived and applied to simulated andreal IASI data. This methodology has allowed us to assess the IASI performance fortemperature, water vapor and ozone based on the full IASI spectral coverage. Theuse of back-transformed spectral radiances (i.e. the filtered radiances obtained bythe truncated expansion) instead of expansion coefficients has also been addressedand assessed. Retrieval exercises performed in simulation and with real observationslead us to conclude that the principal components space-based inverse approach ispotentially superior over the current practice of using sparse channels. Copyrightc© 2011 Royal Meteorological Society

Key Words: infrared; satellite; remote sensing; inverse methodology; dimensionality reduction; temperature;water vapor; ozone

Received 31 March 2011; Revised 22 May 2011; Accepted 17 July 2011; Published online in Wiley Online Library

Citation: Masiello G, Serio C, Antonelli P. 2011. Inversion for atmospheric thermodynamical parameters ofIASI data in the principal components space. Q. J. R. Meteorol. Soc. DOI:10.1002/qj.909

1. Introduction

The infrared atmospheric sounding interferometer (IASI)provides data of unprecedented spectral quality andaccuracy (see, for example, Richter and Wagner, 2009)and the assimilation of its spectral radiances has produceda significant positive impact on forecast quality (e.g.Collard and McNally, 2009). IASI has been developedin France by the Centre National d’Etudes Spatiales(CNES) and flies on board the Metop-A (MeteorologicalOperational Satellite) platform, the first of three satellitesof the European Organization for the Exploitation of

Meteorological Satellite (EUMETSAT) European PolarSystem (EPS). The instrument has a spectral coverageextending from 645 to 2760 cm−1, which with a samplinginterval �σ = 0.25 cm−1 gives 8461 data points or channelsfor each single spectrum. Data samples are taken at intervalsof 25 km along and across track, each sample having aminimum diameter of about 12 km. With a swath widthon the Earth’s surface of about 2000 km, global coverageis achieved in 12 h, during which the instrument recordsabout 650 000 spectra.

This large amount of data still hampers the fullexploitation of IASI data at numerical weather prediction

Copyright c© 2011 Royal Meteorological Society

G. Masiello et al.

(NWP) centers, and in all those situations in which wehave the constraint of processing the data in near real time.As an example, to reduce the significant computationalburden, of the M = 8461 channels received at the EuropeanCentre for Medium Range Weather Forecasts (ECMWF)only 366 are routinely monitored (Collard and McNally,2009) and even fewer (168) are actively assimilated. Becauseof this limitation, IASI capability to improve temperatureand water vapor retrieval has not yet been fully exploited. Theproblem then is posed of how to perform a dimensionalityreduction of the IASI data space, which should not be at theexpense of the IASI spectral coverage, and allow end-users totake advantage of processing the full IASI spectral coveragewithin a level 2 processor aimed at inverting the observationsfor temperature, water vapor and ozone. Dimensionalityreduction is usually performed by considering a suitableorthogonal or unitary transform of the original data space.In the case where the data variability can be reproduced by atruncated expansion, we can use the coefficients (in numberof Mτ ≤ M) of this truncated expansion instead of theoriginal M spectral radiances. The orthogonality or unitaryproperty of the transform ensures that a suitable form ofthe Parseval theorem, such as that valid for the Fouriertransform, can be defined, and therefore the coefficientscan be ordered in such a way as to have an increasingexplanation of the data variance. Truncating the expansionat a given order allows us to retain the data variability withina prescribed threshold. One obvious unitary transform is thewell-known Fourier transform, which has been used in manyapplications for data compression and reduction. For IASIthe Fourier transform has a special meaning, since IASI is aFourier transform spectrometer, and as such measures theFourier transform of the spectrum, which is usually referredto as the interferogram. Exploiting the Fourier transform ofIASI spectra, Grieco et al. (2010) has shown how to performa dimensionality reduction of IASI data, which preserves theIASI spectral coverage. Once compared to the usual way ofreducing the IASI data by simply considering a selection ofIASI sparse channels, the Fourier transform methodologyhas shown a better performance mostly for the retrieval ofwater vapor.

In this paper, we exploit a different unitary transform,i.e. the principal components (PC) transform (e.g. Jolliffe,2002), the use of which to reduce the dimensionality ofthe radiative transfer equation has been exemplified byMasiello and Serio (2004) and Liu et al. (2007, 2009).Today, forward models for IASI capable of computing PCscores directly are available (see, for example, Masiello andSerio, 2004; Liu et al., 2006; Matricardi, 2010), which couldspeed up the process of adapting the assimilation processfor IASI PC scores. Such assimilation experiments arecurrently performed at ECMWF (M. Matricardi, personalcommunication 2010).

The advantage of the PC transform (also known as theempirical orthogonal functions (EOF) expansion) is that,by truncation, we not only save the full spectral coverageas in the case of the Fourier transform, but we retain mostof the spectral quality, because of the effectiveness of theEOF transform to represent complex but highly redundantsignals (as in the case of atmospheric spectra) with fewcoefficients or PC scores. However, when we EOF transformthe full spectrum, the result is to retain all those factors,which may potentially interfere with the temperature andwater vapor retrieval process and add biases to the final

products. This implies that the PC transform has to bejudiciously performed over those spectral segments, whichare less affected by interfering factors, such as minor speciesthat are not included within the retrieved state vectors.The PC or EOF series expansion is now as common asthe Fourier transform, and therefore we do not deal inthis paper with the basics of this mathematical tool. Theinterested reader is referred to Huang and Antonelli (2001),Goldberg et al. (2003), and Serio et al. (2008) for manyaspects and details of the transform in the context of highspectral resolution infrared observations, and to Serio et al.(2009) for a review on the application of EOF to the field ofsatellite meteorology.

In this paper we are not interested in the PC transform foritself or to produce compressed and/or filtered radiances.This last issue has been addressed recently in EUMETSAT(2010), where PC bases for IASI exploiting IASI clear-sky observations, collected globally over the full disk, havebeen developed. The primary objective of the paper is toprovide a methodological framework that can be used byIASI end-users aiming at developing inversion algorithms,which work directly in the PC domain. The driver of thepaper is the inverse/forward methodology rewritten withinthe PC space. We also address the problem of using filteredradiances within a given inverse scheme and how and whenthis usage becomes equivalent to the direct inversion withinthe PC space. The use of filtered radiances for the assimilationof high-resolution infrared sounder observations has beenrecently analyzed and described in Collard et al. (2010). Theemphasis of our paper is rather on the direct assimilationof PC scores and therefore it contains complementaryinformation.

The paper is organized as follows. In section 2 we describethe basic aspects of the inversion process within the spectralradiance data space. Section 2.2 describes how to pass fromthe original data space to the PC space, while sections2.3 and 2.4 show some important aspects of using filteredradiances. Section 2.5 discusses the problem of forwardmodel error. The problem of how to modify the forwardmodel to work directly in the PC space is addressed insection 2.6, which is limited to the case of monochromaticforward models. Section 3.1 is devoted to a reassessment ofthe IASI retrieval performance for temperature, water vaporand ozone in case we use the full IASI spectral coverage. Acomparison is also provided with the case of IASI usage withsparse channels. Section 3.2 shows the application of thePC-based inversion to real observations. Conclusions aredrawn in section 4.

2. Methodological background

2.1. The inverse problem in the spectral domain

Let us assume that the inverse solution of the radiativetransfer equation is sought through a Gauss–Newtoniterative sequence. Then, without any loss of generality,let us assume that we are in a region around the first guessin which the problem is linear. If not, the scheme hasto be further iterated according to the usual Gauss–Newtonscheme. Remaining in the context of Rodgers’ regularization(Rodgers, 1976), at a given step of the Gauss–Newton schemethe estimate, x, of atmospheric parameters is obtained bysolving the system of linear equations (e.g. Carissimo et al.,

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

Inversion of IASI Data in the Principal Components Space

2005):

(γ S−1a + KtS−1

ε K)x = KtS−1ε y + γ S−1

a xa, (1)

where Xt indicate the transpose of the generic variable X,and Sε , Sa indicate the observational (measurement error)and a priori covariance matrices, respectively. Here

x = v − v0; xa = va − v0; y = (robs − r0), (2)

where v, va, v0 are the state vector (to be estimated),background vector and first-guess parameters state vector(the size of these vectors will be denoted by N). Moreover,robs is the observed radiance vector (whose size will bedenoted by M), and r0 = F(v0) with F the forward modelfunction. Finally, the M × N derivative matrix or Jacobian,K, is computed as

K = ∂F(v)

∂v|v=v0 .

The parameter γ in Eq. (1) is introduced in such away as to consider different optimization schemes for thefinal solution: γ = 0 corresponds to the unconstrained leastsquares minimization and γ = 1 to the usual Rodgers’regularization (Rodgers, 1976). A suitable value for γ canbe also chosen by the user, e.g. in the direction in whichthe χ 2-constraint decreases. In addition, objective selectioncould be obtained using, for example, the so-called L-curvecriterion (Hansen, 1992). With the L-curve approach, thephilosophy is to try to minimize the root mean square errorE

((xtrue − x)2

), where E(·) means expectation value.

Under the usual assumption that the observations arenot correlated with the background state, we derive for thea posteriori covariance matrix of the final estimate, v, theexpression shown below:

Sv = A−1(γ 2S−1

a + KtS−1ε K

)A−1, (3)

with

A = (γ S−1

a + KtS−1ε K

), (4)

which for γ = 1 reduces to the usual form of the a posteriorimatrix for Rodgers’ regularization (Rodgers, 1976). A formaldemonstration of Eq. (3), for γ �= 1 and in general how toderive the various a posteriori covariance matrices presentedin this paper is provided in the Appendix.

In addition to the covariance matrix, for the purpose oflater comparison with the inverse problem embedded in thePC space, we quote also the expression for the averagingkernels:

∂ v

∂v= (

γ S−1a + KtS−1

ε K)−1

KtS−1ε K. (5)

Considering the normalized quantities

x = S− 1

2a x; xa = S

− 12

a xa; y = S− 1

2ε y, (6)

we have that the problem (Eq. (1)) can be put in a form ofridge regression (see, for example, Carissimo et al., 2005):

(γ I + GtG)x = Gt y + γ xa, (7)

Pressure (hPa)

Pre

ssur

e (h

Pa)

(K)2

01002003004005006007008009001000

0

100

200

300

400

500

600

700

800

900

1000−5

0

5

10

15

20

Figure 1. A priori covariance matrix for temperature. This figure is availablein colour online at wileyonlinelibrary.com/journal/qj

with I the identity matrix and G defined by

G = S− 1

2ε KS

12a . (8)

We prefer to deal with the dimensionless problem(Eq. (7)) because the PC transform is normally built upfor covariance-normalized spectral radiances, i.e. for theparameter y rather than for y. Furthermore, the systemof Eq. (7) can be more efficiently solved, since we donot need to perform the inversion operation for Sa, andthe kernel GtG can be efficiently decomposed via singularvalued decomposition, so that the only inversion neededinvolves a diagonal matrix (see, for example, Carissimo etal., 2005).

As stated before, the matrix Sε is here identified withthe observational covariance matrix, i.e. it accounts for themeasurement noise affecting radiances. How to deal withthe presence and effect of forward model errors is discussedin section 2.5. However, the methodology we illustrate inthe next two sections is independent of the interpretation wegive to Sε , and holds for any valid covariance matrix. Whendealing with application to IASI, unless otherwise stated thematrix Sε is identified with the full IASI level 1C covariancematrix.

The background matrix, Sa, can be any genericbackground covariance matrix and can depend on the user’schoice. For the purpose of application to IASI and forthe retrieval exercises shown here, Sa has been obtainedfrom functional sliced inverse regression (FSIR) regressionprediction errors. The FSIR (Amato et al., 2009) scheme isused to generate the first-guess state vector to initialize theinverse problem. Coherently with this approach, the a prioricovariance matrix is based on FSIR prediction errors. Theseare obtained for temperature, water vapor and ozone onthe basis of a suitable training dataset. The training datasetis a selection of profiles from the well-known Chevalierdatabase (Chevalier, 2001). For illustrative purposes the apriori covariance matrices for temperature, water vapor andozone are shown in Figures 1–3, respectively. Once again,it should be stressed that the theory we illustrate in thefollowing three sections is independent of the specific userdefinition for Sε and Sa.

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

G. Masiello et al.

Pressure (hPa)

Pre

ssur

e (h

Pa)

(g/kg)2

01002003004005006007008009001000

0

100

200

300

400

500

600

700

800

900

1000 −1

−0.5

0

0.5

1

1.5

2

Figure 2. A priori covariance matrix for water vapour. This figure isavailable in colour online at wileyonlinelibrary.com/journal/qj

Pressure (hPa)

Pre

ssur

e (h

Pa)

(ppmv2)

01002003004005006007008009001000

0

100

200

300

400

500

600

700

800

900

1000−0.06

−0.04

−0.02

0

0.02

0.04

0.06

0.08

0.1

0.12

Figure 3. A priori covariance matrix for ozone. This figure is available incolour online at wileyonlinelibrary.com/journal/qj

2.2. Dimensionality reduction: transforming the inverseproblem to the PC space

The problem in Eq. (7) is set up within the IASI spectralradiance space. In this space the data vectors have sizeM = 8461. Although modern computing platforms can dealwith this dimensionality, the cost is a high computationalburden and the impossibility to process IASI data in realtime. Thus a methodology which allows us to reduce thedimensionality of the problem, with a minimal loss of thespectral quality of the data and no further limitations on theIASI spectral coverage, is highly desirable.

In deriving the equations shown below, we onlyneed the assumption to deal with an unitary matrix.Thus the methodology could be applied to any unitarytransformation, including, for example, the Fouriertransform.

Let U be a suitable unitary matrix of size M x M. Thecolumns of this matrix are unitary vectors of size M, whichspan the data space and form an orthogonal basis for thespectral radiances domain. Remaining in the context of thePC transform, the matrix could be obtained, for example, by

performing the singular value decomposition of a suitablecovariance matrix (see, for example, Serio et al., 2009). Asstated, the matrix U defines a new coordinate system, i.e.an orthogonal set of vectors in which terms any spectralradiance vector can be expressed. Thus, any generic vectorof size M can be expressed as a linear combination ofthese orthogonal vectors. The coefficients (PC scores) of thisexpansion are given by

c = Ut y, (9)

and the original data vector can be transformed back throughthe inverse:

y = Uc. (10)

Using the transform, U, as shown by Masiello and Serio(2004) the problem (Eq. (7)) can be mapped into the PCspace:

(γ I + JtJ)x = Jtc + γ xa, (11)

where we have used J = UtG. Because the transform U hasan inverse, it realizes a bijective mapping from the radiancesto the PC scores space and vice versa, which implies that theproblem (Eq. (7)) is perfectly equivalent to that in Eq. (11).Thus there is no particular advantage in solving the systemin Eq. (11) instead of the problem in Eq. (7). As shown inMasiello and Serio (2004), the advantage of the linear system(Eq. (11)) is that we can truncate the vector c in order toretain only those PC that explain most of the variance of thespectral radiances. In doing so, we can effectively reduce thedimensionality of the inverse problem without sacrificingthe accuracy of the inverse products.

Let Uτ be the truncated operator. This is defined to bethe matrix of size M × Mτ with Mτ ≤ M whose columnsare exactly the first Mτ columns of the matrix U. With thisdefinition, we have

cτ = Utτ y, (12)

where the vector cτ has size Mτ and its components areexactly the first Mτ components of the vector c defined inEq. (9). In the same way, the Jacobian, J, is truncated to yieldJτ = Ut

τ G, and the truncated inverse problem reads

(γ I + Jtτ Jτ )xτ = Jt

τ cτ + γ xa. (13)

Note that now Jτ has size Mτ × N , where Mτ can beas low as Mτ ≈ 100, leading to a reduction factor M/Mτ

which for the case of IASI can be ∼ 80. It should be notedthat the series of transforms leave unchanged the size ofthe normalized state vector, which has been renamed xτ inorder to avoid confusion with the solution of the problemin Eq. (11).

Using Eq. (12), which defines the vector cτ , it is easilyshown that the covariance matrix of cτ , i.e. the expectationvalue E(cτ ct

τ ), is given by

E(cτ ctτ ) = Ut

τ Uτ = I, (14)

where I is the identity matrix of size Mτ × Mτ . In addition,the a posteriori covariance of the estimated parametersvectors can be written as

Sv = A−1R

(γ 2S−1

a + KtcKc

)A−1

R , (15)

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

Inversion of IASI Data in the Principal Components Space

with

AR = (γ S−1

a + KtcKc

), (16)

and

Kc = Utτ S

− 12

ε K. (17)

Transforming the dimensionless xτ vector back to itsnon-normalized version xτ allows us to derive the averagingkernels for the inversion in the PC space, i.e.

∂ v

∂v= (

γ S−1a + Kt

cKc)−1

KtcKc. (18)

The Jacobian term KtcKc can be expressed as

KtcKc = KtS

− t2

ε Uτ Utτ S

− 12

ε K. (19)

With the help of Eq. (19), the above averaging kernelscan be compared with those of Eq. (5) in the case that notruncation is applied. It is seen that the difference arisesbecause of the term Uτ Ut

τ . This term is the identity matrixonly in the case Mτ = M and hence no truncation is applied-in which case, of course, the two coincide. However, becausethe PC decomposition is highly effective in representingIASI spectra with a few components, it is expected that theretrieval performance of the PC-based inverse methodologyis good also for modest values of Mτ .

2.3. The case of filtered radiances

Using the unitary transform U, we can build up a vector offiltered radiances, yF , whose size is the same as the originalradiance vector, y. The reconstructed radiance vector isobtained by transforming back to the radiance space asuitably truncated PC vector:

yF = Uτ cτ . (20)

We can use this new filtered vector instead of the originaldata, y, to perform the inversion for geophysical parameters.However, since in general in Eq. (20) Mτ < M, i.e. we havemore channels than PC scores, the transform (Eq. (20))introduces collinearity among the filtered radiances; henceit does not yield a bijection from the original radiance dataspace to that of the reconstructed radiances. This can beeasily demonstrated by combining Eq. (20) with Eq. (12).We have

yF = Uτ Utτ y. (21)

The relation above has to be consistent with the otherobvious one:

yF = Uτ Utτ yF , (22)

otherwise we could apply the transform over and over andthe error could, in the end, be zeroed, which is a physicalnonsense. Thus the composite transform Uτ Ut

τ has noinverse in the strict sense. This means that the reconstructedradiances are linearly dependent and the inverse problemwith filtered channels has to be carefully addressed andanalyzed. In fact, the covariance matrix of the vector yF maywell be singular (its inverse may not exist). It is possible to

show that this singularity may be easily removed if the casethat we consider is the inversion of all the filtered radiances.This case might seem to be merely academic, since ourprimary objective is to reduce the dimensionality of thedata space. However, the corresponding inverse problemhas important theoretical implications, because it showsthat the case of all the filtered channels (for IASI this meansM = 8461) is formally equivalent to that of Mτ PC scores, i.e.the number of PC scores used to yield the M reconstructedradiances. In the two sections below, first we will deal withthe case M = 8461 and then we will discuss the case in whichwe use a set of sparse filtered channels.

2.3.1. The case with all filtered radiances

An elementary calculation shows that the M × M covariancematrix, E(yF yt

F), is given by (compare with the equivalentcovariance in the PC space, i.e. Eq. (14))

E(yF ytF) = Uτ Ut

τ �= I. (23)

The above matrix has no inverse, which means that wemight not build up an optimal least squares scheme for theretrieval problem with all the M = 8461 filtered radiances.The singularity arises from the fact that the two vectors cτ

and yF are completely equivalent: yF has the same degrees offreedom as cτ . It is as if we wanted to perform the retrievalwith Mτ degrees of freedom, but in a space with higherdimensionality, i.e. M.

Nevertheless, resorting to the Moore–Penrose pseudo-inverse definition, we can introduce a form of a regularizedinverse for the matrix Uτ Ut

τ by resorting to singular valuedecomposition. The matrix Uτ Ut

τ has rank Mτ ; thereforeit has Mτ eigenvalues equal to 1 and the remaining equalto zero. Exploiting this property, a little algebra shows thatthe regularized inverse of Uτ Ut

τ coincides exactly with thematrix itself. Combining Eq. (21) with Eq. (22) we have therelation

(Uτ Utτ )2 = Uτ Ut

τ ,

which states that if we apply twice the transform Uτ Utτ to the

data the result does not change. Matrices that obey the aboverelation are called idempotent. Using the regularized inverse,we can build the optimal least squares inverse problem forthe filtered radiances. We have

(γ I + Gt(Uτ Utτ )G)xF = Gt(Uτ Ut

τ )yF + γ xa, (24)

where we have inserted the regularized inverse covariancematrix of yF to introduce the correct dimensionality of thefiltered radiances.

However, if we consider that

Utτ yF = cτ ,

and thatUt

τ G = Jτ ,

we have that Eq. (24) transforms exactly to the inversesolution of Eq. (13), i.e.

xτ ≡ xF. (25)

The above theoretical result formally shows that M filteredradiances are not equivalent to M unfiltered radiances, butto Mτ PC scores with Mτ < M, i.e. to the number of PCscores used to perform the filtering.

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

G. Masiello et al.

2.3.2. Inverting a subset of filtered radiances

The inverse problem with a subset of filtered radiances canbe formalized as follows. Let us suppose we want to retain forthe purpose of mathematical inversion Mρ filtered channelsor rows. The linear transform, which maps the original spacein the new filtered one, is defined by

yFρ = Uρτ Utτ y = Uρτ cτ , (26)

where yFρ is the set of sparse filtered channels (this vectorhas size Mρ) and Uρτ (whose size is Mρ × Mτ ) is the matrixthat results from choosing the rows corresponding to thedesired channels from Uτ . Using Eq. (26), we can show thatthe covariance matrix Su of the Mρ filtered radiances is givenby

Su = E(yFρ ytFρ) = Uρτ Ut

ρτ . (27)

The inverse problem for the set of sparse filtered channelsreads

(γ I + GtFS−1

u GF)xFρ = GtFS−1

u yFρ + γ xa, (28)

with the kernel GF given by

GF = Uρτ Utτ G, (29)

where the matrix Uτ has size M × Mτ and the normalizedJacobian matrix G has size M × N .

The a posteriori covariance matrix becomes

Sv = A−1u

(γ 2S−1

a + KtFS−1

u KF)

A−1u , (30)

with

Au = (γ S−1

a + KtFS−1

u KF)

, (31)

and the averaging kernel

∂ v

∂v= (

γ S−1a + Kt

FS−1u KF

)−1Kt

FS−1u KF. (32)

In the above equations we have introduced the kernel KF

given by

KF = Uρτ Utτ S

− 12

ε K. (33)

At this point it has to be stressed that equations from Eq.(28) to Eq. (32) hold provided that Su has an inverse.

For Mρ = M we have, as discussed above, Su = S−1u =

Uτ Utτ and the problem is formally equivalent to that of

inverting directly Mτ PC scores.For Mτ < Mρ < M, in general, the singularity in Su

cannot be removed, because the generation of Mρ filteredradiances from Mτ PC scores leads to collinearity, i.e. theresulting set of filtered radiances is linearly dependent. Theresult, Su = S−1

u , is no longer valid, because the kernel Uρτ ,unlike Uτ , has columns, which are no longer orthogonalvectors.

For Mρ ≤ Mτ < M the situation is a bit different. Whileit is true that we cannot build up a bijection between yF

and y, it is true that this kind of mapping could existbetween a subset of the elements of yF and the vector cτ ,because Mτ < M. As suggested by Collard et al. (2010),

since the Mτ PC scores are linearly independent, there couldexist a subset of filtered radiances, which is itself linearlyindependent and, therefore, the covariance matrix Su couldhave an inverse. Although there may exist a subset of filteredradiances with Mρ = Mτ , which admits an inverse for Su,in general this is not expected for any choice of the filteredradiances. According to our experience, for a generic choiceof the filtered radiances, Su has an inverse, within machineprecision, only when Mρ < Mτ .

2.4. Illustrative examples of retrieval performance

To exemplify the retrieval performance, which can beachieved with original (unfiltered) radiances, truncated PCscore vectors or filtered radiances, this section will provide alinear analysis of the performance expected for the variousinverse tools we have been dealing with in this section.

According to our mathematical derivation, we willconsider the following four cases, which are definedbelow and referred to as FI (full IASI spectral coverageand data points), PC (principal component scores), SR(sparse radiance channels) and FSR (filtered sparse radiancechannels), respectively:

FI: the full IASI spectrum, i.e. all M = 8461 channelsover the spectral coverage 645–2760 cm−1. This is thereference case. The equations relevant to this case areEqs (1)–(8).

PC: The representation of the Full IASI spectrum with atotal of Mτ =290 PC scores; the equations relevant tothis case are Eqs (12)–(19).

SR: A selection of Ms = 300 IASI sparse channels (Collard,2007). The equations relevant to this case are againEqs (1)–(8), but with number of channels Ms = 300.

FSR: A subset of Mρ = 150 channels extracted from the 300channels corresponding to the case SR above, but PCfiltered. If rs denotes the SR channels vector, whoseelements are indicated by (rs(j), j = 1, . . . , 300), thenthe subset of 150 FSR channels corresponds to theelements within rs with even index, j . In this way theoriginal spectral coverage of the 300 sparse channelsis preserved. For this last case, the PC filter has beenbuilt up with the Mτ = 290 PC-scores. The equationsrelevant to this case are Eqs (26)–(33).

The sparse channels we consider for the SR case are thoseselected by Collard (2007) for use in numerical weatherprediction. As stated, these channels number 300 and areshown in Figure 4. For comparison, the same figure alsoshows the representation of the IASI spectrum with 290 PCscores. This is obtained by back-transforming to the spectralradiance space the selected 290 PC scores. This subset ofsparse channels may not be optimal for the given Sa we usein this paper. However, the purpose of the comparisons weare going to show is mainly illustrative and the matrix Sa isthe same for all inverse modes intercompared in this paper.

Why do we consider 150 filtered channels (case FSR)alone and not 300? This has to do with the problem ofdefining a Su matrix (see section 2.3), which has an inverse.Since the filtered radiances are obtained from Mτ = 290PC scores, the inverse of Su has been found to exist withinmachine accuracy at a number of channels, Mρ = 150. Thisis, of course, a technicality. In case one wanted to have 300filtered channels, the obvious thing would be to start from,say, Mτ = 500. The fact that we use Mτ = 290 is because we

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

Inversion of IASI Data in the Principal Components Space

1000 1500 2000 2500200

250

300

wave number (cm−1)

wave number (cm−1)

BT

spe

ctru

m (

K)

IASI spectrumselected channels

1000 1500 2000 2500200

250

300

BT

Spe

ctru

m (

K)

(a)

(b) IASI spectrumPC representation

Figure 4. (a) The 300 sparse channels compared to the IASI spectrum. (b)The IASI spectrum rendered by 290 PC scores compared with the originalspectrum.

want to intercompare unfiltered sparse channels against PCscores on the basis of a common size, i.e. Mτ ≈ Ms.

Concerning the U-bases, they have been built uponsynthetic IASI radiances generated for the IASI fields ofview and for a set of 377 profiles for temperature, watervapor and ozone, extracted from the Chevalier database(Chevalier, 2001). The synthetic IASI spectra have beenused to produce covariance matrices for each single band.These matrices have been SVD decomposed to producethree different basis, one for each IASI band, respectively.The radiance vectors have been centered and normalized tothe square root of the full IASI covariance matrix for thelevel 1C. The PC space retrieval approach has been designedfor this exercise to use 90 PC scores for IASI band 1, 120 forIASI band 2 and 80 for IASI band 3, for a total of 290 PCscores.

Having defined the four cases we want to intercompare,Figure 5 shows the temperature-averaging kernels for a rangeof pressure from 950 to 800 hPa and for a tropical modelof atmosphere. We focus on the lower troposphere for thesake of brevity and because this part of the atmosphere is ofmajor concern for IASI.

From Figure 5 it is seen that the averaging kernelsfor the PC are perfectly equivalent to those applyingto the full set of IASI channels. In other words, eitherwe use all the IASI channels (M = 8461) or we use arepresentation of them with 290 PC scores, the averagingkernels remain almost exactly the same. It is interestingto see that case FSR performs almost as well as FI andPC. This is a very important result because it says that the150 filtered channels also convey in the inverse probleminformation from the remaining M − Mρ IASI channels.The SR case of (unfiltered) sparse IASI channels (Figure 5(c))is outperformed by the other three.

The fact that FSR (150 filtered channels) is almostequivalent to the full set of IASI data is better appreciated ifwe consider the retrieval performance, which for the variouscases at hand is shown in Figure 6. We limit the case to thatof temperature, since equivalent results hold for water vaporand ozone. We stress that the results shown in Figure 6assume that the inverse problem is linear, since they have

been obtained by a direct use of the a posteriori covariancematrices shown in Eqs (3), (15) and (30).

As expected, we have perfect equivalence for cases FI andPC. However, once again a number of 150 filtered sparsechannels (the FSR case) performs almost as well as FI andPC. These three cases outperform case SR, with 300 sparsechannels.

In the end, this simple exercise exemplifies, in case weneed to work with filtered radiances, that a good choice is tohave fewer channels than PC scores, but with Mρ ≈ Mτ . Mρ

has to be as close to Mτ as to still allow the matrix Su to beinverted. This implies proceeding by trial and error, but theadvantage is to build up an optimal inverse problem, directlywithin the domain of the spectral (filtered) radiances, whichretains most of the information of the reference case (FI) withM = 8461 channels. It should be stressed that no attemptto optimize the set of filtered channels has been made.With a proper optimization and using the correct formof observational covariance matrix, i.e. Eq. (27), filteredradiances would behave in the same way as PC scores as faras the retrieval performance and quality are concerned.

2.5. Accounting for forward model noise

As stated before, the matrix Sε contains observational noise(i.e. measurement noise) alone, whereas in practical casesthe total covariance matrix Stot should also contain ormodel the noise coming from other sources, which canaffect the synthetic radiances computed with the forwardmodel such as discretization of the radiative transferequation, spectroscopy, and unmodeled minor and trace gasspace–time variability. Other sources of error could directlyaffect IASI observations such as cloud contamination anddrift in the calibration parameters.

It is customary to deal with this extra variability as anadditive term in the observational covariance matrix:

Stot = Sε + Sφ , (34)

where Sφ models the noise, which cannot be directly chargedto the radiometric noise. The details of how Sφ is handledmay depend on the user. The procedure of bias removal, forexample, is a very specialized task within the community ofdata assimilation (e.g. Bormann et al., 2010). Our treatmentis not exhaustive and we want only to show that in the casewhere the size of the hyper-volume spanned by the retrievalnoise matrix modeling radiometric noise, Sε is increasedwhen we build up the total covariance matrix, Stot, we havea smoothing effect over the retrieval.

As stated before, the rationale for increasing the hyper-volume of the retrieval noise covariance matrix has to do withthe presence of errors in the spectral radiances, whose sourceis different from that concerning the radiometric calibration.An inflated retrieval noise helps in not overfitting the data,which otherwise could lead to biases in the retrieval. Tomake the point, let us take the very pragmatic assumptionthat the additional noise term Sφ is proportional to Sε , i.e.

Sφ = f 2Sε and Stot = (1 + f 2)Sε. (35)

Introducing the extra error term the way we do impliesthat the inverse problem has to be scaled accordingly. Sincethe Sε normalization is introduced in Eqs (6) and (8) forobservations and Jacobians, respectively, it is not difficult to

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

G. Masiello et al.

0 0.2 0.4 0.6

0 0.2 0.4 0.6

0

200

400

600

800

1000

Averaging Kernels

Pre

ssur

e, (

hPa)

FIP=966.358hPa

P=952.902hPa

P=939.469hPa

P=929.016hPa

P=919.045hPa

P=906.533hPa

P=887.462hPa

P=862.503hPa

P=841.851hPa

P=823.878hPa

P=804.588hPa

0 0.2 0.4 0.6

0

200

400

600

800

1000

Averaging Kernels

Pre

ssur

e, (

hPa)

PCP=966.358hPa

P=952.902hPa

P=939.469hPa

P=929.016hPa

P=919.045hPa

P=906.533hPa

P=887.462hPa

P=862.503hPa

P=841.851hPa

P=823.878hPa

P=804.588hPa

0

200

400

600

800

1000

Averaging Kernels

Pre

ssur

e, (

hPa)

SRP=966.358hPa

P=952.902hPa

P=939.469hPa

P=929.016hPa

P=919.045hPa

P=906.533hPa

P=887.462hPa

P=862.503hPa

P=841.851hPa

P=823.878hPa

P=804.588hPa

0 0.2 0.4 0.6

0

200

400

600

800

1000

Averaging Kernels

Pre

ssur

e, (

hPa)

FSRP=966.358hPa

P=952.902hPa

P=939.469hPa

P=929.016hPa

P=919.045hPa

P=906.533hPa

P=887.462hPa

P=862.503hPa

P=841.851hPa

P=823.878hPa

P=804.588hPa

Figure 5. Example of IASI averaging kernels for temperature and for a tropical model of the atmosphere. The averaging kernels are shown for thelower troposphere for a range of pressure from 950 to 800 mbar. See text for a definition of the four cases FI, PC, SR, and FSR intercompared. All thecomputations shown in this figure have been obtained with γ = 1. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0

200

400

600

800

1000

Error for Temperature (K)

Pre

ssur

e (h

Pa)

FI

PC

SR

FSR

Figure 6. Example of temperature retrieval performance for the inverseproblems at hand and in the case of a tropical model of the atmosphere. Allthe computations shown in this figure have been obtained with γ = 1.

show that our general inverse problem expressed by Eq. (7)transforms to(

(1 + f 2)γ I + GtG)

x = Gt y + (1 + f 2)γ xa, (36)

which is the original problem (Eq. (7)) where we haveimposed a stronger smoothing by a factor (1 + f 2).Translating this in terms of the a priori Sa, it means that thismatrix is scaled to become

Sa → Sa

(1 + f 2), (37)

and hence it will have a major conditioning role indetermining the final solution. Using the substitution abovein the expressions for a posteriori covariances and averagingkernels we have derived in the previous sections, we can findthe appropriate scaled forms for these quantities.

Although very pragmatic, the approach of dealing witha scalar f works. An effective value of f is a matter of trialand error. For the current IASI observations, good results(in terms of χ 2 convergence) are obtained for f in therange 0.5–1.5. However, we caution that the use of a scalingfactor f > 0) in the observational noise always occurs at theexpense of retrieval quality.

To cope with the more complicated way in which the termSφ is normally handled, e.g. in assimilation processes, wehave to define an f -factor depending on the wave number, insuch a way as to have a diagonal matrix of f -factors. It is alsopossible to deal with Sφ as a pure independent additive term.However, whatever Sφ may be, in the case that it increasesthe volume of the ellipsoid of the error with respect to thatof the observational (radiometric) noise, the result is moresmoothing in the final solution.

2.6. Embedding the forward model in the PC space

Normally, the PC transform is built up for a centered andstandardized variate of the type

− 12(

X − X)

, (38)

where X denotes a generic variate, with X its mean and

its covariance matrix. In adapting the forward model to the

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

Inversion of IASI Data in the Principal Components Space

PC space it is therefore important to be consistent with theseries of transforms we have introduced within the inverseproblem. Failing to do so could introduce ambiguity inthe covariance matrix of data and parameters, which couldnegatively affect the final estimates, included the calculationof the a posteriori covariance matrix. Keeping this in mind,the series of transforms that the linearized forward modelundergoes within the inversion step are here exemplified:

{Ut

τ S− 1

2ε

}((robs − r) − (r0 − r)) =

={

Utτ S

− 12

ε K

}S

12a

{S

− 12

a (v − v0)

},

(39)

where r is the ensemble average of the set of radiances usedto build up the U-basis. Thus the forward model has to yielddirectly the two quantities

cτ = Utτ S

− 12

ε (r0 − r); Kc = Utτ S

− 12

ε K. (40)

Also, note that the normalized jacobian, Jτ=KcS12a , can be

calculated within the inverse scheme, which gains flexibilityto the forward model because in this way it will not dependon the a priori background, which could be left to the user.

Conversely, it should be stressed that the product Utτ S

− 12

ε isnot commutative even in the case that Sε is diagonal! Thus,we cannot reverse the order of the factors, which means thatthe covariance normalization cannot be changed after thevector c and the Jacobian Jτ have been computed.

Let us suppose that our forward model yields monochro-matic radiances and Jacobian matrices. Let R(σ ) be themonochromatic radiance at wave number σ , then the spec-tral radiance r(σ ) at the instrument spectral resolutioninvolves a linear transform:

r(σ ) =∫ +∞

−∞I(σ ; σo)R(σo)dσo, (41)

where the filter I(σ ; σo) is determined by the spectralcharacteristics of the given instrument. For IASI, thefunction I is a shift-invariant filter obtained from the Fouriertransform of a suitable truncated Gaussian function in theoptical path difference domain. Once put into discrete formfor numerical evaluation, the operation (Eq. (41)) can beperformed with a linear operator, A:

r = AR, (42)

where the instrument resolution radiance vector r has sizeM, the monochromatic radiance vector R has size Mh, andthe convolution kernel A has size M × Mh. Note that Mh

can be of the order of 106; however, the matrix A is a typicalsparse matrix.

To obtain the PC vector, we simply have to perform thefollowing matrix operation:

cτ = Uτ S− 1

2ε (r − r) = Uτ S

− 12

ε AR − Uτ S− 1

2ε r. (43)

The product P = Uτ S− 1

2ε A can be pre-computed and

stored, and therefore the forward model can yield directlythe vector cτ by applying the kernel P to the monochromaticradiances. It is worth noting that the matrix P has size

Mτ × Mh; since in practice we are interested in Mτ ≈ 100,the generation of a few PCs is much faster that the generationof the full IASI synthetic spectrum.

For the generation of the PC transformed Jacobian, itis safe in order to save CPU memory to perform thetransformation for each single column at a time:

kc(j) = Pkh(j), (44)

where kh(j) denotes the jth column of the monochromaticJacobian Kh and kc(j) is the jth column of the JacobianKc. Note that j = 1, . . . , np × Nlayer + ns, where np is thenumber of atmospheric vector parameters to be retrieved,Nlayer is the number of atmospheric layers, i.e. the size ofeach single vector parameter and ns is the number of scalarparameters to be retrieved (e.g. the surface temperature). Asan example, for our σ -IASI forward model (Amato et al.,2002) Nlayer = 60, so that in the case that we retrieve theskin temperature, the temperature, water vapor and ozoneprofiles, the operation for the Jacobian has to be repeated3 × 60 + 1 = 181 times.

For IASI, because the size of its radiance vectors is stillformidably large –M = 8461 –in practice it is advisable tosplit the forward model into the three IASI bands (see,for example, EUMETSAT, 2010), and then consider theappropriate concatenation of the PC scores and Jacobianswithin the inverse model.

For the work here shown forward calculations have beenbased on our σ -IASI package (Amato et al., 2002; Masielloet al., 2009; Grieco et al., 2007), which has been adaptedto perform also the appropriate computations describedabove to yield PC scores directly. The forward model σ -IASI is a monochromatic radiative transfer package, whichis based on a suitable look-up table of monochromaticoptical depths. For the present work, the look-up table hasbeen generated by using the Line-by-Line Radiative TransferModel (LBLRTM) version 11.7 (documentation available atthe web site http://www.rtweb.aer.com/.

3. Application to IASI

In this section we will apply the basic methodology shownin section 2 to IASI observations. We first deal with a case insimulation and then we proceed to illustrate the applicationto real IASI observations.

The reference inverse problem (FI case) will be comparedto the three different inverse modes, PC, SR and FSR, asdefined in section 2.4. As anticipated in section 2, the first-guess state vector has been generated via FSIR regression(Amato et al., 2009). The retrieved state vector includes skintemperature, temperature, water vapor and ozone profiles.The a priori covariance matrix and the first-guess statevector are the same for the four modes –a choice that allowsus to intercompare the diverse inversion algorithms on acommon basis.

3.1. Retrieval performance for temperature, water vapor andozone

The simulation case study has been obtained by consideringa set of atmospheric state vectors for skin temperature,temperature, water vapor and ozone profiles. This set hasbeen selected from the Chevalier database (Chevalier, 2001)and consists of the same 377 sea-surface, clear-sky, tropical

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

G. Masiello et al.

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0

200

400

600

800

1000

Error Temperature (K)

Pre

ssur

e (h

Pa)

FI

PC

SR

FSR

First Guess

Figure 7. Retrieval performance for temperature with the four diverseinverse modes analyzed in this paper.

profiles as those used to generate the U-bases. The set wasused to yield IASI spectra, which once properly added withnoise were processed to retrieve the atmospheric parameters.The random error added to the synthetic radiances wasproduced according to the IASI Level 1C noise figures. Inthis way, the noise is generated within the interferogramdomain, Gaussian apodized and back-transformed to thespectral domain.

The retrieval exercise here shown provides a case forwhich the true state of the atmosphere is perfectly knownand there are no forward and spectroscopic biases, but onlyobservational noise.

In contrast to the linear retrieval performance analysisprovided in Figure 6, the retrieval performance is hereassessed on the basis of an end-to-end retrieval exercise,which can address nonlinear aspects of the inverse problem.To this end, the retrieved parameters were compared to thetest ones to derive the root mean square error (RMSE) fortemperature, water vapor and ozone, respectively.

Figure 7 compares the results for temperature. It is clearlyseen that the FI, PC and FSR-based retrievals outperformthe sparse channels approach. The retrieval performancefor temperature improves everywhere and is clearly below1 K also in the lower part of the atmosphere. The resultsexemplify that for temperature we should gain a lot ofinformation by conveying within the retrieval process thefull IASI spectral coverage, rather than only a few insulatedchannels. The full IASI spectral coverage can be conveyeddirectly by using the M = 8461 channels, or by considering aset of suitable PC scores or a set of suitable filtered radiances.

Results, which are equivalent to those shown fortemperature, are also obtained for water vapor (see Figure 8).In comparison to the SR mode, FI, PC and FSR achieve abetter H2O retrieval performance in the lower part of thetroposphere, where the accuracy comes close the IASI goalof 10%. This better performance is expected, since with theFI, PC, and FSR inversion methods we take advantage of thewhole water vapor absorption band.

FI, PC and to a less extent SRF also achieve an impressiveimprovement for the case of ozone, shown in Figure 9. Forozone the SR mode does not improve over the first guess.

It has to be stressed that the quality of the retrieval alsoimproves in terms of the vertical spatial resolution. Thiscan be seen by comparing the averaging kernels for the

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0

200

400

600

800

1000

Error for H2O (g kg−1)

Pre

ssur

e (h

Par

)

FIPCSRFSRFirst Guess

Figure 8. Retrieval performance for water vapor with the diverse inversemodes analyzed in this paper.

0 0.1 0.2 0.3 0.4

10−1

100

101

102

103

Error for O3 (ppmv)

Pre

ssur

e (h

Pa)

FIPCSRFSRFirst Guess

Figure 9. Retrieval performance for ozone with the diverse inverse modesanalyzed in this paper.

diverse inversions modes. For the sake of brevity, these areexemplified for water vapor in Figure 10, for the pressurerange 950–900 hPa, in the case of PC and SR modes. Wesee that for the sparse channels retrieval (SR mode) theaveraging kernels fail to peak at the right pressure level.They peak at about the same pressure level. Conversely, forthe case of PC mode it is seen that the averaging kernelsare located at approximately the right level from 950 to900 hPa, which means that the H2O retrieval has a bettervertical spatial resolution.

3.2. Application to the JAIVEx case study

The Joint Airborne IASI validation experiment (JAIVEx)(Taylor, 2010) was carried out over the Gulf of Mexicoduring April and May 2007. We have a series of six spectrafor 29 April 2007, 16 spectra for 30 April 2007, and finallythree spectra for 4 May 2007. The total of 25 soundings aretime and space colocated with dropsonde observations. TheJAIVEx spectra were recorded during the day; therefore theshort-wave band is contaminated by solar emission. For thisreason we have redefined both the set of sparse channels

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

Inversion of IASI Data in the Principal Components Space

−0.1 0 0.1 0.2 0.3 −0.1 0 0.1 0.2 0.3

0

100

200

300

400

500

600

700

800

900

1000

Averaging Kernels

Pre

ssur

e, (

hPa)

(a)

P=966.358hPaP=952.902hPaP=939.469hPaP=929.016hPaP=919.045hPaP=906.533hPa

0

100

200

300

400

500

600

700

800

900

1000

Averaging Kernels

Pre

ssur

e, (

hPa)

(b)

P=966.358hPaP=952.902hPaP=939.469hPaP=929.016hPaP=919.045hPaP=906.533hPa

Figure 10. H2O averaging kernels for a tropical model of atmosphere. Panel (a) refers to the PC mode, whereas panel (b) refers to the SR mode. Thisfigure is available in colour online at wileyonlinelibrary.com/journal/qj

and the IASI bands to be used for PC space-based inversemethodology.

With reference to Figure 4, for the case of sparsechannels we limit these to those wave numbers that arebelow 2150 cm−1. In this way we have 272 channelswhose contamination by solar radiation can be considerednegligible. The set of filtered sparse channels is changedaccordingly, but the number of radiance data points is leftunchanged at Mρ = 150.

For the case of the PC space-based methodology, the IASIspectral segments we consider for the PC expansion aredefined in such a way as to limit the effect of interferingfactors such as solar emission. We define two spectralsegments, which are subsets of the IASI bands:

• spectral interval 1: 645–1200 cm−1, for a total of 2221IASI channels;

• spectral interval 2: 1450–2150 cm−1, for a total of2801 IASI channels.

In total we have 5022 IASI channels. The first spectralsegment is represented with 120 PC scores, and the secondwith 170 PC scores, for a total of 290 PC scores.

As in the previous section we will consider and compareall four inversion modes defined in section 2.4. To havea proper comparison, the four inversion methods wereinitialized with the same first-guess atmospheric state vector.As already stated, this was obtained with the FSIR regressionscheme (Amato et al., 2009). Once again, the retrieved statevector includes skin temperature, temperature, water vaporand ozone profiles. For the present exercise the emissivitywas not retrieved. Since we deal with sea-surface soundings,we used Masuda’s emissivity model (Masuda et al., 1988).

Dropsonde and ECMWF model data, which have beenused to obtain a best estimate of the atmospheric state duringeach IASI overpass, have been prepared and made availableto us by the JAIVEx team (Taylor, 2010). The JAIVex datasetcontains dropsonde profiles of temperature, water vaporand ozone, which extend up to 400 hPa. Above 400 hPa onlyECMWF model data are available. The atmospheric state

vectors used in this study use JAIVex dropsonde data up to400 hPa supplemented by colocated ECMWF forecasts oftemperature, water vapor and ozone from 400 to 0.1 hPa(corresponding to about 65 km). These atmospheric statevectors where considered to represent the truth state of theatmosphere against which the retrieved parameters werecompared to derive the bias and standard deviation of theretrieval.

Figure 11 summarizes the bias and standard deviation forthe case of temperature. The results for temperature showthat the four inversion modes are all capable of reducingthe large bias at the tropopause level and in the lowertroposphere. In the lower troposphere the full IASI, PC andFSR modes are more effective than the remaining SR inimproving the bias. All the modes add some biases at thelevel of 600 hPa and between 200 and 400 hPa, althoughthe RMSE remains limited to ∼ 1. At the level of 600 hPathe retrieval performance of FI, PC and FSR modes, in thatorder, is slightly better than that of SR mode. As far as therandom variability of the retrieval is concerned, we can saythat all modes are capable of improving the final solutionwith respect to the variance.

The results for water vapor are summarized, once againin terms of bias and standard deviation, in Figure 12.

First, we see that the four modes are capable of improvingthe final solution between 500 and 800 hPa, where thebias is reduced to close to zero. This is expected becausethe water vapor Jacobian matrix has peaks at that level.The best improvement is seen for the case of the FI mode.In the lower troposphere only the FI mode is capable ofimprovement over the first guess, although in terms ofvariance alone. The bias remains almost the same as thatof the first guess, with a modest improvement in the lowertroposphere. The other three modes add bias to the finalsolution in the lower troposphere. Both PC and FSR modesare capable of largely improving the variance, so that theRMSE is improved. Overall, the best performance for watervapor is attained with the FI mode, followed by PC, FSRand SR, in that order. The modest improvement for the

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

G. Masiello et al.

−2 0 2 4 6

0

500

1000

Error for Temperature (K)

Pre

ssur

e (h

Par

)

(a)

−2 0 2 4 6

0

500

1000

Error for Temperature (K)

Pre

ssur

e (h

Par

)

(b)

−2 0 2 4 6

0

500

1000

Error for Temperature (K)

Pre

ssur

e (h

Par

)

(c)

−2 0 2 4 6

0

500

1000

Error for Temperature (K)

Pre

ssur

e (h

Pa)

(d)

bias R

bias FG

sd R

sd FG

bias R

bias FG

sd R

sd FG

bias R

bias FG

sd R

sd FG

bias R

bias FG

sd R

sd FG

Figure 11. Bias and standard deviation (sd) for the case of temperature. The final retrieved solution is indicated by R and the first guess by FG, respectively.(a) FI mode; (b) PC mode; (c) SR mode; (d) FSR mode.

−2 0 2 4 6

0

500

1000

Error for H2O (g kg−1)

Pre

ssur

e (h

Pa)

(a)

−2 0 2 4 6

0

500

1000

Error for H2O (g kg−1)

Pre

ssur

e (h

Pa)

(b)

−2 0 2 4 6

0

500

1000

Error for H2O (g kg−1)

Pre

ssur

e (h

Par

)

(c)

−2 0 2 4 6

0

500

1000

Error for H2O (g kg−1)

Pre

ssur

e (h

Pa)

(d)

bias R

bias FG

sd R

sd FG

bias R

bias FG

sd R

sd FG

bias R

bias FG

sd R

sd FG

bias R

bias FG

sd R

sd FG

Figure 12. Bias and standard deviation (sd) for the case of water vapor. The final retrieved solution is indicated by R and the first guess by FG, respectively.(a) FI mode; (b) PC mode; (c) SR mode; (d) FSR mode.

case of water vapor is due to the failure to capture thefine structure of the vertical distribution of water vapor inthe lower troposphere. This is true especially for 29 April2007, when the water vapor profile shows a double layerstructure with a very dry layer between 990 and 820 hPa.To explain the results for water vapor it is important atthis point to recall that for the JAIVEx experiment, i.e. forIASI real observations, we had to increase the f -factor to thevalue of 1 (see section 2.6), and in parallel we had also to

optimize, through the L-curve criterion (Hansen, 1992), thevalue of the smoothing parameter γ . For the JAIVEx casestudy at hand the optimal value of γ varied from mode tomode and ranged in the interval 1–2.5. With these combinedsmoothing factors the averaging kernels for water vapor areheavily smoothed. The findings for water vapor exemplifyhow important is the correct characterization of the totalnoise affecting IASI observations. In particular, a treatmentof the forward model bias, which does not remove the bias

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

Inversion of IASI Data in the Principal Components Space

800 1000 1200 1400 1600 1800 2000 2200 2400 2600−1

−0.5

0

0.5

1x 10−3

wave number, σ (cm−1)

Obs

–Cal

c(W

m−2

sr−1

(cm

−1)−1

)

Obs. Calc.

± 1 σ

800 1000 1200 1400 1600 1800 2000 2200 2400 26000

1

2

3

wave number (cm−1)

NE

DT

at 2

80 K

(K

)

IASI NEDTEstimated from the data

Figure 13. Bias (a) and standard deviation (b) of the spectral residual from the 25 JAIVEx IASI soundings. In panel (b) the standard deviation has beenconverted to units of NEDT at 280 K to allow a proper comparison with the IASI level 1C noise.

itself from the synthetic radiances, but just increase the totalnoise modeled through the observational covariance matrix,leads to an over-smoothing of the retrieval.

The retrieval for ozone is not shown for the sake of brevity.For this parameter, we could reach some improvement uponthe first guess with the FI mode alone, whereas the otherthree modes showed no improvement at all.

Before concluding this section, we show an analysis of thespectral residual for the case of the FI mode. The analysis canprovide insight into understanding the level of extra noise(in addition to the radiometric noise), which affects ourforward/inverse system. We show here just one case sinceequivalent conclusions are achieved by analyzing the otherthree modes.

Figure 13 shows the mean and standard deviation ofthe spectral residual (Observations − Calculations) averagedover the 25 JAIVEx spectra. The spectral residual has beenfurther smoothed with a moving average window with a filterlength of 25 wave numbers. The filtering is applied to thespectral residual in order to smooth random fluctuations,which are inherent when analyzing a sample of few elements(25 IASI spectra).

Figure 13(a) shows the forward model bias and we seethat apart from the evident model parameter error at themethane band at 7.6 µm (1306.2 cm−1) (methane is notretrieved), the bias oscillates within the IASI level 1C noise.This confirms that we were right in running the inversionwith an equivalent error twice the IASI noise, whichallowed us not to fit the bias. In passing, Figure 13(a) alsoexhibits an important forward model error in the short-waveCO2 absorption band because of non-local thermodynamicequilibrium (non-LTE) effects, which are driven by solarradiation.

Figure 13(b) shows the standard deviation of the spectralresidual, i.e. the strength of the random component affectingthe difference Observations − Calculations. This has beenconverted into noise equivalent difference temperature(NEDT) at 280 K in order to have a proper comparison

with the IASI level 1C noise. We see that agreement with theIASI radiometric noise is impressive.

To understand the result shown in Figure 13(b) it isimportant to stress that the 25 IASI soundings at hand arestrongly localized in time and space; therefore the variabilitydue, for example, to forward model parameters and evenspectroscopy, which may depend on the state vector itself,is minimized. In this situation, Figure 13(b) just shows thatthe larger source of random variability is that related to theradiometric noise. In the end, this figure provides a checkthat our inversion strategy is capable of extracting fromthe observations all the variability supposedly introducedby the state of the atmosphere. In fact, the inversemethodology leaves behind a random component, whichis easily identified with the IASI radiometric noise.

To sum up, the spectral residual analysis shown inFigure 13 may provide a powerful tool to check for thequality of a given forward/inverse scheme. For this test towork, it is important to consider soundings, which are bothspace and time localized. If we extended the analysis tothe full disk, we would find a space–time variability, forexample, of spectroscopic noise –trace gases –which wouldjust add variability and hence increase the variance of thespectral residual. If needed, we also stress that, because of itsspace–time localized origin, the bias shown in Figure 13(a)cannot be universally used to be subtracted from thesynthetic radiances in order to better match observations tocalculations.

4. Conclusions

We have developed and presented a methodologicalframework, which can be used to develop IASI dataretrieval algorithms for meteorological applications, whichare embedded in the PC space and PC filtered radiancespace.

In addition, we have shown how to modify the forwardmodel in order to yield PC scores directly. We have also

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

G. Masiello et al.

shown that radiometric noise is not the major source oferrors when inverting IASI radiances. Other sources suchas forward model, spectroscopy and cloud contaminationhave to be properly addressed and modeled within theobservational covariance matrix before the retrieval isproduced. Based on our analysis, we can say that the levelof bias in state-of-the-art forward models appears to be aslarge as the strength of the variance of the IASI radiometricnoise. Failure to properly address this source of extra noisemay lead to on over-smoothing of final inverted products.

The methodology has been illustrated and exemplifiedby simulated and real IASI observations. It has been shownthat the PC space-based retrieval methodology allows one towork with the full IASI spectral coverage in a domainof reduced dimensionality, which greatly improves thecomputational burden of forward calculations and can savetransmission bandwidth.

Compared to the present usage of IASI sparse channels,the use of the full IASI spectral coverage can further improvethe retrieval performance for temperature and water vapor.

The paper has also addressed the issue of filteredradiances (i.e. the spectral radiances, which are obtainedby transforming back to the spectral domain a PC truncatedvector), and we have derived the correct inverse problem bya suitable transformation of the equivalent problem in thespectral domain. In deriving the inverse problem for filteredradiances we have introduced no approximation, whichis at least a formal, mathematical step forward from thecurrent practice of using the filtered radiances as they werethe original observations (same observational covariancematrix and Jacobian matrices). Then, an important findinghas been found, that PC filtered sparse radiances have thesame capability as PC scores in effectively reducing thedimensionality of the IASI data space, while still preservingthe full IASI spectral coverage. This result can be obtainedby a proper selection of a number Mρ of filtered sparsechannels, with Mρ that can be as low as Mρ < 300.

In conclusion, we have shown, and demonstrated withpractical applications to IASI, how to transform theoriginal inverse problem embedded in the spectral radiancedata space to new spaces of lower dimensionality, whichcould solve the practical problem of saving transmissionbandwidth. The transformed problems are not expected tobe superior (in terms of retrieval performance) over theoriginal problem, since the original and transformed spacesare mapped one onto the other by linear operations. Thebest we can do is to achieve a similar performance betweenthe original and transformed space. In the end, which spacesto use (PC or back-transformed radiances) is a matter ofpractical usefulness, constraints and, finally, quality of theforward model.

Appendix

Deriving the a posteriori covariance matrix

The various inverse problems we have described in thepaper have the same form; therefore here we exemplify thecalculation of the a posteriori covariance for the generalinverse problem expressed by Eq. (1). With reference toEq. (1) and to simplify notations, let us define

{A = (γ S−1

a + KtS−1ε K)

B = KtS−1ε

. (45)

Then we can write Eq. (1) as

A(v − v0) = By + γ S−1a (va − v0), (46)

which considering expectation values becomes

A (E(v) − v0) = BE(y) + γ S−1a (E (va) − v0) . (47)

Subtracting Eq. (47) from Eq. (46), we have

A (v − E(v)) = B(

y − E(y)) + γ S−1

a (va − E(va)) , (48)

which by defining

δv = v − E(v)δy = y − E(y)δva = va − E(va)

, (49)

becomes

Aδv = Bδy + γ S−1a δva. (50)

Multiplying the above equation on both sides forthe transpose term, considering expectations values andassuming that the observations and background are notcorrelated, we have

AE(δvδvt)At = BE(δyδyt)Bt + γ 2S−1a E(δvaδvt

a)S−ta , (51)

which, considering that A and Sa are symmetric matrices,and remembering that because of their definition we have

E(δvδvt) = Sv

E(δyδyt) = Sε

E(δvaδvta) = Sa

, (52)

can be written as

ASvA = BSεBt + γ 2S−1a . (53)

Remembering the definition for B, Eq. (53) becomes

ASvA = (γ 2S−1a + KtS−1

ε K), (54)

and finally

Sv = A−1(γ 2S−1a + KtS−1

ε K)A−1. (55)

Acknowledgements

We would like to thank an anonymous referee, whose criticalreview led us to greatly improve the paper. IASI has beendeveloped and built under the responsibility of the CentreNational d’Etudes Spatiales (CNES, France). It is flownon board the Metop satellites as part of the EUMETSATPolar System. The IASI L1 data are received throughthe EUMETCast near-real-time data distribution service.We thank Dr Stuart Newman (Met Office) for providingthe JAIVEx data. The JAIVEx project has been partiallyfunded under EUMETSAT contract Eum/CO/06/1596/PS.FAAM BAe 146 is jointly funded by the Met Officeand the Natural Environment Research Council. The USJAIVEx team was sponsored by the National Polar-orbitingOperational Environmental Satellite System (NPOESS)Integrated Program Office (IPO) and NASA.

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)

Inversion of IASI Data in the Principal Components Space

References

Amato U, Masiello G, Serio C, Viggiano M. 2002. The σ -IASI code forthe calculation of infrared atmospheric radiance and its derivatives.Environ. Model. Software 17: 651–667.

Amato U, Antoniadis A, De Feis I, Masiello G, Matricardi M, Serio C.2009. Technical note: functional sliced inverse regression to infertemperature, water vapour and ozone from IASI data, Atmos. Chem.Phys. 9: 5321–5330.

Bormann N, Collard A, Bauer P. 2010. Estimates of spatial andinterchannel observation-error characteristics for current sounderradiances for numerical weather prediction. II: Application to AIRSand IASI data. Q. J. R. Meteorol. Soc. 136: 1051–1063.

Carissimo A, De Feis I, Serio C. 2005. The physical retrieval methodologyfor IASI: the δ-IASI code. Environ. Model. Software 20: 1111–1126.

Chevalier F. 2001. Sampled database of 60 levels atmospheric profiles fromthe ECMWF analysis. ECMWF EUMETSAT SAF programme ResearchReport 4. ECMWF, Reading, UK.

Collard AD. 2007. Selection of IASI channels for use in numericalweather prediction. Q. J. R. Meteorol. Soc. 133: 1977–1991.

Collard AD, McNally AP. 2009. The assimilation of infrared atmosphericsounding interferometer radiances at ECMWF. Q. J. R. Meteorol. Soc.135: 1044–1058.

Collard AD, McNally AP, Hilton FI, Healy SB, Atkinson NC. 2010.The use of principal component analysis for the assimilation ofhigh-resolution infrared sounder observations for numerical weatherprediction Q. J. R. Meteorol. Soc. 136: 2038–2050.

EUMETSAT. 2010. IASI Level 1 Principal Component CompressionPPF (Part 1). Doc. No. EUM/OPS-EPS/REP/10/0148. EUMETSAT,Darmstadt, Germany.

Goldberg MD, Qu Y, McMillin LM, Wolf W, Zhou L, Divakarla M.2003. AIRS near-real-time products and algorithms in support ofoperational numerical weather prediction. IEEE Trans. Geosci. RemoteSensing 41: 379–389.

Grieco G, Masiello G, Matricardi M, Serio C, Summa D, Cuomo V.2007. Demonstration and validation of the φ-IASI inversion schemewith NAST-I data. Q. J. R. Meteorol. Soc. 133: 217–232.

Grieco G, Masiello G, Serio C. 2010. Interferometric vs spectralIASI radiances: effective data-reduction approaches for the satellitesounding of atmospheric thermodynamical parameters. Remote Sens.2: 2323–2346.

Hansen C. 1992. Analysis of discrete ill-posed problems by means of theL-curve. SIAM Rev. 34: 561–580.

Huang H-L, Antonelli P. 2001. Application of principal component

analysis to high-resolution infrared measurements compression andretrieval. J. Appl. Metereol. 40: 365–388.

Jolliffe IT. 2002. Principal Component Analysis. Springer: New York.Liu X, Smith WL, Zhou DK, Larar A. 2006. Principal component-based

radiative transfer model for hyperspectral sensors: theoretical concept.Appl. Opt. 45: 201–209.

Liu X, Zhou DK, Larar A, Smith WL, Mango SA. 2007. Case-study ofa principal-component-based radiative transfer forward model andretrieval algorithm using EAQUATE data. Q. J. R. Meteorol. Soc.133(S3): 243–256.

Liu X, Zhou DK, Larar AM, Smith WL, Schluessel P, Newman SM,Taylor JP, Wu W. 2009. Retrieval of atmospheric profiles and cloudproperties from IASI spectra using super-channels. Atmos. Chem.Phys. 9: 9121–9142.

Masiello G, Serio C. 2004. Dimensionality-reduction approach to thethermal radiative transfer equation inverse problem. Geophys. Res.Lett. 31: L11105, DOI: 10.1029/2004GL019845.

Masiello G, Serio C, Carissimo A, Grieco G, Matricardi M. 2009.Application of ϕ-IASI to IASI: retrieval products evaluation andradiative transfer consistency. Atmos. Chem. Phys. 9: 8771–8783.

Masuda K, Takashima T, Takayama Y. 1988. Emissivity of pure andsea waters for the model sea surface in the infrared window regions.Remote Sens. Environ. 24: 313–329.

Matricardi M. 2010. A principal component based version of theRTTOV fast radiative transfer model. Q. J. R. Meteorol. Soc. 136:1823–1835.

Richter A, Wagner T. 2009. The IASI instrument onboard theMETOP Satellite: First results. Atmos. Chem. Phys. Special Issue.http://www.atmos-chem-phys.net/special issue123.html

Rodgers CD. 1976. Retrieval of atmospheric temperature andcomposition from remote measurements of thermal radiation. Rev.Geophys. Space Phys. 14: 609–624.

Serio C, Esposito F, Masiello G, Pavese G, Calvello MR, Grieco G,Cuomo V, Buijs HL, Roy CB. 2008. Interferometer for ground-based observations of emitted spectral radiance from the troposphere:evaluation and retrieval performance. Appl. Optics 47(21):3909–3919.

Serio C, Masiello G, Grieco G. 2009. EOF regression analytical modelwith applications to the retrieval of atmospheric temperature andgas constituents concentration from high spectral resolution infraredobservations. In Environmental Modelling: New Research, Findley PN(ed.). Nova Science: Hauppagauge, NY; 51–88.

Taylor JP. 2010. Joint Airborne IASI Validation Experiment.http://badc.nerc.ac.uk/data/jaivex/

Copyright c© 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. (2011)