O. Gervasi et al. (Eds.): ICCSA 2009, Part I, LNCS 5592, pp. 107–117, 2009. © Springer-Verlag Berlin Heidelberg 2009

An Evaluation of the Performance of the CHIMERE Model over Spain Using Meteorology from MM5 and

WRF Models

Marta G. Vivanco1, Inmaculada Palomino1, Fernando Martín1, Magdalena Palacios1, Oriol Jorba2, Pedro Jiménez2, José María Baldasano2, and Oier Azula3

1 Atmospheric Pollution Unit, Environmental Department, CIEMAT, Avda. Complutense 22, 28040 Madrid, Spain

2 Earth Science Department, Barcelona Supercomputing Center-Centro Nacional de Supercomputación (BSC-CNS), Barcelona, Spain

3 Departamento de Química Fisica, UPV, Paseo de la Universidad, 7, 01006 Vitoria-Gasteiz, Spain

Abstract. The presence of high pollution levels in the atmosphere can produce damages to human health and ecosystems. Because of this reason, the prediction of air pollutant concentration is important to prevent any potential damage. Chemistry-transport models constitute a useful tool to quantify the presence of pollutants in the atmosphere. Input information, such as meteorology and emissions, has a strong influence on model results. Many research activities are focused on trying to reduce errors affecting input information. In this paper we have applied the CHIMERE photochemical model to simulate ozone, NO2 and SO2 in Spain for two sets of meteorological fields obtained with the MM5 and WRF meteorological models. An evaluation of the performance of the CHIMERE model for both meteorological data sets is presented. Very similar air pollutant concentrations were found for the three pollutants and for the two sets of meteorological information.

Keywords: Photochemical modeling, air quality forecast, evaluation of model performance.

1 Introduction

Predicting air pollutants levels has become an important goal in environmental management in order to prevent potential damage to human health or ecosystems. Air quality models are being extensively applied as a tool to obtain predicted pollutant concentrations in the troposphere. Large efforts trying to reduce discrepancies between model predictions and observations are being done by air quality modeling community. The evaluation of the performance of models has become a common and very important practice to assess the quality of model predictions.

In Spain the use of air quality models has significantly increased over the last years ([1], [2], [3], [4], [5], [6]). A provisional forecast system in Spain was presented in [7], in the framework of the CALIOPE project financed by the Environment Ministry of Spain. Four institutions participate in this project: the Barcelona Supercomputing

108 M.G. Vivanco et al.

Center (BSC), the CIEMAT, the Earth Sciences Institute “Jaume Almera” (IJA-CSIC) and the CEAM Foundation. Air quality forecast for the Iberian Peninsula at a 4-km resolution is planned to be provided using the BSC “Mare Nostrum” computer (Figure 1), one of the most powerful supercomputers in Europe (10240 processors, 92.21 Tflops). Two air quality models have been proposed to predict air quality for the CALIOPE project: CMAQ and CHIMERE chemistry-transport models. A comparison between them at a 20-km resolution using EMEP emissions was presented also in [7]. Regarding ozone, the authors showed statistical scores under the EPA’s proposed range for both models at background EMEP stations.

Fig. 1. MareNostrum supercomputing, located in the Barcelona Supercomputing Center (BSC-CNS)

The CHIMERE chemistry-transport model has been extensively used over Europe.

A detailed description of the model configuration and performances over Europe are presented in previous studies, such as those in [8],[9],[10] and [11]. In Spain, there are several recent publications showing an evaluation of the performance of the CHIMERE model for simulations carried out covering the Iberian Peninsula ([1]). Also some studies have been focused on smaller areas of Spain at a higher resolution ([2]). In those publications meteorological fields were obtained from the MM5 model [12]. Here, we evaluate the CHIMERE air quality predictions using meteorology coming also from another model, the Weather Research and Forecasting (WRF) meteorological model ([13]). A comparison between predictions based on both MM5 and WRF models using different physical schemes for Microphysics, PBL and LSM cumulus descriptions is presented.

2 Setup of the Model

One of the sources of air quality models uncertainties is related to model input information, such as meteorological fields, emissions or land-use information. The improvement of these data is of concern for many research groups all around the world and involving different science areas.

An Evaluation of the Performance of the CHIMERE Model over Spain 109

For this study, two different meteorological models were used to obtain the meteorological fields required to simulate air pollutant concentrations: MM5 and WRF models. Table 2 presents the physical schemes for Microphysics, PBL and LSM cumulus, used for each model simulation.

Table 1. Options used in MM5 and WRF meteorological models

MM5 WRF

Dynamics ARW MM5

Microphysics WSM-3 class Reisner graupel

Radiation RRTM, Dudhia scheme RRTM, cloud radiation

PBL YSU MRF PBL

LSM (Land Soil model) Noah Land Surface Model Five-Layer LSM

Cumulus Kain-Fritsch Grell

Inititialization and

boundary conditions

NCEP-GFS analysis NCEP-GFS analysis

The V2006-par version of the CHIMERE model calculates the concentration of 44

gaseous species and both inorganic and organic aerosols of primary and secondary origin. Using both sets of meteorological fields, the CHIMERE model was applied for 2004 at a 0.2º resolution over the Iberian Peninsula (Figure 2), and 14 vertical sigma-pressure levels extending up to 500 hPa. This domain was nested to a coarser one covering an area from 10.5W to 22.5E and from 35N to 57.5 N at a 0.5 degree horizontal resolution. Boundary conditions for the coarse domain were provided from monthly 2003 climatology from LMDz-INCA model [14] for gases concentrations and from monthly 2004 GOCART model [15] for particulate species. The emissions for all the simulations were derived from the annual totals of the EMEP database for 2003 [16]. Original EMEP emissions were spatially disaggregated taking into account the land use information.

Figure 3 shows the O3 concentration maps calculated by the CHIMERE model using both sets of meteorological fields. Although in overall a similar spatial pattern is found, some differences in the ozone spatial distribution can be observed.

In order to evaluate the performance of the CHIMERE model for both meteorological models some statistics were calculated. Table 2 presents the metrics used and their definition. Parameters such as mean bias, mean normalized bias, mean normalized absolute error, root mean square error and root mean normalized square error were calculated.

Regarding ozone, only statistics for moderate-to-high ozone concentration cases (more important for human health protection) were considered by selecting predicted-observed value pairs when hourly observations were equal to or greater than the cut-off of 80 μgm-3. For NO2 and SO2 a cut-off value of 5 μgm-3 was used. The US EPA [17] suggested a range of ±5–15% as an acceptable guideline of model performance for mean normalized bias (MNB) and 35% for the mean normalized absolute error (MNAE).

110 M.G. Vivanco et al.

Fig. 2. Domain simulated with the CHIMERE model

Fig. 3. Ozone concentration over the domain at June, 30; 14:00hr

Table 2. Definition of the metrics used in the evaluation of the CHIMERE model performance

Mean bias ∑ −=−= OMOMMB iiN

)(1

Mean normalized bias ( ) ( )∑ ∑ −== − 111

i

i

i

ii

O

M

NO

OM

NMNB

Mean normalized absolute error ( )∑= −

i

ii

O

OM

NMNAE 1

Root mean square error [ ]21

21 )(∑ −= iiN OMRMSE

Root mean normalized square error [ ]21

21 )(∑= −

i

ii

O

OM

NRMNSE

N: pairs of modeled and observed concentrations Mi and Oi. The index i is over time series and over all the locations in the domain.

-10 -5 0 535

38

41

44

SP (0.2º X 0.2º)

MM5

WRF

-10 -5 0 5 10 15 2035

38

41

44

47

50

53

56

EUR (0.5º X 0.5º)

CHIMERE MODEL

0.4 -8.4 -6.4 -4.4 -2.4 -0.4 1.6 3.60

30

60

90

120

150

180

240

360

500

700

O3 - 20040630 14:00 h (microgr/m3)

MM5 + CHIMERE

-10.4 -8.4 -6.4 -4.4 -2.4 -0.4 1.6 3.635

36

37

38

39

40

41

42

43

44

O3 - 20040630 14:00 h (microgr/m3)

WRF + CHIMERE

An Evaluation of the Performance of the CHIMERE Model over Spain 111

As the model was run at a 0.2º resolution (~20 km) just background stations were selected for the comparison: 38 rural background stations, 19 suburban background stations and 22 urban background sites. The location of these background monitoring sites is presented in Figure 4.

-10.4 -8.4 -6.4 -4.4 -2.4 -0.4 1.6 3.635

36

37

38

39

40

41

42

43

44

BACKGROUND STATIONS

RURAL BACKGROUND STATIONS SUBURBAN BACKGROUND STATIONS

URBAN BACKGROUND STATIONS

Fig. 4. Location of all the background stations used in the evaluation process

3 Results of the Evaluation of the Model Performance

Statistics results for ozone are presented in Table 3, using all the available hourly 2004 observations with the cut-off mentioned before.

Very similar scores are found for the two meteorological scenarios. In general, slightly lower errors are found when using the MM5 physic parameterizations. MNB and MNAE values meet clearly the EPA-proposed criteria for both meteorological models, indicating an acceptable quality of ozone predictions. This behaviour can be observed at all the station types. The seasonal behaviour of the CHIMERE model was also evaluated by comparing hourly predictions to observations for the four seasons. Table 4 shows the statistics taking into the recorded information at all the background stations and the modelled values predicted by the model at those stations. Again, a

112 M.G. Vivanco et al.

Table 3. Evaluation of the performance of the CHIMERE model for the two sets of meteorological fields at the different background stations

Statistical measure OZONE MM5 +

CHIMERE WRF +

CHIMERE RURAL BACKGROUND MB (μg m-3) -9.6 -11.1 MNB (%) -8.4 -10 MNAE (%) 13.8 14.9 RMSE (μg m-3) 18.4 19.7 RMNSE (%) 17.0 18.3 Number of samples 111161 SUBURBAN BACKGROUND MB (μg m-3) -8.6 -9.7 MNB (%) -6.8 -8.0 MNAE (%) 13.6 14.7 RMSE (μg m-3) 18.9 20.1 RMNSE (%) 16.8 18.2 Number of samples 38860 URBAN BACKGROUND MB (μg m-3) -3.7 -5.0 MNB (%) -2.0 -3.4 MNAE (%) 13.7 14.5 RMSE (μg m-3) 17.7 18.7 RMNSE (%) 16.7 17.8 Number of samples 31977 ALL THE BACKGROUND STATIONS MB (μg m-3) -8.3 -9.7 MNB (%) -6.8 -8.3 MNAE (%) 13.5 14.5 RMSE (μg m-3) 18.1 19.3 RMNSE (%) 16.6 17.9

better agreement of the CHIMERE ozone predictions with observations is found when using MM5 meteorological fields for the four periods. Worse results are found for summer and autumn seasons.

In Figure 5 we analyse the performance of the model for each background station for ozone, NO2 and SO2. The graphic in the top of the figure shows the MNAE values for ozone at all the background stations. For all them, the MNAE presents values below the 35% EPA- recommended limit and for both sets of meteorology fields. The graphic illustrates the fact that a better agreement between model and observations is found when using the MM5 meteorology.

The other two graphics of Figure 4 include the MNAE for NO2 and SO2 at all the background stations. Again, these results represent mean values obtained using all the

An Evaluation of the Performance of the CHIMERE Model over Spain 113

Table 4. Evaluation of the seasonal behaviour of the CHIMERE model for the two sets of meteorological fields at all the background stations

Statistical measure OZONE MM5 + CHIMERE WRF + CHIMERE

WINTER MB (μg m-3) --5.0 -6.3 MNB (%) -5.1 -6.7 MNAE (%) 10.8 11.6 RMSE (μg m-3) 12.4 13.3 RMNSE (%) 13.5 14.4 Number of samples 21028 SPRING MB (μg m-3) -5.3 -6.9 MNB (%) -3.9 -5.5 MNAE (%) 12.1 12.9 RMSE (μg m-3) 16.3 17.3 RMNSE (%) 15.0 16.1 Number of samples 81341 SUMMER MB (μg m-3) -12.1 -13.0 MNB (%) -9.9 -10.8 MNAE (%) 16.2 17.3 RMSE (μg m-3) 22.0 23.2 RMNSE (%) 19.31 20.6 Number of samples 63781 AUTUMN MB (μg m-3) -14.5 -15.1 MNB (%) -14.8 -15.6 MNAE (%) 16.9 18.2 RMSE (μg m-3) 19.6 20.6 RMNSE (%) 19.8 21.0 Number of samples 15838

hourly concentrations higher than the chosen cut-off at each background station. Normalized errors have higher values than those for ozone, as observation values are much lower. Regarding NO2, better results can be obtained for the MM5 meteorological fields. For SO2 errors are more similar for both meteorological fields. As these results were obtained for 1-year period, further research would be interesting to be done for specific meteorology conditions and seasons.

Figure 6 has been included to illustrate the very similar temporal behavior of the model for the two meteorological data sets. The figure contains time series showing observed and predicted hourly ozone concentrations at two rural background stations (“Barcarrota” and “Campisábalos” EMEP stations), for a 5-days period in 2004. This period starts on July, 29.

114 M.G. Vivanco et al.

Ozone: MNAE (%) - 2004

0

10

20

30

RB SB UB

%

MM5 + CHIMERE WRF + CHIMERENO2 : MNAE (%) - 2004

0

25

50

75

100

125

RB SB UB

%

MM5 + CHIMERE WRF + CHIMERESO2 : MNAE (%) - 2004

0

25

50

75

100

125

RB SB UB

%

MM5 + CHIMERE WRF + CHIMERE

Fig. 5. Mean Normalized absolute error for O3, NO2 and SO2 at all the background sites taking into account hourly information in 2004

RB: Rural Background SB: Suburban background UB: Urban Background

An Evaluation of the Performance of the CHIMERE Model over Spain 115

2004-07-29 2004-07-30 2004-07-31 2004-08-01 2004-08-02 2004-08-03TIME

0

20

40

60

80

100

120

140

160

180

200

220

240

260

280

300

O3

CO

NC

EN

TR

AT

ION

(μg

/m3)

OzoneObservationsMM5-CHIMEREWRF-CHIMERE

06016999 - BARCARROTA

2004-07-29 2004-07-30 2004-07-31 2004-08-01 2004-08-02 2004-08-03TIME

0

20

40

60

80

100

120

140

160

180

200

220

240

260

280

300

O3

CO

NC

EN

TR

AT

ION

(μg

/m3 )

OzoneObservationsMM5-CHIMEREWRF-CHIMERE

19061999 - CAMPISABALOS

Fig. 6. Time series of observed and predicted ozone for an episode using WRF and MM5 meteorological fields

4 Conclusions

The use of an air quality model for air quality management purposes requires a previous analyse of the quality of the model results in order to rely on model predictions. This analyse must be based on a comparison of modelled pollutant concentrations with observations for past events. In Spain an air-quality forecast system is being prepared in the framework of the CALIOPE project. The CHIMERE is one of the models planned to be used to provide air quality predictions at a 4-km resolution. In this paper we present an evaluation of the predictions of the CHIMERE model using meteorological fields coming from two different meteorological models.

116 M.G. Vivanco et al.

The aim of this study was to determine if predicted pollutant concentrations were significantly different for the two meteorological models. The evaluation was done considering all the 2004 hourly predicted and observed air pollutant concentrations at background stations. The CHIMERE model predicts acceptable ozone levels for both sets of meteorological fields, showing statistical errors inside the range mentioned in the EPA 2000 guidelines. For NO2 and SO2 normalized errors are higher, as observed values present lower values. A comparison of model predictions for the two meteorological models indicates that very similar pollutant concentrations are found for the two meteorological models. Just slightly lower errors are found for the MM5 model configuration. As this evaluation has been done for 1-year period, further research should be done for specific meteorological conditions.

Acknowledgments. This work is funded by the CALIOPE project of the Spanish Ministry of the Environment (441/2006/3-12.1 and A357/200/2-12.1).

References

1. Vivanco, M.G., Palomino, I., Vautard, R., Bessagnet, B., Martín, F., Menut, L., Jiménez, S.: Multi-year assessment of photochemical air quality simulation over SPAIN. Environmental Modelling & Software 24(1) (2009); ISSN:1364-8152

2. Vivanco, M.G., Correa, M., Azula, O., Palomino, I., Martín, F.: Influence of Model Resolution on Ozone Predictions over Madrid Area (Spain). In: Gervasi, O., Murgante, B., Laganà, A., Taniar, D., Mun, Y., Gavrilova, M.L. (eds.) ICCSA 2008, Part I. LNCS, vol. 5072, pp. 165–178. Springer, Heidelberg (2008)

3. Vivanco, M.G., Palomino, I., Plaza, J., Artíñano, J., Pujadas, M., Bessagnet, B.: An evaluation of SOA modelling in the Madrid Metropolitan area. In: Air Pollution XVI, pp. 115–123. WIT Press (2008)

4. Jiménez, P., Lelieveld, J., Baldasano, J.M.: Multiscale modeling of air pollutants dynamics in the northwestern Mediterranean basin during a typical summertime episode. Journal of Geophysical Research 111, D18306 (2006), doi:10.1029/2005JD006516

5. Pérez, C., Nickovic, S., Baldasano, J.M., Sicard, M., Rocadenbosh, F., Cachorro, V.E.: A long Saharan dust event over the western Mediterranean: Lidar, sun photometer observations and regional dust modelling. Journal of Geophysical Research 111, D15214 (2006), doi:10.1029/2005JD006579

6. Palacios, M., Martín, F., Aceña, B.: Estimate of potentially high ozone concentration areas in the centre of the Iberian Peninsula. Int. J. Environment and Pollution 24(1/2/3/4), 260–271 (2005)

7. Baldasano, J.M., Jiménez-Guerrero, P., Jorba, O., Pérez, C., López, E., Güereca, P., Martín, F., Vivanco, M.G., Palomino, I., Querol, X., Pandolfi, M., Sanz, M.J., Diéguez, J.J.: An operational air quality forecasting system for the Iberian Peninsula, Balearic Islands and Canary Islands. First annual evaluation and ongoing developments: Advances in Science and Research 2, 89–98 (2008)

8. Schmidt, H., Derognat, C., Vautard, R., Beekmann, M.: A comparison of simulated and observed ozone mixing ratios for the summer of 1998 in western Europe. Atmospheric Environment 35, 6277–6297 (2001)

9. Bessagnet, B., Hodzic, A., Vautard, R., Beekmann, M., Rouil, L., Rosset, R.: Aerosol modelling with CHIMERE –first evaluation at continental scale. Atmospheric Environment 38, 2803–2817 (2004)

An Evaluation of the Performance of the CHIMERE Model over Spain 117

10. Vautard, R., Martin, D., Beekmann, M., Drobinski, P., Friedrich, R., Jaubertie, A., Kley, D., Lattuati, M., Moral, P., Neininger, B., Theloke, J.: Paris emission inventory diagnostics from ESQUIF airborne measurements and a chemistry transport model. Journal of Geophysics Research 108(D17), 8564–8585 (2003)

11. Hodzic, A., Vautard, R., Bessagnet, B., Lattuati, M., Moreto, F.: On the quality of long-term urban particulate matter simulation with the CHIMERE model. Atmospheric Environment 39, 5851–5864 (2005)

12. Grell, G.A., Dudhia, J., Stauffer, D.R.: A Description of the Fifth-Generation Penn State/NCAR Mesoscale Model (MM5).NCAR/TN-398 + STR. NCAR TECHNICAL NOTE (1995)

13. Skamarock, W.C., Klemp, J.B., Dudhia, J., Gill, D.O., Barker, D.M., Duda, M.G.: Xiang-Yu Huang, Wang, W., Powers, J.G.: A Description of the Advanced Research WRF Version 3 Mesoscale & Microscale Meteorology Division · National Center for Atmospheric Research (2009)

14. Hauglustaine, D.A., Hourdin, F., Jourdain, L., Filiberti, M.-A., Walters, S., Lamarque, J.-F., Holland, E.A.: Interactive chemistry in the Laboratoire de Météorologie Dynamique general circulation model: Description and background tropospheric chemistry evaluation. J. Geophys. Res. 109 (2004), doi:10.1029/2003JD003957

15. Chin, M., Ginoux, P., Kinne, S., Holben, B.N., Duncan, B.N., Martin, R.V., Logan, J.A., Higurashi, A., Nakajima, T.: Tropospheric aerosol optical thickness fromt he GOCART model and comparisons with satellite and sunphotometer measurements. J. Atmos. Sci. 59, 461–483 (2002)

16. Vestreng, V., Breivik, K., Adams, M., Wagener, A., Goodwin, J., Rozovskkaya, O., Pacyna, J.M.: Inventory Review 2005, Emission Data reported to LRTAP Convention and NEC Directive, Initial review of HMs and POPs, Technical report MSC-W 1/2005 (2005), ISSN 0804-2446

17. Tesche, T.W., Georgopoulos, P., Seinfeld, J.H., Cass, G., Lurmann, F.W., Roth, P.M.: Improvement of procedures for evaluating photochemical models. Draft final report prepared for California Air Resources Board, Sacramento, CA by Radian (1990)