HYDROLOGICAL PROCESSESHydrol. Process. (2013)Published online in Wiley Online Library(wileyonlinelibrary.com) DOI: 10.1002/hyp.9882

Implementation of an automatic calibration procedure forHYDROTEL based on prior OAT sensitivity and

complementary identifiability analysis

Médard Bouda,1 Alain N. Rousseau,1* Silvio J. Gumiere,2 Patrick Gagnon,1

Brou Konan3 and Roger Moussa41 Institut National de la Recherche Scientifique (Centre Eau, Terre et Environnement), Université du Québec, 490 rue de la Couronne, Québec City,

QC, Canada, G1K 9A92 Département des sols et de génie agroalimentaire, 2480, boulevard Hochelaga, Université Laval, Québec City, QC, Canada, G1V 0A6

3 Hydroelectric forecasts and data quality, Production Planning Directorate, Hydro-Québec, 75 boul. René-Lévesque ouest, 9e étage, Montréal,Québec City, QC, Canada, H2Z 1A4

4 INRA-LISAH, Institut National de la Recherche Agronomique, 2, Place Pierre Viala, Montpellier, France, 34060

*CSci490E-m

Co

Abstract:

Efficiency of hydrological models mostly depends on the quality of the calibration performed prior to use. In this paper, anautomatic calibration framework for the distributed hydrological model HYDROTEL is proposed. The calibration procedurewas performed for three watersheds characterized with different hydroclimatological conditions: the Sassandra located in IvoryCoast, Africa, and the Montmorency and Beaurivage watersheds located in Quebec (Canada). Results of one-a-time (OAT)sensitivity analysis showed that the order of the most sensitive parameters differs for each watershed. Thus, the sensitivitydepends on the hydroclimatic and physiographic characteristics of the watersheds. Co-linearity indices showed that all modelparameters were identifiable, that is, none of the studied parameters could be explained by a combination of the otherparameters. Following these findings, an automatic calibration was run. Results indicated there was good agreement betweensimulated and measured streamflows at the outlet of each watershed; Nash–Sutcliffe efficiency (NSE) ranging between 0.77and 0.92 and R2 ranging from 0.87 to 0.97. When comparing NSE and R2 values obtained using a process-oriented,multiple-objective, manual calibration strategy, a slight increase in model efficiency was reached with the automaticcalibration procedure (4.15% for NSE and 2.95% for R2) improving predictions of peak flows for the Montmorency andBeaurivage watersheds (temperate climate conditions) and flows beyond the rainfall season in the Sassandra watershed. Theproposed automatic calibration procedure introduced in this paper may be applied to other distributed hydrological model.Copyright © 2013 John Wiley & Sons, Ltd.

KEY WORDS sensitivity; identifiability; HYDROTEL; automatic calibration; parameter estimation

Received 6 June 2012; Accepted 29 April 2013

INTRODUCTION

Distributed hydrological models such as HYDROTEL(Fortin et al., 2001a, b; Turcotte et al., 2007, 2003) arevaluable tools to understand, reproduce, and predict thebehavior of watersheds (e.g. Moussa et al., 2002; Quilbéet al., 2008; Savary et al., 2009; Gumiere et al., 2010;Rousseau et al., 2011b, 2013). For example, hydrologicalmodels have been developed to: (i) improve ourunderstanding of hydrological processes at the watershedscale; (ii) predict reservoir inflows for various water uses

orrespondence to: Alain N. Rousseau, Institut National de la Rechercheentifique (Centre eau, terre et environnement), Université du Québec,rue de la Couronne, Québec, Qc, Canada, G1K 9A9ail: alain.rousseau@ete.inrs.ca

pyright © 2013 John Wiley & Sons, Ltd.

(e.g. hydroelectric production, water supply for domestic,recreational, agricultural, and industrial activities); (iii)assess public safety issues such as flood risk management;and (iv) evaluate impacts of human activities on waterresources (e.g. land cover and climate changes). Theapplication of a hydrological model in a new study oftenrequires calibration. Various calibration procedures andalgorithms have been extensively documented in theliterature (e.g. Duan et al., 1992; Yapo et al., 1998; Guptaet al., 2003). These parameter estimation procedures canbe classified into two categories: the manual or trial-and-error calibration and the automatic calibration. Manualcalibration approaches require the user to adjust a fewparameters interactively in successive model runs. Thiscan be time consuming for new users of complex models.Its success depends highly on the user’s experience and

M. BOUDA ET AL.

knowledge of the watershed, the modeling assumptions,and the computational algorithms. Nonetheless, manualcalibration has some advantages as well, such as,benefiting from the users’ past experience with the modelunder various forcing conditions and prior knowledge ofrobust parameter sets (Bahremand and De Smedt, 2008).Unlike manual calibration, automatic calibration uses asearch algorithm to set parameter values and offers anumber of advantages such as: speed, objectivity, andextent in terms of searching the parameter space.Nevertheless, when compared to manual calibration,Boyle et al. (2000) have shown that automatic proceduresare not always satisfactory. Regardless of the calibrationprocedure adopted (manual or automatic), it is alwaysrecommended to proceed a priori with a sensitivityanalysis before undertaking model calibration (Spear andHornberger, 1980; Saltelli et al., 2000a, b; Tarantola andSaltelli, 2003; Saltelli et al., 2004; Yang, 2011).Several sensitivity analysis methods have been

proposed, and they are generally categorized (Saltelliet al., 1999, 2004) into global or local methods. Localmethods rely on differential analysis or nominal rangemethods (Helton, 1993; Frey and Patil, 2002). Differentialanalysis approximates partial derivatives of model outputswith respect to the imposed perturbations in model inputsand then takes the numerical values of these derivatives asmeasures of sensitivity. Nominal range methods startfrom a reference output obtained with a referenceparameter set, then express sensitivity as the ratio ofrelative output variations to relative input variations. Aknown drawback of local sensitivity analysis methods istheir incapacity to identify interactions between inputparameters, at the risk of erroneous sensitivityestimations, especially for models with largeparameterization or highly non-linear characteristics.Under such conditions, global sensitivity methods areoften more suitable.Global sensitivity methods such as regional sensitivity

analysis (Young, 1978), variance-based methods (Saltelliet al., 2000a, b), and regression-based approaches (Spearet al., 1994), to name a few, are designed to fully explorethe parameter space within predefined, plausibleparameter ranges. It should be noted that it is virtuallyimpossible to explore the entire parameter space withoutsimplifying the assumptions for complex models whichoften weakens the technique. Furthermore, globalsensitivity methods may require enormous computationaltime (Scott and Saltelli, 1997; Saltelli, 1999).The sensitivity analysis provides helpful information

to guide the calibration procedure, but in order to avoidor limit the non-uniqueness of model parameters set aco-linearity index from an identifiability analysis mustalso be computed. A parameter set is said identifiableif the two following conditions are fulfilled

Copyright © 2013 John Wiley & Sons, Ltd.

(Beven, 2000; Brun et al., 2001; Reichert, 2006a, b):(i) the model output has to be sufficiently sensitive toindividual changes of each parameter in the set, and(ii) changes in the model output due to changes insingle parameters may not be coincidently cancelled byappropriate changes in other parameters in the set.Identifiability problems related to the use of mechanis-tic models are widely reported in the literature(Kleissen et al., 1990; Brun et al., 2001, 2002). Oneof the strategies commonly used to perform anidentifiability analysis requires the partitioning ofmodel parameters into subsets of parameters that needto be estimated from given observations, and a subset ofparameters that are fixed at prior values. This strategyensures that the remaining model parameters areidentifiable.Although automatic calibration, sensitivity, and

identifiability analyses are beneficial to hydrologicalsimulations, these tools are not widely used. For example,for the distributed hydrological model HYDROTEL,Turcotte et al. (2003) introduced a four-step manualtrial-and-error, process-oriented, multiple-objective cali-bration strategy whereby parameters affecting objectivescharacterized by long time scales (e.g. minimization ofannual runoff errors and spring runoff errors) arecalibrated first while those characterized by short timescales are calibrated last (e.g. minimization of summerand fall peak flow errors). In Turcotte et al.’s procedure(2003), the calibration parameters are grouped accordingto each one of the following objectives: (i) prolongedsummer drought recessions, (ii) annual and monthly flowvolumes, (iii) summer and fall high flows, (iv) high flowsynchronization, and (v) spring runoff resulting fromsnowmelt. Any loss in model performance is compensat-ed by readjusting previously calibrated parameters.Although these developments have proven to be quiteuseful for subsequent model applications (e.g. Rousseauet al., 2011b, 2013), they were based solely on themodelers’ and users’ experience and applied withoutformerly performing complete sensitivity andidentifiability analyses.The objective of this article is to introduce an automatic

calibration procedure for the distributed hydrologicalmodel HYDROTEL. The automatic calibration is pre-ceded by formal sensitivity and identifiability analyses todetermine parameters to calibrate. Note that the proposedcalibration procedure could be applied to any distributedhydrological model. The article is structured as follows:‘Materials and methods’ describes: (i) HYDROTEL andits parameters, (ii) the sensitivity and identifiabilityanalyses, (iii) the automatic calibration procedure applied,and (iv) the three watersheds used for this study. This isfollowed by the ‘Results and discussion’ section and thenthe ‘Conclusion’.

Hydrol. Process. (2013)

Table I. The key parameters of HYDROTEL

Symbol Parameters Units

FFCO Melt factor for coniferous forestsa mm/d.�CFFFE Melt factor for deciduous forestsa mm/d.�CFFFO Melt factor for open areasa mm/d.�CSTCO Threshold air temperature for

melt in coniferous forestsa�C

STFE Threshold air temperature formelt in deciduous forestsa

�C

STFO Threshold air temperature formelt in open areasa

�C

TFN Melt rate at the snow–soil interfacea mm/dCC Compaction coefficienta -FETP Potential evapotranspiration

multiplication factorb-

Z1 Depth of the lower boundary ofsoil layer#1b

m

Z2 Depth of the lower boundary ofsoil layer#2b

m

Z3 Depth of the lower boundary ofsoil layer#3b

m

REC Recession coefficientb m/hDES Extinction coefficientb -VHMAX Maximum variation of humidityb -TPPN Threshold air temperature for

partitioning solid andliquid precipitation

�C

GVP Vertical gradient of precipitation mm/100mGVT Vertical gradient of temperature �C/100m

a For a complete description of snow parameters, the reader is referred toTurcotte et al. (2007).b For a complete description of soil parameters, the reader is referred toFortin et al. (2001a).

AUTOMATIC CALIBRATION FRAMEWORK FOR HYDROTEL

MATERIAL AND METHODS

HYDROTEL

HYDROTEL is a process-based, continuous, distribut-ed hydrological model, and it is currently used for inflowand hydrological forecasting by Hydro-Quebec, Quebec’smajor power utility, and the Quebec HydrologicalExpertise Centre which is in charge of the managementand safety of publicly owned dams (Turcotte et al., 2004).It was designed to use available remote sensing andGIS data at either a 3-h or a daily time step. It isbased on the spatial segmentation of a watershed into‘relatively homogeneous hydrological units’ (RHHUs)and interconnected river segments (RSs) draining theaforementioned units. A semi-automatic, GIS-basedframework called PHYSITEL (Turcotte et al., 2001;Rousseau et al., 2011a) allows easy watershedsegmentation and parameterization of the hydrologicalobjects (RHHUs and RSs).The model is composed of six computational modules,

which run in successive steps. Each module simulates aspecific hydrological process. The first module deals withthe interpolation of meteorological variables, includingliquid or solid precipitation, minimum and maximumtemperatures, relative humidity, wind speed, and solarradiation. It is performed for each RHHU, with due carefor vertical gradients, if desired, using either Thiessenpolygons or weighted mean of nearest three stations. Thesecond module is the snow module. It simulates dailychanges of mean snowpack characteristics (thickness,water equivalent, mean density, thermal deficit, liquidwater content, temperature) using a modified energybudget approach developed by Riley et al. (1973) (for acomplete description, see Turcotte et al., 2007). The thirdmodule calculates the potential evapotranspiration (PET)for each RHHU using a choice of equations depending onavailable meteorological data: (i) Thornthwaite (1948),(ii) Linacre (1977), (iii) Penman–Monteith (Monteith1965), (iv) Priestley and Taylor (1972), or (v) anotherequation developed by Hydro-Quebec. In the fourthmodule, the soil column moisture module, rainfall, and/orsnow melt are partitioned into infiltration and runoff. Forsimulation purposes, the soil column is divided into threelayers. The first surface layer is relatively shallow so as tocorrespond to the soil layer affected by evaporation overbare ground and to that, much smaller, for which soilmoisture can be estimated by remote sensing. While thefirst layer controls infiltration, the second layer can beassociated with interflow and the third layer with baseflow. In the fifth module, the surface and subsurface flowtransfer module, the available amount of free water fromeach soil layer of a RHHU is summed up and routed tothe corresponding downslope RS using an RHHU-specific geomorphological unit hydrograph (GUH). This

Copyright © 2013 John Wiley & Sons, Ltd.

hydrograph is determined once for each RHHU using thekinematic wave approximation of the complete Saint-Venant system of equations, but it can be recomputed ifnecessary, that is when there is a change in land cover forexample. The shape of the specific GUH is determined byrouting a reference water depth over all cells of a RHHUbased on the slope and roughness of each cell or a land-covered, weighted-average of the latter parameters. Theresulting values are considered as lateral inflow to the RSdraining the RHHU. The sixth and last module is thechannel routing module. Two simulation options areavailable: the diffusive and the kinematic wave equations,both being approximations of the Saint-Venant equations.For more details on these aspects of HYDROTEL, thereader is referred to Fortin et al. (2001a) and Turcotteet al. (2007).

Model parameters. The main calibration parameters ofHYDROTEL can be subdivided into two groups, forpractical issue; they are presented in Table I. The firstgroup includes the snow parameters (the first eight), andthe second group includes the soil parameters (the

Hydrol. Process. (2013)

M. BOUDA ET AL.

following seven). The last three individual parameters arethe interpolations of temperature and precipitationaccording to the mean of three nearest stations. Eachparameter was analyzed independently by keepingconstant all other parameters. However, the thresholdair temperature that separates solid and liquid precipita-tion was included in the snow parameter group, whereasthe vertical gradient of precipitation and the verticalgradient of temperature were included as parts of the soilparameter group.

The proposed calibration procedure

Figure 1 presents the flowchart of the proposedcalibration procedure. The procedure may be viewed asa three-step procedure as follows: (i) sensitivity analysis,(ii) identifiability analysis, and (iii) automatic calibrationbased on the findings of steps (i) and (ii). The procedurewas performed using UNCertainty SIMulation(UNCSIM), a parameter estimator and uncertaintyanalysis tool (Reichert, 2001, 2005, 2006a, b) byadjusting the most sensitive model parameters of eachwatershed. The model parameters were both estimatedand analyzed according to routines provided byUNCSIM.

Sensitivity analysis: local method. The one-parameter-at-a-time (OAT) approach used in this work is particularly

Figure 1. Flowchart of the proposed calibration procedure. Theequifinality concept conveys the idea that acceptable simulation results

can be obtained from different parameter sets

Copyright © 2013 John Wiley & Sons, Ltd.

well suited for complex models with little or noidentifiability issues (Cheviron et al., 2010). In thefollowing paragraphs, a brief description of the localapproach used is provided.A deterministic hydrologic model can be written in the

form of a function:

y ¼ y yð Þ (1)

by assuming that model results y = (y1,....,yn) are adeterministic function of the model parameters y= (y1,ym). In general form, model results are a function of bothmodel inputs and parameters, but for convenience ofpresentation, replaced here by a simplified formulation. Afirst-order Taylor development approximates Equation (1)in the vicinity of y* by:

y yð Þ � y y�ð Þ þ @y

@y

����y�

� �Ty� y�ð Þ (2)

Associated with Equation (2), Brun et al. (2002)defined the sensitivity function as:

Si;j ¼ ΔyjSCi

@yi@yj

i ¼ 1; . . . n; j ¼ 1; . . .m (3)

where Δyj represents the uncertainty range of theparameter j, SCi is a scale factor with the same physicaldimension as the corresponding observation and S= (Sij)represents the sensitivity matrix. The norm of eachcolumn Sj= (S1,j, . . .,Sn,j)

T provides a measure of theimportance of individual parameters. This norm iscomputed according to Equation (4) below (Brun et al., 2002):

dj yð Þ ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1n

Xni¼1

S2i;j

s¼ 1ffiffiffi

np Sj�� �� (4)

A large norm ‖Sj‖ indicates that a change of Δyj in theparameter yj will have an important effect on the modeloutcome. Parameters are ranked according to the djmeasures in decreasing order.Based on the findings of Brun et al. (2001), we chose

Δyj as� 25% of the absolute value of y*, the nominalvalue being the optimum parameter determined during apreliminary manual calibration procedure. Also accordingto Brun et al. (2001), SCi was chosen as the measuredstreamflow. The assessment of the ranges of parameters,the scale factor, and the reference values of the parametersreflects the users’ knowledge of the model.

Identifiability of a parameter subset. The sensitivityanalysis is not sufficient to estimate model parameters. In

Hydrol. Process. (2013)

AUTOMATIC CALIBRATION FRAMEWORK FOR HYDROTEL

some cases, parameters are strongly correlated, and anidentifiability analysis is often recommended to determinethe subset of parameters which are identifiable.The identifiability of a parameter depends on the

uniqueness and stability of the inverse solution and on thequantity and quality of the observation data. Identifiabilityanalysis techniques were originally developed to assessthe uniqueness and accuracy with which the parameters ofa given model could be determined for a givenmeasurement layout.Various definitions of parameter identifiability within a

distributed parameter system can be found in the literature(e.g. Brun et al., 2001; Matott et al., 2009). The idea hereis to identify those parameters that have a cancelling effecton model output. This may be either symptomatic of a poormodel structure or problem of parameter identification.Either way, the parameters are poorly identifiable becausethey have a dependence behavior (Brun et al., 2001). Iftheir values cannot be easily measured, it is possible to linka priori information on the parameters to a priordistribution. This approach does not require the parametersto be identifiable. Another way is to partition the parameterset into subsets where one subset is estimated with theobserved data and the other is fixed to a priori values.However, the high degree of subjectivity used to build thesubsets represents the main drawback.From a general mathematical point of view, the

primary objective is to establish whether the parameterset is linearly dependent, that is, if there is a vector b suchthat Sb= 0, where b= (b1,b2, . . .,bm)T with ‖b‖ 6¼ 0 and Sis the sensitivity matrix of dimension n�m. If Sb= 0 isnot satisfied, an alternative approach must be sought.Several approaches have been proposed, but the most

effective is to find the minimum of the Sb vector normunder ‖b‖ = 1 constraint (Belsley, 1991). This minimum isreached if b is equal to the normalized eigenvectorassociated with the smallest eigenvalue l of STS. The co-linearity index g for a subset of parameters K is computedas follows (Brun et al., 2002):

gK ¼ 1

min bk k¼1eS Kð Þ

b��� ��� ¼ 1ffiffiffiffiffiffi

lKp (5)

where eS is the matrix whose elements are given by eSi;j ¼Si;jSjk k and eS Kð Þ is the subset of eS containing only the column

referring to the subset of parameters K. This procedure canbe applied on any subset in the parameters space. Brunet al. (2002) established that if g exceeds an empiricalthreshold (between 10 and 15), then the correspondingparameter subset is not easily identifiable; in fact, suchsmall eigenvalues make numerical inversion impossible.Brun et al. (2001) indicated that due to the normalization,the collinearity index g does not depend on the choice of

Copyright © 2013 John Wiley & Sons, Ltd.

Δyj’s. The determinant measure r introduced in Brun et al.(2002) is another important measure that combinesinformation of the sensitivity and identifiability analyses.It is defined for the whole parameter set as

r ¼ det STS� �1=2m ¼

Ymj¼1

lj

!1=2m

(6)

where, det( ) represents the determinant function and the ljare the eigenvalues of STS. Themeasure r can be calculatedfor any subset of parameters as well. As described by Brunet al. (2002), a high value of r indicates high sensitivity andgood identifiability of the parameter set or the parametersubset. As mentioned before, sensitivity and identifiabilityanalyses, as well parameter estimation can be performedwith UNCSIM (Reichert, 2005, 2006a, b). The outcomes y(y) are obtained after running HYDROTEL.

Automatic calibration by maximum likelihood estimation.The automatic calibration procedure was performedby coupling HYDROTEL with UNCSIM maximumlikelihood parameter estimation procedure. It uses theshuffled complex optimization algorithm (Duan et al.,1993), which includes the simplex algorithm developedby Nelder andMead (1965). According to Reichert (2005),it is a very robust algorithm that can be used to performa global search. The best set of parameters is selectedwithin reasonable ranges by adjusting the values untildiscrepancies between model outcomes and thosemeasured in the field are reduced to aminimum (Bahremandand De Smedt, 2008).UNCSIM communicates with HYDROTEL using the

model input and output files. During the calibration, modelparameters are adjusted automatically according to theUNCSIM optimization, minimizing objective functions.The process is repeated until a criterion is satisfied (e.g.maximum number of iterations; relative tolerance of the lossfunction).The principal prerequisite of UNCSIM is that a

deterministic hydrologic simulation model can be de-scribed in the form of Equation (1). Then, assumingresiduals between observations and model results areidentically, independently, and normally distributed, thelikelihood function is defined as:

f YM Yj y yjð Þ ¼Yni¼1

1ffiffiffiffiffiffi2p

p 1sexp � 1

2

yti � yMti yð Þh i

s2

20B@1CA

264375(7)

where yti and yMti yð Þ are the observed and the simulatedstreamflow values at time ti, respectively; and s is thestandard deviation of the residuals. However, some of the

Hydrol. Process. (2013)

M. BOUDA ET AL.

above assumptions may be violated. That is why somerecent techniques have been developed to resolve thisproblem (Yang et al., 2007a). In this study, theautoregressive error model proposed by Yang et al.(2007a, b) was used. In this case, the likelihood functionis defined as follows:

f YM jY yjyð Þ ¼ 1ffiffiffiffiffiffi2p

p 1sexp �1

2

g yti� �� g yMti yð Þ

h i2s2

0B@1CA:

dg

dy

���� ����y¼yt1

:Yni¼2

1ffiffiffiffiffiffi2p

p 1

s

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1� exp �2

ti � ti�1

t

r exp �12

g yti� �� g yMti yð Þ

� g yti�1

� �� g yMti�1yð Þ

h iexp �ti � ti�i

t

h i2s2 1� exp �2

ti � ti�1

t

0B@

1CA:dg

dy

���� ����y¼yti

26643775(8)

where t is the characteristic correlation time, and thefunction g(y) represents the Box–Cox transformation withparameters a1 and a2 (Box and Cox, 1982):

g yð Þ ¼yþ a2ð Þa1 � 1

a1a1 6¼ 0

ln yþ a2ð Þ a1 ¼ 0

8><>: ;

anddg

dy¼ yþ a2ð Þa1�1

(9)

In the present study, statistical assumptions of thelikelihood function (8) were tested using the empiricaldistribution of the standardized observed ‘innovations’ oftransformed observations g

�yti�

and the transformedmodel results g

�yMti yð Þ� , as suggested by Yang et al.

(2007b):

iti y; yð Þ ¼g yti� �� g yMti yð Þ

� g yti�1

� �� g yMti�1yð Þ

h iexp � ti�ti�i

t

� �sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1� exp �2 ti�ti�1

t

� �q (10)

As proposed by Yang et al. (2007b), the tests includeplots of the innovation time series, the autocorrelationfunction of innovations, the cumulative periodogram, anda normal quantile–quantile plot of innovations. Tocomply with the statistical assumptions of the likelihoodfunction, we performed the proposed tests (see Boudaet al., 2012). The initial parameter set used for the

Copyright © 2013 John Wiley & Sons, Ltd.

automatic calibration procedure was provided by themanual trial-and-error, process-oriented, multiple-objective calibration strategy developed by Turcotteet al. (2003).The objective criteria chosen to calibrate HYDROTEL

were:

NSE ¼ 1�

Xni¼1

yti � yMti yð Þ 2Xni¼1

yti � y� �2 (11)

and

R2 ¼

Xni¼1

yti � y� �

yMti yð Þ � yM yð Þ

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXni¼1

yti � y� �2Xn

i¼1

yMti yð Þ � yM yð Þ 2s

2666643777752

(12)

where NSE is the Nash–Sutcliffe coefficient, R2 is thecoefficient of determination between modeled andobserved values, y is the mean of observed streamflowvalues, and yM yð Þ is the mean of modeled values. As

discussed by Schaefli and Gupta (2007) and Krause et al.(2005), the use of NSE is not paramount although widelyused by hydrological modelers. Indeed, when the manualcalibration on the study watersheds was performed it wasnot exclusively used. An initial calibration was performedmanually as described by Turcotte et al. (2003). Themetrics used to meet the calibration objectives which

Hydrol. Process. (2013)

AUTOMATIC CALIBRATION FRAMEWORK FOR HYDROTEL

focused on sequentially minimizing errors betweenobserved and modeled (i) prolonged summer droughtrecessions, ( ii) annual and monthly flow volumes, (iii)summer and fall high flows, (iv) high flowsynchronization, and (v) spring runoff resulting fromsnowmelt, were residuals of flow volumes for (i) and (ii)and RMSE for the others. Specific groups of parameterswere assigned to each one of these objectives. Parametersaffecting objectives characterized by long time scaleswere calibrated first while those characterized by shorttime scales were calibrated last. Any loss in modelperformance was compensated by readjusting previouslycalibrated parameters. Repeating the process until asatisfactory model performance was reached.Furthermore, PBIAS (percent bias) values were also usedas supplemental means of assessing the calibration. Aseloquently illustrated by Schaefli and Gupta (2007), anothermeans of assessing a model would be to compare NSEvalues with those characterizing simple benchmark models.Although quite interesting, application of this approach toassess HYDROTEL was beyond the scope of this paper.

Case study: three watersheds located in substantiallydifferent hydroclimatic regions

Montmorency watershed. With an area of 1150 km2,the Montmorency watershed headwaters are at Lac DesNeiges in the north. The Montmorency River discharges

Figure 2. Locations of the Montmoren

Copyright © 2013 John Wiley & Sons, Ltd.

into the St. Lawrence River to the south, via theMontmorency Falls in Quebec, Canada (Figure 2). Keycharacteristics of the watershed, such as climate,topography, land cover, number of RHHUs, andhydrometeorological data are listed in Table II(Rousseau et al., 2008). The model incorporates informa-tion from a 25-m resolution digital elevation model (DEM)created from altitude information sheets available fromGeobase (http://www.geobase.ca/geobase/fr/) and eightmeteorological stations. The maximum elevation within thewatershed is 1127m above sea level with a mean slope ofapproximately 14%. The annual average precipitation overthe watershed is 975mm but, approximately 1500mm, in thenorthern portion of the watershed. The southern part of thebasin is urbanized and receives less precipitation withapproximately 320mm of solid precipitation compared to720mm in the northern part (Conseil de Bassin de la rivièreMontmorency, 2006). As shown in Table II, the watershedwas discretized into 366RHHUswith average area of 314ha.

Beaurivage watershed. The Beaurivage watershed islocated south of Québec City, Canada (Figure 3). It is an87-km long tributary of the Chaudière River. Thewatershed has an area of 718 km2, 36% of which beingused for agricultural activities. This watershed isrepresentative of Quebec’s agricultural landscape(Rousseau et al., 2013). The land cover classes and other

cy watershed and observation sites

Hydrol. Process. (2013)

Table II. Main characteristics of each watershed: Montmorency, Beaurivage, and Sassandra

Montmorency Beaurivage Sassandra

Hydroclimaticcharacteristics

Annual liquidprecipitation (mm/yr)

975 875 1750

Annual solidprecipitation (mm/yr)

340 250 0

Rain period May–October May–October June–OctoberSnow period November–April November–April -Climate Humid continental Humid continental TropicalAnnual averagetemperature (�C)

2.4 4.3 27

Topographic andland use characteristics

Drainage area (km2) a 1150 (21, 33, 46) 709 (45, 25, 30) 35 400 (45, 30, 25)Mean slope 0.14 0.01 0.06Elevation range (m) 246–1127 60–685 130–650Drainage density (km�1) 0.728 0.975 0.12% agricultural land 0.3 36 9% forest land 83 57 55Location Quebec (Canada) Quebec (Canada) Ivory Coast (West Africa)

Simulation Data record period 2001–2006 1980–1985 1988–1993Number ofRHHUsb (km2)d

366 (3.14) 675 (1.05) 407 (86.97)

DEMd Resolution 25 m 20 m 50 m

a (% of open area, deciduous forest, coniferous forest);b Relatively homogeneous hydrological unit (HYDROTEL’s computational units for water balance)c Average sized Digital elevation model

Figure 3. Location of the Beaurivage watershed and observation sites

M. BOUDA ET AL.

Copyright © 2013 John Wiley & Sons, Ltd. Hydrol. Process. (2013)

AUTOMATIC CALIBRATION FRAMEWORK FOR HYDROTEL

characteristics of the Beaurivage watershed are providedin Table II. Streamflow has been monitored dailysince 1972 using an automated gauging station locatedin Saint-Étienne-de-Lauzon, 5 km upstream of thewatershed outlet. A 20-m resolution DEM was used inorder to describe the topographic variability of thewatershed. The annual precipitation is approximately875mm with solid precipitation of 250mm. A detaileddescription of the Beaurivage watershed can be found inRousseau et al. (2012). As shown in Table II, theBeaurivage was discretized into 675 RHHUs with anaverage area of 105 ha (Rousseau et al., 2012). Thirtymeteorological stations were used for this study.

Sassandra watershed. The Sassandra watershed is atrans-boundary basin overlapping Ivory Coast andGuinea, West Africa (Figure 4). It is located betweenthe ninth and fifth parallel with a drainage area of 73500 km2 of which 30% is located north of the eighthparallel, and 15% in the mountainous West (Avenardet al., 1971). Our study is limited to a sub-basin, upstreamfrom Lake Buyo (Ivory Coast), with a drainage area of 35400 km2. The altitude of the Sassandra basin varies from

Figure 4. Location of the Sassandra watershed and observation sites

Copyright © 2013 John Wiley & Sons, Ltd.

130 to 650m above sea level for 90% of its area, with anaverage of 385m asl. The vegetation consists of savannahwoodlands in the sub-sudanian’s sector (roughly north ofthe eighth parallel), Guinean’s savannah, and semi-deciduous rainforest in the mesophilic guinean sector(Central and South–East Basin). The northern part of thesub-basin is a tropical system with a dry season fromNovember to April and a rainy season from May toOctober and an annual average precipitation of 1750mm.Further south, the rainy season is longer and more intense,September being the rainiest month. The most abundantrains are recorded in the western part of the basin withinter-annual averages (1960–1996), based on Sociétéd’Exploitation et de Développement Aéroportuaire,Aéronautique et Météorologique (SODEXAM) data, of1600mm and 1725mm at Man and Duékoué, respective-ly. Twenty-five meteorological stations operated by theSODEXAM were used for this study. Six of these stationsare located outside the basin. The watershed wasmodelled using 407 RHHUs and four years of hydrome-teorological data (see Table II). Soil information wasprovided by the Institut français de RecherchesScientifiques et Techniques pour le développement enCoopération (Avenard et al., 1971).

Major differences between the three watersheds.Table II presents the main characteristics of eachwatershed. The Montmorency and Beaurivage watershedsare located in North America near Québec city, Canadawith a dominant humid continental climate. On the otherhand, the Sassandra watershed is located in Africa IvoryCoast with a dominant tropical climate characterized byhigh intensity and short duration of the rainfall events. BothNorth American watersheds have the same precipitationperiod and similar amounts of annual average liquid andsolid precipitations while the African watershed receivesalmost twice as much liquid precipitation (1750mm) andno snow.The three watersheds are also quite different in their

topographic and land use attributes. The Montmorencywatershed has the largest mean surface slope (14%) andelevation range (246–1127m) with a propensity to highprecipitation gradients with varied altitudes. TheBeaurivage is relatively flat (1%), has a smaller elevationrange (60–685m), and the smallest maximum altitude.These figures are somewhat similar to those of theSassandra (6% and 650m). The Beaurivage and Mont-morency watersheds have similar drainage densities(0.728 and 0.925 km�1 much larger than these of theSassandra 0.12 km�1). Regarding land use, theBeaurivage watershed can be characterized as anagricultural watershed with 36% of its surface areaoccupied by cropland land having an impact onevapotranspiration properties. Meanwhile, the Montmo-

Hydrol. Process. (2013)

Table III. Symbols, units, prior values (y�j ), prior uncertainty (Δyj) of HYDROTEL parameters for each watershed

Parameters Units

Montmorency Watershed Beaurivage Watershed Sassandra Watershed

y�j Δyj y�j Δyj y�j Δyj

FFCO mm/d. �C 6.89 1.72 8.00 2.00 - -FFFE mm/d. �C 7.78 1.95 10.00 2.50 - -FFFO mm/d. �C 12.98 3.25 13.00 3.25 - -STCO �C 1.97 0.49 1.25 0.31 - -STFE �C 0.33 0.08 0.75 0.19 - -STFO �C �2.49 0.62 �2.00 0.50 - -TFN mm/d 0.85 0.21 0.50 0.13 - -CC - 0.71 0.18 0.01 0.00 - -FETP - 0.68 0.17 0.70 0.18 1.00 0.25Z1 m 0.11 0.03 0.10 0.03 0.29 0.07Z2 m 0.26 0.07 0.20 0.05 1.36 0.34Z3 m 1.42 0.36 1.50 0.38 1.87 0.47REC - 3.0� 10�6 7.50� 10�7 1.05� 10�5 2.63� 10�6 1.0� 10�5 2.50� 10�6

DES m/h 0.86 0.22 0.60 0.15 0.60 0.15VHMAX - 0.43 0.11 0.31 0.08 0.02 0.01TPPN �C �1.77 0.44 �0.25 0.06 - -GVP mm/100m 0.75 0.19 0.50 0.13 0 -GVT �C/100m �0.65 0.16 �0.50 0.13 �0.5 -

Table IV. Sensitivity ranking of each HYDROTEL parameter

MontmorencyWatershed

BeaurivageWatershed

SassandraWatershed

M. BOUDA ET AL.

rency watershed has only 0.3% of agricultural land andmost of its surface is occupied by forest (83%). TheSassandra watershed has 9% of its surface covered byagricultural land and 55% by forest.

Parameters rank dj rank dj rank dj

Z2 1 0.54 2 0.66 3 0.32FETP 2 0.53 1 0.77 2 0.5Z3 3 0.49 4 0.60 1 2.72GVT 4 0.47 17 0.34 9 0.001FFCO 5 0.46 5 0.55 - -Z1 6 0.45 7 0.47 6 0.03REC 7 0.45 3 0.61 4 0.13GVP 8 0.44 8 0.43 8 0.002VHMAX 9 0.44 11 0.39 7 0.003TFN 10 0.43 12 0.39 - -DES 11 0.43 6 0.54 5 0.04STCO 12 0.42 14 0.37 - -CC 13 0.41 9 0.42 - -TPPN 14 0.40 15 0.37 - -FFFE 15 0.38 16 0.37 - -STFE 16 0.38 18 0.25 - -STFO 17 0.35 10 0.41 - -FFFO 18 0.34 13 0.39 - -Average 0.43 0.46 0.41Standard deviation 0.05 0.13 0.88Variation coefficient 0.12 0.28 2.11

RESULTS AND DISCUSSION

Sensitivity and identifiability analyses

The sensitivity analysis was performed using thecalclocsen routine in UNCSIM. This routine computesthe linear sensitivity measure dj according to Equation(4). After fixing the prior parameter values y�j and priorparameter uncertainties Δyj (Table III), the sensitivitymeasures for all parameters introduced in Table I werecalculated from variations in the target output:streamflow.The results of the sensitivity analysis are provided in

Table IV. Indeed, an analysis of the results shows that inthe Montmorency watershed, the depth of the lowerboundary of the second soil layer (Z2) was the mostsensitive parameter with a d value of 0.54. For theBeaurivage watershed, it is the potential evapotranspira-tion multiplication factor (FETP) that is the most sensitiveparameter with d= 0.77, whereas for the Sassandrawatershed, it is the depth of the lower boundary of thethird soil layer (Z3) that is most sensitive with d = 2.72.The vertical gradient of temperature (GVT) is the leastsensitive parameter for the Sassandra watershed,(d= 0.001), while for the Beaurivage watershed, the leastsensitive parameter is the threshold air temperature for

Copyright © 2013 John Wiley & Sons, Ltd.

snow/ice melt in deciduous forests (STFE) with d = 0.25.For the Montmorency watershed, the least sensitiveparameter is the melt factor for open areas (FFFO) withd= 0.34. This result reflects the fact that the Montmoren-cy watershed, which is quite different than the Beaurivage

Hydrol. Process. (2013)

AUTOMATIC CALIBRATION FRAMEWORK FOR HYDROTEL

watershed is characterized by a very small percentage ofopen areas, 0.3% versus 36% for the latter watershed. Asshown in Table IV, the sensitivity measure of the verticalgradient of temperature (GVT) parameter is more sensitivein the case of the Montmorency watershed than in that forthe Beaurivage watershed. This is caused by the differingelevations (and highest elevations) between the two areas:246–1127m for the Montmorency watershed against60–685m for the Beaurivage watershed. For these threewatersheds, this suggests that the most sensitiveparameters in HYDROTEL are Z2, FETP, and Z3.Figure 5 shows a summary of the sensitivity analyses,

plotting all input parameters sensitivity (ordinate) for thethree tested watersheds (abscissa). It is noteworthy thatthe Montmorency watershed has a little variability ofsensitivity (d) values, ranging from 0.34 to 0.54.

Figure 5. Boxplot of sensitivity values for each tested watershed

Table V. Ranges of g and r and f

Subsetsize

Montmorency Watershed B

g� range r� range g� ra

2 1.1 – 1.4 15.0 – 24.0 1.0 –3 1.2 – 1.5 15.4 – 23.0 1.1 –4 1.3 – 1.6 15.5 – 21.8 1.2 –5 1.3 – 1.6 15.5 – 20.9 1.3 –6 1.3 – 1.6 15.6 – 20.2 1.3 –7 1.4 – 1.7 15.6 – 19.6 1.4 –8 1.4 – 1.7 15.6 – 19.1 1.5 –9 1.5 – 1.7 15.6 – 18.7 1.6 –10 1.5 – 1.7 15.6 – 18.3 1.7 –11 1.5 – 1.7 15.6 – 18.0 1.8 –12 1.6 – 1.7 15.6 – 17.7 1.9 –13 1.6 – 1.7 15.7 – 17.5 2.1 –14 1.6 – 1.8 15.8 – 17.2 2.1 –15 1.7 – 1.8 15.8 – 16.9 2.2 –16 1.7 – 1.8 15.9 – 16.7 2.4 –17 1.7 – 1.8 16.0 – 16.5 2.4 –

T6–T8

Copyright © 2013 John Wiley & Sons, Ltd.

It denotes that the model is somewhat sensitive to all testedparameters. For the Beaurivage watershed, the variance isslightly larger, ranging from 0.25 to 0.77; showing, that themodel responds differently to the parameter variation. Forthe Sassandra, the variance is much larger than the other twowatersheds, ranging from0.002 to 2.72. The difference in thesensitivity indices may be explained by the hydroclimatic,characteristics, which have the rainfall season characterizedby short and high intensity rainfall events and a relativelylong recession period without precipitation.To get information on the identifiability of different

parameter subsets, co-linearity indices g were calculated forall subsets of the full parameter set. Table V presents theranges of g and r for all possible parameter subsets with rcombining information provided by d and g (Brun et al.,2002). The values of the co-linearity index g for allparameter subsets were less than 10 for all cases studied.Hence, all parameterswere deemed identifiable according tothe Brun et al. (2002) criterion. As expected, the resultsshow that the co-linearity index increaseswith the size of theparameter subset (Table V) for all three watersheds. Theidentifiable subsets are provided in Tables VI–VIII whichcorrespond to theMontmorency, Beaurivage, and Sassandrawatersheds, respectively. The results also show that themaximum value of the co-linearity index is strongly linkedto the type of watershed being analyzed; this is logicalbecause g is calculated from the sensitivity matrix, whereasthe parameter sensitivity depends on the watershed itself.

Automatic calibration of HYDROTEL

The calibration algorithm is specified through modelparameterization, selection of calibration parameters,

or all possible parameter subsets

eaurivage Watershed Sassandra Watershed

nge r� range g� range r� range

2.0 10.7 – 30.0 1.0 – 4.0 0.1 – 29.82.2 10.8 – 28.1 1.1 – 4.4 0.1 – 17.22.3 11.1 – 25.5 1.2 – 4.5 0.1 – 11.42.4 11.3 – 23.1 1.4 – 4.6 0.2 – 6.22.4 11.3 – 21.5 1.9 – 4.7 0.3 – 3.92.5 11.4 – 20.3 3.2 – 4.8 0.5 – 2.22.5 11.5 – 19.3 4.5 – 4.8 0.7 – 1.52.6 11.6 – 18.5 - -2.6 11.8 – 17.8 - -2.6 12.0 – 17.2 - -2.6 12.2 – 16.7 - -2.7 12.5 – 16.1 - -2.7 12.7 – 15.7 - -2.7 12.8 – 15.3 - -2.7 12.9 – 14.8 - -2.7 13.1 – 14.4 - -

Hydrol. Process. (2013)

Table VII. The g and r for selected parameter subsets -Beaurivage watershed

Subsets size Parameters gk rk

2 FETP, Z2 1.15 30.003 FETP, Z2, REC 1.20 28.064 FETP, Z2, Z3, REC 2.01 24.765 FFCO, FETP, Z2, Z3, REC 2.06 22.636 FFCO, FETP, Z2, Z3, REC, DES 2.32 21.177 FFCO, FETP, Z1, Z2, Z3, REC, DES 2.33 20.108 GVP, FFCO, FETP, Z1, Z2, Z3,

REC, DES2.36 18.86

9 GVP, FFCO, CC FETP, Z1,Z2, Z3, REC, DES

2.38 17.87

Table VIII. The g andr for selected parameter subsets - Sassandrawatershed

Subsetssize Parameters gk rk

2 Z3, FETP 4.02 22.73 Z3, REC, FETP 4.37 12.34 GVT, Z3, REC, FETP 4.57 3.25 GVP, GVT, Z3, REC, FETP 4.62 1.56 Z1, Z2, Z3, REC, DES, VHMAX 4.72 2.07 GVP, Z1, Z2, Z3, REC, DES, VHMAX 4.77 1.28 GVP, GVT, Z1, Z2, Z3, REC, DES,

VHMAX4.79 0.8

9 GVP, GVT, Z1, Z2, Z3, REC,FETP, DES, VHMAX

4.83 0.9

Figure 6. Comparison of observed (hollow circles) and calibratedstreamflows in the Montmorency Watershed. The manual calibration isrepresented by the dotted line, and the automatic calibration is represented

by the continuous black line

Table VI. The g and r for selected parameter subsets -Montmorency watershed

Subsets size Parameters gk rk

2 Z2, Z3 1.10 23.953 FETP, Z2, Z3 1.43 23.044 GVT, FETP, Z2, Z3 1.43 21.865 GVT, FETP, Z2, Z3, VHMAX 1.51 20.896 GVT, TFCO, FETP, Z2, Z3, VHMAX 1.51 20.257 GVT, TFCO, FETP, Z2, Z3,

REC, VHMAX1.57 19.63

8 GVT, TFCO, FETP,Z1, Z2,Z3, REC, VHMAX

1.58 19.14

9 GVP, GVT, TFCO, FETP,Z1, Z2,Z3, REC, VHMAX

1.59 18.72

*The subsets were chosen based on the results file and identifiabilitycriteria. The results file contains the general ranking of the variouspossible subsets.

M. BOUDA ET AL.

defining intervals of parameter values, and assigningprior information to a group of parameters. Thecalibration period were 1988–1993, 2001–2006, and1980–1985 for the Sassandra, Montmorency, andBeaurivage watersheds, respectively (Table II). The

Copyright © 2013 John Wiley & Sons, Ltd.

selected hydrological years go from the beginning ofOctober to the end of September the next year. To reducethe number of parameters in the calibration process,parameter estimation was performed with only the mostsensitive parameters identified in ‘Sensitivity andidentifiability analyses’. Since automatic calibration andsensitivity analysis can be affected by initial parametervalues as mentioned previously, a primary manualcalibration was completed to obtain a set of reasonablevalues. The initial global model parameter values wereselected specifically based on previous studies (Fortinet al., 2001b; Turcotte et al., 2003; Lavigne, 2007). Theautomatic calibration was performed using the five mostsensitive parameters for each watershed (Montmorency:Z2, FETP, Z3, GVT, FFCO; Beaurivage: FETP, Z2, REC,R3, FFCO; Sassandra: Z3, FETP, Z2, REC, DES). Asmentioned, in the maximum likelihood method, thelikelihood function selected and statistical hypothesesmade can substantially affect the calibration results. Theresults from the testing diagnostic of the statisticalassumptions made in the likelihood function (8) areanalyzed and provided in a complementary study (Boudaet al., 2012). In general, these assumptions did not seemto be violated. However, there was a small autocorrelationof the errors in the first 15 steps of simulations for theSassandra and Beaurivage watersheds. Figures 6–8 illus-trate the daily streamflows for each watershed over onehydrological year. Each figure shows the simulated flowsresulting from the manual and the automatic calibrationexercises and the observed/measured flows at the basinoutlets.Similar to the manual calibration, the results of the

automatic calibration using a daily time step showed verygood agreement between measured and simulatedstreamflows, for different hydrological years. In general,the various recession and flood periods were fairly well

Hydrol. Process. (2013)

Figure 7. Comparison of observed (hollow circles) and calibratedstreamflows in the Beaurivage Watershed. The manual calibration isrepresented by the dotted line and the automatic calibration is represented

by the continuous black line

Figure 8. Comparison of observed (hollow circles) and calibratedstreamflows in the Sassandra Watershed. The manual calibration isrepresented by the dotted line, and the automatic calibration is represented

by the continuous black line

AUTOMATIC CALIBRATION FRAMEWORK FOR HYDROTEL

simulated, but it appears that HYDROTEL has a tendencyto underestimate some floods. This is likely due, in part,to the uncertainties related to errors in input data, andperhaps the model structure. Also, some hydrologicalevents were not well reproduced by the model. After theautomatic calibration, most predicted daily streamflowvalues improved according to the NSE which were for the

Table IX. Performance criteria: the Nash

Catchments (period)

Nash–Sutcliffe (NSE)

Manual calibration Auto calibratio

Montmorency (2001–2006) 0.76 0.77Beaurivage (1980–1985) 0.74 0.78Sassandra (1988–1993) 0.87 0.92

*diff is the difference between manual and automatic calibration efficiencies

Copyright © 2013 John Wiley & Sons, Ltd.

Montmorency, Sassandra, and Beaurivage watersheds0.77, 0.92, and 0.78, respectively (Table IX). In general,results for the Sassandra were affected by some missinginput data, particularly for hydrological year 1989–1990.This caused some difficulties during the calibrationprocess. From Table IX, it is observed that modelefficiency slightly increased following the automaticcalibration; the mean between NSE and R2 increases is1.2%, 4.4%, and 4.9% for the Montmorency, Sassandra,and Beaurivage watersheds, respectively. Most improve-ments were related to peak flows for the Montmorencyand Beaurivage watersheds (temperate climate condi-tions) and flows beyond the rainfall season in theSassandra watershed. This shows that additional gainscan be obtained when an automatic calibration of the mostsensitive parameters of HYDROTEL is performed afterapplying a well-structured, process-oriented, multiple-objective, manual calibration.

CONCLUSION

The objective of this paper was to introduce an automaticcalibration procedure for the distributed hydrological modelHYDROTEL based on the OAT-sensitive analysis ap-proach and the eigenvalue identifiability framework. Theprocedure was applied to three distinct watersheds: theSassandra located in Ivory Coast, West Africa, andthe Montmorency and Beaurivage watersheds located inQuebec (Canada). The UNCSIM package was used toperform sensitivity and identifiability analyses as well as toestimate themodel parameters. The calibrationwas based onthe maximum likelihood parameter estimation accountingfor autocorrelation by using an autoregressive error model.The sensitivity analyses revealed that sensitivity

depended on watershed characteristics and the dominanthydrological processes involved. Clearly, there weredifferent results between watersheds; linked to differencesin climate, quality of input data, and soil and landproperties. It is noteworthy that van Griensven et al.(2006) reached similar conclusions. Thus, it is advisedthat for each new watershed studied, a parametersensitivity analysis should be conducted to select the

–Sutcliffe and R2 for each watershed

Coefficient of determination (R2)

n diff* Manual calibration Auto calibration diff*

+1.31% 0.87 0.88 +1.15%+5.40% 0.89 0.93 +4.50%+5.74% 0.94 0.97 +3.19%

, calculated as follows: diff ¼ Efficiencyautomatic�EfficiencymanualEfficiencymanual

Hydrol. Process. (2013)

M. BOUDA ET AL.

subset of parameters to be used for model calibration oruncertainty analysis. These results confirm that theuniqueness of place and process principia proposed byBeven (2000). Each watershed is unique in its particularcharacteristics of topography, soil, vegetation, groundwa-ter processes, and anthropogenic modifications. Thus, thetransfer of a calibrated dataset of parameters, from onewatershed to another, needs to be done with caution.The identifiability analysis was used to indicate which

model parameters were clearly identifiable. This analysisindicated that HYDROTEL does not seem to be over-parameterized, that is for the three watersheds used, eachparameter carries its own relevant information and nocombination of other parameters can explain a singleparameter.The results of the automatic calibration using a daily

time step showed very good agreement between measuredand simulated streamflows. Since, this technique isintrinsically linked to prior knowledge of basin charac-teristics; a poor choice of initial parameter values andtheir magnitude can often lead to absurd results. Thus, astructured manual approximation of parameters providedrealistic values as a first step; then, automatic calibrationwas used in a second step to fine tune the parametersvalues. Model efficiency from automatic calibration of themost sensitive parameters of each watershed was slightlyincreased when compared to the efficiency obtained bythe structured trial-and-error procedure proposed byTurcotte et al. (2003). The coupling between sensitivityand identifiability analyses with the automatic calibrationprocedures seems to perform well, not only regarding theachievements of standard performance criteria (NSE andR2), but in reducing calibration time which is foremost fordeterministic, hydrological forecasting distributed modellike HYDROTEL.Future studies should consider a global method

for sensitivity analysis. It is worth noting that localmethods, such as the method implemented in UNCSIM,only consider local information which may not representthe entire parameter space. Also, special attentionshould be focused on improving the autocorrelationresults, by using seasonal analysis. In a complementarypaper, the uncertainty of model outcomes due to modelparameters and a predictive analysis is carried outusing the UNCSIM package to estimate the range ofsimulated flows resulting from model uncertainties(Bouda et al., 2012).

ACKNOWLEDGEMENTS

The authors would like to acknowledge the contributions ofJin Yang of Singapore-MIT Alliance for Research andTechnology (SMART); AlainRoyer,Martin-Pierre Lavigne,

Copyright © 2013 John Wiley & Sons, Ltd.

Simon Ricard and Stéphane Savary of INRS-ETE; andRichard Turcotte of the Centre d’Expertise Hydrique duQuébec. Financial support was provided by Hydro-Québec,Ouranos, NSERC (Natural Sciences and EngineeringResearch Council of Canada), Agriculture and AgrifoodCanada, Environment Canada, and Quebec Ministry ofSustainable Development, Environment and Parks.

REFERENCES

Avenard JM, Eldin M, Girard G, Sircoulon J, Touchebeuf P, Guillamet JL,Adjanohoun E, Perraud A. 1971. Le milieu naturel de la Côte-d’Ivoire.Paris: ORSTOM. (Mémoires ORSTOM, n� 50).

Bahremand A, De Smedt F. 2008. Distributed hydrological modeling andsensitivity analysis in Torysa watershed, Slovakia. Water ResourcesManagement 22: 393–408. DOI: 10.1007/s11269-007-9168-x.

Belsley DA. 1991. Conditioning Diagnostics - Collinearity and WeakData in Regression. J. Wiley: New York.

Beven KJ. 2000. Uniqueness of place and the presentation of hydrologicalprocesses. Hydrology and Earth System Sciences 4: 203–213.

Bouda M, Rousseau AN, Konan B, Gagnon P, Gumiere SJ. 2012.Uncertainty analysis of the distributed hydrological modelHYDROTEL: Application of a Bayesian method. Accepted forpublication in Journal of Hydrologic Engineering. DOI: 10.1061/(ASCE)HE.1943-5584.0000550

Box GEP, Cox DR. 1982. An analysis of transformations revisited, rebutted.Journal of the American Statistical Association 77(377): 209–210.

Boyle DP, Gupta HV, Sorooshian S. 2000. Toward improved calibrationof hydrologic models: Combining the strengths of manual andautomatic methods. Water Resources Research 36(12): 3663–3674.DOI: 10.1029/2000WR900207.

Brun R, Reichert P, Künsch RH. 2001. Practical identifiability analysisof large environmental simulation models. Water Resources Research37(4): 1015–1030.

Brun R, Kühni M, Siegrist H, Gujera W, Reichert P. 2002. Practicalidentifiability of ASM2d parameters – systematic selection and tuningof parameter subsets. Water Research 36: 4113–4127.

Cheviron B, Gumiere SJ, Le Bissonnais Y, Moussa R, Raclot D. 2010.Sensitivity analysis of distributed erosion models: Framework. WaterResources Research 46: W08508. DOI: 10.1029/2009WR007950.

Conseil de Bassin de la rivière Montmorency. 2006. Plan Directeur del’eau – Protrait, pp 4.

Duan Q, Sorooshian S, Gupta V. 1992. Effective and efficient globaloptimisation for conceptual rainfall-runoff models. Water ResourcesResearch 28: 1015–1031.

Duan QY, Gupta VK, Sorooshian S. 1993. Shuffled complex evolutionapproach for effective and efficient global minimization. Journal ofOptimization Therory and Application 76(3): 501–521.

Fortin JP, Turcotte R, Massicotte S, Moussa R, Fitzback J, Villeneuve JP.2001a. A Distributed Watershed Model Compatible with Remote Sensingand GIS data. Part I: Description of Model. Journal of HydrologicEngineering, American Society of Civil Engineering 6(2): 91–99.

Fortin JP, Turcotte R,Massicotte S,MoussaR, Fitzback J, Villeneuve JP. 2001b.A Distributed Watershed Model Compatible with Remote Sensing and GISdata. Part II: Application to Chaudière watershed. Journal of HydrologicEngineering, American Society of Civil Engineering 6(2): 100–108.

Frey HC, Patil SR. 2002. Identification and review of sensitivity analysismethods. Risk Analysis 22(3): 553–578.

Gumiere SJ, Raclot D, Cheviron B, Davy G, Louchart X, Fabre J-C,Moussa R, Bissonnais YL. 2010. MHYDAS-Erosion: a distributedsingle-storm water erosion model for agricultural catchments Hydro-logical Processes, in press. DOI: 10.1002/hyp.7931

Gupta H, Sorooshian S, Hogue T, Boyle D. 2003. Advances in theautomatic calibration of watershed models, in: Calibration of watershedmodels, Water Science and Application, American Geophysical Union,Washington DC, USA, 6, 9–28.

Helton J C. 1993. Uncertainty and sensitivity analysis techniques for usein performance assessment for radioactive waste disposal. ReliabilityEngineering and System Safety 42: 327–367.

Hydrol. Process. (2013)

AUTOMATIC CALIBRATION FRAMEWORK FOR HYDROTEL

Kleissen FM, Beck MB, Wheater HS. 1990. The identifiability ofconceptual hydrochemical models. Water Resources Research 26(12):2979–2992.

Krause P, Boyle DP, Bäse F. 2005. Comparison of different efficiencycriteria for hydrological model assessment. Advances in Geosciences 5:89–97. DOI: 10.5194/adgeo-5-89-2005

Lavigne MP. 2007. Modélisation du régime hydrologique et de l’impactdes coupes forestières sur l’écoulement du ruisseau des eaux-volées àl’aide d’HYDROTEL. Mémoire de maîtrise, Université du Québec,INRS-ETE, Canada.

Linacre ET. 1977. A simple formula for estimating evaporation rates invarious climates, using temperature data alone. Agricultural Meteorol-ogy 18: 409–424.

Matott LS, Babendreier JE, Purucker ST. 2009. Evaluating uncertainty inintegrated environmental models: A review of concepts and tools. WaterResources Research 45: W06421. DOI: 10.1029/2008WR007301.

Monteith JL. 1965. Evaporation and environment. Symposia of the Societyfor Experimental Biology 19: 205–234.

Moussa R, Voltz M, Andrieux P. 2002. Effects of the spatial organizationof agricultural management on the hydrological behaviour of a farmedcatchment during flood events. Hydrological Processes 16: 393–412.

Nelder JA, Mead R. 1965. A simplex method for function minimization.The Computer Journal 7: 308–313. DOI: 10.1093/comjnl/7.4.308.s

Priestley CHB, Taylor RJ. 1972. On the assessment of surface heat flux andevaporation using large scale parameters. Monthly Weather Reviews 100:81–92.

Quilbé R, Rousseau AN, Moquet J-S, Trinh NB, Dikibi Y, Gachon P,Chaumont D. 2008. Assessing the effect of climate change on river flowusing general circulation models and hydrological modelling -Application to the Chaudière River (Québec, Canada). Canadian WaterResources Journal 33(1): 73–94.

Reichert P. 2001. UNCSIM—A Program Package for UncertaintyAnalysis and Bayesian Inference: User Manual, EAWAG, Dübendorf,Switzerland (2001) p. 36.

Reichert P. 2005. UNCSIM – A computer program for statistical inferenceand sensitivity, identifiability and uncertainty analysis. In: TeixeiraJMF, Carvalho-Brito AE (eds). Proceedings of the 2005 EuropeanSimulation and Modelling Conference (ESM 2005), October 24-26,Porto, Portugal, EUROSIS-ETI., pp 51–55.

Reichert P. 2006a. A standard interface between simulation programsand systems analysis software. Water Science and Technology 53(1):267–275. DOI: 10.2166/wst.2006.029

Reichert P. 2006b. A computer program for statistical inference andsensitivity, identifiability and uncertainty analysis. User’s ManuelVersion 2.0, EAWAG, Dübendorf, Switzerland.

Riley J.P., Israelsen E. K. and Eggleston, K.O. 1973. Some approaches tosnowmelt prediction. The Role of Snow and Ice in Hydrology, IAHSpub 107, IAHS, pp. 956–971.

Rousseau AN, Savary S, Konan B. 2008. Implantation du modèleHYDROTEL sur le bassin de la rivière Montmorency afin de simulerles débits observés et de produire des scénarios de crues du printemps2008. Rapport R-921. Centre Eau, Terre et Environnement, Institutnational de la recherche scientifique, INRS-ETE. Québec, PQ 54 pages.

Rousseau AN, Fortin JP, Turcotte R, Royer A, Savary S, Quévy F, Noël P,Paniconi C. 2011a. PHYSITEL, a specialized GIS for supporting theimplementation of distributed hydrological models.Water News - OfficialMagazine of the Canadian Water Resources Association 31(1): 18–20.

Rousseau AN, Lafrance P, Lavigne M-P, Savary S, Konan B, Quilbé R,Amrani M, Jiapizan P. 2011b. A hydrological modelling framework fordefining watershed-scale achievable performance standards of pesti-cides beneficial management practices. Journal of EnvironmentalQuality 41(1): 52–63. DOI: 10.2134/jeq2010.0281

Rousseau AN, Savary S, Hallema DW, Gumiere SJ, Foulon E. 2013.Modeling the effects of agricultural BMPs on sediments, nutrients andwater quality of the Beaurivage River watershed (Quebec, Canada).

Copyright © 2013 John Wiley & Sons, Ltd.

Canadian Water Resources Journal (in press). DOI: 10.1080/07011784.2013.780792.

Saltelli A. 1999. Sensitivity analysis. Could better methods be used?Journal of Geophysical Research 104(D3): 3789–3793.

SaltelliA,Tarantola SChanK. 1999.Aquantitative,model independentmethodfor global sensitivity analysis of model output. Technometrics 41: 39–56.

Saltelli A, Chan K, Scott M. 2000a. Sensitivity Analysis. Probability andStatistics Series. John Wiley & Sons Ltd.: New York, NY, USA; 475.

Saltelli A, Tarantola S, Campolongo F. 2000b. Sensitivity analysis as aningredient of modeling. Statistical Science 15: 377–395.

Saltelli A, Tarantola S, Campolongo F, RattoM. 2004. Sensitivity analysis inpractice. A guide to assessing scientific models. John Wiley & Sons: NY.

Savary S, Rousseau AN, Quilbé R. 2009. Assessing the effects ofhistorical land cover changes on runoff and low flows using remotesensing and hydrological modeling. Journal of Hydrologic Engineering14(6): 575–587. DOI: 10.1061/(ASCE)HE.1943-5584.0000024.

Schaefli B, Gupta HV. 2007. Do Nash values have value? HydrologicalProcesses 21: 2075–2080

Scott M, Saltelli A (eds). 1997. Special issue on sensitivity analysis.Journal of Statistical Computation and Simulation 57(1-4): 1–364.

Spear RC, Hornberger GM. 1980. Eutrophication in Peel Inlet - IIIdentification of critical uncertainties via generalized sensitivityanalysis. Water Research 14: 43–49.

Spear R, Grieb TM, Shang N. 1994. Parameter uncertainty and interaction incomplex environmental models.Water Resources Research 30: 3159–3169.

Tarantola S, Saltelli A. 2003. SAMO 2001: methodological advances andinnovative applications of sensitivity analysis. Reliability Engineeringand System Safety 79: 121–122.

Thornthwaite CW. 1948. An approach toward a rational classification ofclimate. Geographical Reviews 38: 55–94.

Turcotte R, Fortin JP, Massicotte S, Fitzback J, Rousseau AN. 2001.Determination of watershed drainage structure using a digital elevation modeland a digital river and lake network. Journal of Hydrology 240: 225–242.

Turcotte R, Rousseau AN, Fortin JP, Villeneuve JP. 2003. A process-oriented, multiple-objective calibration strategy accounting for modelstructure. In Advances in Calibration of watershed models, WaterScience & Application 6, American Geophysical Union WaterResources Monograph Series, Duan Q, Gupta H, Sorooshian S,Rousseau AN, Turcotte R (eds); 153–163.

Turcotte R, Lacombe P, Dimnik C, Villeneuve J-P. 2004. Distributedhydrological prediction for the management of Quebec’s public dams |[Prevision hydrologique distribuee pour la gestion des barrages publicsdu Quebec]. Canadian Journal of Civil Engineering 31(2): 308–320

Turcotte R, Fortin L-G, Fortin V, Fortin J-P, Villeneuve J-P. 2007.Operational analysis of the spatial distribution and the temporalevolution of the snowpack water equivalent in southern Québec,Canada. Nordic Hydrology 38(3): 211–234

Van Griensven A, Meixner T, Grunwald S, Bishop T, Diluzio M,Srinivasan R. 2006. A global sensitivity analysis tool for parameters ofmutli-variable catchment models. Journal of Hydrology 324: 10–23.

Yang J. 2011. Convergence and uncertainty analyses in Monte-Carlo basedsensitivity analysis. Environmental Modelling & Software 26: 444–457.

Yang J, Reichert P, Abbaspour KC, Yang H. 2007a. Hydrologicalmodelling of the Choahe Basin in China: Statistical model formulationand Bayesian inference. Journal of Hydrology 340: 167–182. DOI:10.1016/j.jhydrol.2007.04.006.

Yang J, Reichert P, Abbaspour KC. 2007b. Bayesian uncertainty analysisin distributed hydrologic modeling: A case study in the Thur Riverbasin (Switzerland). Water Resources Research 43: W10401. DOI:10.1029/2006WR005497.

Yapo P, Gupta H, Sorooshian S. 1998. Multi-objective global optimiza-tion for hydrologic models. Journal of Hydrology 204: 83–97.

Young PC. 1978. Modeling, Identification and Control in EnvironmentalSystems. North Holland, Amsterdam, Ch. A general theory ofmodelling for badly defined dynamic systems, pp. 103–135.

Hydrol. Process. (2013)