EGIM14: Computational Case Study
“Estimation of Parameter Uncertainty”
Module Lecturer: Dr. Yunqing Xaun
Student Name: Mmeka Felix Uchenna
Student Number: 663327
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 1
Contents
1. Introduction…………………………………………………………………………...…..2
2. Literature Review…………………………………………………………………………5
3. Methods for accounting for Parameter Uncertainty………………………………...…….7
3.1) Mathematical Representation of Parameter Uncertainty
3.2) Generalized Likelihood Uncertainty Estimation (GLUE)
3.3) Differential Evolution Adaptive Metropolis (DREAM)
3.4) Major Differences Between the GLUE and DREAM Estimation Methods
4. THE CATCHMENT: Description……………………………………………………...18
4.1) Work done in 3rd
year thesis
5. Hydrological Predictions for the Environment (HYPE)……………………………......21
6. Applications of Parameter Estimation Methods to Case Study……………….......…....25
6.1) GLUE method
6.2) DREAM method
7. Conclusion…………………………………………………………………………..…..26
7. References…………………………………………………………………………..…..27
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 2
1) Introduction
Hydrological modeling is the creation of simulations though mathematical
analogues/simplifications with the use of given data and computing techniques that best
represents complex physical, spatially distributed, and highly interrelated water, energy, and
vegetation processes in a watershed/catchment thereby predicting future data of the given
catchment. A hydrological model consists of 2 parts; the hydrological core and technological
shell. The core is a set of mathematical equations that produces an output with a given input
while the shell is the programming, user interface and pre and post processing facilities.
Rainfall runoff model can be divided into various stages of production and they are
perceptual model, conceptual model, procedural model, model calibration and validation.
Model Calibration is the setting of parameter values through the process of adjusting
parameters to fit a real catchment data while Model Validation/Verification is the checking of
the model flow against that of a predicted flow to model a similar hydrograph therefore
predicting future data [1]. Although model calibration attempts to replicate observed
hydrograph by making predicted simulations, due to the presence of significant
errors/uncertainty in the simulated hydrograph, this makes the model’s prediction not to be a
true representation of the observed. These predictions could be questioned in decision making
thereby hindering accurate judgment to be made from the model’s predictions; to what degree
the simulated hydrograph represents the observed one and this dilemma is a major problem.
The uncertainty estimation techniques have been developed to quantify these
uncertainties/errors in the model simulation, thereby aid better decisions made with respect to
the model. Uncertainty in hydrological modeling in recent years has been an increasing
consensus in hydrologic literatures; it can be defined as the presence of errors in hydrological
models that forces a derivation in the model’s predicted hydrograph from the observed
hydrograph. Uncertainty appears in models could be as a result of errors in measurement of
observed data, model structure errors (Over-parameterization), errors in initial and boundary
conditions of the model, output measurement errors, errors form potential evapotranspiration
or temperature and model parameters errors, conceptually limitations or errors in the model,
etc. These reasons for uncertainty can be further classified into various types of uncertainty
like structural, Input (forcing/state) and model parameter uncertainty (Model
parameterization) [2].
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 3
As previously stated, to aid judgments with model simulations, these uncertainties
have to be in some way quantified and the research of Uncertainty quantification has received
a very significant surge in recent decades. Over the years, various methods to quantify
uncertainty in models have been researched and they can be broadly split into parameter,
structural and Input/forcing data uncertainty estimations. Methods for parameter uncertainty
estimation is broadly divided into deterministic model calibration methods and stochastic
methods, while examples of deterministic model calibration methods (these methods are
normally used for models with many parameters calibration) are dynamic identifiability
analysis framework (DYNIA), multiobjective complex optimization method (MOCOM), etc.
and examples of stochastic methods bayesian model averaging (BMA), generalized
likelihood uncertainty estimation (GLUE), sequential uncertainty fitting (SUFI-2), first-order
second moment (FOSM), uncertainty estimation based on local errors and clustering
(UNEEC), differential evolution adaptive metropolis (DREAM), Shuffled Complex
Evolution Metropolis algorithm (SCEM), maximum likelihood bayesian averaging method
(MLBMA), bayesian recursive estimation technique (BaRE), all other markov chain monte
carlo (MCMC) sampling approaches, etc. Most of these models consider any errors between
the simulated and observed to be down to parameter uncertainty without any considering to
other sources of error. Structural and Input uncertainty are a bit difficult to estimate in
practice and therefore there are no established schemes for their estimation. Although this is
the case, Multi-model approach are used for Structural uncertainties because they are better
than single-model approach due to Equifinality. The concept of Equifinality is an assumption
of multiple possibilities for acceptable predictions for a particular simulation. While
examples of methods of Input (forcing) estimation developed over the years are state-space
filtering, kalman filtering, particle filtering, Variational data assimilation (VDA), dual state-
parameter estimation methods, etc [3]. Finally, some techniques have also been developed to
investigate the combination of Input and structural errors and Input and model parameters
errors and they include Bayesian model averaging (BMA), the integrated Bayesian
uncertainty estimator (IBUNE), the Bayesian total error analysis (BATEA) and closely
related developments, Framework for Understanding Structural Errors (FUSE), etc. and
Simultaneous Optimization and Data Assimilation (SODA), state augmentation, etc.
respectively. However, these schemes are not been commonly applied in practice besides
from each of the techniques’ limitations, is partly because of their computational
requirements [6].
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 4
Although the ideal case in uncertainty estimation and thereby aiding decision making
with hydrological modeling, would be the ability to quantify all the various uncertainties in a
particular model, however, one could argue that a full uncertainty analyses would require a
very comprehensive knowledge on the processes involved and presence of these uncertainties
show deficiencies in the current models (conceptually, assumptions made and limitations of
the model in general) and data (limitations in observations, measurements techniques and
performance criterions), also practically due to time constraint on this research and level of
experience in the field needed, this is not the goal for this Case study and Dissertation. This
case study for my Dissertation will take an in-depth look at Model parameter uncertainty
using the generalized likelihood uncertainty estimation (GLUE) uncertainty estimation
method and will use the recently developed differential evolution adaptive metropolis
(DREAM) uncertainty estimation method to serve as a contrast to the GLUE method due to
their different philosophies in uncertainty estimation analysis and this analysis will be applied
to the Upper Medway catchment. Model parameterization is the representation of
hydrological processes in Hydrological models with parameters; this is extremely hard due
lack of accurate knowledge about the processes (can be due to poor measurement techniques,
etc.) and these differences between Model simulations and Observed data bring about
errors/uncertainties. Model Parameter uncertainty was chosen was chosen in the uncertainty
analysis because it typically dominates in many catchment analysis due to model
parameterization and as stated above, an analysis for the entire uncertainty distribution will
be too complex. While stochastic models (GLUE and DREAM) were chosen over
deterministic model calibration methods (Manual and Automatic calibration) for this analysis
because they not only make predication by estimating errors but also state how uncertain their
predictions are, in general deterministic methods are not really referred to as an uncertainty
analysis but a form of calibration for model with many parameters. Furthermore, this study
discusses GLUE and DREAM method in the context of both methods having different
philosophies but producing similar predictions.
The paper structure is organized as follows. Section 2 is the Literature review which
describes some previous work done in uncertainty estimation. Section 3 will give a detailed
general description of the GLUE and DREAM uncertainty estimation methods. Section 4
introduces the Upper Medway catchment and previous case study in calibration techniques.
Section 5 gives a brief introduction on the Hydrological Predictions for the Environment
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 5
(HYPE) hydrological model. Finally, section 6 talks about the applications of GLUE and
DREAM model uncertainty estimation methods to the HYPE model.
2) Literature review
So much work has been done on the topic of uncertainty estimation and there are a
couple of them that are relevant to this research. K. Bevan’s 2000 paper titled “Equifinality,
data assimilation and uncertainty estimation in mechanistic modeling of complex
environmental systems using the GLUE methodology”, this paper was focused on the
investigation of effect of data length on calibration of model (Topmodel) parameters within
GLUE methodology. The paper started by the explanation of the concept of equifinality,
which is the acceptance of many predictions as acceptable prediction and its application to
GLUE method, which it refer to as the basis of the Method. Then the paper explained the
concept of a likelihood function and how it distinctly shows behavioral models with a
particularly threshold limit, how this pertains to GLUE and finally, GLUE method and the
problem was applied to the Maimai M8 catchment in New Zealand. The paper concluded by
saying that an increase in behavioral threshold would reduce the number of behavioral
models, prediction limits are dependent on model, initial conditions and measurement of
observations and likelihood measure and that the likelihood weight assigned to each model
will implicitly reflect on the complex interactions between parameter values [4].
Another paper related to this case study is the one by L. Xiong titled “An empirical
method to improve the prediction limits of the GLUE methodology in rainfall-runoff
modeling”. The paper focuses on the implementation of GLUE method to the King and
Qingjiang catchment using the SMAR conceptual model, more specifically; the paper
primarily focused on increasing the number of behavioral models without any change to the
threshold frequency by a modification to the GLUE method and this was then applied to the
both catchments. The paper started by introducing the problem, which is that the number of
observations that fall within the prediction limits are normally much less than the prediction
limit used, which was majorly as a result of error in the simulation series produced by the
model and the purposed solution was a modification of the GLUE method by the addition of
a simulating-correcting module to produce better predictions. Then the paper described the
step-wise implementation of the GLUE method and the Modified GLUE method for any
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 6
problem and stated that the both step-wise differences between the both methods was that in
the modified method, a simulation bias curve is plotted for every behavioral parameter set
and at every time interval of the simulated hydrograph, all prediction values of all intervals
for the same prediction are corrected by dividing them with their median bias value before
the derivation of the prediction limits. The SMAR model was then talked about and finally
the entire process was applied to the both catchments. At the end of the analysis, it was
concluded the Modified GLUE method substantially improve in the percentage of the
observed runoff data within the prediction limits than the GLUE method [5].
Another paper related to this case study is the one by J. A. Vrugt’s 2008 paper titled
“Equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic
modeling?”. The paper focuses the application of the GLUE and DREAM method to the Leaf
river and French Broad watersheds using the HYMOD conceptual model, more specifically;
the paper primarily focused in comparing and establishing common grounds between the two
different approaches. The paper began by stating that uncertainty is the reason for the both
models and move on to a brief introduction on GLUE method. Then it then discussed the
concept of inverse problem (likelihood function) and then derives a likelihood function. Then
the paper goes on to define the likelihood mostly used that was developed by K. Bevan, the
implementation of GLUE method and then it moved MCMC with DREAM where it
discussed the implementation and predictive inference. Finally, the paper ran analysis on the
both catchments and concluded with the following: although the both methods have different
philosophies, they still produce roughly same estimates, DREAM method has better coverage
of observations but smaller spread of predictions than GLUE, GLUE method can reveal when
the observation given the available input data without compensation by statistical error model
or input changes, can’t be reproduced by any model, Parameter uncertainty is usually large in
GLUE implementations because of the inclusion of implicit representation of model error and
finally, GLUE method has an inability to separate various individual error sources [3].
Finally, another paper related to this case study is the one by J. A. Vrugt’s 2008 paper
titled “Treatment of input uncertainty in hydrologic modelling: Doing hydrology backward
with Markov chain Monte Carlo simulation”. This paper focuses on the application of the
DREAM method to the Leaf river and French Broad watersheds using the HYMOD
conceptual model, more specifically; the effect of rainfall multipliers on model performance.
The paper began by explaining why just calibration is not enough instead of DREAM method
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 7
due to the presence of errors, then describes the rainfall forcing data error, where an option of
rainfall multipliers can be used for more realistic simulations of the data, then the paper
talked about the MCMC methods and DREAM method theorem and finally the case studies
of the experiment. In the Case study, the paper tried to compare estimation of parameters and
hydrograph without the use of rainfall multipliers against estimation of parameters and
hydrograph with them and from the analysis, the second experiment (with multipliers) as
predicted had better values. The paper concluded by stated that explicit treatment of forcing
error during model calibrations significantly alters the posterior distribution of model
parameters, The DREAM algorithm provides an accurate estimate of the Posterior probability
density function (PDF) of model parameters, Rainfall multipliers help give better estimation
for hydrograph and Rainfall multipliers provide important diagnostic information to quantify
rainfall error [10].
3) Methods for accounting for Parameter Uncertainty.
As stated above, this paper will take an in-depth study of model uncertainty using
GLUE and DREAM parameter estimation methods applied to the Upper Medway catchment.
Also as stated previously, deterministic model calibration methods validity has been
questioned due to the presence of significant errors/uncertainty in the output hydrograph
(which could be improved by better understanding of the physical processes and high
resolution simulations) and because of this problem, the uncertainty estimation techniques
have been developed to help quantify errors in the model simulation. Below further explains
concept of uncertainty estimation;
3.1) Mathematical Representation of Parameter Uncertainty
Mathematically, a model is normally composed on 7 different components and they
are system boundary (B), inputs (u), initial states (xo), system properties (θ), structure (M),
states (x), and outputs (y). The input and output (u and y respectively) are considered as
fluxes of mass/energy in the system across the system boundary B while the system
properties (θ) is considered to be time-invariant. As shown in the Fig. 1 below;
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 8
Fig. 1: a diagram showing the mathematical components of a model
Using a set of model parameters (will be refer to as Model/parameter set from now on), the
model simulates a response Y (which is a function of all its components) while the observed
flow for the catchment is �� . The error will be denoted as;
∈𝐢= ��𝒊 − 𝐘𝒊 Where i = 1,……No of time steps
It can also be observed that 5 of the 7 components (system boundary (B), inputs (u), initial
states (xo), system properties (θ) and structure (M)) are needed to be specified, estimated or
defined before the model can run while the other two (states (x) and outputs (y)) are obtained
by running the model. The five assumed components may be uncertain and it would lead to
errors in the other 2 dependent components and that is why the 5 components are the major
sources of uncertainty in hydrological modelling. The point of an uncertainty measure is to
estimate accurately this error distribution, to ideally be able to make an error-free model for
analysis. The above point is also the basic concept of likelihood measure (will be later
discussed in the paper) [3].
As previously stated, this error is can be quantified by various methods but this
research aims to use the GLUE method and then compare with results obtained from
DREAM method in the Upper Medway catchment.
3.2) Generalized Likelihood Uncertainty Estimation (GLUE)
The GLUE method is an informal bayesian method that was developed Keith Bevan
and it is inspired by the Hornberger and Spear method of sensitivity analysis; it operates with
the Monto Carlo simulations with bayesian estimation and propagation of uncertainty. The
method has found a widespread because it’s conceptually simple and easy to implementation.
The fundamental assumption of the method is that it rejects the concept of an optimal method
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 9
because of the presence of uncertainties and adopts concept of Equifinality, which believes in
multiple models as an acceptable predictions due to imperfect knowledge about the model.
GLUE rejects typical statistical principle for likelihood functions (measure of how well a
model fit with the observed data) in favor of finding behavioral models. For GLUE to be
implemented, many different models have to be run and these models are assigned likelihood
function values. Based on the assign threshold value (this could be a certain value or a
percentage of the total number of models), the behavioral models are then selected from the
all the models ran. The behavioral models are then used to provide predictions of the system
behavior by obtaining the cumulative distribution (CDF) for each prediction. To access
uncertainties in predictions, weights of behavioral models are calculated by normalizing their
corresponding likelihood values so that all weights sum to 1. Finally for any comparisons, the
deterministic model prediction is practically assumes to be the median of output prediction
and upper and lower limits for the distribution could be found (usually 5% - 95 %). The main
improvement of this method over deterministic calibration methods is the ability to account
for uncertainties in model parameter (structural uncertainties could also be argued) by using
multiple models. With this all said, the 4 key areas are choice of a likelihood measure, choice
of hydrological model, method of generating parameter set (models) and the threshold value
definition [4].
Choice of Hydrological model: The choice is a model for the GLUE analysis is completely
subjective on the modeler in question due to the availability of various models that are
capable for the analysis. Some of the reasons for choice may be; number of model parameters
(due to model sampling), good user interface (due to numerous runs of the Hydrological
model), control of the model (convenient for modelers), etc. This decision is total up to
choice of modeler choice and application the model is needed for.
Generating parameter sets: The GLUE method separates the sampling of parameters to
ensure independence of model selection process been used to select models for the analysis.
There are various techniques that have been developed for this, such as Monte Carlo
methods, Latin-Hypercube simulations (LHS), Uniform Distribution method, Variance-based
methods, etc. Monte Carlo sampling methods aim to randomly sample models due to their
likelihood function values with hope to saving computer time by the sampling of the most
probable models first (highest likelihood function). The sampling method can be very
effective in a well-defined surface but not so great for surfaces with multi local peak values.
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 10
The Latin-Hypercube simulations (LHS) is very robust and may require a lot of sampling, it
could be described as a stratified sampling approach Monte Carlo method because it allows
adequate estimation of output statistics. Some of LHS advantages are robustness and
efficiency while a major drawback is its assumption of linearity when sampling parameters.
Uniform distribution method is a methodical way of sampling, where every possible
combination for all the parameters is modeled to make various models. The major advantage
is that the method leaves no room for subjective assumptions but the major drawbacks are
that the method is time consuming and not feasible at times. The Variance-based methods
aim to decompose the variance, when all inputs are varying into partial variances. With all
the above choices, one method will be chosen for the analysis [8].
Choice of likelihood function: Likelihood functions as previously discussed tries to estimate
errors and furthermore assign a value of likelihood (trust) to each model which is obtained by
comparing the simulated against observed data. There are many likelihood functions that
have been developed such as Mean absolute error (MAE), Root mean square error (RMSE),
Correlation (R2), Nash-Sutcliffe efficiency criterion, Auto correlated Gaussian error model,
Inverse error variance, Exponential transformation of error variance, etc. All these likelihood
functions have their various ways to relate the likelihood of a simulated hydrograph against
their observed counterparts [9].
Threshold value definition: This value is chosen with respect to the likelihood function
chosen for the analysis and it can be very subjective to the modeler except if otherwise stated
from an earlier requirement or something.
Detailed Procedure for GLUE method analysis
This is a step-wise method of the GLUE method;
Step 1: Select a hydrological model for the analysis and determine the range of every
parameter of the model.
Step 2: Select a likelihood measure, likelihood threshold value and total sampling
time or number of behavioral models needed.
Step 3: Repeatedly run the hydrological model for the sampling time or till the
number of behavioral models are achieved. Behavioral models are indexed
by r (1 ≤ 𝑟 ≤ 𝑀)and each model is associated with its corresponding
likelihood function.
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 11
Step 4: This is the core of the GLUE method
a) run the hydrological model M times with the behavioral models to obtain
M different predictions [Qs(t,r)], then each prediction is assigned a weight
which is normally the likelihood value of the model.
b) The weighted predictions are then ranked in ascending order denoted by
{Q[t, r(i)], l[r(i)]}and then a cumulative distributed distribution (CDF) is
made with the formula below;
Pr{𝑄 ≤ Q[t, r(i)]} =∑ l[r(j)]𝑖
𝑗=1
∑ l[r(j)]𝑀𝑗=1
c) For each certainty level, find the two quartiles and Median value.
The GLUE method normally has a low efficiency in its prediction limits capturing more
observed discharge was applied to a catchment due to 2 groups of errors. The first are those
erors that are independent of the method like uncertainties in the model structure, input data,
etc. while the second group are based on GLUE induced subjective decisions such as the
choice of threshold value, assumption of the unknown probability distribution of model
parameters, etc. The improved/proposed method is aims to treat the errors in the first group
because these errors are passed down into the prediction limits therefore affecting the
percentage of observed flow. The above stated step-wise can be improved as stated in the L.
Xiong paper [5] by the addition simulation-correcting module to reduce the errors in the
simulation output of models. The first 3 steps of the modified GLUE method are same to the
GLUE method while the steps differ slightly from then on and the steps are stated below;
Detailed Procedure for the Modified GLUE method analysis.
Step 1: Select a hydrological model for the analysis and determine the range of every
parameter of the model.
Step 2: Select a likelihood measure, likelihood threshold value and total sampling
time or number of behavioral models needed.
Step 3: Repeatedly run the hydrological model for the sampling time or till the
number of behavioral models are achieved. Behavioral models are indexed
by r (1 ≤ 𝑟 ≤ 𝑀)and each model is associated with its corresponding
likelihood function.
Step 4: The aim is to plot a simulation bias curve that compares the simulated against
observed data
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 12
a) For the rth
(each) behavioral model, divide the predicted runoff into equal
intervals and denoted the number of intervals as ψ .
b) For the Kth interval [between Vk and Vk+1] (1 ≤ 𝑘 ≤ 𝛙), identify all the
simulated runoff values that fall within the interval and calculate the sum
(SSk), then do the same for the corresponding observed values (SOk: observed)
c) For the interval, a simulation bias curve can be achieved by plotting br(k)
against their corresponding interval for the each model
𝑏𝑟 (𝑘) =𝑆𝑆𝑘
𝑆𝑂𝑘
Step 5: a) The hydrological model is run M times with the M behavioral parameter
sets with constant inputs and observations, thereby producing M predictions
[Qs(t,r)], each prediction is then assigned a weight which is normally its
corresponding likelihood function.
b) For each prediction Qs(t,r) (1 ≤ 𝑟 ≤ 𝑀) , estimate the prediction’s Br(t)
value by looking at the rth
parameter set’s constructed simulation bias curve
c) Rank all value of Br(t) (1 ≤ 𝑟 ≤ 𝑀) in ascending order and find the
median value Bmedian(t).
d) Correct all the weighted prediction values with the formula below;
Qs(t, r) ←Qs(t, r)
𝐵𝑚𝑒𝑑𝑖𝑎𝑛(𝑡)
e) Rank the M corrected predictions Qs(t,r) in ascending order, then construct
a cumulative probability distribution (CDF) for the corrected runoff
prediction
f) For a given certainty level, find the two quartiles and Median value
As previously stated, the major difference between the GLUE model and the Modified GLUE
model addition simulation-correcting module which could be seen in the step-wise
application of the both methods; it could be observed that in the modified method, a
simulation bias curve is plotted for every behavioral parameter set and at every time interval
of the simulated hydrograph, then all prediction values of all intervals for the same prediction
are corrected by dividing them with their median bias value before the derivation of the
prediction limits.
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 13
Example of the GLUE method used in previous studies: GLUE method has been
implemented in many papers/case studies but particularly in the L. Xiong paper, it showed
comparisons between the GLUE method and the Modified GLUE method with their
application to the King and Qingjiang catchment using the SMAR conceptual model. The
paper concluded that the Modified GLUE method increased the number of behavioral models
without any change to the threshold value over the GLUE method.
This paper firstly shows the implementation of GLUE method in general then
secondly showed the superiority of the Modified version to the normal version [5]. The data
below show the implementation of the above points;
In the King catchment:
Table 1: Showing the both versions in the King catchment.
Where Containing ratio (CR) means the proportion of the observed data contained within
the prediction limits.
Fig. 2: Showing the CR values of the both versions in the King catchment.
From the Table 1 and Fig. 2 above, it can be observed that they both agree with the paper’s
conclusion for the King catchment.
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 14
In the Qingjing catchment:
Table 2: Showing the both versions in the Qingjing catchment.
Fig. 3: Showing the CR values of the both versions in the Qingjing catchment.
From the Table 2 and Fig. 3 above, it can be observed that they both agree with the paper’s
conclusion for the Qingjing catchment.
3.3) Differential Evolution Adaptive Metropolis (DREAM)
The DREAM method is a formal Bayesian method that is an adaptive from the
MCMC sampling method. The formal Bayesian method is a purely statistical method that
offers a simple way to combine several probability distributions using Bayes theorem, in
hydrological terms, the method has the ability to systematically estimate uncertainty within a
single cohesive and consolidated method. To implement the method, sampling methods are
needed to produce the posterior density function (PDF) of parameters which combines the
model likelihood with the assumed parameter distribution. Unfortunately, the PDF in most
cannot be analytical achieved so therefore Monte Carlo (MCMC) sampling is used to
generate a sample of it. The heart of an MCMC method is the Markov chain that generates a
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 15
random stepping throughout the parameter with their limits with a stable frequency. After
experiment and analysis, it is observed that the convergence to a posterior distribution is
often very slow which normally results from the inappropriate initial choice parameter
distribution used to generate trial moves in the Markov chain. To improve this phenomenon
and make the model more efficient, the proposed distribution’s orientation and scale is tuned
during the iterations to make the sample PDF to its target form by the help of information
from the previous step and this self-adaptive randomized subspace sampling is what makes
DREAM an adaptive MCMC algorithm. In practice, the MCMC evolution is run/repeated
until the procedure shows convergence to a stationary posterior distribution (target PDF). The
DREAM method has received a surge of popularity over other MCMC methods (e.g. STEM,
RWM) before of the adaptive form that gives it efficiency in solving highly non-linear and
multimodal target distributions, advantage of maintaining detailed balance and ergodicity.
With all this said, DREAM and MCMC methods in general assume that error are
addictive and fundamentally random and this may not be the case in reality but will instead
be a lack of knowledge and understanding of the processes [10].
Detailed Procedure for DREAM MCMC method analysis
Step 1: Draw an initial population X of size N, typically N = d or 2d (where
represents the number of parameters to be estimated) using the specified
distribution.
Step 2: Compute the density p(Xi|Y, Φ)of each point of X, where i=1,….,N using the
antilog of the likelihood function.
FOR i = 1,….,N DO (CHAIN EVOLUTION)
Step 3: Generate a candidate point, zi in chain i,
𝒛𝒊 = 𝒙𝒊 + 𝜸(𝜹). (∑ 𝒙𝒓(𝒋)
𝜹
𝒋=𝟏
− ∑ 𝒙𝒓(𝒏)
𝜹
𝒏=𝟏
)
Where δ represents the number of pairs used to achieve the candidate point
r(j) ≠ r(n) ≠ i.
γ value depends on the number of pairs to create proposal, a good choice is
𝛾 = 2.38/√2𝛿𝑑𝑒𝑓𝑓 , with deff = d, but potentially decreased in next step.
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 16
Step 4: Replace each element, j = 1,….,d of the proposal Zij with x
ij using the
binominal scheme with crossover probability CR,
𝑧𝑖𝑗 = {
𝑥𝑖𝑗 𝑖𝑓 𝑈 ≤ 1 − 𝐶𝑅, 𝑑𝑒𝑓𝑓 = 𝑑𝑒𝑓𝑓 − 1
𝑧𝑖𝑗 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
j = 1,…,d
Where U є [0,1] is a draw from the uniform distribution
Step 5: Compute p(Xi|Y, Φ) and accept the candidate point with Metropolis
acceptance probability, ∝ (𝑥𝑖 , 𝑧𝑖)
∝ (𝑥𝑖 , 𝑧𝑖) = {𝑚𝑖𝑛 (
p(zi|Y, Φ)
p(xi|Y, Φ), 1) 𝑖𝑓 p(Xi|Y, Φ) > 0
1 𝑖𝑓 p(Xi|Y, Φ) = 0
Step 6: If the candidate point is accepted, move the chain, xi = z
i; otherwise remain at
the old location, xi.
END FOR (CHAIN EVOLUTION)
Step 7: Remove potential outlier chains using the inter-quartile range.
Step 8: Compute the Gelman-Rubin (Rstat) convergence diagnostic.
Step 9: If Rstat ≤ 1.2, stop, otherwise go to CHAIN EVOLUTION.
As discussed in the J. A. Vrugt’s 2008 paper, The DREAM method could be
improved by the use of a single rainfall multiplier for each storm event to describe the
Rainfall forcing (input) data. By the use of these multipliers to vary in hydrologic period
ranges, system based errors in the rainfall input data can be corrected and parameter inference
and stream flow predictions can also be improved [10].
Example of the DREAM method used in previous studies: The DREAM method has been
applied in many papers but the J. A. Vrugt’s 2008 paper, it compared the DREAM method
with and without the use of rainfall multipliers in the Leaf river and French Broad watersheds
and the paper concluded by stating that the analysis with the multipliers produced better
results [10].
This paper firstly shows the implementation of DREAM method in general then
secondly showed the superiority of the Method with multipliers (Modified) version to the
normal version [10]. The table below shows the implementation of the above points;
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 17
Table 2: Showing the both versions in the both catchments.
From the table above, it can be observed that the results support the conclusion made by the
paper because the Case study 2 (DREAM with rainfall multipliers) have better values in the
both catchment than Case study 1 (DREAM without rainfall multipliers).
3.4) Major Differences Between the GLUE and DREAM Estimation Methods.
Although the both methods could be used to quantify parameter errors, as previously
stated, they both have their differences. With the obvious being that the GLUE method takes
an informal Bayesian approach while the DREAM takes a formal Bayesian approach, there
are still some more in-depth differences. The table below states some in-depth differences
from research from other papers (Literature review) and Note that this list will be improved
upon after more research with the both models in the Dissertation;
Table 1: Showing the differences between the DREAM and GLUE method.
Properties GLUE method DREAM method
Concept GLUE results are based on weighted
simulations from the set of behavioral
models.
DREAM simulations uses a first-order
auto correlated error model with explicit
information form the stream flow
observations.
Separation of errors
sources
GLUE can’t individually separate errors
sources
DREAM can individually separate
errors sources
Errors This is not the case for GLUE Assumes erors are addictive and
fundamentally random.
Ease of
Implementation
Easier of Implementation. Harder of Implementation.
Efficiency Not that good for finding behavioral
models.
Much better at finding behavioral
models due to adaptive updating
component.
User casualties The method could be very subjective with
the choice of the threshold value.
This is not the case for DREAM.
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 18
4) THE CATCHMENT: Description
The study catchment is the Upper Medway catchment which is located on the south of
London, 50km from the Thurnham radar which has an area of around 220km2 .The Medway
river flows for about 112 km in the county of Kent, starting flow from East Grinstead in
Sussex and flowing into the Thames estuary which is a main tributary to the lower basin. The
catchment contains complex topography and a combination of permeable (chalk) and
impermeable (clay) geologies. The average annual rainfall and annual potential
evapotranspiration is about 729mm and 633mm respectively. Predicting and managing fluvial
flooding has been historically referred to as mostly difficult in the Medway catchment and
this is due to the convergence of sub catchments with very different responses to rainfall and
tidal influences from the wide Thames Estuary. The sluice gates at Leigh control the high
flows of the River Medway which is operational during high rainfalls on the Upper Medway
catchment. These gates control the flow before the Chafford water level gauge but without
consistent functional regulations, it is hard to calculate its influence on the Chafford
discharge. The presence of many natural springs in the catchment, soft and muddy soils,
increases the speed of the rainfall-runoff procedure as the soil is almost always fully saturated
during autumn and winter and this brings an assumption that the ground water flow is
relatively high and these ground water levels are very close to the surface during both
periods. There is a water storage area that was confirmed by Google maps and window live
map, near the upstream of the River Medway. The structure is located close to Forest Row
with a 1.5 mile long stretch of water and a full capacity of 1,237 gallons of water and area of
2.8km2
[9].
It was found by Dr D.Zhr [9] during his conducted visit to the catchment that it had
accessible hydrological gauges including 9 tipping buckets rain gauges, 4 water level gauges
and 3 stream-flow gauges. Rainfall in the catchment is measured by the tipping bucket rain
gauge (TBR), telemeters and storage gauges while rainfall data was obtained automatically
from the TBR but it is calibrated using data from rainfall storage gauges which is next to it,
there is normally about 8% discrepancy in the calibration. Manual measurements of water
level are taken once a month and are compared to that of the logger and these values are
plotted on a graph. The diagram below gives a visual explanation to the Catchment.
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 19
Fig. 2: Showing the Upper Medway catchment.
4.1) Work done in 3rd
year thesis
Last year during my final project with the same catchment and data, I did my research
on Hydrological forecasting and Model Optimization where my research was primarily
focused on the impact of data length in model calibration and effect of selective calibration
on model performance. The Case study used the Probability Distributed Model (PDM) to
analyze the data from Upper Medway catchment. The rainfall data is gotten from the tipping
bucket rain gauge and calibrated with the storage gauges while the water level or outflow of
catchment is gotten from the water level gauges and stream-flow gauges. It can be noticed
that limited amount of recording devices were present during the readings, therefore data was
lumped data [5c]. The model data for the catchment was aimed at Collier’s land bridge and
the readings were done for a period from 1 July 2006 to 8 December 2007 (about 1.5 years).
The input data was initially ran in PDM to get the simulated flow data and this is compared to
the real data and the comparison criterions are then used to analyze how the simulated data is
similar to the real data. When this was done for the entire data set, The R2 value was 0.3966
(40% correlation) and RSME was 3.0025 (7.7% error), as it can be noticed that according to
the criterions, the simulated data is a poor comparison to the real data.
This previous case study is related to the present GLUE method analysis because
although, last year’s case study did not estimate errors, the both case studies are still model
optimization techniques. Model calibration (Manual and Automatic calibration) as previously
defined in this paper, is the adjusting of model parameters to fit a real catchment while Model
uncertainty estimation measures (GLUE and DREAM) try fit the real data through the
removal of errors. This shows that the both techniques although try to achieve the same goal,
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 20
model uncertainty estimation are better because they consider errors while calibration
techniques don’t [Please refer to Section 2]. The uncertainty estimation techniques have been
developed to quantify these uncertainties/errors in the model simulation, thereby aid better
decisions made with respect to just model calibration methods in a model.
Firstly in the research, the 2 year long data was divided into 2 non-equal parts for
calibration and verification (Experiment 1) and then the model was then selectively calibrated
(Experiment 2) where the data length included for calibration and verification were just
High/peak flow (since flooding was the point of the case study) and in this calibration data
range, the data was strategically divided to change length at every calibration. The
experiment was achieved as shown in the Fig 3 below;
Figure 3: the calibration and verification data for Experiment 2.
The paper concluded by saying that Experiment 2 showed better results than
Experiment 1 which meant that selective calibration worked while The 3 calibrations in
Experiment 2 showed a contrary results to the expected result which was the longer the data,
the better the model. Experiment 2 was concluded that the reason for this subjective of
automatic calibration, longer data is needed and presence of errors in data and results errors.
At this point of the analysis, Uncertainty estimation techniques like GLUE method should
have been implemented to check the basis of the conclusion of this case study [11].
0
10
20
30
40
50
1 10001 20001 30001 40001 50001
Flo
w (
m^3
/s)
Time step
CALIBRATION 3
VERIFICATION CALIBRATION 2
CALIBRATION 3
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 21
5) Hydrological Predictions for the Environment (HYPE)
The proposed model for this analysis is the Hydrological Predictions for the
Environment (HYPE) model. It is a dynamic, semi-distributed, process-based and integrated
catchment model recently developed by the Swedish Meteorological and Hydrological
Institute (SMHI). It can be both used for small and large (E-HYPE, India-HYPE, etc.) scale
assessments in hydrological, nutrient and water transport applications. The model code is
written in FORTRAN computer language which runs on either Windows or Linux and is
open sourced under the GNU Lesser General Public license. The code is divided into the
hydrological simulation system (HYSS) and the actual hydrological model (HYPE). HYSS is
the computational framework component that runs the input and output to the model and also
its time stepping procedures while The HYPE part is the main hydrological model that
contains defined parameters, output variables and procedures of simulating various
hydrological sections [12].
HYPE model has a very simplified file structure which is a text file format for all
input and outputs which contain 203 various variables. The input are grouped into time series
input data (which is forcing data and observation series) and geographical input data while
the Output is grouped into time series of water and nutrients, and performance criteria (can be
likelihood function). The following are the five mandatory input files for the a successful run
of the model; Pobs.txt, Tobs.txt, par.txt, GeoData.txt and GeoClass.txt. The Pobs.txt
and Tobs.txt files contain forcing time series of perception and temperature, the GeoData.txt
file contain the geographical input data for each subbasin while GeoClass.txt files contain
their classification information (calculation units) and the par.txt file contain the model
parameters that determine the function of the model (which can be land use type, soil type or
a general value). The output file with not special request is the log-file with information about
running time and files used but the model can also provide the performance criteria (such as
Nash-Sutcliffe Efficiency, Correlation Coefficient, Mean Absolute Error, etc.) for every
subbasin in the subass1.txt file [13].
The basic concept of HYPE model which describes the flow and transformation of
water, nutrients and organic carbon in the soil, lake and rivers can be broken down into the
following; Processes above the ground, Land routines, Rivers and lake, Nitrogen and
phosphorus in land routines, Nitrogen and phosphorus in rivers and lakes, water management,
deep processes and organic carbon. All these hydrological processes are the basic concepts of
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 22
the HYPE model which are explained in detail in the reference [13]. These processes are
shown in great detail in the Fig. 4 and Fig. 5 below;
Fig. 4: Showing the various components of the HYPE model
Fig. 5: Showing the entire concept of the HYPE model
Although the model has a wide range of applications, it needed for this case study for its
rainfall-runoff modelling application and to tackle runoff modelling, the 4 basic concepts of
the HYPE model used are Processes above the ground, Land routines, Rivers and lake and
Deep processes. The Processes above the ground concept is mostly divided into temperature,
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 23
precipitation and evaporation; the temperature and precipitation is adjusted through various
methods in the model (please refer to reference [13]) but the model assumes mixed (rain and
snow) precipitation occurs when the air temperature is around than threshold temperature and
purely liquid (rain) and solid (snow) when the air temperature is greater and lesser
respectively to the threshold temperature. While in evaporation, the potential evaporation
only occurs when the air temperature is greater than threshold value and evaporation form the
soil is assumed to occur only in the 2 upper layers (if more than 1) and it decreases with
depth. The land routines could be divided into soil water, snow routines and glaciers but with
respect to the case study, we focus mainly on Soil water (Rainfall). The model contains up to
3 soil layers with each layer having their various properties (such as water retention,
thickness, porosity, etc.). Groundwater runoff in the model only occurs when the soil water
surpasses the field capacity of the soil layers and this runoff occurs in all number of layers
while if infiltration is greater than zero, it is added to the upper layer but surface runoff
occurs when the high ground water table reaches the surface. The figure below shows the soil
classifications in the model;
Fig. 6: Showing the soil classifications in the HYPE model.
In the HYPE model, there are two types of rivers and they are local stream and main river
while lakes are also divided into local and outlet lakes and all these classification all have
threshold/percentage values that above which flow occurs. The model assumes that all local
runoff enters the local stream then some of the local runoff flows into the local flow while the
rest goes directly to the main river. The main river receives all the local runoff from the
combination of the local lakes and local stream and the inflow from upstream subbasins and
the main river empties in the outlet lakes which finally flow into downstream subbasins.
Either the river or lake could be represented to be 1-dimensional (no precipitation added) or
soil land-use/area. The diagram below shows the described process;
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 24
Fig. 7: Showing the Lake and River classifications in the HYPE model.
Finally, Deep processes below ground are divided into regional groundwater flow and
aquifers. The regional groundwater flow model simulates groundwater flow between
subbasins but no explicit storage of deep groundwater while the aquifer model simulates
storages and delay before regional groundwater reaches its destination [13].
As the modeller, the HYPE model could be applied to a lot of hydrological
applications river levels application, transport of pollutants, water quantity, etc. but in this
case study, it is needed to verify the Catchment river flow data. The HYPE model was chosen
for this analysis mainly because of the Open source GNU Lesser General Public license
which gives the modeller, the ability to modify the program to suit the particular problem.
Although this would be the case, a possible drawback will be the over 200 model parameters
(although most might not be used in a particular application) which might make sampling of
the model lengthy and the time for each run might be long thereby increase computational
cost.
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 25
6) Applications of Parameter Estimation Methods to Case Study
6.1) GLUE method
The Modified GLUE method will be used for this analysis as preferred to the
normally GLUE method for reasons previously stated in this paper and The application of the
method to the Upper Medway catchment using the HYPE model could be easily summarized
in the Detailed Procedure for GLUE method analysis in Section (3.2) previously discussed.
The Choices for the GLUE method application is as follows;
Choice of Hydrological model: As previously stated, the Hydrological Predictions for the
Environment (HYPE) model will be used for this analysis mainly because of Open source
GNU Lesser General Public license.
Generating parameter sets: The Monte Carlo methods and Latin-Hypercube simulations
(LHS) will be both applied in the GLUE method application because of their efficiency and
robustness, to compare which sampling would produce the best model.
Choice of likelihood function: The Nash-Sutcliffe efficiency likelihood coefficient will be
used for this analysis because it quantitatively describes the accuracy of model simulations to
the observed hydrograph. The coefficient varies from −∞ to 1 where an efficiency of 1
corresponds to a perfect match of modelled discharge to the observed flow, then an efficiency
of 0 indicates that the model simulations are as accurate as the mean of the observed data,
whereas an efficiency of less than zero occurs when the observed mean is better than the
model simulations. Essentially, the closer the model efficiency is to 1, the more accurate the
model.
Threshold value definition: During previous research, the maximum achieved likelihood
function was 0.82, so therefore, the analysis will have a likelihood value of 0.7 (this value is
totally subjective).
6.2) DREAM method
The application of the DREAM method with rainfall multipliers to the Upper
Medway catchment using the HYPE model could be easily summarized in the Detailed
Procedure for DREAM method analysis in Section (3.3) previously discussed.
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 26
7) Conclusion
At this point, this case study has introduced the concept of model parameter
uncertainty estimation, went though some literature that was very related to the topic, in great
detail explained the GLUE and DREAM uncertainty estimation methods (assumptions/basic
concepts, stet-wise method, etc.), introduced the catchment and discussed my previous case
study in model calibration in same catchment, gave a concise but detailed knowledge of the
Hydrological Predictions for the Environment (HYPE) model and finally, explained the
application of the both methods to the HYPE model and catchment in general.
With all this said, the research will continue in my Dissertation by the actual
application of the GLUE and DREAM methods to the Upper Medway catchment using the
HYPE model and then results obtained will be analyzed and then compared to results from
other case studies and a suitable conclusion of all the analysis will be done with respect to the
expected results.
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 27
References
[1] Beven K. Rainfall-runoff modelling : the primer. 2nd ed. New York: J. Wiley; 2000;: 1-23
[2] Vrugt, Jasper A. et al. 'Equifinality Of Formal (DREAM) And Informal (GLUE) Bayesian Approaches In Hydrologic
Modeling?'. Stoch Environ Res Risk Assess 23.7 (2008): 1011-1026. Web.
[3] Liu, Yuqiong, and Hoshin V. Gupta. 'Uncertainty In Hydrologic Modeling: Toward An Integrated Data Assimilation
Framework'. Water Resour. Res. 43.7 (2007): n/a-n/a. Web.
[4] Beven, Keith, and Jim Freer. 'Equifinality, Data Assimilation, And Uncertainty Estimation In Mechanistic Modelling Of
Complex Environmental Systems Using The GLUE Methodology'. Journal of Hydrology 249.1-4 (2001): 11-29. Web.
[5] Xiong, Lihua, and Kieran M. O’Connor. 'An Empirical Method To Improve The Prediction Limits Of The GLUE
Methodology In Rainfall–Runoff Modeling'. Journal of Hydrology 349.1-2 (2008): 115-124. Web.
[6] He, Minxue et al. 'Corruption Of Parameter Behavior And Regionalization By Model And Forcing Data Errors: A
Bayesian Example Using The SNOW17 Model'. Water Resour. Res. 47.7 (2011): n/a-n/a. Web.
[7] YouTube,. 'Dr. Keith Beven - Breakthroughs In Uncertainty Estimation'. N.p., 2015. Web. 13 May 2015.
[8] van Griensven, A. et al. 'A Global Sensitivity Analysis Tool For The Parameters Of Multi-Variable Catchment
Models'. Journal of Hydrology 324.1-4 (2006): 10-23. Web.
[9] Zhu D. A Comparative Study of Single and Dual Polarisation Quantitative Weather Radar in the Context of Hydrological
Modelling Structure. 2010;:103-109
[10] Vrugt, Jasper A. et al. 'Treatment Of Input Uncertainty In Hydrologic Modeling: Doing Hydrology Backward With
Markov Chain Monte Carlo Simulation'. Water Resour. Res. 44.12 (2008): n/a-n/a. Web.
[11] Mmeka F., Xaun Y. ‘Hydrological Forcasting and Model Optimisation: The Impact of Data Length and Selective
Calibration’. 2014.
[12] Hype.sourceforge.net,. 'HYPE - Hydrological Predictions For The Environment'. N.p., 2015. Web. 19 May 2015.
[13] Smhi.se,. 'HYPE | SMHI'. N.p., 2013. Web. 19 May 2015.
[14] Georgakakos, Konstantine P et al. 'Towards The Characterization Of Streamflow Simulation Uncertainty Through
Multimodel Ensembles'. Journal of Hydrology 298.1-4 (2004): 222-241. Web.
[15] Vrugt, J. A. et al. 'Accelerating Markov Chain Monte Carlo Simulation By Differential Evolution With Self-Adaptive
Randomized Subspace Sampling'. International Journal of Nonlinear Sciences and Numerical Simulation 10.3 (2009): n.
pag. Web.
[16] Ferraris, Luca, Roberto Rudari, and Franco Siccardi. 'The Uncertainty In The Prediction Of Flash Floods In The
Northern Mediterranean Environment'. J. Hydrometeor 3.6 (2002): 714-727. Web.
[17] Raftery, Adrian E., David Madigan, and Jennifer A. Hoeting. 'Bayesian Model Averaging For Linear Regression
Models'. Journal of the American Statistical Association 92.437 (1997): 179-191. Web.
[18] Adel Saleh, Ahmed. 'UNCERTAINTY ANALYSIS OF HYDROLOGICAL MODELS: NZOIA RIVER CASE
STUDY'. (2008): n. pag. Print.
[19] Rochester, Rebekah Elizabeth Lyn. Uncertainty In Hydrological Modelling: A Case Study In The Tern Catchment,
Shropshire, Uk (2010): N. Pag. Print.
[20] Montanari, Alberto. 'What Do We Mean By ‘Uncertainty’? The Need For A Consistent Wording About Uncertainty
Assessment In Hydrology'. Hydrol. Process. 21.6 (2007): 841-845. Web.
EGIM14: Computational Case Study Mmeka Felix (663327)
Dr. Yunqing Xaun Page 28
[21] Jin, Xiaoli et al. 'Parameter And Modeling Uncertainty Simulated By GLUE And A Formal Bayesian Method For A
Conceptual Hydrological Model'. Journal of Hydrology 383.3-4 (2010): 147-155. Web.
[22] Li, Lu et al. 'Evaluation Of The Subjective Factors Of The GLUE Method And Comparison With The Formal Bayesian
Method In Uncertainty Assessment Of Hydrological Models'. Journal of Hydrology390.3-4 (2010): 210-221. Web.
[23] Blasone, Roberta-Serena et al. 'Generalized Likelihood Uncertainty Estimation (GLUE) Using Adaptive Markov Chain
Monte Carlo Sampling'. Advances in Water Resources 31.4 (2008): 630-648. Web.
[24] Shen, Z. Y., L. Chen, and T. Chen. 'Analysis Of Parameter Uncertainty In Hydrological And Sediment Modeling Using
GLUE Method: A Case Study Of SWAT Model Applied To Three Gorges Reservoir Region, China'. Hydrology and Earth
System Sciences 16.1 (2012): 121-132. Web.
[25] Stedinger, Jery R. et al. 'Appraisal Of The Generalized Likelihood Uncertainty Estimation (GLUE) Method'. Water
Resour. Res. 44.12 (2008): n/a-n/a. Web.
[26] Li, Zhijie et al. 'Generalized Likelihood Uncertainty Estimation Method In Uncertainty Analysis Of Numerical
Eutrophication Models: Take BLOOM As An Example'. Mathematical Problems in Engineering2013 (2013): 1-9. Web.
[27] Pechlivanidis, I. G., and B. Arheimer. 'Large-Scale Hydrological Modelling By Using Modified PUB
Recommendations: The India-HYPE Case'. Hydrology and Earth System Sciences Discussions 12.3 (2015): 2885-2944.
Web.
[28] an Vliet, Michelle T.H. 'Ensemble Of European Water Resources Projections'. Future European water resources: A
demonstration case-study at the European scale D7.5 (2013): n. pag. Print.
[29] Strömqvist, Johan et al. 'Water And Nutrient Predictions In Ungauged Basins: Set-Up And Evaluation Of A Model At
The National Scale'. Hydrological Sciences Journal 57.2 (2012): 229-247. Web.
[30] Pechlivanidis, I. G., and B. Arheimer. 'Large-Scale Hydrological Modelling By Using Modified PUB
Recommendations: The India-HYPE Case'. Hydrology and Earth System Sciences Discussions 12.3 (2015): 2885-2944.
Web.
[31] Vrugt, Jasper A. et al. 'Response To Comment By Keith Beven On “Equifinality Of Formal (DREAM) And Informal
(GLUE) Bayesian Approaches In Hydrologic Modeling?”'. Stoch Environ Res Risk Assess 23.7 (2008): 1061-1062. Web.
Top Related