Post on 27-Apr-2023
AGU Fall Meeting Dec 9, 2013 1
INCREASING LEAD TIME IN SHORT-
RANGE STREAMFLOW FORECASTING
VIA THE HYDROLOGIC ENSEMBLE
FORECAST SERVICE (HEFS)
Dong-Jun Seo1, Manabendra Saharia1,*, Bob Corby2, Frank Bell2 and
James Brown3
1Dept of Civil Eng, The University of Texas at Arlington, Arlington, TX 2NWS West Gulf River Forecast Center, Fort Worth, TX 3Hydrologic Solutions Limited, Bournemouth, United Kingdom *Now at University of Oklahoma
Short-Range Ensemble streamflow forecasting
AGU Fall Meeting Dec 9, 2013 2 AGU Fall Meeting Dec 9, 2013
In this presentation
• Motivation
• Questions
• Approach
• Tools used
• Community Hydrologic Prediction System (CHPS)
• Hydrologic Ensemble Forecast Service (HEFS)
• Meteorological Ensemble Forecast Processor (MEFP)
• Ensemble Streamflow Prediction (ESP)
• Ensemble Post-Processor (EnsPost)
• Ensemble Verification System (EVS)
• See Demargne et al. in Jan 2014 issue of Bulletin of Amer.
Meteorol. Soc.
• Results
• Conclusions and recommendations
Short-Range Ensemble streamflow forecasting
AGU Fall Meeting Dec 9, 2013 3 AGU Fall Meeting Dec 9, 2013
Why ensemble forecasting?
• Quantify forecast uncertainty
• Improve forecast accuracy
• Extend forecast lead time
• Improve cost-effectiveness of investment
Short-Range Ensemble streamflow forecasting
In 2006, National Research Council
recommended that NWS produce uncertainty-
quantified products, expand verification and make
information easily available to all users in near
real time.
AGU Fall Meeting Dec 9, 2013 4 AGU Fall Meeting Dec 9, 2013
Motivation
• The current practice at the NWS/WGRFC
• Quantitative Precipitation Forecast (QPF) is produced out to
Day 3, but only Day-1 (or less) QPF is input to hydrologic
models (zero precipitation assumed beyond)
• In the single-valued forecasting paradigm, such a practice is
inevitable to avoid highly erroneous river forecasts
• In the ensemble forecasting paradigm, one may input longer-
lead QPF to potentially increase the lead time of river forecast
• Utilizes all available skill in short-range QPF
Short-Range Ensemble streamflow forecasting
AGU Fall Meeting Dec 9, 2013 5 AGU Fall Meeting Dec 9, 2013
Questions
• What is the bang from the Hydrologic Ensemble
Forecast Service (HEFS) for short-range river
forecasting under the existing QPF process?
• Compared to using Day 1 QPF only, what does using
Day 1-3 QPF via HEFS bring to short-range
streamflow forecasting?
• What is the quality of:
•The ensemble QPF (EQPF) generated by MEFP
from RFC-produced single-valued QPF
•The resulting short-range ESP forecast
•The ESP forecast post-processed via EnsPost
Short-Range Ensemble streamflow forecasting
AGU Fall Meeting Dec 9, 2013 6 AGU Fall Meeting Dec 9, 2013
Study area
• 5 headwater basins in the
Upper Trinity River Basin in
North Texas
• SAC-SMA, UHG operations
Short-Range Ensemble streamflow forecasting
JAKT2
BRPT2 SGET2
GLLT2
DCJT2
6
AGU Fall Meeting Dec 9, 2013 7 AGU Fall Meeting Dec 9, 2013
Approach
• Ensemble hindcasting and verification using CHPS and
HEFS
• Generate EQPF from the RFC-produced single-valued
QPF using MEFP (Schaake et al. 2007, Wu et al. 2011)
• Input uncertainty processor
• Feed EQPF to ESP
• Post-process ESP ensembles using EnsPost (Seo et al.
2007)
• Hydrologic uncertainty processor
• Verify EQPF and short-range ESP ensembles using
EVS (Brown et al. 2010)
Short-Range Ensemble streamflow forecasting
AGU Fall Meeting Dec 9, 2013 8 AGU Fall Meeting Dec 9, 2013
Ensemble hindcasting experiments
• Ensemble precipitation hindcasts are generated using MEFP for a
period of 7 years (2004-2010)
• 46 members, max lead time of 14 days
• Experiment 1
• Day 1 EQPF from Day 1 single-valued QPF
• Climatological ensembles for Days 2-14
• Experiment 2
• Day 1-3 EQPF from Day 1-3 single-valued QPF
• Climatological ensembles for Days 4-14
• The EQPFs are ingested into ESP to produce raw streamflow
hindcasts
• The ESP hindcasts are processed by EnsPost to produce post-
processed ensemble streamflow hindcasts
Short-Range Ensemble streamflow forecasting
AGU Fall Meeting Dec 9, 2013 9 AGU Fall Meeting Dec 9, 2013
Results
• Reliability diagram
• Measures reliability (or unbiasedness in probability)
• Requisite for ensemble forecasts
• Relative Operating Characteristic (ROC)
• Measures discrimination
• Not sensitive to reliability
• Closely related to economic value (Zhu et al. 2002)
• Allows interpretation in the single-valued sense through
probability of detection (POD) and false alarm rate (FAR)
• ROC area is related to Pearson’s correlation of ensemble
mean forecast with verifying obs. via joint distribution
among ensemble member, ensemble mean and verifying
obs.
Short-Range Ensemble streamflow forecasting
AGU Fall Meeting Dec 9, 2013 10
Short-Range Ensemble streamflow forecasting
Reliability diagram for Day-1 EQPF, BRPT2
MEFP
ensembles
are, in general,
reasonably
reliable
AGU Fall Meeting Dec 9, 2013 11
Short-Range Ensemble streamflow forecasting
Reliability diagram for Day-1 ESP forecast, BRPT2
Raw ESP
ensembles are
generally not
very reliable
AGU Fall Meeting Dec 9, 2013 12
Short-Range Ensemble streamflow forecasting
Reliability diagram for Day-1 ESP forecast w/ EnsPost, BRPT2
Post-processed
ESP ensembles
are reasonably
reliable (a hint of
underforecasting
seen here)
AGU Fall Meeting Dec 9, 2013 13
Short-Range Ensemble streamflow forecasting
Reliability diagram for Day-1 EQPF, GLLT2
MEFP
ensembles
are, in general,
reasonably
reliable
AGU Fall Meeting Dec 9, 2013 14
Short-Range Ensemble streamflow forecasting
Reliability diagram for Day-1 ESP forecast, GLLT2
ESP
ensemble is
relatively of
high quality
AGU Fall Meeting Dec 9, 2013 15
Short-Range Ensemble streamflow forecasting
Reliability diagram for Day-1 ESP forecast w/ EnsPost, GLLT2
EnsPost renders
ESP ensembles
reliable
AGU Fall Meeting Dec 9, 2013 16
Short-Range Ensemble streamflow forecasting
N Improvement due to
Day 2-3 QPF
BRPT2, threshold = 95th percentile flow
Increase in ~ 1 day in lead time
AGU Fall Meeting Dec 9, 2013 17
Short-Range Ensemble streamflow forecasting
Improvement due to Day
2-3 QPF & EnsPost
BRPT2, threshold = 95th percentile flow
EnsPost needs improvement
AGU Fall Meeting Dec 9, 2013 18
Short-Range Ensemble streamflow forecasting
Improvement due
to Day 2-3 QPF
GLLT2, threshold = 95th percentile flow
AGU Fall Meeting Dec 9, 2013 19
Short-Range Ensemble streamflow forecasting
Improvement due to Day
2-3 QPF & EnsPost
GLLT2, threshold = 95th percentile flow
AGU Fall Meeting Dec 9, 2013 20 AGU Fall Meeting Dec 9, 2013
Translating increase in ROC score
• To relate increase in ROC score to improvement
in forecast quality in a single-valued sense, we
compare ROC’s in terms of increase in probability
of detection (POD) at user-specified false alarm
rate (FAR)
• The more conservative the user is, the lower
the acceptable FAR is.
Short-Range Ensemble streamflow forecasting
AGU Fall Meeting Dec 9, 2013 21
Short-Range Ensemble streamflow forecasting
Acceptable FAR is 5%
ROC curves for Day-1 ESP forecast forced by Day-1 QPF ROC curves for Day-1 ESP forecast forced by Day1 QPF
AGU Fall Meeting Dec 9, 2013 22
Short-Range Ensemble streamflow forecasting
Acceptable FAR is 5%
ROC curves for Day-1 post-processed ESP forecast forced by Day 1-3 QPF
AGU Fall Meeting Dec 9, 2013 23
Short-Range Ensemble streamflow forecasting
Increase in POD in ESP forecast due to adding Day 2-3 QPF FAR = 5 %
About 10% increase in POD for Day 3-4
AGU Fall Meeting Dec 9, 2013 24
Short-Range Ensemble streamflow forecasting
Increase in POD due to adding Day 2-3 QPF & EnsPost (FAR=5%)
EnsPost has the largest positive impact for Day 1.
AGU Fall Meeting Dec 9, 2013 25
Short-Range Ensemble streamflow forecasting
Increase in POD due to adding Day 2-3 QPF (FAR=5%)
As the threshold flow decreases, the positive impact of Day 2-3 QPF decreases.
AGU Fall Meeting Dec 9, 2013 26
Short-Range Ensemble streamflow forecasting
Increase in POD due to adding Day 2-3 QPF & EnsPost (FAR=5%)
As the threshold flow decreases, the positive impact of EnsPost increases.
AGU Fall Meeting Dec 9, 2013 27 AGU Fall Meeting Dec 9, 2013
Conclusions and recommendations
• Compared to using Day1-only QPF, using RFC-produced
Day 1-3 single-valued QPF via HEFS significantly
increases skill in short-range ESP forecast
• Increases POD by about 10% for Day 3-4 forecasts
• Extends lead time by about a day
• The margin of improvement is impacted by the quality of
streamflow simulation
• Demonstrated large-sample ensemble hindcasting using
CHPS and HEFS outside of NWS
Short-Range Ensemble streamflow forecasting
AGU Fall Meeting Dec 9, 2013 28
0102030405060708090
100
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
Pro
bab
ilit
y o
f D
ete
cti
on
in
%
0500100015002000250030003500400045005000
Nu
mb
er
of
Even
ts
Probability of Detection Number of Events
Source
NWS/OCWWS
Perspective
Short-Range Ensemble streamflow forecasting
Flash flood warning Probability Of Detection (POD) trend
AGU Fall Meeting Dec 9, 2013 29 AGU Fall Meeting Dec 9, 2013
Conclusions and recommendations
(cont.) • Apply to a large number of basins for larger-sample
verification, particularly for large events
• Carry out similar hindcasting experiments using the
GEFS reforecast data set
• Enhance EnsPost to deal with ephemeral basins
• Implement and evaluate ensemble data assimilation
within HEFS
• Include parametric uncertainty modeling
• Reduce reliance on stochastic modeling
Short-Range Ensemble streamflow forecasting
AGU Fall Meeting Dec 9, 2013 30
Short-Range Ensemble streamflow forecasting
Elements of a Hydrologic Ensemble Prediction System
Ensemble Pre-Processor
Parametric
Uncertainty
Processor
Data
Assimilator
Ensemble Post-
Processor
Hydrology & Water Resources
Ensemble Product Generator
Hydrology & Water
Resources Models
QPF, QTF QPE, QTE,
Soil Moisture
Streamflow
Ensem
ble
Verific
atio
n S
yste
m
Input
Uncertainty
Processor
Hydrologic
Uncertainty
Processor
AGU Fall Meeting Dec 9, 2013 31
THANK YOU
Q/A, Discussion
For more info, please contact djseo@uta.edu.
Short-Range Ensemble streamflow forecasting