Rainfall and river flow ensemble verification: Phase 2. Final report

Anderson, Seonaid R.; Moore, Robert J.; Cole, Steven J.; Csima, Gabriella; Cole, Sebastian; Crocker, Ric; Mittermaier, Marion. 2021 Rainfall and river flow ensemble verification: Phase 2. Final report. Wallingford, UK, UK Centre for Ecology & Hydrology, 47pp. (Unpublished)

Before downloading, please read NORA policies.

Preview	Text (Main Report) Ensemble Verification Phase 2 Final Report Main.pdf Download (3MB) | Preview
Preview	Text (Appendix) Ensemble Verification Phase 2 Final Report Appendix.pdf - Supplemental Material Download (48MB) | Preview
	Text (Appendix B.2.2 Supplementary Plots (zip file)) Appendix_B_2_2_Overall_Verification_Summary_River_Flow_plots.zip - Supplemental Material Download (438MB)
	Text (Appendix B.5.2 Supplementary Plots (zip file)) Appendix_B_5_2_Compare_G2G_Rainfall_Source_Plots.zip - Supplemental Material Download (21MB)
	Text (Appendix C.3.2 Supplementary Plots (zip file)) Appendix_C_3_2_Case_Study_Analysis_Hydrological_Impacts_plots.zip - Supplemental Material Download (671MB)

Abstract/Summary

This document is the Final Report for the “Rainfall and River Flow Ensemble Verification: Phase 2” (EnsVerP2) project forming part of the project “Improving confidence in Flood Guidance through verification of rainfall and river flow ensembles”. Forecasting the weather and floods is a challenging task and inherently uncertain. Acknowledging and accounting for the uncertainty in precipitation and flood forecasts has become increasingly important. This has partly been driven by the move of warning and guidance services to risk-based approaches that combine the likelihood of flooding with its potential impact on society and the environment. In the UK, such risk-based approaches underpin the National Severe Weather Warning Service delivered by the Met Office, and the Flood Guidance Statements produced by the Flood Forecasting Centre (FFC) and Scottish Flood Forecasting Service (SFFS). A standard approach to accounting for forecast uncertainty is to use ensemble methods. For a number of years, FFC and SFFS have used precipitation ensembles coupled with the national Grid-to-Grid (G2G) model of river flow to underpin the Flood Guidance Statement. However, the performance of the overall end-to-end ensemble precipitation and river flow forecasting system is currently not verified routinely. This ensemble verification information and evidence is essential: its absence can limit end-user confidence and inhibit full exploitation for flood-risk guidance. In addition, the local flood forecasting systems - used by the Environment Agency (EA), Scottish Environment Protection Agency (SEPA) and Natural Resources Wales (NRW) - are planned to transition to ensemble forecasting and will have similar requirements for verification information. A first step in addressing this operational gap has been to bring together existing expertise in meteorological and hydrological model performance assessment to design and develop a new, holistic Ensemble Verification Framework. Then to consider how this Framework could be used to develop an operational end-end interactive Ensemble Forecast Visualisation and Verification System. The Framework has been designed so that the operational system developed from it would help forecasters answer the following two key questions. • How well has the ensemble precipitation and flood forecasting system performed in the (recent) past? Particularly for flood events of interest. • What does this mean for interpreting today’s forecast? Forecasters could then make more informed decisions and increase their confidence in the use of ensembles for forecasting the severity and likelihood of precipitation and flooding. To develop and test the potential verification approaches and operational displays, 16-months of precipitation and river flow ensemble forecasts have been processed and verified. Specific case-studies, identified with the help of stakeholders, have been used to prototype, demonstrate, assess and refine the verification tools. The Best Medium Range (BMR) precipitation ensemble is used as input to the national-scale G2G model of river flow across Great Britain and to a small selection of catchment-scale PDM local models of river flow. This approach has allowed rigorous scientific exploration of how to provide robust verification statistics of the ensemble precipitation inputs to the river flow modelling and of its ensemble river flow outputs. The scientific analysis allowed identification of several points relevant to the underpinning verification methodology part of the Framework. • Three different precipitation accumulation time-intervals were evaluated: 15 min (the temporal resolution of the river flow model and its precipitation inputs), hourly and daily. Daily precipitation accumulations appear to provide the best guidance in terms of rain volume for hydrological impacts. One reason for this may well be because it removes the impact of timing errors at the sub-daily scale. Sub-daily precipitation can be more closely related to river flow on an ensemble member-by-member basis. • The source of observed precipitation (raingauge, radar or merged raingauge-radar) has an impact on the verification analyses and G2G river flow performance. • The change in precipitation-intensity characteristics with lead-time between the STEPS, MOGREPS-UK and MOGREPS-G components of the BMR precipitation ensemble, are evident in both rainfall and river flow analysis. • The length of period used for ensemble verification is an important factor: generally longer than two years is recommended if possible. The 16-month test period was sufficient for generating enough precipitation threshold-exceedances for the 95th percentile thresholds: but insufficient for higher thresholds and for considering river flow thresholds above one half the median annual maximum flood at sub-regional scales. • New methods of presenting the precipitation forecast probabilities have been developed for precipitation thresholds that are hydrologically relevant. The verification of these Time-Window Probabilities (TWPs) has shown that the probabilities are larger, and also more reliable: so users can have greater confidence in using them. For new real-time displays to be of value in operational settings, it is important that users (e.g. FFC hydrometeorologists or flood forecasting officers) find the displays understandable and easy to deploy in support of flood guidance and warning. Operational users have been engaged in co-design of the real-time forecast displays through the Project Board and a Workshop. These interactions have identified that the real-time displays need to be flexible and informative, with varying layers of detail. Viewing the precipitation and river flow together, however, is the most important ingredient along with using common methods for conveying information on both. Prototype joint rainfall and river flow displays have been created. Further co-design of interactive displays is recommended during future implementation and interactions should include operational users, researchers and system developers. Case-studies have been used to highlight the potential benefits of these new real-time displays. They have demonstrated how the ensemble verification information can help users make more informed decisions when ensemble verification information is included. For example, knowing whether a forecast is over- or under-confident for different lead-times and severity-thresholds can be very helpful, particularly in marginal cases. That is if a forecast has a tendency to predict too high or low a probability of precipitation, or of river flow, exceeding a given level of severity. Summary and key recommendation: Realising the benefit and value of probabilistic flood-risk information for decision-making was a key motivator for the “Rainfall and River Flow Ensemble Verification: Phase 2” project. The project succeeded in bringing together the meteorology and hydrology to define, test and demonstrate a joint Ensemble Verification Framework for ensemble precipitation and river flow forecasts. The outcomes of the project demonstrate how the subsequent verification information can be used to enhance the user’s perception and ability to deploy ensemble forecasts and derived probabilities in day-to-day flood risk decision-making. Overall, the key finding is that joint precipitation and river flow ensemble verification is possible and useful. The primary recommendation is that an end-to-end interactive Ensemble Forecast Visualisation and Verification System for FFC (and SFFS) be implemented as soon as is practicable. The Ensemble Verification Framework provides the blueprint for the system and the Joint Coding Framework developed and applied here provides the basis for the algorithm and code. A detailed set of recommendations have been provided, including what is required for operational implementation. This also includes a priority list of recommendations for developing a minimum system. The proposed system would address the current urgent operational gap in ensemble forecast verification capability for FFC and SFFS. It would mark a significant addition to the forecasters’ toolkit by providing real-time displays that incorporate ensemble verification information for the first time, and in a usable form. In turn, this will facilitate enhanced and more informed decision-making at times of potential flood-risk. Local model systems have ensemble and probabilistic flood forecasting as an aspiration in their future plans. These systems would eventually benefit from the operationally urgent developments recommended here for the national-scale G2G model used by FFC and SFFS. Local model users could play an early and active part in system co-design as part of a staged implementation process for local model systems.

Item Type:	Publication - Report (Project Report)
UKCEH and CEH Sections/Science Areas:	Hydro-climate Risks (Science Area 2017-)
Additional Pages:	Appendices
Funders/Sponsors:	Flood Forecasting Centre, Environment Agency, Scottish Environment Protection Agency, Natural Resources Wales
Additional Information. Not used in RCUK Gateway to Research.:	This report should be cited as follows: Anderson, S.R., Moore, R.J., Cole, S.J., Csima, G., Cole, S., Crocker, R., Mittermaier, M. (2021) Rainfall and River Flow Ensemble Verification: Phase 2. Final Report. Contract Report to FFC/SEPA/EA/NRW. UK Centre for Ecology & Hydrology and Met Office, Wallingford, UK, 47pp plus Appendices.
Additional Keywords:	verification, precipitation, river flow, flood, hydrological, model, forecast, ensemble, uncertainty, operational, EnsVerP2
NORA Subject Terms:	Hydrology Meteorology and Climatology Atmospheric Sciences
Date made live:	03 Aug 2021 12:04 +0 (UTC)
URI:	https://nora.nerc.ac.uk/id/eprint/530828

Actions (login required)

View Item

Document Downloads

More statistics for this item...