Imputation of missing sub-hourly precipitation data in a large sensor network: a machine learning approach
Chivers, Benedict D.; Wallbank, John; Cole, Steven J.; Sebek, Ondrej; Stanley, Simon; Fry, Matthew ORCID: https://orcid.org/0000-0003-1142-4039; Leontidis, Georgios. 2020 Imputation of missing sub-hourly precipitation data in a large sensor network: a machine learning approach. Journal of Hydrology, 588, 125126. 12, pp. https://doi.org/10.1016/j.jhydrol.2020.125126
Before downloading, please read NORA policies.
Text
Chivers Imputation of missing sub-hourly precipitation data in a large sensor (1) (1).pdf Restricted to NORA staff only Download (1MB) |
Abstract/Summary
Precipitation data collected at sub-hourly resolution represents specific challenges for missing data recovery by being largely stochastic in nature and highly unbalanced in the duration of rain vs non-rain. Here we present a two-step analysis utilising current machine learning techniques for imputing precipitation data sampled at 30-minute intervals by devolving the task into (a) the classification of rain or non-rain samples, and (b) regressing the absolute values of predicted rain samples. Investigating 37 weather stations in the UK, this machine learning process produces more accurate predictions for recovering precipitation data than an established surface fitting technique utilising neighbouring rain gauges. Increasing available features for the training of machine learning algorithms increases performance with the integration of weather data at the target site with externally sourced rain gauges providing the highest performance. This method informs machine learning models by utilising information in concurrently collected environmental data to make accurate predictions of missing rain data. Capturing complex non-linear relationships from weakly correlated variables is critical for data recovery at sub-hourly resolutions. Such pipelines for data recovery can be developed and deployed for highly automated and near instantaneous imputation of missing values in ongoing datasets at high temporal resolutions.
Item Type: | Publication - Article |
---|---|
Digital Object Identifier (DOI): | https://doi.org/10.1016/j.jhydrol.2020.125126 |
UKCEH and CEH Sections/Science Areas: | Water Resources (Science Area 2017-) Hydro-climate Risks (Science Area 2017-) |
ISSN: | 0022-1694 |
Additional Keywords: | machine learning, data imputation, gradient boosted trees, environmental sensor networks, precipitation, soil moisture |
NORA Subject Terms: | Hydrology Data and Information |
Date made live: | 08 Jun 2020 10:39 +0 (UTC) |
URI: | https://nora.nerc.ac.uk/id/eprint/527887 |
Actions (login required)
View Item |
Document Downloads
Downloads for past 30 days
Downloads per month over past year