SAGE : final report on the AWS nowcast and forecasting system and research advances

Beggan, C.D.; Eastwood, J.; Forsyth, C.; Freeman, M. ORCID: https://orcid.org/0000-0002-8653-8279; Heyns, M.; Huebert, J.; Richardson, G.S.; Smith, A.. 2023 SAGE : final report on the AWS nowcast and forecasting system and research advances. Nottingham, UK, British Geological Survey, 55pp. (OR/24/013) (Unpublished)

Before downloading, please read NORA policies.

Preview

Text OR24013.pdf Download (4MB) | Preview

Abstract/Summary

This report describes the inputs, processing and outputs of the SWIMMR N4 (SAGE) Met Office Amazon Web Services (AWS) Elastic Cloud Compute (EC2) system. The SAGE system ingests real time magnetic measurements from the British Geological Survey’s (BGS) UK geomagnetic observatories and solar wind data measured by satellites at the L1 Lagrange point, provided by the NOAA Space Weather Prediction Centre (SWPC). The data are captured and disseminated by through a Met Office SWIMMR database via a password-protected API. Four different machine learning models process the L1 solar wind data to produce: (i) a magnetic field forecast for up to one hour ahead of time at the three UK observatories, known as SPIDER; (ii) the probability of a substorm occurring in the next hour (named M1 or the Substorm Forecast); (iii) the probability of a sudden storm commencement in the next hour (denoted M2 or Shock Impact Assessment); and (iv) the probability of the magnetic field rate of change exceeding a series of threshold values at each of the UK observatories in the next hour (called M3 or Extreme Threshold Exceedance Forecast). A magnetohydrodynamic (MHD) model of the magnetosphere, driven by L1 solar wind data, is run separately on a Met Office HPC system before its data are passed into the SWIMMR API. The code, called GorgonOps, produces magnetic field estimates at the three UK observatories as well as maps of the ionospheric current systems for visualisation purposes. GorgonOps is the operational version of the more general Gorgon global magnetosphere model, and the two names are used interchangeably in this document. The magnetic field data – either measured for nowcasting or modelled for forecasting – are convolved with a ground conductivity model to produce a set of geoelectric field maps based on the rate of change of the magnetic field which induces a geoelectric field. The snapshot geoelectric field map of Britain is applied to models of (i) the high voltage power grid, (ii) high pressure gas network and (iii) model of the railway system to compute the Geomagnetically Induced Currents (GICs) flowing through them. The model outputs are delivered back to the Met Office API via an automated collection process on the AWS. The processing is set to run every five minutes, assuming that nowcast and forecast magnetic field data from GorgonOps, and L1 solar wind data are available to the Met Office API (which feeds into SPIDER/Substorm Forecast, Shock Impact Assessment and Extreme Threshold Exceedance Forecast models). The code to process the data has been set up via docker and docker-compose, allowing modular changes to be made in future. The system is provided as a Github repository which provides full version control and integration into a process that allows the build and deploy to AWS to occur automatically upon triggering a Github commit. Finally, the report describes the initial validation of the models either within AWS or offline.

Item Type:	Publication - Report
Funders/Sponsors:	British Geological Survey, British Antarctic Survey, Imperial College London, University College London
Additional Information. Not used in RCUK Gateway to Research.:	This item has been internally reviewed, but not externally peer-reviewed.
Date made live:	18 Jul 2024 11:11 +0 (UTC)
URI:	https://nora.nerc.ac.uk/id/eprint/537731

Actions (login required)

View Item

Document Downloads

More statistics for this item...