On the choice of training data for machine learning of geostrophic mesoscale turbulence

Data plays a central role in data-driven methods, but is not often the subject of focus in investigations of machine learning algorithms as applied to Earth System Modeling related problems. Here we consider the problem of eddy-mean interaction in rotating stratified turbulence in the presence of lateral boundaries, where it is known that rotational components of the eddy flux plays no direct role in the sub-grid forcing onto the mean state variables, and its presence is expected to affect the performance of the trained machine learning models. While an often utilized choice in the literature is to train a model from the divergence of the eddy fluxes, here we provide theoretical arguments and numerical evidence that learning from the eddy fluxes with the rotational component appropriately filtered out, achieved in this work by means of an object called the eddy force function, results in models with comparable or better skill, but substantially reduced sensitivity to the presence of small-scale features. We argue that while the choice of data choice and/or quality may not be critical if we simply want a model to have predictive skill, it is highly desirable and perhaps even necessary if we want to leverage data-driven methods to aid in discovering unknown or hidden physical processes within the data itself.