Validation metrics for ice edge position forecasts

Abstract

The ice edge is a simple quantity in the form of a line that can be derived from a spatially varying sea ice concentration field. Due to its long history and relevance for operations in the Arctic, the position of the ice edge should be an essential element in any system that is designed to monitor or provide forecasts for the physical state of the Arctic Ocean and adjacent ocean regions. Users of monitoring and forecast products for sea ice must be provided with complementary information on the expected accuracy of the data or model results. Such information is traditionally available as a set of metrics that provide an assessment of the information quality. In this study we provide a survey of metrics that are presently included in the product quality assessment of the Copernicus Marine Environment Monitoring Service (CMEMS) Arctic Marine Forecasting Center sea ice edge position forecast. We show that when ice edge results from different products are compared, mismatching results for polynya and local freezing at the coasts of continents and archipelagos have a large impact on the quality assessment. Such situations, which occur regularly in the products we examine, have not been properly acknowledged when sets of metrics for the quality of ice edge position results are constructed. We examine the quality of ice edge forecasts using a total of 15 metrics for the ice edge position. These metrics are analysed in synthetic examples, as well as in selected cases of actual forecasts, and finally for a full year of weekly forecast bulletins. Using necessity and simplicity of information as a guideline, we recommend using a set of four metrics that sheds light on the various aspects of product quality that we consider. Moreover, any user is expected to be interested in a limited part of the geographical domain, so metrics derived as domain-wide integrated quantities may be of limited value. Consequently, we recommend that metrics also be made available for an appropriate set of sub-domains. Furthermore, we find that the metrics decorrelation timescales are much longer than the present forecast range. Hence, our final recommendation is to include depictions of gridded mismatching ice edge positions using maps for the integrated ice edge error.