Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciarán

Charlton-Perez, A. J. ORCID: https://orcid.org/0000-0001-8179-6220, Dacre, H. F. ORCID: https://orcid.org/0000-0003-4328-9126, Driscoll, S., Gray, S. L. ORCID: https://orcid.org/0000-0001-8658-362X, Harvey, B. ORCID: https://orcid.org/0000-0002-6510-8181, Harvey, N. J. ORCID: https://orcid.org/0000-0003-0973-5794, Hunt, K. M. R. ORCID: https://orcid.org/0000-0003-1480-3755, Lee, R. W. ORCID: https://orcid.org/0000-0002-1946-5559, Swaminathan, R. ORCID: https://orcid.org/0000-0001-5853-2673, Vandaele, R. and Volonté, A. ORCID: https://orcid.org/0000-0003-0278-952X (2024) Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciarán. npj Climate and Atmospheric Science, 7. 93. ISSN 2397-3722 doi: 10.1038/s41612-024-00638-w

Abstract/Summary

There has been huge recent interest in the potential of making operational weather forecasts using machine learning techniques. As they become a part of the weather forecasting toolbox, there is a pressing need to understand how well current machine learning models can simulate high-impact weather events. We compare forecasts of Storm Ciarán, a European windstorm that caused sixteen deaths and extensive damage in Northern Europe, made by machine learning and numerical weather prediction models. The four machine learning models considered (FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2) produce forecasts that accurately capture the synoptic-scale structure of the cyclone including the position of the cloud head, shape of the warm sector and location of warm conveyor belt jet, and the large-scale dynamical drivers important for the rapid storm development such as the position of the storm relative to the upper-level jet exit. However, their ability to resolve the more detailed structures important for issuing weather warnings is more mixed. All of the machine learning models underestimate the peak amplitude of winds associated with the storm, only some machine learning models resolve the warm core seclusion and none of the machine learning models capture the sharp bent-back warm frontal gradient. Our study shows there is a great deal about the performance and properties of machine learning weather forecasts that can be derived from case studies of high-impact weather events such as Storm Ciarán.

Item Type	Article
URI	https://reading-clone.eprints-hosting.org/id/eprint/115971
Identification Number/DOI	10.1038/s41612-024-00638-w
Refereed	Yes
Divisions	Science > School of Mathematical, Physical and Computational Sciences > National Centre for Earth Observation (NCEO) Science > School of Mathematical, Physical and Computational Sciences > NCAS Science > School of Mathematical, Physical and Computational Sciences > Department of Meteorology
Uncontrolled Keywords	weather forecasting; AI; machine learning; storm Ciarán
Publisher	Nature Publishing Group
Download/View statistics	View download statistics for this item