Skill of data-based predictions versus dynamical models

Siegert, S., Bröcker, J. and Kantz, H. (2015) Skill of data-based predictions versus dynamical models. In: Chavez, M., Ghil, M. and Urrutia-Fucugauchi, J. (eds.) Extreme Events — Observations, Modeling, and Economics. John Wiley & Sons, Inc, pp. 35-50. ISBN 9781119157014

Abstract/Summary

We compare probabilistic predictions of extreme temperature anomalies issued by two different forecast schemes. One is a dynamical physical weather model, the other a simple data model. We recall the concept of skill scores in order to assess the performance of these two different predictors. Although the result confirms the expectation that the (computationally expensive) weather model outperforms the simple data model, the performance of the latter is surprisingly good. More specifically, for some parameter range, it is even better than the uncalibrated weather model. Since probabilistic predictions are not easily interpreted by the end user, we convert them into deterministic yes/no statements and measure the performance of these by ROC statistics. Scored in this way, conclusions about model performance partly change, which illustrates that predictive power depends on how it is quantified.