On the limitations of comparing mean squared forecast errors

Clements, M. P. ORCID: https://orcid.org/0000-0001-6329-1341 and Hendry, D. F. (1993) On the limitations of comparing mean squared forecast errors. Journal of Forecasting, 12 (8). pp. 617-637. ISSN 1099-131X doi: 10.1002/for.3980120802

Abstract/Summary

Linear models are invariant under non-singular, scale-preserving linear transformations, whereas mean square forecast errors (MSFEs) are not. Different rankings may result across models or methods from choosing alternative yet isomorphic representations of a process. One approach can dominate others for comparisons in levels, yet lose to another for differences, to a second for cointegrating vectors and to a third for combinations of variables. The potential for switches in ranking is related to criticisms of the inadequacy of MSFE against encompassing criteria, which are invariant under linear transforms and entail MSFE dominance. An invariant evaluation criterion which avoids misleading outcomes is examined in a Monte Carlo study of forecasting methods.