Sparse modeling using orthogonal forward regression with PRESS statistic and regularization

Chen, S., Hong, X. ORCID: https://orcid.org/0000-0002-6832-2298, Harris, C. J. and Sharkey, P. M. (2004) Sparse modeling using orthogonal forward regression with PRESS statistic and regularization. IEEE Transactions on Systems Man and Cybernetics Part B-Cybernetics, 34 (2). pp. 898-911. ISSN 1083-4419 doi: 10.1109/tsmcb.2003.817107

Abstract/Summary

The paper introduces an efficient construction algorithm for obtaining sparse linear-in-the-weights regression models based on an approach of directly optimizing model generalization capability. This is achieved by utilizing the delete-1 cross validation concept and the associated leave-one-out test error also known as the predicted residual sums of squares (PRESS) statistic, without resorting to any other validation data set for model evaluation in the model construction process. Computational efficiency is ensured using an orthogonal forward regression, but the algorithm incrementally minimizes the PRESS statistic instead of the usual sum of the squared training errors. A local regularization method can naturally be incorporated into the model selection procedure to further enforce model sparsity. The proposed algorithm is fully automatic, and the user is not required to specify any criterion to terminate the model construction procedure. Comparisons with some of the existing state-of-art modeling methods are given, and several examples are included to demonstrate the ability of the proposed algorithm to effectively construct sparse models that generalize well.

Item Type	Article
URI	https://reading-clone.eprints-hosting.org/id/eprint/15171
Item Type	Article
Refereed	Yes
Divisions	Science > School of Mathematical, Physical and Computational Sciences > Department of Computer Science
Uncontrolled Keywords	Bayesian learning, cross validation, orthogonal forward regression, predicted residual sums of squares (PRESS), statistic, regularization, sparse data modeling, BASIS FUNCTION NETWORKS, LEAST-SQUARES, SYSTEM-IDENTIFICATION, MODEL, SELECTION, ALGORITHM, VALIDATION
Download/View statistics	View download statistics for this item