Sparse kernel density construction using orthogonal forward regression with leave-one-out test score and local regularization

Chen, S., Hong, X. ORCID: https://orcid.org/0000-0002-6832-2298 and Harris, C. J. (2004) Sparse kernel density construction using orthogonal forward regression with leave-one-out test score and local regularization. IEEE Transactions on Systems Man and Cybernetics Part B-Cybernetics, 34 (4). pp. 1708-1717. ISSN 1083-4419 doi: 10.1109/tsmcb.2004.828199

Abstract/Summary

This paper presents an efficient construction algorithm for obtaining sparse kernel density estimates based on a regression approach that directly optimizes model generalization capability. Computational efficiency of the density construction is ensured using an orthogonal forward regression, and the algorithm incrementally minimizes the leave-one-out test score. A local regularization method is incorporated naturally into the density construction process to further enforce sparsity. An additional advantage of the proposed algorithm is that it is fully automatic and the user is not required to specify any criterion to terminate the density construction procedure. This is in contrast to an existing state-of-art kernel density estimation method using the support vector machine (SVM), where the user is required to specify some critical algorithm parameter. Several examples are included to demonstrate the ability of the proposed algorithm to effectively construct a very sparse kernel density estimate with comparable accuracy to that of the full sample optimized Parzen window density estimate. Our experimental results also demonstrate that the proposed algorithm compares favorably with the SVM method, in terms of both test accuracy and sparsity, for constructing kernel density estimates.

Item Type	Article
URI	https://reading-clone.eprints-hosting.org/id/eprint/15167
Item Type	Article
Refereed	Yes
Divisions	Science > School of Mathematical, Physical and Computational Sciences > Department of Computer Science
Uncontrolled Keywords	cross validation, leave-one-out test score, orthogonal least squares, Parzen window estimate, probability density function, regularization, sparse kernel modeling, LEAST-SQUARES, IDENTIFICATION, ALGORITHM
Download/View statistics	View download statistics for this item