Probability density estimation with tunable kernels using orthogonal forward regression

Chen, S., Hong, X. ORCID: https://orcid.org/0000-0002-6832-2298 and Harris, C. J. (2010) Probability density estimation with tunable kernels using orthogonal forward regression. IEEE Transactions on Systems Man and Cybernetics Part B-Cybernetics, 40 (4). pp. 1101-1114. ISSN 1083-4419 doi: 10.1109/TSMCB.2009.2034732

Abstract/Summary

A generalized or tunable-kernel model is proposed for probability density function estimation based on an orthogonal forward regression procedure. Each stage of the density estimation process determines a tunable kernel, namely, its center vector and diagonal covariance matrix, by minimizing a leave-one-out test criterion. The kernel mixing weights of the constructed sparse density estimate are finally updated using the multiplicative nonnegative quadratic programming algorithm to ensure the nonnegative and unity constraints, and this weight-updating process additionally has the desired ability to further reduce the model size. The proposed tunable-kernel model has advantages, in terms of model generalization capability and model sparsity, over the standard fixed-kernel model that restricts kernel centers to the training data points and employs a single common kernel variance for every kernel. On the other hand, it does not optimize all the model parameters together and thus avoids the problems of high-dimensional ill-conditioned nonlinear optimization associated with the conventional finite mixture model. Several examples are included to demonstrate the ability of the proposed novel tunable-kernel model to effectively construct a very compact density estimate accurately.

Item Type	Article
URI	https://reading-clone.eprints-hosting.org/id/eprint/16726
Item Type	Article
Refereed	Yes
Divisions	Science > School of Mathematical, Physical and Computational Sciences > Department of Computer Science
Uncontrolled Keywords	diagonal covariance matrix , multiplicative nonnegative quadratic programming , nonlinear optimization , nonnegative constraint , orthogonal forward regression , probability density function estimation , tunable kernel , unity constraint
Download/View statistics	View download statistics for this item