Nonlinear identification using orthogonal forward regression with nested optimal regularization

Downloads

[1] A. E. Ruano, Ed., Intelligent Control Systems Using Computational Intelligence Techniques. London, U.K.: IEE Publishing, 2005. [2] R. Murray-Smith and T. A. Johansen, Multiple Model Approaches to Modeling and Control. Bristol, PA, USA: Taylor and Francis, 1997. [3] S. G. Fabri and V. Kadirkamanathan, Functional Adaptive Control: An Intelligent Systems Approach. London, U.K.: Springer, 2001. [4] S. Chen, C. F. N. Cowan, and P. M. Grant, “Orthogonal least squares learning algorithm for radial basis function networks,” IEEE Trans. Neural Netw., vol. 2, no. 2, pp. 302–309, Mar. 1991. [5] S. R. Gun, “Support vector machines for classification and regression,” Dept. Electron. Comput. Sci., ISIS Group, University of Southampton, Southampton, U.K., May 1998. [6] G. C. Cawley and N. L. C. Talbot, “On over-fitting in model selection and subsequent selection bias in performance evaluation,” J. Mach. Learn. Res., vol. 11, pp. 2079–2107, Jul. 2010. [7] D. Du, K. Li, X. Li, M. Fei, and H. Wang, “A multi-output twostage locally regularized model construction method using the extreme learning machine,” Neurocomputing, vol. 128, pp. 104–112, Mar. 2014. [8] M. Stone, “Cross-validatory choice and assessment of statistical predictions,” J. Royal Stat. Soc. B, vol. 36, no. 2, pp. 111–147, 1974. [9] G. H. Golub, M. Heath, and G. Wahba, “Generalized cross-validation as a method for choosing good ridge parameter,” Technometrics, vol. 21, no. 2, pp. 215–223, 1979. [10] S. Chen, Y. Wu, and B. L. Luk, “Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks,” IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 1239–1243, Sep. 1999. [11] M. J. L. Orr, “Regularisation in the selection of radial basis function centers,” Neural Comput., vol. 7, no. 3, pp. 606–623, 1995. [12] X. Hong and S. A. Billings, “Parameter estimation based on stacked regression and evolutionary algorithms,” IEE Proc. Control Theory Appl., vol. 146, no. 5, pp. 406–414, Sep. 1999. [13] L. Ljung and T. Glad, Modeling of Dynamic Systems. Englewood Cliffs, NJ, USA: Prentice Hall, 1994. [14] S. Chen, S. A. Billings, and W. Luo, “Orthogonal least squares methods and their applications to non-linear system identification,” Int. J. Control, vol. 50, no. 5, pp. 1873–1896, 1989. [15] M. J. Korenberg, “Identifying nonlinear difference equation and functional expansion representations: The fast orthogonal algorithm,” Ann. Biomed. Eng., vol. 16, no. 1, pp. 123–142, 1988. [16] D. Du, X. Li, M. Fei, and G. W. Irwin, “A novel locally regularized automatic construction method for RBF neural models,” Neurocomputing, vol. 98, pp. 4–11, Dec. 2012. [17] L. Wang and J. M. Mendel, “Fuzzy basis functions, universal approximation, and orthogonal least-squares learning,” IEEE Trans. Neural Netw., vol. 3, no. 5, pp. 807–814, Sep. 1992. [18] X. Hong and C. J. Harris, “Neurofuzzy design and model construction of nonlinear dynamical processes from data,” IEE Proc. Control Theory Appl., vol. 148, no. 6, pp. 530–538, Nov. 2001. [19] Q. Zhang, “Using wavelets network in nonparametric estimation,” IEEE Trans. Neural Netw., vol. 8, no. 2, pp. 227–236, Mar. 1993. [20] S. A. Billings and H. L. Wei, “The wavelet-NARMAX representation: A hybrid model structure combining polynomial models with multiresolution wavelet decompositions,” Int. J. Syst. Sci., vol. 36, no. 3, pp. 137–152, 2005. [21] R. H. Myers, Classical and Modern Regression with Applications, 2nd Ed. Boston, MA, USA: PWS-KENT, 1990. [22] X. Hong, P. M. Sharkey, and K. Warwick, “Automatic nonlinear predictive model construction using forward regression and the PRESS statistic,” IEE Proc. Control Theory Appl., vol. 150, no. 3, pp. 245–254, May 2003. [23] S. Chen, X. Hong, C. J. Harris, and P. M. Sharkey, “Sparse modeling using orthogonal forward regression with PRESS statistic and regularization,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 34, no. 2, pp. 898–911, Apr. 2004. [24] S. A. Billings, S. Chen, and R. Backhouse, “The identification of linear and nonlinear models of a turbocharged automotive diesel engine,” Mech. Syst. Signal Process., vol. 3, no. 2, pp. 123–142, 1989. [25] S. S. Chen, D. L. Donoho, and M. A. Saunders, “Atomic decomposition by basis pursuit,” SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33–61, 1998. [26] R. Tibshirani, “Regression shrinkage and selection via the lasso,” J. Royal Stat. Soc. B, vol. 58, no. 1, pp. 267–288, 1996. [27] B. Efron, I. Johnstone, T. Hastie, and R. Tibshirani, “Least angle regression,” Ann. Stat., vol. 32, no. 2, pp. 407–451, 2004. [28] S. Chen, “Locally regularized orthogonal least squares algorithm for the construction of sparse kernel regression models,” in Proc. 6th Int. Conf. Signal Process., Beijing, China, 2002, pp. 1229–1232. [29] D. J. C. MacKay, “Bayesian methods for adaptive models,” Ph.D. dissertation, Dept. Comput. Neural Syst., California Inst. Technol., Pasadena, CA, USA, 1992. [30] S. Chen, X. Hong, and C. J. Harris, “Sparse kernel regression modeling using combined locally regularized orthogonal least squares and D-optimality experimental design,” IEEE Trans. Autom. Control, vol. 48, no. 6, pp. 1029–1036, Jun. 2003. [31] S. Chen, X. Hong, and C. J. Harris, “Non-linear system identification using particle swarm optimization tuned radial basis function models,” Int. J. Bio-Ins. Comput., vol. 1, no. 4, pp. 246–258, 2009. [32] S. Chen, X. Hong, and C. J. Harris, “Construction of tunable radial basis function networks using orthogonal forward selection,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 39, no. 2, pp. 457–466, Apr. 2009. [33] S. Chen, X. Hong, and C. J. Harris, “Particle swarm optimization aided orthogonal forward regression for unified data modeling,” IEEE Trans. Evol. Comput., vol. 14, no. 4, pp. 477–499, Aug. 2010. [34] S. Chen and S. A. Billings, “Representation of nonlinear systems: The NARMAX model,” Int. J. Control, vol. 49, no. 3, pp. 1013–1032, 1989. [35] S. A. Billings and W. Voon, “A prediction-error and stepwise-regression estimation algorithm for nonlinear systems,” Int. J. Control, vol. 44, no. 3, pp. 803–822, 1986. [36] G. E. P. Box and G. M. Jenkins, Time Series Analysis, Forecasting and Control. San Francisco, CA, USA: Holden-Day, 1976. [37] A. Frank and A. Asuncion. (2010). UCI Machine Learning Repository. [Online]. Available: http://archive.ics.uci.edu/ml