Sparse Covariance Estimates for High Dimensional Classification Using the Cholesky Decomposition (original) (raw)

Abstract

Results in time series analysis literature state that through the Cholesky decomposition, covariance estimates can be stated as a sequence of regressions. Furthermore, these results imply that the inverse of the covariance matrix can be estimated directly. This leads to a novel approach for approximating covariance matrices in high dimensional classification problems based on the Cholesky decomposition. By assuming that some of the targets in these regressions can be set to zero, simpler estimates for class-wise covariance matrices can be found. By reducing the number of parameters to estimate in the classifier, good generalization performance is obtained. Experiments on three different feature sets from a dataset of images of handwritten numerals show that simplified covariance estimates from the proposed method is competitive with results from conventional classifiers such as support vector machines.

Chapter PDF

Similar content being viewed by others

References

  1. Dempster, A.: Covariance selection. Biometrics (1), 157–175 (1972)
    Article Google Scholar
  2. Golub, G.H., Van Loan, C.F.: Matrix computations, 3rd edn. John Hopkins University Press (1996)
    Google Scholar
  3. Smith, M., Kohn, R.: Parsimonius covariance matrix estimation for longitudinal data. Journal of the American Statistical Association 97(460), 1141–1153 (2002)
    Article MATH MathSciNet Google Scholar
  4. Bilmes, J.A.: Factored sparse inverse covariance matrices. In: Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2000), vol. 2, pp. 1009–1012 (2000)
    Google Scholar
  5. Pouhramadi, M.: Foundations of Time Series Analysis and Prediction Theory. Wiley, Chichester (2001)
    Google Scholar
  6. Whittaker, J.: Graphical models in applied multivariate statistics. Wiley, Chichester (1990)
    MATH Google Scholar
  7. Hastie, T., Tibshirani, R., Buja, A.: Flexible discriminant analysis and mixture models. Journal of the American Statistical Association 89(428), 1255–1270 (1994)
    Article MATH MathSciNet Google Scholar
  8. Jain, A.K., Duin, R.P., Mao, J.: Statistical pattern recognition: A review. IEEE Trans. Pattern Anal. Machine Intell. 22(1), 4–37 (2000)
    Article Google Scholar

Download references

Author information

Authors and Affiliations

  1. Department of Informatics, University of Oslo, Norway
    Asbjørn Berge & Anne Schistad Solberg

Authors

  1. Asbjørn Berge
  2. Anne Schistad Solberg

Editor information

Editors and Affiliations

  1. Hong Kong University of Science and Technology,
    Dit-Yan Yeung
  2. Department of Computer Science, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China
    James T. Kwok
  3. Instituto de Telecomunicações, Instituto Superior Técnico, Lisbon, Portugal
    Ana Fred
  4. Department of Electrical and Electronic Engineering, University of Cagliari, Piazza d’Armi, 09123, Cagliari, Italy
    Fabio Roli
  5. Faculty of Electrical Engineering, Mathematics and Computer Science, Information and Communication Theory Group, Delft University of Technology, Delft, The Netherlands
    Dick de Ridder

Rights and permissions

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Berge, A., Solberg, A.S. (2006). Sparse Covariance Estimates for High Dimensional Classification Using the Cholesky Decomposition. In: Yeung, DY., Kwok, J.T., Fred, A., Roli, F., de Ridder, D. (eds) Structural, Syntactic, and Statistical Pattern Recognition. SSPR /SPR 2006. Lecture Notes in Computer Science, vol 4109. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11815921\_92

Download citation

Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Publish with us