Bayesian Non-negative Matrix Factorization (original) (raw)

Abstract

We present a Bayesian treatment of non-negative matrix factorization (NMF), based on a normal likelihood and exponential priors, and derive an efficient Gibbs sampler to approximate the posterior density of the NMF factors. On a chemical brain imaging data set, we show that this improves interpretability by providing uncertainty estimates. We discuss how the Gibbs sampler can be used for model order selection by estimating the marginal likelihood, and compare with the Bayesian information criterion. For computing the maximum a posteriori estimate we present an iterated conditional modes algorithm that rivals existing state-of-the-art NMF algorithms on an image feature extraction problem.

Preview

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Paatero, P., Tapper, U.: Positive matrix factorization: A nonnegative factor model with optimal utilization of error-estimates of data values. Environmetrics 5(2), 111–126 (1994)
    Article Google Scholar
  2. Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)
    Article Google Scholar
  3. Sajda, P., Du, S., Parra, L.: Recovery of constituent spectra using non-negative matrix factorization. In: Wavelets: Applications in Signal and Image Processing, Proceedings of SPIE, vol. 5207, pp. 321–331 (2003)
    Google Scholar
  4. Gaussier, E., Goutte, C.: Relation between PLSA and NMF and implication. In: Proceedings of the International SIGIR Conference on Research and Development in Information Retrieval, pp. 601–602 (2005)
    Google Scholar
  5. Schmidt, M.N., Olsson, R.K.: Single-channel speech separation using sparse non-negative matrix factorization. In: ISCA International Conference on Spoken Language Processing, (INTERSPEECH) (2006)
    Google Scholar
  6. Ochs, M.F., Stoyanova, R.S., Arias-Mendoza, F., Brown, T.R.: A new method for spectral decomposition using a bilinear bayesian approach. Journal of Magnetic Resonance 137, 161–176 (1999)
    Article Google Scholar
  7. Moussaoui, S., Brie, D., Mohammad-Djafari, A., Carteret, C.: Separation of non-negative mixture of non-negative sources using a Bayesian approach and MCMC sampling. IEEE Transactions on Signal Processing 54(11), 4133–4145 (2006)
    Article Google Scholar
  8. Chib, S.: Marginal likelihood from the Gibbs output. Journal of the American Statistical Association 90(432), 1313–1321 (1995)
    Article MathSciNet MATH Google Scholar
  9. Besag, J.: On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society 48(3), 259–302 (1986)
    MathSciNet MATH Google Scholar
  10. Sajda, P., Du, S., Brown, T.R., Stoyanova, R., Shungu, D.C., Mao, X., Parra, L.C.: Nonnegative matrix factorization for rapid recovery of constituent spectra in magnetic resonance chemical shift imaging of the brain. IEEE Transactions on Medical Imaging 23(12), 1453–1465 (2004)
    Article Google Scholar
  11. Schmidt, M.N., Laurberg, H.: Nonnegative matrix factorization with gaussian process priors. Computational Intelligence and Neuroscience (February 2008)
    Google Scholar
  12. Graham, D.B., Allinson, N.M.: Characterizing virtual eigensignatures for general purpose face recognition. In: Face Recognition: From Theory to Applications. Computer and Systems Sciences, vol. 163, pp. 446–456 (1998)
    Google Scholar
  13. Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: Advances in Neural Information Processing Systems (NIPS), pp. 556–562 (2000)
    Google Scholar
  14. Kim, D., Sra, S., Dhillon, I.S.: Fast Newton-type methods for the least squares nonnegative matrix approximation problem. In: Proceedings of SIAM Conference on Data Mining (2007)
    Google Scholar

Download references

Author information

Authors and Affiliations

  1. Department of Engineering, University of Cambridge, UK
    Mikkel N. Schmidt
  2. DTU Informatics, Technical University of Denmark, Denmark
    Ole Winther & Lars Kai Hansen

Authors

  1. Mikkel N. Schmidt
  2. Ole Winther
  3. Lars Kai Hansen

Editor information

Editors and Affiliations

  1. Department of Computer Science and Electrical Engineering, ITE 324, University of Maryland, Baltimore County, 1000 Hilltop Circle, MD 21250, Baltimore, USA
    Tülay Adali
  2. Domaine Universitaire, GIPSA-lab, BP 46, 38402, Saint Martin d’Hères Cedex, France
    Christian Jutten
  3. Departamento de Microonda e Óptica (DMO), FEEC / Unicamp, Avenida Albert Einstein 400, 13083-852, Campinas, Sao Paulo, Brazil
    João Marcos Travassos Romano
  4. Centro Tecnológico, Curso de Engenharia Elétrica, Universidade Federal do Maranhão, Avenida dos Portugueses, s/n, Bacanga, 65080-040, São Luís, MA, Brazil
    Allan Kardec Barros

Rights and permissions

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Schmidt, M.N., Winther, O., Hansen, L.K. (2009). Bayesian Non-negative Matrix Factorization. In: Adali, T., Jutten, C., Romano, J.M.T., Barros, A.K. (eds) Independent Component Analysis and Signal Separation. ICA 2009. Lecture Notes in Computer Science, vol 5441. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-00599-2\_68

Download citation

Publish with us