Sparse Coding Neural Gas for the Separation of Noisy Overcomplete Sources (original) (raw)

Abstract

We consider the problem of separating noisy overcomplete sources from linear mixtures, i.e., we observe N mixtures of M > N sparse sources. We show that the “Sparse Coding Neural Gas” (SCNG) algorithm [1] can be employed in order to estimate the mixing matrix. Based on the learned mixing matrix the sources are obtained by orthogonal matching pursuit. Using artificially generated data, we evaluate the influence of (i) the coherence of the mixing matrix, (ii) the noise level, and (iii) the sparseness of the sources with respect to the performance that can be achieved on the representation level. Our results show that if the coherence of the mixing matrix and the noise level are sufficiently small and the underlying sources are sufficiently sparse, the sources can be estimated from the observed mixtures.

Preview

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Labusch, K., Barth, E., Martinetz, T.: Learning Data Representations with Sparse Coding Neural Gas. In: Verleysen, M. (ed.) Proceedings of the 16th European Symposium on Artificial Neural Networks, pp. 233–238. D-Side Publishers (2008)
    Google Scholar
  2. Bell, A.J., Sejnowski, T.J.: An information-maximization approach to blind separation and blind deconvolution. Neural Computation 7(6), 1129–1159 (1995)
    Article Google Scholar
  3. Hyvärinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks 10(3), 626–634 (1999)
    Article Google Scholar
  4. Hyvarinen, A., Cristescu, R., Oja, E.: A fast algorithm for estimating overcomplete ica bases for image windows. In: Proceedings of the International Joint Conference on Neural Networks, IJCNN 1999, vol. 2, pp. 894–899 (1999)
    Google Scholar
  5. Lee, T.W., Lewicki, M., Girolami, M., Sejnowski, T.: Blind source separation of more sources than mixtures using overcomplete representations. IEEE Signal Processing Letters 6(4), 87–90 (1999)
    Article Google Scholar
  6. Lewicki, M.S., Sejnowski, T.J.: Learning Overcomplete Representations. Neural Computation 12(2), 337–365 (2000)
    Article Google Scholar
  7. Theis, F., Lang, E., Puntonet, C.: A geometric algorithm for overcomplete linear ICA. Neurocomputing 56, 381–398 (2004)
    Article Google Scholar
  8. Hyvärinen, A.: Gaussian moments for noisy independent component analysis. IEEE Signal Processing Letters 6(6), 145–147 (1999)
    Article Google Scholar
  9. Hyvarinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. Wiley-Interscience, Chichester (2001)
    Google Scholar
  10. Pati, Y., Rezaiifar, R., Krishnaprasad, P.: Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In: Proceedings of the 27 th Annual Asilomar Conference on Signals, Systems (November 1993)
    Google Scholar
  11. Donoho, D.L., Elad, M., Temlyakov, V.N.: Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Transactions on Information Theory 52(1), 6–18 (2006)
    Article MathSciNet Google Scholar
  12. Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing 20(1), 33–61 (1998)
    Article MathSciNet Google Scholar
  13. Rebollo-Neira, L., Lowe, D.: Optimized orthogonal matching pursuit approach. IEEE Signal Processing Letters 9(4), 137–140 (2002)
    Article Google Scholar
  14. Oja, E.: A simplified neuron model as a principal component analyzer. J. Math. Biol. 15, 267–273 (1982)
    Article MATH MathSciNet Google Scholar
  15. Martinetz, T., Berkovich, S., Schulten, K.: “Neural-gas” Network for Vector Quantization and its Application to Time-Series Prediction. IEEE-Transactions on Neural Networks 4(4), 558–569 (1993)
    Article Google Scholar

Download references

Author information

Authors and Affiliations

  1. Institute for Neuro- and Bioinformatics, University of Lübeck, Ratzeburger Alle 160, 23538, Lübeck, Germany
    Kai Labusch, Erhardt Barth & Thomas Martinetz

Authors

  1. Kai Labusch
  2. Erhardt Barth
  3. Thomas Martinetz

Editor information

Véra Kůrková Roman Neruda Jan Koutník

Rights and permissions

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Labusch, K., Barth, E., Martinetz, T. (2008). Sparse Coding Neural Gas for the Separation of Noisy Overcomplete Sources. In: Kůrková, V., Neruda, R., Koutník, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87536-9\_81

Download citation

Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Publish with us