Natural Conjugate Gradient on Complex Flag Manifolds for Complex Independent Subspace Analysis (original) (raw)

Abstract

We study the problem of complex-valued independent subspace analysis (ISA). We introduce complex flag manifolds to tackle this problem, and, based on Riemannian geometry, propose the natural conjugate gradient method on this class of manifolds. Numerical experiments demonstrate that the natural conjugate gradient method yields better convergence compared to the natural gradient geodesic search method.

Preview

Unable to display preview. Download preview PDF.

References

  1. Abrudan, T., Eriksson, J., Koivunen, V.: Steepest Descent Algorithms for Optimization under Unitary Matrix Constraint (preprint)
    Google Scholar
  2. Adali, T., Li, H.: A practical formulation for computation of complex gradients and its application to maximum likelihood. In: Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing (ICASSP), Honolulu, Hawaii (April 2007)
    Google Scholar
  3. Amari, S.: Natural gradient works efficiently in learning. Neural Computation 10, 251–276 (1998)
    Article Google Scholar
  4. Brandwood, D.H.: A Complex Gradient Operator and its Application in Adaptive Array Theory. IEE Proceedings H (Microwaves, Optics, and Antennas) 130(1), 11–16 (1983)
    MathSciNet Google Scholar
  5. Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications 20(2), 303–353 (1998)
    Article MATH MathSciNet Google Scholar
  6. Fiori, S.: A Study on Neural Learning on Manifold Foliations: The Case of the Lie Group SU(3). Neural Computation 4, 1091–1197 (2008)
    Article MathSciNet Google Scholar
  7. Honkela, A., Tornio, M., Raiko, T., Karhunen, J.: Natural Conjugate Gradient in Variational Inference. In: Ishikawa, M., Doya, K., Miyamoto, H., Yamakawa, T. (eds.) ICONIP 2007, Part II. LNCS, vol. 4985. Springer, Heidelberg (to appear, 2008)
    Chapter Google Scholar
  8. Hyvärinen, A., Hoyer, P.O.: Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces. Neural Computation 12(7), 1705–1720 (2000)
    Article Google Scholar
  9. Kreutz-Delgado, K.: The Complex Gradient Operator and the CR -Calculus, Technical report, University of California, San Diego (2003)
    Google Scholar
  10. Manton, J.: Optimization algorithms exploiting unitary constraints. IEEE Transactions on Signal Processing 50(3), 635–650 (2002)
    Article MathSciNet Google Scholar
  11. Clemente, R.M., Puntonet, C.G., Catalina, J.I.A.: Blind Signal Separation Based on the Derivatives of the Output Cumulants and a Conjugate Gradient Algorithm. In: Proceedings of Int. Conf. Independent Component Analysis Signal Separation, San Diego, pp. 390–393 (2001)
    Google Scholar
  12. Nishimori, Y., Akaho, S.: Learning algorithms utilizing quasi-geodesic flows on the Stiefel manifold. Neurocomputing 67, 106–135 (2005)
    Article Google Scholar
  13. Nishimori, Y., Akaho, S., Plumbley, M.D.: Riemannian optimization method on the flag manifold for independent subspace analysis. In: Rosca, J.P., Erdogmus, D., Príncipe, J.C., Haykin, S. (eds.) ICA 2006. LNCS, vol. 3889, pp. 295–302. Springer, Heidelberg (2006)
    Chapter Google Scholar
  14. Nishimori, Y., Akaho, S., Abdallah, S., Plumbley, M.D.: Flag manifolds for subspace ICA problems. In: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2007), Honolulu, Hawaii, USA, April 15-20, vol. IV, pp. 1417–1420 (2007)
    Google Scholar
  15. Plumbley, M.D.: Algorithms for non-negative independent component analysis. IEEE Transactions on Neural Networks 14(3), 534–543 (2003)
    Article Google Scholar
  16. Szabo, Z., Lorincz, A.: Real and Complex Independent Subspace Analysis by Generalized Variance, arXiv:math/0610438v1
    Google Scholar
  17. van den Bos, A.: Complex Gradient and Hessian. IEE Proc.-Vis. Image Signal Processing 141(6), 380–382 (1994)
    Article Google Scholar

Download references

Author information

Authors and Affiliations

  1. Neuroscience Research Institute, National Institute of Advanced Industrial Science and Technology (AIST), , AIST Central2, 1-1-1, Umezono, Tsukuba, Ibaraki, 305-8568, Japan
    Yasunori Nishimori & Shotaro Akaho
  2. Department of Electronic Engineering, Queen Mary University of London, Mile End Road, London, E1 4NS, UK
    Mark D. Plumbley

Authors

  1. Yasunori Nishimori
  2. Shotaro Akaho
  3. Mark D. Plumbley

Editor information

Véra Kůrková Roman Neruda Jan Koutník

Rights and permissions

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Nishimori, Y., Akaho, S., Plumbley, M.D. (2008). Natural Conjugate Gradient on Complex Flag Manifolds for Complex Independent Subspace Analysis. In: Kůrková, V., Neruda, R., Koutník, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87536-9\_18

Download citation

Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Publish with us