Optimal Learning Rates for Clifford Neurons (original) (raw)

Abstract

Neural computation in Clifford algebras, which include familiar complex numbers and quaternions as special cases, has recently become an active research field. As always, neurons are the atoms of computation. The paper provides a general notion for the Hessian matrix of Clifford neurons of an arbitrary algebra. This new result on the dynamics of Clifford neurons then allows the computation of optimal learning rates. A thorough discussion of error surfaces together with simulation results for different neurons is also provided. The presented contents should give rise to very efficient second–order training methods for Clifford Multi-layer perceptrons in the future.

Preview

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Arena, P., Fortuna, L., Muscato, G., Xibilia, M.G.: Concepts in User Interfaces. LNCIS, vol. 234. Springer, Heidelberg (1998)
    MATH Google Scholar
  2. Arena, P., Fortuna, L., Occhipinti, L., Xibilia, M.G.: Neural networks for quaternion valued function approximation. In: IEEE Int. Symp. on Circuits and Systems, London, vol. 6, pp. 307–310. IEEE Computer Society Press, Los Alamitos (1994)
    Google Scholar
  3. Baldi, P.F., Hornik, K.: Learning in linear neural networks: A survey. IEEE Transactions on Neural Networks 6(4), 837–858 (1995)
    Article Google Scholar
  4. Buchholz, S.: A theory of neural computation with Clifford algebra. Technical Report Number 0504, Universität Kiel, Institut für Informatik (2005)
    Google Scholar
  5. Buchholz, S., Le Bihan, N.: Optimal separation of polarized signals by quaternionic neural networks. In: EUSIPCO 2006. 14th European Signal Processing Conference, Florence, Italy (September 4-8, 2006)
    Google Scholar
  6. Hirose, A.: Complex–Valued Neural Networks. Studies in Computational Intelligence, vol. 32. Springer, Heidelberg (2006)
    MATH Google Scholar
  7. LeCun, Y., Bottou, L., Orr, G.B., Müller, K.–R.: Efficient BackProp. In: Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 1524, Springer, Heidelberg (1998)
    Chapter Google Scholar
  8. LeCun, Y., Kanter, I., Solla, S.A.: Eigenvalues of Covariance Matrices: Application to Neural–Network Learning. Physical Review Letters 66(18), 2396–2399 (1991)
    Article Google Scholar
  9. Leung, H., Haykin, S.: The Complex Backpropagation Algorithm. IEEE Transactions on Signal Processing 3(9), 2101–2104 (1991)
    Article Google Scholar
  10. Lounesto, P.: Clifford Algebras and Spinors. Cambridge University Press, Cambridge (1997)
    MATH Google Scholar
  11. Matsui, N., Isokawa, T., Kusamichi, H., Peper, F., Nishimura, H.: Quaternion neural network with geometrical operators. Journal of Intelligent & Fuzzy Systems 15(3–4), 149–164 (2004)
    MATH Google Scholar
  12. Porteous, I.R.: Clifford Algebras and the Classical Groups. Cambridge University Press, Cambridge (1995)
    MATH Google Scholar
  13. Rojas, R.: Neural Networks. Springer, Heidelberg (1996)
    MATH Google Scholar
  14. Yaglom, I.M.: Complex Numbers in Geometry. Academic Press, New York (1968)
    Google Scholar

Download references

Author information

Authors and Affiliations

  1. Institute of Computer Science, University of Kiel, Germany
    Sven Buchholz
  2. Graduate School of Engineering, Nagoya University, Japan
    Kanta Tachibana
  3. Department of Applied Physics, University of Fukui, Japan
    Eckhard M. S. Hitzer

Authors

  1. Sven Buchholz
  2. Kanta Tachibana
  3. Eckhard M. S. Hitzer

Editor information

Joaquim Marques de Sá Luís A. Alexandre Włodzisław Duch Danilo Mandic

Rights and permissions

© 2007 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Buchholz, S., Tachibana, K., Hitzer, E.M.S. (2007). Optimal Learning Rates for Clifford Neurons. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D. (eds) Artificial Neural Networks – ICANN 2007. ICANN 2007. Lecture Notes in Computer Science, vol 4668. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74690-4\_88

Download citation

Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Publish with us