Predictive Modeling with Echo State Networks (original) (raw)

Abstract

A lot of attention is now being focused on connectionist models known under the name “reservoir computing”. The most prominent example of these approaches is a recurrent neural network architecture called an echo state network (ESN). ESNs were successfully applied in several time series modeling tasks and according to the authors they performed exceptionally well. Multiple enhancements to standard ESN were proposed in the literature. In this paper we follow the opposite direction by suggesting several simplifications to the original ESN architecture. ESN reservoir features contractive dynamics resulting from its’ initialization with small weights. Sometimes it serves just as a simple memory of inputs and provides only negligible “extra-value” over much simple methods. We experimentally support this claim and we show that many tasks modeled by ESNs can be handled with much simple approaches.

This work was supported by the grants VG-1/0848/08 and VG-1/0822/08.

Preview

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks. Technical Report GMD 148, German National Research Center for Information Technology (2001)
    Google Scholar
  2. Jaeger, H.: Adaptive nonlinear system identification with echo state networks. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15, pp. 593–600. MIT Press, Cambridge (2003)
    Google Scholar
  3. Jaeger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667), 78–80 (2004)
    Article Google Scholar
  4. Frank, S.L.: Learn more by training less: Systematicity in sentence processing by recurrent networks. Connection Science (in press, 2006)
    Google Scholar
  5. Prokhorov, D.: Echo state networks: Appeal and challenges. In: Proceedings of International Joint Conference on Neural Networks, IJCNN 2005, Montreal, Canada, pp. 1463–1466 (2005)
    Google Scholar
  6. Jaeger, H.: Reservoir riddles: Suggestions for echo state network research. In: Proceedings of International Joint Conference on Neural Networks IJCNN 2005, Montreal, Canada, pp. 1460–1462 (2005)
    Google Scholar
  7. Čerňanský, M., Makula, M.: Feed-forward echo state networks. In: Proceedings of International Joint Conference on Neural Networks IJCNN 2005, Montreal, Canada, pp. 1479–1482 (2005)
    Google Scholar
  8. Jaeger, H.: Adaptive nonlinear system identification with echo state networks. In: Proceedings of Neural Information Processing Systems NIPS 2002, Vancouver, Canada (2002)
    Google Scholar
  9. Xue, Y., Yang, L., Haykin, S.: Decoupled echo state network with lateral inhibition. IEEE Transactions on Neural Network (January 2007) (in press)
    Google Scholar
  10. Wierstra, D., Gomez, F.J., Schmidhuber, J.: Modeling systems with internal state using evolino. In: GECCO 2005: Proceedings of the 2005 conference on Genetic and evolutionary computation, pp. 1795–1802. ACM, New York (2005)
    Chapter Google Scholar

Download references

Author information

Authors and Affiliations

  1. Faculty of Informatics and Information Technologies, STU Bratislava, Slovakia
    Michal Čerňanský
  2. School of Computer Science, University of Birmingham, United Kingdom
    Peter Tiňo

Authors

  1. Michal Čerňanský
  2. Peter Tiňo

Editor information

Véra Kůrková Roman Neruda Jan Koutník

Rights and permissions

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Čerňanský, M., Tiňo, P. (2008). Predictive Modeling with Echo State Networks. In: Kůrková, V., Neruda, R., Koutník, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87536-9\_80

Download citation

Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Publish with us