Modular Neural Networks for Model-Free Behavioral Learning (original) (raw)

Abstract

Future robots/agents will need to perform situation specific behaviors for each user. To cope with diverse and unexpected situations, model-free behavioral learning is required. We have constructed a modular neural network model based on reinforcement learning and demonstrated that the model can learn multiple kinds of state transitions with the same architectures and parameter values, and without pre-designed models of environments. We recently developed a modular neural network model equipped with a modified on-line modular learning algorithm, which is more suitable for neural networks and more efficient in learning. This paper describes the performances of constructed models using the probabilistically fluctuated Markov decision process including partially observable conditions. In the test transitions, the observed states probabilistically fluctuated. The new learning model is able to function in those complex transitions without specific adjustments for each transition.

Preview

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
    Google Scholar
  2. Takeuchi, J., Shouno, O., Tsujino, H.: Modular neural networks for reinforcement learning with temporal intrinsic rewards. In: Proceedings of 2007 International Joint Conference on Neural Networks (IJCNN). CD-ROM (2007)
    Google Scholar
  3. Schmidhuber, J.: Self-motivated development through rewards for predictor errors / improvements. In: Developmental Robotics 2005 AAAI Spring Symposium (2005)
    Google Scholar
  4. Doya, K., Samejima, K., Katagiri, K.-I., Kawato, M.: Multiple model-based reinforcement learning. Neural Computation 14, 1347–1369 (2002)
    Article MATH Google Scholar
  5. Haruno, M., Wolpert, D.M., Kawato, M.: Mosaic model for sensorimotor learning and control. Neural Computation 13, 2201–2220 (2001)
    Article MATH Google Scholar
  6. Jaeger, H.: The ‘echo state’ approach to analysing and training recurrent neural networks. Gmd report 148, German National Research Center for Information Technology (2001)
    Google Scholar
  7. Nishida, S., Ishii, K., Furukawa, T.: An online adaptation control system using mnsom. In: King, I., Wang, J., Chan, L.-W., Wang, D. (eds.) ICONIP 2006. LNCS, vol. 4232, pp. 935–942. Springer, Heidelberg (2006)
    Chapter Google Scholar
  8. Kohonen, T.: Self-Organizing Maps. Springer, Heidelberg (1995)
    Google Scholar
  9. Berglund, E., Sitte, J.: The parameterless self-organizing map algorithm. IEEE transactions on neural networks 17(2), 305–316 (2006)
    Article Google Scholar

Download references

Author information

Authors and Affiliations

  1. Honda Research Institute Japan Co., Ltd., , 8-1 Honcho, Wako-shi, Saitama, 351-0188, Japan
    Johane Takeuchi, Osamu Shouno & Hiroshi Tsujino

Authors

  1. Johane Takeuchi
  2. Osamu Shouno
  3. Hiroshi Tsujino

Editor information

Véra Kůrková Roman Neruda Jan Koutník

Rights and permissions

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Takeuchi, J., Shouno, O., Tsujino, H. (2008). Modular Neural Networks for Model-Free Behavioral Learning. In: Kůrková, V., Neruda, R., Koutník, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87536-9\_75

Download citation

Keywords

Publish with us