Finding Nominally Conditioned Multivariate Polynomials Using a Four-Layer Perceptron Having Shared Weights (original) (raw)

Abstract

We present a method for discovering nominally conditioned polynomials to fit multivariate data containing numeric and nominal variables using a four-layer perceptron having shared weights. A polynomial is accompanied with the nominal condition stating a subspace where the polynomial is applied. To get a succinct neural network, we focus on weight sharing, where a weight is allowed to have one of common weights. A near-zero common weight can be eliminated. Our method iteratively merges and splits common weights based on 2nd-order criteria, escaping from local optima. Moreover, our method selects the optimal number of hidden units based on cross-validation. The experiments showed that our method can restore the original sharing structure for an artificial data set, and discovers rather succinct rules for a real data set.

Preview

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Haykin, S.: Neural Networks, 2nd edn. Prentice-Hall, Englewood Cliffs (1999)
    MATH Google Scholar
  2. Tanahashi, Y., Saito, K., Nakano, R.: Piecewise Multivariate Polynomials Using a Four-Layer Perceptron. In: Negoita, M.G., Howlett, R.J., Jain, L.C. (eds.) KES 2004. LNCS (LNAI), vol. 3214, pp. 602–608. Springer, Heidelberg (2004)
    Chapter Google Scholar
  3. Lloyd, S.P.: Least squares quantization in PCM. IEEE Trans. Information Theory IT-28(2), 129–137 (1982)
    Article MathSciNet Google Scholar
  4. Saito, K., Nakano, R.: Structuring neural networks through bidirectional clustering of weights. In: Proc. 5th Int. Conf. on Discovery Science, pp. 206–219 (2002)
    Google Scholar
  5. Tanahashi, Y., Saito, K., Nakano, R.: Model Selection and Weight Sharing of Multi-layer Perceptrons. In: Khosla, R., Howlett, R.J., Jain, L.C. (eds.) KES 2005. LNCS (LNAI), vol. 3684, pp. 716–722. Springer, Heidelberg (2005)
    Chapter Google Scholar
  6. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
    Google Scholar

Download references

Author information

Authors and Affiliations

  1. Nagoya Institute of Technology, Gokiso-cho, Showa-ku, Nagoya, 466-8555, Japan
    Yusuke Tanahashi, Daisuke Kitakoshi & Ryohei Nakano
  2. NTT Communication Science Laboratories, NTT Corporation, 2-4 Hikaridai, Seika, Soraku, Kyoto, 619-0237, Japan
    Kazumi Saito

Authors

  1. Yusuke Tanahashi
  2. Kazumi Saito
  3. Daisuke Kitakoshi
  4. Ryohei Nakano

Editor information

Editors and Affiliations

  1. School of Design, Engineering and Computing, Bournemouth University, UK
    Bogdan Gabrys
  2. Centre for SMART Systems, School of Environment and Technology, University of Brighton, BN2 4GJ, Brighton, UK
    Robert J. Howlett
  3. School of Electrical and Information Engineering, Knowledge Based Intelligent Engineering Systems Centre, University of South Australia, SA, 5095, Mawson Lakes, Australia
    Lakhmi C. Jain

Rights and permissions

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Tanahashi, Y., Saito, K., Kitakoshi, D., Nakano, R. (2006). Finding Nominally Conditioned Multivariate Polynomials Using a Four-Layer Perceptron Having Shared Weights. In: Gabrys, B., Howlett, R.J., Jain, L.C. (eds) Knowledge-Based Intelligent Information and Engineering Systems. KES 2006. Lecture Notes in Computer Science(), vol 4252. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11893004\_124

Download citation

Publish with us