Non-Euclidean or Non-metric Measures Can Be Informative (original) (raw)

Abstract

Statistical learning algorithms often rely on the Euclidean distance. In practice, non-Euclidean or non-metric dissimilarity measures may arise when contours, spectra or shapes are compared by edit distances or as a consequence of robust object matching [1,2]. It is an open issue whether such measures are advantageous for statistical learning or whether they should be constrained to obey the metric axioms.

The _k_-nearest neighbor (NN) rule is widely applied to general dissimilarity data as the most natural approach. Alternative methods exist that embed such data into suitable representation spaces in which statistical classifiers are constructed [3]. In this paper, we investigate the relation between non-Euclidean aspects of dissimilarity data and the classification performance of the direct NN rule and some classifiers trained in representation spaces. This is evaluated on a parameterized family of edit distances, in which parameter values control the strength of non-Euclidean behavior. Our finding is that the discriminative power of this measure increases with increasing non-Euclidean and non-metric aspects until a certain optimum is reached. The conclusion is that statistical classifiers perform well and the optimal values of the parameters characterize a non-Euclidean and somewhat non-metric measure.

Chapter PDF

References

  1. Dubuisson, M., Jain, A.: Modified Hausdorff distance for object matching. In: ICPR, vol. 1, pp. 566–568 (1994)
    Google Scholar
  2. Jacobs, D., Weinshall, D., Gdalyahu, Y.: Classification with Non-Metric Dist.: Image Retrieval and Class Representation. TPAMI 22, 583–600 (2000)
    Google Scholar
  3. Pękalska, E., Duin, R.: The dissimilarity representation for pattern recognition. Foundations and applications. World Scientific, Singapore (2005)
    Book MATH Google Scholar
  4. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, Heidelberg (2001)
    MATH Google Scholar
  5. Pękalska, E., Paclík, P., Duin, R.: A Generalized Kernel Approach to Dissimilarity Based Classification. JMLR 2, 175–211 (2002)
    Article MATH Google Scholar
  6. Schölkopf, B., Smola, A.: Learning with Kernels. MIT Press, Cambridge (2002)
    Google Scholar
  7. Veltkamp, R., Hagedoorn, M.: State-of-the-art in shape matching. Technical Report UU-CS-1999-27, Utrecht University, The Netherlands (1999)
    Google Scholar
  8. Pękalska, E., Duin, R., Günter, S., Bunke, H.: On not making dissimilarities Euclidean. In: S+SSPR, pp. 1145–1154 (2004)
    Google Scholar
  9. Pękalska, E., Duin, R., Paclík, P.: Prototype selection for dissimilarity-based classifiers. Pattern Recognition 39, 189–208 (2005)
    Article Google Scholar
  10. Haasdonk, B.: Feature space interpretation of SVMs with indefinite kernels. TPAMI 25, 482–492 (2005)
    Google Scholar
  11. Laub, J., Müller, K.R.: Feature discovery in non-metric pairwise data. JMLR, 801–818 (2004)
    Google Scholar
  12. Goldfarb, L.: A unified approach to pattern recognition. Pattern Recognition 17, 575–582 (1984)
    Article MATH MathSciNet Google Scholar
  13. Graepel, T., Herbrich, R., Schölkopf, B., Smola, A., Bartlett, P., Müller, K.R., Obermayer, K., Williamson, R.: Classification on proximity data with LP-machines. In: ICANN, pp. 304–309 (1999)
    Google Scholar
  14. Tax, D., Veenman, C.: Tuning the hyperparameter of an auc-optimized classifier. In: BNAIC, pp. 224–231 (2005)
    Google Scholar
  15. Andreu, G., Crespo, A., Valiente, J.M.: Selecting the toroidal self-organizing feature maps (TSOFM) best organized to object recogn. In: ICNN, pp. 1341–1346 (1997)
    Google Scholar
  16. Bunke, H., Bühler, U.: Applications of approximate string matching to 2D shape recognition. Pattern recognition 26, 1797–1812 (1993)
    Article Google Scholar
  17. Bunke, H., Sanfeliu, A. (eds.): Syntactic and Structural Pattern Recognition Theory and Applications. World Scientific, Singapore (1990)
    MATH Google Scholar
  18. Devijver, P., Kittler, J.: Pattern recognition: A statistical approach. Prentice Hall, Englewood Cliffs (1982)
    MATH Google Scholar

Download references

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering, Mathematics and Computer Sciences, Delft University of Technology, The Netherlands
    Elżbieta Pękalska, Artsiom Harol & Robert P. W. Duin
  2. School of Computer Science, University of Manchester, United Kingdom
    Elżbieta Pękalska
  3. Institute of Computer Science and Applied Mathematics, University of Bern, Switzerland
    Barbara Spillmann & Horst Bunke

Authors

  1. Elżbieta Pękalska
  2. Artsiom Harol
  3. Robert P. W. Duin
  4. Barbara Spillmann
  5. Horst Bunke

Editor information

Editors and Affiliations

  1. Hong Kong University of Science and Technology,
    Dit-Yan Yeung
  2. Department of Computer Science, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China
    James T. Kwok
  3. Instituto de Telecomunicações, Instituto Superior Técnico, Lisbon, Portugal
    Ana Fred
  4. Department of Electrical and Electronic Engineering, University of Cagliari, Piazza d’Armi, 09123, Cagliari, Italy
    Fabio Roli
  5. Faculty of Electrical Engineering, Mathematics and Computer Science, Information and Communication Theory Group, Delft University of Technology, Delft, The Netherlands
    Dick de Ridder

Rights and permissions

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Pękalska, E., Harol, A., Duin, R.P.W., Spillmann, B., Bunke, H. (2006). Non-Euclidean or Non-metric Measures Can Be Informative. In: Yeung, DY., Kwok, J.T., Fred, A., Roli, F., de Ridder, D. (eds) Structural, Syntactic, and Statistical Pattern Recognition. SSPR /SPR 2006. Lecture Notes in Computer Science, vol 4109. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11815921\_96

Download citation

Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Publish with us