A Kernel Method for the Optimization of the Margin Distribution (original) (raw)

Abstract

Recent results in theoretical machine learning seem to suggest that nice properties of the margin distribution over a training set turns out in a good performance of a classifier. The same principle has been already used in SVM and other kernel based methods as the associated optimization problems try to maximize the minimum of these margins.

In this paper, we propose a kernel based method for the direct optimization of the margin distribution (KM-OMD). The method is motivated and analyzed from a game theoretical perspective. A quite efficient optimization algorithm is then proposed. Experimental results over a standard benchmark of 13 datasets have clearly shown state-of-the-art performances.

Preview

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Reyzin, L., Schapire, R.: How boosting the margin can also boost classifier complexity. In: Proceedings of the 23rd International Conference on Machine Learning (ICML) (2006)
    Google Scholar
  2. Garg, A., Har-Peled, S., Roth, D.: On generalization bounds, projection profile, and margin distribution. In: Proceedings of the 11th International Conference on Machine Learning (ICML) (2002)
    Google Scholar
  3. Shawe-Taylor, J., Cristianini, N.: Further results on the margin distribution. In: Proceedings of the 15th International Conference on Machine Learning (ICML) (2003)
    Google Scholar
  4. Garg, A., Roth, D.: Margin distribution and learning algorithms. In: Proceedings of the 12th Conference on Computational Learning Theory (COLT) (1999)
    Google Scholar
  5. Mason, L., Bartlett, P., Baxter, J.: Improved generalization trough explicit optimization of margins. Machine Learning 38(3), 243–255 (2000)
    Article MATH Google Scholar
  6. Aiolli, F., Sperduti, A.: A re-weighting strategy for improving margins. Artifical Intelligence Journal 137, 197–216 (2002)
    Article MATH MathSciNet Google Scholar
  7. Pelckmans, K., Suykens, J., Moor, B.D.: A risk minimization principle for a class of parzen estimators. In: Advances in Neural Information Processing Systems (2007)
    Google Scholar
  8. Siu, K.Y., Roychowdhury, V., Kailath, T.: Discrete Neural Computation. Prentice Hall, Englewood Cliffs (1995)
    MATH Google Scholar
  9. Bhattacharyya, C., Keerthi, S., Murthy, K., Shevade, S.: A fast iterative nearest point algorithm for support vector machine classifier design. IEEE Transactions on Neural Networks (2000)
    Google Scholar
  10. Scholkopf, B., Smola, A.: Learning with Kernels. MIT Press, Cambridge (2002)
    Google Scholar
  11. Rätsch, G., Onoda, T., Müller, K.R.: Soft margins for adaboost. Mach. Learn. 42(3), 287–320 (2001)
    Article MATH Google Scholar
  12. Mika, S., Rätsch, G., Müller, K.R.: A mathematical programming approach to the kernel fisher algorithm. In: NIPS, pp. 591–597 (2000)
    Google Scholar

Download references

Author information

Authors and Affiliations

  1. Dept. of Pure and Applied Mathematics, , Via Trieste 63, 35131, Padova, Italy
    Fabio Aiolli, Giovanni Da San Martino & Alessandro Sperduti

Authors

  1. Fabio Aiolli
  2. Giovanni Da San Martino
  3. Alessandro Sperduti

Editor information

Véra Kůrková Roman Neruda Jan Koutník

Rights and permissions

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Aiolli, F., Da San Martino, G., Sperduti, A. (2008). A Kernel Method for the Optimization of the Margin Distribution. In: Kůrková, V., Neruda, R., Koutník, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87536-9\_32

Download citation

Keywords

Publish with us