A Speedup Method for SVM Decision (original) (raw)

Abstract

In this paper, we proposed a method to speed up the test phase of SVM based on Feature Vector Selection method (FVS). In the method, the support vectors (SVs) appeared in the decision function of SVM are replaced with some feature vectors (FVs) which are selected from support vectors by FVS method. Since it is a subset of SVs set, the size of FVs set is normally smaller than that of the SVs set, therefore the decision process of SVM is speeded up. Experiments on 12 datasets of IDA show that the number of SVs can be reduced from 20% to 99% with only a slight increase on the error rate of SVM by the proposed method. The trade-off between the generalization ability of obtained SVM and the speedup ability of the proposed method can be easily controlled by one parameter.

Chapter PDF

Similar content being viewed by others

References

  1. Vapnik, V.N.: The Natural of Statistical Learning Theory. Springer, New York (1995)
    Google Scholar
  2. Burges, C.J.C.: Simplified support vector decision rules. In: Proc. 13th International Conference on Machine Learning, San Mateo, CA, pp. 71–77 (1996)
    Google Scholar
  3. Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2, 121–167 (1998)
    Article Google Scholar
  4. Burges, C.J.C., Schoelkopf, B.: Improving the accuracy and speed of support vector learning machines. In: Mozer, M., Jordan, M., Petsche, T. (eds.) Advances in neural information processing systems, vol. 9, pp. 375–381. MIT Press, Cambridge (1997)
    Google Scholar
  5. Zhu, Y.S., Zhang, Y.Y.: A new type SVM—Projected SVM. Science in China G: Physics, Mechanics & Astronomy supp. 47, 21–28 (2004)
    Article Google Scholar
  6. Osuna, E., Girosi, F.: Reducing the run-time complexity of Support Vector Machines. In: Proc.14th International Conference on Pattern Recognition, Btisbane, Austrilia (1998)
    Google Scholar
  7. Tipping, M.E.: Sparse Bayesian Learning and the Relevance Vector Machine. Journal of Machine Learning Research 1, 211–244 (2001)
    Article MATH MathSciNet Google Scholar
  8. Lee, Y.-J., Mangasarian, O.L.: RSVM: reduced support vector machines. In: Proc. 1st SIAM Int. Conf. Data Mining, Chicago (2001)
    Google Scholar
  9. Lin, K.-M.: A Study on Reduced Support Vector Machines. IEEE Trans. Neural Networks 14, 1449–1459 (2003)
    Article Google Scholar
  10. Schoelkopf, B., Mika, S., Burges, C.J.C., Knirsch, P., Muller, K., Ratsch, G., Smola, A.J.: Input space versus feature space in kernel-based methods. IEEE Trans. Neural Networks 10, 1000–1017 (1999)
    Article Google Scholar
  11. Nguyen, D.D., Ho, T.B.: An Efficient Method for Simplifying Support Vector Machines. In: The 22nd International Conference on Machine Learning (ICML 2005), Bonn, Germany (2005)
    Google Scholar
  12. Downs, T., Gates, K.E., Masters, A.: Extract simplification of support vector solutions. Journal of Machine Learning Research 2, 293–297 (2001)
    Article Google Scholar
  13. Suykens, J.A.K., Lukas, L., Vandewalle, J.: Sparese approximation using least squares support vector machines. In: Proc. of the IEEE International Symposium on Circuits and Systems, Geneva, Switzerland, pp. II757–II760 (2000)
    Google Scholar
  14. Baudat, G., Anouar, F.: Feature Vector selection and projection using kernels. Neuro-computing 55, 21–38 (2003)
    Google Scholar
  15. The datasets are from website: http://ida.first.gmd.de/~raetsch/data/benchmarks.htm
  16. Zhu, Y.S., Li, C.H., Zhang, Y.Y.: A practical parameters selection method for SVM. In: Yin, F.-L., Wang, J., Guo, C. (eds.) ISNN 2004. LNCS, vol. 3173, pp. 518–523. Springer, Heidelberg (2004)
    Chapter Google Scholar

Download references

Author information

Authors and Affiliations

  1. Key Laboratory of Education Ministry for Modern Design and Rotor-Bearing System, Xi’an Jiaotong University, Xi’an, 710049, China
    Yongsheng Zhu, Junyan Yang & Youyun Zhang
  2. Network & Information Technology Center of Library, Xi’an Jiaotong University, Xi’an, 710049, China
    Jian Ye

Authors

  1. Yongsheng Zhu
  2. Junyan Yang
  3. Jian Ye
  4. Youyun Zhang

Editor information

Editors and Affiliations

  1. Hong Kong University of Science and Technology,
    Dit-Yan Yeung
  2. Department of Computer Science, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China
    James T. Kwok
  3. Instituto de Telecomunicações, Instituto Superior Técnico, Lisbon, Portugal
    Ana Fred
  4. Department of Electrical and Electronic Engineering, University of Cagliari, Piazza d’Armi, 09123, Cagliari, Italy
    Fabio Roli
  5. Faculty of Electrical Engineering, Mathematics and Computer Science, Information and Communication Theory Group, Delft University of Technology, Delft, The Netherlands
    Dick de Ridder

Rights and permissions

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zhu, Y., Yang, J., Ye, J., Zhang, Y. (2006). A Speedup Method for SVM Decision. In: Yeung, DY., Kwok, J.T., Fred, A., Roli, F., de Ridder, D. (eds) Structural, Syntactic, and Statistical Pattern Recognition. SSPR /SPR 2006. Lecture Notes in Computer Science, vol 4109. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11815921\_54

Download citation

Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Publish with us