Support vector regression with ANOVA decomposition kernels (original) (raw)

A Computationally Efficient SUPANOVA: Spline Kernel Based Machine Learning Tool

Advances in Soft Computing, 2007

Many machine learning methods just consider the quality of prediction results as their final purpose. To make the prediction process transparent (reversible), spline kernel based methods were proposed by Gunn. However, the original solution method, termed SUpport vector Parsimonious ANOVA (SUPANOVA) was computationally very complex and demanding. In this paper, we propose a new heuristic to compute the optimal sparse vector in SUPANOVA that replaces the original solver for the convex quadratic problem of very high dimensionality. The resulting system is much faster without the loss of precision, as demonstrated in this paper on two benchmarks: the iris data set and the Boston housing market data benchmark.

A note on the decomposition methods for support vector regression

2001

Abstract The dual formulation of support vector regression involves with two closely related sets of variables. When the decomposition method is used, many existing approaches use pairs of indices from these two sets as the working set. Basically they select a base set first and then expand it so that all indices are pairs. This makes the implementation different from that for support vector classification. In addition, a larger optimization sub-problem has to be solved in each iteration.

The support vector decomposition machine

Proceedings of the 23rd international conference on Machine learning - ICML '06, 2006

In machine learning problems with tens of thousands of features and only dozens or hundreds of independent training examples, dimensionality reduction is essential for good learning performance. In previous work, many researchers have treated the learning problem in two separate phases: first use an algorithm such as singular value decomposition to reduce the dimensionality of the data set, and then use a classification algorithm such as naïve Bayes or support vector machines to learn a classifier. We demonstrate that it is possible to combine the two goals of dimensionality reduction and classification into a single learning objective, and present a novel and efficient algorithm which optimizes this objective directly. We present experimental results in fMRI analysis which show that we can achieve better learning performance and lower-dimensional representations than two-phase approaches can.

Properties of support vector machines for regression

1999

In this report we show that the -tube size in Support Vector Machine (SVM) for regression is 2 = p 1 + jjwjj 2 . By using this result we show that, in the case all the data points are inside the -tube, minimizing jjwjj 2 in SVM for regression is equivalent to maximizing the distance between the approximating hyperplane and the farest points in the training set. Moreover, in the most general setting in which the data points live also outside the -tube, we show that, for a xed value of , minimizing jjwjj 2 is equivalent to maximizing the sparsity of the representation of the optimal approximating hyperplane, that is equivalent to minimizing the number of coe cients di erent from zero in the expression of the optimal w. Then, the solution found by SVM for regression is a tradeo between sparsity of the representation and closeness to the data. We also include a complete derivation of SVM for regression in the case of linear approximation.

Training suport vector regression: Theory and algorithms

2002

We discuss the relation betweenϵ-support vector regression (ϵ-SVR) and v-support vector regression (v-SVR). In particular, we focus on properties that are different from those of C-support vector classification (C-SVC) and v-support vector classification (v-SVC). We then discuss some issues that do not occur in the case of classification: the possible range of ϵ and the scaling of target values. A practical decomposition method for v-SVR is implemented, and computational experiments are conducted.

Kernel functions for support vector machines

2008

Boonserm Kijsirikul, for his continuous guidance and excellent support throughout this research. Since he had accepted me to study in the Doctorial of Philosophy Program, he granted many good opportunities to my life. He provided the financial support for my study, and encourages me for all activities that are the good experiences. During my research period, he helped me to correct all of my publications. His motivation and suggestion have inspired me at times of difficulty. His timely support has helped me at various stages of this work.

Comparative Study of the Performance of Support Vector Machines with Various Kernels

2021

A support vector machine (SVM) is a state-of-the-art machine learning model rooted in structural risk minimization. SVM is underestimated with regards to its application to real world problems because of the difficulties associated with its use. We aim at showing that the performance of SVM highly depends on which kernel function to use. To achieve these, after providing a summary of support vector machines and kernel function, we constructed experiments with various benchmark datasets to compare the performance of various kernel functions. For evaluating the performance of SVM, the F1-score and its Standard Deviation with 10cross validation was used. Furthermore, we used taylor diagrams to reveal the difference between kernels. Finally, we provided Python codes for all our experiments to enable re-implementation of the experiments. 1. Motivation and Goal SVMs are state-of-the-art machine learning techniques with their root in structural risk minimization [57, 58]. Additionally, SVM...