A new method to determine separated hyper-plane for non-parametric sign test in multivariate data (original) (raw)

The Signed Distance Function: A New Tool for Binary Classification

ArXiv, 2005

From a geometric perspective most nonlinear binary classification algorithms, including state of the art versions of Support Vector Machine (SVM) and Radial Basis Function Network (RBFN) classifiers, and are based on the idea of reconstructing indicator functions. We propose instead to use reconstruction of the signed distance function (SDF) as a basis for binary classification. We discuss properties of the signed distance function that can be exploited in classification algorithms. We develop simple versions of such classifiers and test them on several linear and nonlinear problems. On linear tests accuracy of the new algorithm exceeds that of standard SVM methods, with an average of 50% fewer misclassifications. Performance of the new methods also matches or exceeds that of standard methods on several nonlinear problems including classification of benchmark diagnostic micro-array data sets.

The support vector machine under test

Neurocomputing, 2003

Support vector machines (SVMs) are rarely benchmarked against other classiÿcation or regression methods. We compare a popular SVM implementation (libsvm) to 16 classiÿcation methods and 9 regression methods-all accessible through the software R-by the means of standard performance measures (classiÿcation error and mean squared error) which are also analyzed by the means of bias-variance decompositions. SVMs showed mostly good performances both on classiÿcation and regression tasks, but other methods proved to be very competitive.

‘Multi-Hyperplane Formulations for Classification and Discrimination Analysis

2006

We introduce two new mixed integer programming formulations for the two-group classification and discrimination problem that make use of multiple separating hyperplanes, yielding an advance having a wide range of practical applications. Our work constitutes an innovation in the area of support vector machines in the context of successive perfect separation decision trees, by constructing a discriminant function that approximates complex non-linear decision units without the need for kernel transformations of the data. Unlike previous piecewise-linear formulations, our models do not require that one of the groups belong to a convex region, making our formulation more effective for more complex data sets. We conduct an in-depth computational study using well known data sets in banking and in cancer detection that shows the merit of our models.

Support vector machines and the multiple hypothesis test problem

IEEE Transactions on Signal Processing, 2001

Two enhancements are proposed to the application and theory of support vector machines. The first is a method of multicategory classification based on the binary classification version of the support vector machine (SVM). The method, which is called the -ary SVM, represents each category in binary format, and to each bit of that representation is assigned a conventional SVM. This approach requires only log 2 ( ) SVMs, where is the number of classes. We give an example of classification on an octaphase-shift-keying (8-PSK) pattern space to illustrate main concepts.

Comparative Analysis of Support Vector Machines Based on Linear and Quadratic Optimization Criteria

2009

We present results from a comparative empirical study on the performance of two methods for constructing support vector machines (SVMs). The first method is the conventional one based on the quadratic programming approach, which builds the optimal separating hyperplane maximizing the margin between two classes (optimal SVM). The second method is based on the linear programming approach suggested by Vapnik to build a separating hyperplane with the minimum number of support vectors (heuristic SVM). Using synthetic data from two classes, we compare the classification performance of these SVMs, with an in-depth geometrical comparison of their separating hyperplanes and support vectors. We show that both classifiers achieve practically identical classification accuracy and generalization performance. However, the heuristic SVM has many fewer support vectors than the optimal SVM. In addition, in contrast to the optimal SVM, its support vectors lie on the furthermost borders of the classes, at the maximum distance from the opposite class. In our future work, we will seek to find a theoretical basis to explain these geometrical patterns of the heuristic SVM. We will also compare these classifiers using real benchmark data.

Tests for high dimensional data based on means, spatial signs and spatial ranks

Tests based on mean vectors and spatial signs and ranks for a zero mean in one-sample problems and for the equality of means in two-sample problems have been studied in the recent literature for high-dimensional data with the dimension larger than the sample size. For the above testing problems, we show that under suitable sequences of alternatives, the powers of the meanbased tests and the tests based on spatial signs and ranks tend to be same as the data dimension tends to infinity for any sample size when the coordinate variables satisfy appropriate mixing conditions. Further, their limiting powers do not depend on the heaviness of the tails of the distributions. This is in striking contrast to the asymptotic results obtained in the classical multivariate setting. On the other hand, we show that in the presence of stronger dependence among the coordinate variables, the spatial-sign-and rank-based tests for high-dimensional data can be asymptotically more powerful than the mean-based tests if, in addition to the data dimension, the sample size also tends to infinity. The sizes of some mean-based tests for high-dimensional data studied in the recent literature are observed to be significantly different from their nominal levels. This is due to the inadequacy of the asymptotic approximations used for the distributions of those test statistics. However, our asymptotic approximations for the tests based on spatial signs and ranks are observed to work well when the tests are applied on a variety of simulated and real datasets.

Learning classifiers without negative examples: A reduction approach

2008

The problem of PU Learning, i.e., learning classifiers with positive and unlabelled examples (but not negative examples), is very important in information retrieval and data mining. We address this problem through a novel approach: reducing it to the problem of learning classifiers for some meaningful multivariate performance measures. In particular, we show how a powerful machine learning algorithm, Support Vector Machine, can be adapted to solve this problem. The effectiveness and efficiency of the proposed approach have been confirmed by our experiments on three real-world datasets.

Prediction with the SVM Using Test Point Margins

Support vector machines (SVMs) carry out binary classification by constructing a maximal margin hyperplane between the two classes of observed (training) examples and then classifying test points according to the half-spaces in which they reside (irrespective of the distances that may exist between the test examples and the hyperplane). Cross-validation involves finding the one SVM model together with its optimal parameters that minimizes the training error and has good generalization in the future. In contrast, in this paper we collect all of the models found in the model selection phase and make predictions according to the model whose hyperplane achieves the maximum separation from a test point. This directly corresponds to the L ∞ norm for choosing SVM models at the testing stage. Furthermore, we also investigate other more general techniques corresponding to different L p norms and show how these methods allow us to avoid the complex and time consuming paradigm of cross-validation. Experimental results demonstrate this advantage, showing significant decreases in computational time as well as competitive generalization error.