Multiclass Proximal Support Vector Machines (original) (raw)
Related papers
Fuzzy multi-category proximal support vector classification via generalized eigenvalues
Soft Computing, 2007
Given a dataset, where each point is labeled with one of M labels, we propose a technique for multicategory proximal support vector classification via generalized eigenvalues (MGEPSVMs). Unlike Support Vector Machines that classify points by assigning them to one of M disjoint half-spaces, here points are classified by assigning them to the closest of M non-parallel planes that are close to their respective classes. When the data contains samples belonging to several classes, classes often overlap, and classifiers that solve for several nonparallel planes may often be able to better resolve test samples. In multicategory classification tasks, a training point may have similarities with prototypes of more than one class. This information can be used in a fuzzy setting. We propose a fuzzy multi-category classifier that utilizes information about the membership of training samples, to improve the generalization ability of the classifier. The desired classifier is obtained by using one-from-rest (OFR) separation for each class, i.e. 1 : M − 1 classification. Experimental results demonstrate the efficacy of the proposed classifier over MGEPSVMs.
Study on proximal support vector machine as a classifier
2012
Proximal Support Vector machine based on Least Mean Square Algorithm classi-fiers (LMS-SVM) are tools for classification of binary data. Proximal Support Vector based on Least Mean Square Algorithm classifiers is completely based on the theory of Proximal Support Vector Machine classifiers (PSVM). PSVM classifies binary pat- terns by assigning them to the closest of two parallel planes that are pushed apart as far as possible. The training time for the classifier is found to be faster compared to their previous versions of Support Vector Machines. But due to the presence of slack variable or error vector the classification accuracy of the Proximal Support Vector Machine is less. So we have come with an idea to update the adjustable weight vectors at the training phase such that all the data points fall out-side the region of separation and falls on the correct side of the hyperplane and to enlarge the width of the separable region.To implement this idea, Least Mean Square (LMS) algo...
Multi-Class L2,1-Norm Support Vector Machine
2011 IEEE 11th International Conference on Data Mining, 2011
Feature selection is an essential component of data mining. In many data analysis tasks where the number of data point is much less than the number of features, efficient feature selection approaches are desired to extract meaningful features and to eliminate redundant ones. In the previous study, many data mining techniques have been applied to tackle the above challenging problem. In this paper, we propose a new 2,1norm SVM, that is, multi-class hinge loss with a structured regularization term for all the classes to naturally select features for multi-class without bothering further heuristic strategy. Rather than directly solving the multi-class hinge loss with 2,1-norm regularization minimization, which has not been solved before due to its optimization difficulty, we are the first to give an efficient algorithm bridging the new problem with a previous solvable optimization problem to do multi-class feature selection. A global convergence proof for our method is also presented. Via the proposed efficient algorithm, we select features across multiple classes with jointly sparsity, i.e., each feature has either small or large score over all classes. Comprehensive experiments have been performed on six bioinformatics data sets to show that our method can obtain better or competitive performance compared with exiting stateof-art multi-class feature selection approaches.
Improved Sparse Multi-Class SVM and Its Application for Gene Selection in Cancer Classification
Cancer Informatics, 2013
Background Microarray techniques provide promising tools for cancer diagnosis using gene expression profiles. However, molecular diagnosis based on high-throughput platforms presents great challenges due to the overwhelming number of variables versus the small sample size and the complex nature of multi-type tumors. Support vector machines (SVMs) have shown superior performance in cancer classification due to their ability to handle high dimensional low sample size data. The multi-class SVM algorithm of Crammer and Singer provides a natural framework for multi-class learning. Despite its effective performance, the procedure utilizes all variables without selection. In this paper, we propose to improve the procedure by imposing shrinkage penalties in learning to enforce solution sparsity. Results The original multi-class SVM of Crammer and Singer is effective for multi-class classification but does not conduct variable selection. We improved the method by introducing soft-thresholdin...
Multi-kernel regularized classifiers
Journal of Complexity, 2007
A family of classification algorithms generated from Tikhonov regularization schemes are considered. They involve multi-kernel spaces and general convex loss functions. Our main purpose is to provide satisfactory estimates for the excess misclassification error of these multi-kernel regularized classifiers. The error analysis consists of two parts: regularization error and sample error. Allowing multi-kernels in the algorithm improves the regularization error and approximation error, which is one advantage of the multi-kernel setting. For a general loss function, we show how to bound the regularization error by the approximation in some weighted L q spaces. For the sample error, we use a projection operator. The projection in connection with the decay of the regularization error enables us to improve convergence rates in the literature even for the one kernel schemes and special loss functions: least square loss and hinge loss for support vector machine soft margin classifiers. Existence of the optimization problem for the regularization scheme associated with multi-kernels is verified when the kernel functions are continuous with respect to the index set. Gaussian kernels with flexible variances and probability distributions with some noise conditions are demonstrated to illustrate the general theory.
Intuitionistic Fuzzy Proximal Support Vector Machines for Pattern Classification
Neural Processing Letters, 2020
Support vector machine is a powerful technique for classification and regression problems. In the binary data problems, it classifies the points by assigning them to one of the two disjoint halfspaces. However, this method fails to handle the noises and outliers present in the dataset and the solution of a large-sized quadratic programming problem is required to obtain the decision surface in input or in feature space. We propose the intuitionistic fuzzy proximal support vector machine (IFPSVM) which classifies the patterns according to its proximity with the two parallel planes that are kept as distant as possible from each other. These two parallel 'proximal' planes can be obtained by solving a system of linear equations only. There is an intuitionistic fuzzy number associated with each training point which is framed by its degree of membership and non-membership. The membership degree of a pattern considers its distance from the corresponding class center and the degree of non-membership of a pattern is given by the ratio of the number of heterogeneous points to the number of total points in its neighborhood. The proposed technique effectively reduces the impact of noises and distinguishes the edge support vectors and outliers. Computational simulations on an artificial and eleven UCI benchmark datasets using linear, polynomial and Gaussian kernel functions, show the effectiveness of the proposed IFPSVM method. The experiments prove that it can handle large datasets with less computational time and yields better accuracy. Keywords Machine learning • Support vector machines • Fuzzy sets • Kernel functions • Quadratic programming 1 Introduction Support vector machines (SVMs) emerged from statistical learning theory by Vapnik [31]. SVM [3,4,7,35] is based on the structural risk minimization principle and is a powerful learning tool for pattern classification. It is widely used in many areas such as bio-medicine
Journal of Biomedicine and Biotechnology, 2009
DNA microarrays provide rich profiles that are used in cancer prediction considering the gene expression levels across a collection of related samples. Support Vector Machines (SVM) have been applied to the classification of cancer samples with encouraging results. However, they rely on Euclidean distances that fail to reflect accurately the proximities among sample profiles. Then, non-Euclidean dissimilarities provide additional information that should be considered to reduce the misclassification errors. In this paper, we incorporate in the ν-SVM algorithm a linear combination of non-Euclidean dissimilarities. The weights of the combination are learnt in a (Hyper Reproducing Kernel Hilbert Space) HRKHS using a Semidefinite Programming algorithm. This approach allows us to incorporate a smoothing term that penalizes the complexity of the family of distances and avoids overfitting. The experimental results suggest that the method proposed helps to reduce the misclassification errors in several human cancer problems.
Class Prediction from Disparate Biological Data Sources Using an Iterative Multi-Kernel Algorithm
2009
For many biomedical modelling tasks a number of different types of data may influence predictions made by the model. An established approach to pursuing supervised learning with multiple types of data is to encode these different types of data into separate kernels and use multiple kernel learning. In this paper we propose a simple iterative approach to multiple kernel learning (MKL), focusing on multi-class classification. This approach uses a block L 1-regularization term leading to a jointly convex formulation. It solves a standard multi-class classification problem for a single kernel, and then updates the kernel combinatorial coefficients based on mixed RKHS norms. As opposed to other MKL approaches, our iterative approach delivers a largely ignored message that MKL does not require sophisticated optimization methods while keeping competitive training times and accuracy across a variety of problems. We show that the proposed method outperforms state-of-the-art results on an important protein fold prediction dataset and gives competitive performance on a protein subcellular localization task.
On L 1 Norm Multiclass Support Vector Machines
Journal of The American Statistical Association, 2007
Binary Support Vector Machines (SVM) have proven effec- tive in classification. However, problems remain with respect to feature selection in multi-class classification. This article proposes a novel multi-class SVM, which performs classifica- tion and feature selection simultaneously via L1-norm penal- ized sparse representations. The proposed methodology, to- gether with our developed regularization solution path, per- mits feature selection within the