ClassSPLOM -- A Scatterplot Matrix to Visualize Separation of Multiclass Multidimensional Data (original) (raw)

Using discriminant analysis for multi-class classification: an experimental investigation

2006

Abstract Many supervised machine learning tasks can be cast as multi-class classification problems. Support vector machines (SVMs) excel at binary classification problems, but the elegant theory behind large-margin hyperplane cannot be easily extended to their multi-class counterparts. On the other hand, it was shown that the decision hyperplanes for binary classification obtained by SVMs are equivalent to the solutions obtained by Fisher's linear discriminant on the set of support vectors.

Visualization in Classification Problems

In the simplest form support vector machines (SVM) define a separating hyperplane between classes generated from a subset of cases, called support vectors. The support vectors "mark" the boundary between two classes. The result is an interpretable classifier, where the importance of the variables to the classification, is identified by the coefficients of the variables defining the hyperplane. This paper describes visual methods that can be used with classifiers to understand cluster structure in data.

ClassiMap: a new dimension reduction technique for exploratory data analysis of labeled data

Multidimensional scaling techniques are unsupervised Dimension Reduction (DR) techniques which use multidimensional data pairwise similarities to represent data into a plane enabling their visual exploratory analysis. Considering labeled data, the DR techniques face two objectives with potentially different priorities: one is to account for the data points’ similarities, the other for the data classes’ structures. Unsupervised DR techniques attempt to preserve original data similarities, but they do not consider their class label hence they can map originally separated classes as overlapping ones. Conversely, the state-of-the-art so-called supervised DR techniques naturally handle labeled data, but they do so in a predictive modeling framework where they attempt to separate the classes in order to improve a classification accuracy measure in the low-dimension space, hence they can map as separated even originally overlapping classes. We propose ClassiMap, a DR technique which optimizes a new objective function enabling exploratory data analysis of labeled data. Mapping distortions known as tears and false neighborhoods cannot be avoided in general due to the reduction of the data dimension. ClassiMap intends primarily to preserve data similarities but tends to distribute preferentially unavoidable tears among the different-label data and unavoidable false neighbors among the same-label data. Standard quality measures to evaluate the quality of unsupervised mappings cannot tell about the preservation of within-class or between-class structures, while classification accuracy used to evaluate supervised mappings is only relevant to the framework of predictive modeling. We propose two measures better suited to the evaluation of DR of labeled data in an exploratory data analysis framework. We use these two label-aware indices and four other standard unsupervised indices to compare ClassiMap to other state-of-the-art supervised and unsupervised DR techniques on synthetic and real datasets. ClassiMap appears to provide a better tradeoff between pairwise similarities and class structure preservation according to these new measures.

Using discriminant analysis for multi-class classification

Proceedings of the Third IEEE International …, 2003

Discriminant analysis is known to learn discriminativefeature transformations. This paper studies its use in multi-classclassification problems. The performance is tested ona large collection of benchmark datasets. ... Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references. ... C. Blake and C. Merz. UCI repository of machine learning databases, 1998. ... GJ McLachlan. Discriminant Analysis and Statistical Pattern Recognition. John Wiley & ...

Polynomial Kernel Discriminant Analysis for 2D visualization of classification problems

Neural Computing and Applications, 2017

In multivariate classification problems, 2D visualization methods can be very useful to understand the data properties whenever they transform the n-dimensional data into a set of 2D patterns which are similar to the original data from the classification point of view. This similarity can be understood as that a classification method works similarly on the original n-dimensional and on the 2D mapped patterns, i.e., the classifier performance should not be much lower on the mapped than on the original patterns. We propose several simple and efficient mapping methods which allow to visualize classification problems in 2D. In order to preserve the structure about the original classification problem, the mappings minimize different class overlap measures, combined with different functions (linear, quadratic and polynomic of several degrees) from R n to R 2. They are also able to map into R 2 new data points (out of sample), not used during the mapping learning. This is one of the main benefits of the proposed methods, since few supervised mappings offer a similar behavior. For 71 data sets of the UCI database, we compare the SVM performance using the original and the 2D mapped patterns. The comparison also includes other 34 popular supervised and unsupervised methods of dimensionality reduction, some of them used for the first time in classification. One of the proposed methods, the Polynomial Kernel Discriminant Analysis of degree 2 (PKDA2), outperforms the remaining mappings. Compared to the original n-dimensional patterns, PKDA2 achieves 82% of the performance (measured by the Cohen kappa), raising or keeping the performance for 26.8% of the data sets. For 36.6% of the data sets, the performance is reduced by less than 10%, and it is reduced by more than 20% only for 22.5% of the data sets. This low reduction in performance shows that the 2D maps created by PKDA2 really represent the original data, whose ability to be classified in 2D is highly preserved. Besides, PKDA is very fast, with times of the same order than LDA. The MATLAB code is available.

Discriminative Dimensionality Reduction for the Visualization of Classifiers

Advances in Intelligent Systems and Computing, 2014

We propose a novel way to use discriminative analysis to project high-dimensional EEG data onto a low-dimensional discriminative space for visualization, analysis, and statistical testing. This multivariate analysis directly controls for the multiple comparison problem (MCP) by effectively reducing the number of test variables. A major advantage of this approach is that it is possible to compare the brain activity across conditions even when the trial count is low, provided that a sufficient number of trials are used to establish the initial hyperplane(s), meaning that error conditions and conditions that divide subtle behavioral differences can be readily compared. Currently these data are either ignored or lumped with other data thereby losing the ability to reveal the neural mechanisms underlying subtle behavioral differences. The proposed method provides a powerful tool to analyze conditions with relatively small numbers of trials from high-dimensional neural recordings.

Visualization of multivariate data with additional class information

Image Analysis, Computer Graphics, Security Systems and Artificial Intelligence Applications, 2005

The goal is to visualize a set of multivariate data in such a way that data vectors belonging to different classes (subgroups) appear differentiated as much as possible. When intending such visualization, the first question should be about the intrinsic dimensionality of the data. The answer may be obtained by evaluating, e.g., the fractal correlation dimension. The projection to a plane is justified when the correlation dimension of the data is about 2. Only in such case the performed visualization is plausible to reflect all the between group and the within group relationships among the data vectors. There are several recognized methods for mapping data to a plane. Our interest lies especially in nonlinear methods. We consider in detail three methods: The canonical discriminant functions, the kernel discriminant functions and the neuroscale mapping. We illustrate our considerations using the Kefallinia erosion data, where each data vector belongs -in a crisp way -to one of five predefined subgroups indicating the severity of the erosion risk. The assignments to the subgroups were performed by an expert GIS system based on logical rules established by experts.

Nonlinear Discriminative Data Visualization

2009

Due to the tremendous increase of electronic information with respect to the size of data sets as well as dimensionality, visualization of high-dimensional data constitutes one of the key problems of data mining. Since embedding in lower dimensions necessarily includes a loss of information, methods to explicitly control the information kept by a specific visualization technique are highly desirable. The incorporation of supervised class information constitutes an important specific case. In this contribution we propose an extension of prototype-based local matrix learning by a charting technique which results in an efficient nonlinear dimension reduction and discriminative visualization of a given labelled data manifold.

Multidimensional Data Visualization Based on the Minimum Distance Between Convex Hulls of Classes

Pattern Recognition and Image Analysis, 2018

The problem of data visualization in the analysis of two classes in a multidimensional feature space is considered. The two orthogonal axes by which the classes are maximally separated from each other are found in the mapping of classes as a result of linear transformation of coordinates. The proximity of the classes is estimated based on the minimum-distance criterion between their convex hulls. This criterion makes it possible to show cases of full class separability and random outliers. A support vector machine is used to obtain orthogonal vectors of the reduced space. This method ensures the obtaining of the weight vector that determines the minimum distance between the convex hulls of classes for linearly separable classes. Algorithms with reduction, contraction, and offset of convex hulls are used for intersecting classes. Experimental studies are devoted to the application of the considered visualization methods to biomedical data analysis.

Class proximity measures – Dissimilarity-based classification and display of high-dimensional data

Journal of Biomedical Informatics, 2011

For two-class problems, we introduce and construct mappings of high-dimensional instances into dissimilarity (distance)-based Class-Proximity Planes. The Class Proximity Projections are extensions of our earlier relative distance plane mapping, and thus provide a more general and unified approach to the simultaneous classification and visualization of many-feature datasets. The mappings display all Ldimensional instances in two-dimensional coordinate systems, whose two axes represent the two distances of the instances to various pre-defined proximity measures of the two classes. The Class Proximity mappings provide a variety of different perspectives of the dataset to be classified and visualized. We report and compare the classification and visualization results obtained with various Class Proximity Projections and their combinations on four datasets from the UCI data base, as well as on a particular highdimensional biomedical dataset.