A comparison of feature extraction and selection techniques (original) (raw)
Related papers
Interdisciplinary Publishing Academia, 2020
Due to sharp increases in data dimensions, working on every data mining or machine learning (ML) task requires more efficient techniques to get the desired results. Therefore, in recent years, researchers have proposed and developed many methods and techniques to reduce the high dimensions of data and to attain the required accuracy. To ameliorate the accuracy of learning features as well as to decrease the training time dimensionality reduction is used as a pre-processing step, which can eliminate irrelevant data, noise, and redundant features. Dimensionality reduction (DR) has been performed based on two main methods, which are feature selection (FS) and feature extraction (FE). FS is considered an important method because data is generated continuously at an ever-increasing rate; some serious dimensionality problems can be reduced with this method, such as decreasing redundancy effectively, eliminating irrelevant data, and ameliorating result comprehensibility. Moreover, FE transacts with the problem of finding the most distinctive, informative, and decreased set of features to ameliorate the efficiency of both the processing and storage of data. This paper offers a comprehensive approach to FS and FE in the scope of DR. Moreover, the details of each paper, such as used algorithms/approaches, datasets, classifiers, and achieved results are comprehensively analyzed and summarized. Besides, a systematic discussion of all of the reviewed methods to highlight authors' trends, determining the method(s) has been done, which significantly reduced computational time, and selecting the most accurate classifiers. As a result, the different types of both methods have been discussed and analyzed the findings.
Annals of Data Science, 2018
Classification of high dimensional data is a very crucial task in bioinformatics. Cancer classification of the microarray is a typical application of machine learning due to the large numbers of genes. Feature (genes) selection and classification with computational intelligent techniques play an important role in diagnosis and prediction of disease in the microarray. Artificial neural networks (ANN) is an artificial intelligence technique for classifying, image processing and predicting the data. This paper evaluates the performance of ANN classifier using six different hybrid feature selection techniques, for gene selection of microarray data. These hybrid techniques use Independent component analysis (ICA), as an extraction technique, popular filter techniques and bio-inspired algorithm for optimization of the ICA feature vector. Five binary gene expression microarray datasets are used to compare the performance of these techniques and determine how these techniques improve the performance of ANN classifier. These techniques can be extremely useful in feature selection because they achieve the highest classification accuracy along with the lowest average number of selected genes. Furthermore, to check the significant difference between these different algorithms a statistical hypothesis test was employed with a certain level of confidence. The experimental result shows that a combination of ICA with genetic bee colony algorithm shows superior performance as it heuristically removes noncontributing features to improve the performance of classifiers. Keywords DNA microarrays • Independent component analysis (ICA) • Artificial neural networks (ANN) • Genetic bee algorithm (GBC) • Cancer classification B Rabia Aziz
A Neural Adaptive Algorithm for Feature Selection and Classification of High Dimensionality Data
2005
In this paper, we propose a novel method which involves neural adaptive techniques for identifying salient features and for classifying high dimensionality data. In particular a network pruning algorithm acting on Multi-Layer Perceptron topology is the foundation of the feature selection strategy. Feature selection is implemented within the back-propagation learning process and based on a measure of saliency derived from bell functions positioned between input and hidden layers and adaptively varied in shape and position during learning. Performances were evaluated experimentally within a Remote Sensing study, aimed to classify hyperspectral data. A comparison analysis was conducted with Support Vector Machine and conventional statistical and neural techniques. As seen in the experimental context, the adaptive neural classifier showed a competitive behavior with respect to the other classifiers considered; it performed a selection of the most relevant features and showed a robust behavior operating under minimal training and noisy situations.
The Use of Feature Selection Techniques in the Context of Artificial Neural Networks
Feature selection is an important issue, especially for classification problems where artificial neural networks are involved. It is known that using large number of inputs can make the network overspecific and require significantly longer time to learn the characteristics of the training data. Such over-specificity also reduces the generalisation capabilities of a neural network, so the network may fail to classify new data outside the range of the training data. Although feature selection methods have been used in remote sensing studies for many years, their use in the context of artificial neural networks has not been fully investigated. This paper sets out some results of an investigation of feature selection techniques, specifically the separability indices, in the problem of determining the optimum network structure in terms of achieved accuracy. For this purpose, separability indices, including divergence, transformed divergence, Bhattacharyya distance and Jeffries-Matusita distance, and the Mahalanobis distance classifier (MDC) based on two accuracy measures are employed to determine the best eight-band combination out of a 24 band multitemporal dataset. Two search procedures, sequential forward selection and the genetic algorithm, have been used to search for the best band combinations using separability measures as evaluation functions.
An Empirical Comparison of Dimensionality Reduction Techniques for Pattern Classification
1997
To some extent or other all classifiers are subject to the curse of dimensionality. Consequently, pattern classification is often preceded with finding a reduced dimensional representation of the patterns. In this paper we empirically compare the performance of unsupervised and supervised dimensionality reduction techniques. The data set we consider is obtained by segmenting cells in cytological preparations and extracting 9 features from each of the cells. We evaluate the performance of 4 dimensionality reduction techniques (2 unsupervised) and (2 supervised) with and without noise. The unsupervised techniques include principal component analysis and self-organizing feature maps, while the supervised techniques include Fisher's linear discriminants and multi-layered feed-forward neural networks. Our results on a real world data set indicate that multi-layered feed-forward neural networks outperform the other three dimensionality reduction techniques and that all techniques are sensitive to noise.
Feature Selection Using Artificial Neural Networks
Lecture Notes in Computer Science, 2008
Machine learning is useful for building robust learning models, and it is based on a set of features that identify a state of an object. Unfortunately, some data sets may contain a large number of features making, in some cases, the learning process time consuming and the generalization capability of machine learning poor. To make a data set easy to learn and understand, it is typically recommended to remove the most irrelevant features from the set. However, choosing what data should be kept or eliminated may be performed by complex selection algorithms, and optimal feature selection may require an exhaustive search of all possible subsets of features which is computationally expensive. This paper proposes a simple method to perform feature selection using artificial neural networks. It is shown experimentally that genetic algorithms in combination with artificial neural networks can easily be used to extract those features that are required to produce a desired result. Experimental results show that very few hidden neurons are required for feature selection as artificial neural networks are only used to assess the quality of an individual, which is a chosen subset of features.
A Survey Of Dimensionality Reduction And Classification Methods
International Journal of Computer Science & Engineering Survey, 2012
Dimensionality Reduction is usually achieved on the feature space by adopting any one of the prescribed methods that fall under the selected technique. Feature selection and Feature extraction being the two aforesaid techniques of reducing dimensionality, the former discards certain features that may be useful at a later stage whereas the latter reconstructs its features into a simpler dimension thereby preserving all its initial characteristics. The sole purpose of this survey is to provide an adequate comprehension of the different dimensionality reduction techniques that exist currently and also to introduce the applicability of any one of the prescribed methods depending upon the given set of parameters and varying conditions as described, under each algorithm's usage statistics. This paper also presents guidelines where in, selection of the best possible algorithm for a specific instance can be determined with ease when a condition arises where in two or more algorithms may be suitable for executing the aforementioned task.
Comparative Analysis of Dimensionality Reduction Techniques for Machine Learning
Dimensionality reduction as a pre-processing step to machine learning is effective in removing irrelevant and redundant data, increasing learning accuracy, and improving result comprehensibility. However, the recent increase of dimensionality of data poses a severe challenge to many existing feature selection and feature extraction methods with respect to efficiency and effectiveness. In the field of machine learning and pattern recognition, dimensionality reduction is important area, where many approaches have been proposed. Aim of this paper is to reduce the dimensionality of the dataset without the loss of any information from the datasets. We have implemented three dimensionality reduction algorithms.So this three algorithms are performed on two datasets, Iris and Wines datasets and the results are analyzed.
Dimensionality Reduction for Classification? Comparison of Techniques and Dimension Choice
We investigate the eects of dimensionality reduction using dierent techniques and dierent dimensions on six two-class data sets with numerical attributes as pre-processing for two classification algo- rithms. Besides reducing the dimensionality with the use of principal components and linear discriminants, we also introduce four new tech- niques. After this dimensionality reduction two algorithms are applied. The first algorithm takes advantage of the reduced dimensionality itself while the second one directly exploits the dimensional ranking. We ob- serve that neither a single superior dimensionality reduction technique nor a straightforward way to select the optimal dimension can be iden- tified. On the other hand we show that a good choice of technique and dimension can have a major impact on the classification power, gen- erating classifiers that can rival industry standards. We conclude that dimensionality reduction should not only be used for visualisation or as pre-processing...