Feature Selection vs. Feature Extraction (original) (raw)
Last Updated : 20 Nov, 2025
Feature selection and feature extraction are two key techniques used in machine learning to improve model performance by handling irrelevant or redundant features. While both works on data preprocessing, feature selection uses a subset of existing features whereas feature extraction transforms data into a new feature.
Feature Selection vs. Feature Extraction
- **Feature selection****:** Involves selecting a subset of the most relevant features that are actually contributing in prediction while discarding the rest features. This helps improve reducing overfitting and increased accuracy. Common techniques include filter, wrapper and embedded methods.
- **Feature extraction: Transforms existing features into a new set of features that captures better underlying patterns in data. It is useful when raw data is in high dimension or complex. Techniques like PCA, LDA and Autoencoders are used for this purpose.
**Difference Feature Selection and Feature Extraction Methods
Feature selection and feature extraction methods have their own advantages and disadvantages depending on the nature of the data and the task they handle.
| **Feature Selection | **Feature Extraction |
|---|---|
| Selects a subset of relevant features from the original set of features. | Transforms original features into a new, more informative set. |
| Reduces dimensionality while keeping original features. | Reduces dimensionality by transforming data into a new space. |
| Methods include Filter, Wrapper and Embedded techniques. | Methods include PCA, LDA, Kernel PCA and Autoencoders. |
| Requires domain knowledge and feature engineering. | Can be applied to raw data without prior feature engineering. |
| Enhances interpretability and reduces overfitting. | Improves performance and handles nonlinear relationships. |
| May lose useful information if important features are removed. | May introduce redundancy and noise if extracted features are not well-defined. |
Use Feature Selection When
- You want to keep the original meaning of features.
- The dataset is not extremely high-dimensional.
- You need a more interpretable model.
- You want to remove redundant or irrelevant features.
- Models like Decision Trees, Random Forest, Logistic Regression benefit from clean input features.
- The dataset is very high-dimensional (e.g., images, text, sensor data).
- You need to capture underlying structure not visible in raw features.
- The raw features are correlated or noisy.
- Deep learning models require dense, informative representations (e.g., Autoencoders).
- Tasks involve dimensionality reduction, such as PCA for visualization or compression.