Implementing L1 and L2 regularization using Sklearn (original) (raw)

Last Updated : 31 Mar, 2026

Regularization is a technique used to prevent overfitting in machine learning models. It works by adding a penalty to the loss function so the model does not become too complex. The two most common types are L1 (Lasso) and L2 (Ridge) regularization. Now let us understand the concept in a simple flow:

• We first control model complexity so it does not overfit the training data.
• We then apply L1 regularization, which can reduce some feature coefficients to zero.
• Next, we apply L2 regularization, which reduces coefficient values but keeps all features.
• Finally, we choose the appropriate method depending on whether we want feature selection (L1) or coefficient shrinkage (L2).

Regularization

Implementation using Scikit Learn

Step 1: Import Required Libraries

• **Ridge : L2 Regularization
• **Lasso : L1 Regularization Python `

import numpy as np import pandas as pd from sklearn.datasets import load_diabetes from sklearn.model_selection import train_test_split from sklearn.linear_model import LinearRegression, Ridge, Lasso from sklearn.metrics import mean_squared_error, r2_score

`

Step 2: Load the Dataset

• load_diabetes loads a built in regression dataset.
• X contains input features.
• y contains the target values. Python `

data = load_diabetes() X = data.data y = data.target

`

Step 3: Split the Dataset

• 80% of data is used for training and 20% for testing.
• random_state=42 ensures reproducible results. Python `

X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42 )

`

Step 4: Building model without Regularization

• No penalty term is applied
• Model may overfit if data is complex
• This serves as our baseline model Python `

linear_model = LinearRegression() linear_model.fit(X_train, y_train)

linear_pred = linear_model.predict(X_test)

linear_mse = mean_squared_error(y_test, linear_pred) linear_r2 = r2_score(y_test, linear_pred)

print("Linear Regression MSE:", linear_mse) print("Linear Regression R2:", linear_r2)

`

**Output:

Evaluating linear regression model

Step 5: Building model with L1 Regularization

• alpha=0.7 controls penalty strength for L1.
• mean_squared_error() calculates average squared prediction error.
• r2_score() shows how well the model explains variation in the target variable. Python `

lasso_model = Lasso(alpha=0.7) lasso_model.fit(X_train, y_train)

lasso_pred = lasso_model.predict(X_test)

lasso_mse = mean_squared_error(y_test, lasso_pred) lasso_r2 = r2_score(y_test, lasso_pred)

print("Lasso (L1) MSE:", lasso_mse) print("Lasso (L1) R2 Score:", lasso_r2)

`

**Output:

Output

Step 6: Building model with L2 Regularization

• alpha=1.0 controls regularization strength. Higher alpha means stronger penalty.
• mean_squared_error() calculates average squared prediction error.
• r2_score() shows how well the model explains variation in the target variable. Python `

ridge_model = Ridge(alpha=1.0) ridge_model.fit(X_train, y_train)

ridge_pred = ridge_model.predict(X_test)

ridge_mse = mean_squared_error(y_test, ridge_pred) ridge_r2 = r2_score(y_test, ridge_pred)

print("Ridge (L2) MSE:", ridge_mse) print("Ridge (L2) R2 Score:", ridge_r2)

`

**Output:

Evaluating ridge model

Download full code from here

Linear Regression vs. L2 vs. L1