Regularization (L1/L2) Cheat Sheet

Covers L1 (Lasso) and L2 (Ridge) regularization for linear models, including Elastic Net, scikit-learn code, and hyperparameter tuning tips.

2 PagesIntermediateMar 5, 2026

Ridge & Lasso in scikit-learn

Fit L2 and L1 regularized linear models with proper feature scaling.

python

from sklearn.linear_model import Ridge, Lassofrom sklearn.preprocessing import StandardScaler# Always scale features before regularizationscaler = StandardScaler()X_train_scaled = scaler.fit_transform(X_train)X_test_scaled = scaler.transform(X_test)# Ridge (L2) - shrinks coefficients toward zeroridge = Ridge(alpha=1.0)  # alpha = lambda, higher = more regularizationridge.fit(X_train_scaled, y_train)# Lasso (L1) - can shrink coefficients to exactly zero (feature selection)lasso = Lasso(alpha=0.1)lasso.fit(X_train_scaled, y_train)print(lasso.coef_)  # some coefficients will be 0.0

Elastic Net & Logistic Regression

Combine L1/L2 penalties and apply regularization to classification models.

python

from sklearn.linear_model import ElasticNet, ElasticNetCV, LogisticRegression# Elastic Net combines L1 and L2 penalties# l1_ratio=1 -> pure Lasso, l1_ratio=0 -> pure Ridgeen = ElasticNet(alpha=0.1, l1_ratio=0.5)en.fit(X_train_scaled, y_train)# Cross-validated search over alpha and l1_ratioen_cv = ElasticNetCV(l1_ratio=[.1, .5, .7, .9, .95, 1], cv=5)en_cv.fit(X_train_scaled, y_train)# Regularized logistic regression (classification)# penalty: 'l1', 'l2', 'elasticnet', None# C = 1 / lambda -> smaller C = stronger regularizationclf_l2 = LogisticRegression(penalty='l2', C=1.0, solver='lbfgs')clf_l1 = LogisticRegression(penalty='l1', C=0.5, solver='liblinear')

Core Concepts

The math and intuition behind L1 and L2 penalties.

L1 penalty (Lasso)- Adds λΣ|wᵢ| to the loss function; produces sparse solutions by driving some weights exactly to 0
L2 penalty (Ridge)- Adds λΣwᵢ² to the loss function; shrinks all weights smoothly toward 0 but rarely to exactly 0
Elastic Net- Combines L1 and L2: λ₁Σ|wᵢ| + λ₂Σwᵢ²; useful when features are correlated
alpha / lambda (λ)- Regularization strength; higher values increase bias and reduce variance
C (scikit-learn)- Inverse of regularization strength in LogisticRegression/SVC; smaller C means a stronger penalty
Bias-variance tradeoff- Regularization increases bias but reduces variance, often lowering test error
Feature scaling- Required before regularization since penalty magnitude depends on coefficient scale

Hyperparameter Tuning

Practical guidance for choosing regularization strength.

GridSearchCV- Search alpha over a log scale, e.g. np.logspace(-4, 4, 50)
RidgeCV / LassoCV- Built-in cross-validated estimators that select alpha automatically
Standardization- Use StandardScaler so all coefficients are penalized on the same scale
Sparse solutions- Use Lasso/L1 when you expect only a subset of features to matter
Multicollinearity- Ridge handles correlated features better than Lasso, which picks one arbitrarily

Pro Tip

When features are highly correlated, prefer Elastic Net over pure Lasso - Lasso tends to arbitrarily select one feature from a correlated group and zero out the rest, which hurts interpretability and stability.

Was this cheat sheet helpful?

Gradient Descent Cheat Sheet

Model Deployment Basics Cheat Sheet