Linear Regression Cheat Sheet
A reference for linear regression covering scikit-learn implementation, the normal equation, regularized variants, and key statistical assumptions to check.
1 PageBeginnerMar 10, 2026
Fitting with scikit-learn
Train and evaluate an ordinary least squares model.
python
from sklearn.linear_model import LinearRegressionfrom sklearn.model_selection import train_test_splitfrom sklearn.metrics import mean_squared_error, r2_scoreX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)model = LinearRegression()model.fit(X_train, y_train)y_pred = model.predict(X_test)print('R2:', r2_score(y_test, y_pred))print('MSE:', mean_squared_error(y_test, y_pred))print('Coefficients:', model.coef_, 'Intercept:', model.intercept_)
Normal Equation
Closed-form solution without gradient descent.
python
# Closed-form solution: beta = (X^T X)^-1 X^T yimport numpy as npX_b = np.c_[np.ones((len(X), 1)), X] # add bias/intercept columnbeta = np.linalg.inv(X_b.T @ X_b) @ X_b.T @ y
Regularized Variants
Ridge, Lasso, and Elastic Net regression.
python
from sklearn.linear_model import Ridge, Lasso, ElasticNetridge = Ridge(alpha=1.0).fit(X_train, y_train) # L2 penaltylasso = Lasso(alpha=0.1).fit(X_train, y_train) # L1 penalty, can zero out coefficientselastic = ElasticNet(alpha=0.1, l1_ratio=0.5).fit(X_train, y_train) # mix of L1 and L2
Key Concepts
Core theory behind linear regression.
- Ordinary Least Squares- Minimizes the sum of squared residuals between predicted and actual values
- R-squared- Proportion of variance in the target explained by the model; 1.0 is a perfect fit
- Multicollinearity- High correlation between features inflates coefficient variance; check with VIF
- Regularization- Ridge (L2) shrinks coefficients smoothly, Lasso (L1) can zero them out entirely
- Homoscedasticity- Assumption that residual variance stays constant across all predicted values
- Residual plot- Plot of residuals vs. predictions used to visually check assumption violations
Pro Tip
Don't rely on R-squared alone to judge model fit — always plot residuals against predicted values, since a high R-squared can still hide non-linearity or heteroscedasticity that biases your standard errors and confidence intervals.
Was this cheat sheet helpful?
Explore Topics
#LinearRegression#LinearRegressionCheatSheet#DataScience#Beginner#FittingWithScikitLearn#NormalEquation#RegularizedVariants#KeyConcepts#MachineLearning#CheatSheet#SkillVeris
