Cross-Validation Cheat Sheet

Methods for reliably estimating model generalization performance, covering k-fold, stratified, time-series, and leave-one-out cross-validation with scikit-learn.

2 PagesIntermediateMar 20, 2026

K-Fold Cross-Validation

Standard k-fold split and scoring.

python

from sklearn.model_selection import KFold, cross_val_scorefrom sklearn.ensemble import RandomForestClassifiermodel = RandomForestClassifier(random_state=42)kf = KFold(n_splits=5, shuffle=True, random_state=42)scores = cross_val_score(model, X, y, cv=kf, scoring="accuracy")print(f"Mean accuracy: {scores.mean():.3f} +/- {scores.std():.3f}")

Stratified & Time Series Splits

Preserve class balance or chronological order.

python

from sklearn.model_selection import StratifiedKFold, TimeSeriesSplit# Preserves class proportions in each fold -- use for classificationskf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)for train_idx, val_idx in skf.split(X, y):    X_train, X_val = X[train_idx], X[val_idx]    y_train, y_val = y[train_idx], y[val_idx]# Respects temporal order -- never shuffle time series datatscv = TimeSeriesSplit(n_splits=5)for train_idx, val_idx in tscv.split(X):    X_train, X_val = X[train_idx], X[val_idx]

CV Strategies

Which splitting strategy to use and when.

K-Fold- splits data into k equal folds; each fold used once as validation
Stratified K-Fold- preserves class distribution in each fold; use for imbalanced classification
Leave-One-Out (LOO)- k = n; expensive but low bias, high variance
Time Series Split- expanding-window splits that respect chronological order, no shuffling
Group K-Fold- ensures samples from the same group (e.g. patient, user) stay in one fold
Repeated K-Fold- repeats k-fold multiple times with different splits for a more stable estimate
Nested CV- outer loop for performance estimation, inner loop for hyperparameter tuning; avoids optimistic bias

Common Pitfalls

Mistakes that invalidate a cross-validation estimate.

Leakage before splitting- scaling/imputing on the full dataset before CV leaks test info into training
Shuffling time series- destroys temporal order and lets the model see the future
Ignoring groups- random k-fold on grouped data (e.g. multiple rows per patient) leaks identity across folds
Tuning on the same folds- using CV both to tune hyperparameters and report final performance overstates accuracy
Too few folds on small data- small k on small datasets gives high-variance, unreliable estimates

Pro Tip

When tuning hyperparameters, use nested cross-validation (or a separate held-out test set) — evaluating on the same folds you tuned on gives an optimistically biased performance estimate.

Was this cheat sheet helpful?

Feature Engineering Cheat Sheet

Hyperparameter Tuning Cheat Sheet

Explore Topics

#CrossValidation#CrossValidationCheatSheet#DataScience#Intermediate#KFoldCrossValidation#Stratified#Time#Series#Functions#MachineLearning#CheatSheet#SkillVeris

Sri Hayavadhana Info-Tech

Professional Web Designing Services