Overfitting

Supervised Learning

Supervised learning is a machine learning approach in which a model learns to map inputs to outputs by training on a dataset of labeled examples.

Overfitting

Overfitting occurs when a machine learning model learns the training data too closely, including its noise and idiosyncrasies, resulting in poor performance on…

Underfitting

Underfitting occurs when a machine learning model is too simple to capture the underlying patterns in the training data, resulting in poor performance on both…

Gradient Descent

Gradient descent is an optimization algorithm that iteratively adjusts a model's parameters in the direction that most reduces its error, or loss, in order to…

Backpropagation

Backpropagation is the algorithm used to train neural networks by efficiently computing how much each parameter contributed to the model's error, so those para…

Model Pruning

Model pruning is a technique that removes redundant or low-impact parameters from a trained neural network, reducing its size and computational cost while aimi…

Catastrophic Forgetting

Catastrophic forgetting is the tendency of a neural network to abruptly lose previously learned knowledge when it is trained on new data or tasks, because the…

Data Augmentation

Data augmentation is the practice of artificially expanding a training dataset by applying transformations to existing examples — such as rotating images or pa…

Cross-Validation

Cross-validation is a model evaluation technique that repeatedly splits a dataset into different training and validation subsets, trains and tests the model on…

Hyperparameter Tuning

Hyperparameter tuning is the process of systematically searching for the configuration values that control a machine learning model's training process — such a…

Feature Selection

Feature selection is the process of choosing a subset of the most relevant input variables from a dataset to use in building a machine learning model, discardi…

Weight Decay

Weight decay is a regularization technique that adds a penalty proportional to the magnitude of a model's weights to the loss function, discouraging overly lar…

Early Stopping

Early stopping is a regularization technique that halts model training once performance on a held-out validation set stops improving, preventing the model from…

Overfitting and Underfitting

The two failure modes of model fitting — memorizing noise in training data versus failing to capture real patterns — and the diagnostic and mitigation techniqu…