Underfitting

Definition

Underfitting occurs when a machine learning model is too simple to capture the underlying patterns in the training data, resulting in poor performance on both training and new data.

Overview

An underfit model fails to learn the relevant relationships in its training data, often because it lacks sufficient capacity, was trained for too few iterations, or is applied to a problem that is genuinely more complex than the model architecture can represent. Unlike overfitting, where a model performs well on training data but poorly on new data, an underfit model performs poorly on both — a clear indicator that the model itself, not generalization, is the bottleneck. Common causes include using an overly simple algorithm for a complex problem, insufficient training time, excessive regularization that overly constrains the model, or poor feature engineering that fails to expose the relevant signal in the data. Addressing underfitting typically involves increasing model complexity or capacity, training for longer, reducing regularization strength, or improving the input features and representations available to the model. Underfitting and overfitting represent the two failure modes of the bias-variance tradeoff: an underfit model has high bias (it makes overly simplistic assumptions), while an overfit model has high variance (it is overly sensitive to the specific training data). Finding the right balance is a central goal of model development in supervised learning.

Key Concepts

• Model performs poorly on both training and unseen data
• Usually caused by insufficient model capacity or training time
• Represents the high-bias end of the bias-variance tradeoff
• Can result from excessive regularization or poor feature representation
• Addressed by increasing model complexity or improving features
• Easier to diagnose than overfitting since training accuracy itself is low

Use Cases

Frequently Asked Questions

Related in AI Fundamentals