Data Drift
Data drift is a change in the statistical distribution of a machine learning model's input data over time, which can degrade model performance even if the underlying relationship between inputs and outputs has not changed.
Definition
Data drift is a change in the statistical distribution of a machine learning model's input data over time, which can degrade model performance even if the underlying relationship between inputs and outputs has not changed.
Overview
Machine learning models assume, implicitly, that the data they see in production resembles the data they were trained on. Data drift happens when that assumption breaks down — new user demographics start using a product, a sensor is recalibrated and shifts its reading range, or a seasonal pattern shifts input distributions — so the model is now being asked to make predictions on inputs that look meaningfully different from what it learned on. There are a few common flavors of data drift. Covariate shift is when the distribution of input features changes but the relationship between inputs and the target stays the same. Label shift is when the distribution of the target outcome itself changes (for example, the overall fraud rate rising), even though the relationship given each input stays constant. Both are distinct from model drift, where the actual relationship between inputs and outputs changes — though in real systems these effects frequently overlap and can be difficult to tell apart without careful monitoring. Detecting data drift typically involves comparing statistical summaries — feature means, distributions, or more formal statistical tests — between the training data and recent production data, and flagging significant divergence. Because it doesn't require waiting for ground-truth labels, data drift monitoring can often catch problems earlier than model drift monitoring, which needs labeled outcomes to measure accuracy directly. Addressing drift may involve retraining, reweighting training data to better reflect the new distribution, or investigating upstream data pipeline changes that caused an unexpected shift — a common topic in MLOps & Model Deployment coursework.
Key Concepts
- Describes a shift in the statistical distribution of input data over time
- Includes covariate shift (input distribution changes) and label shift (outcome distribution changes)
- Distinct from model drift, where the input-output relationship itself changes
- Detectable without ground-truth labels, using statistical comparisons of feature distributions
- Often an early warning sign that precedes measurable accuracy degradation
- Addressed through retraining, data reweighting, or fixing upstream pipeline issues