Confusion Matrix
A confusion matrix is a table that summarizes the performance of a classification model by showing the counts of correct and incorrect predictions broken down by actual and predicted class, revealing exactly which categories the model…
Definition
A confusion matrix is a table that summarizes the performance of a classification model by showing the counts of correct and incorrect predictions broken down by actual and predicted class, revealing exactly which categories the model confuses with one another.
Overview
For a binary classification problem, a confusion matrix is a simple 2x2 grid: true positives (correctly predicted positive), true negatives (correctly predicted negative), false positives (predicted positive but actually negative, a 'false alarm'), and false negatives (predicted negative but actually positive, a 'miss'). For a multi-class problem, the matrix extends to an N x N grid, with rows representing actual classes and columns representing predicted classes, so the diagonal shows correct predictions and off-diagonal cells show specific confusions, such as a model that frequently mistakes one dog breed for another. The confusion matrix is valuable precisely because a single number like overall accuracy can be misleading, especially on imbalanced datasets. A model that always predicts 'no fraud' on a dataset where 99% of transactions are legitimate would be 99% accurate while being completely useless — the confusion matrix exposes this immediately by showing zero true positives for the fraud class. It is the foundation from which more targeted metrics are calculated, including precision and recall, the F1 score, and specificity, each of which emphasizes a different tradeoff between catching positives and avoiding false alarms. Because it breaks results down by class rather than collapsing everything into one number, the confusion matrix is also a common starting point for spotting AI bias — computing a separate confusion matrix for different demographic subgroups can reveal that a model performs much worse for one group than another, even when overall accuracy looks acceptable. It's one of the first evaluation tools introduced in most applied machine learning coursework, including Machine Learning Fundamentals.
Key Concepts
- Breaks predictions into true positives, true negatives, false positives, and false negatives
- Extends to an N x N grid for multi-class classification problems
- Reveals specific misclassification patterns that a single accuracy number hides
- Exposes weaknesses of overall accuracy on imbalanced datasets
- Serves as the basis for computing precision, recall, F1 score, and specificity
- Can be computed per demographic subgroup to help detect AI bias