Accuracy, precision, recall and F1: measuring classification models

Updated October 21, 2026

Classification models are evaluated using metrics derived from the confusion matrix: accuracy, precision, recall, and F1 score. Accuracy measures overall correctness, but can be misleading for imbalanced datasets. Precision and recall focus on positive predictions, and F1 score balances them.

The confusion matrix and accuracy

Classification models produce four outcomes for binary tasks: true positives, true negatives, false positives, and false negatives. These four categories form the confusion matrix, which is the basis for most evaluation metrics. Accuracy is the proportion of all predictions that are correct, calculated as (TP + TN) divided by the total number of predictions, as described in the Google classification metrics guide.

Accuracy is intuitive and useful when classes are balanced and error costs are similar. However, it can be misleading for imbalanced datasets. For example, a model that always predicts the majority class can achieve high accuracy while failing to identify any positive cases, as noted in the Google machine learning glossary.

In many real-world applications, one type of error is more costly than the other. For instance, in medical diagnosis, a false negative can be more dangerous than a false positive. In such cases, accuracy alone is insufficient, and other metrics should be considered.

Precision and recall

Precision measures the proportion of positive predictions that are actually correct. It is calculated as TP divided by (TP + FP). High precision means that when the model predicts a positive class, it is likely correct, reducing false alarms. The IBM model evaluation overview illustrates precision with a stop sign detection example: if a model flags six stop signs but also four nonexistent ones, its precision is 60%.

Recall, also called the true positive rate, measures the proportion of actual positives that the model correctly identifies. It is calculated as TP divided by (TP + FN). High recall means the model catches most positive cases, which is critical for applications like disease screening. The same IBM source explains that recall can be maximized by predicting positive more often, but this may lower precision.

Precision and recall are often in tension. Improving recall may increase false positives, reducing precision, and vice versa. The choice between them depends on the task: for spam detection, high precision avoids blocking legitimate emails, while for cancer detection, high recall ensures no cases are missed.

F1 score and other derived metrics

The F1 score is the harmonic mean of precision and recall, providing a single number that balances both. It is calculated as 2 * (precision * recall) / (precision + recall). F1 is especially useful when the class distribution is imbalanced and both false positives and false negatives matter. The scikit-learn model evaluation guide discusses various scoring functions and emphasizes choosing a metric aligned with the task's goal.

Other metrics include the false positive rate (FPR), which measures the proportion of actual negatives incorrectly classified as positive, and balanced accuracy, which averages recall across classes. The Microsoft AutoML evaluation documentation recommends using macro recall or balanced accuracy for imbalanced classification, and notes that accuracy can hide minority-class failures.

The choice of metric also depends on the decision threshold. Metrics like precision, recall, and F1 are calculated at a fixed threshold, while ROC AUC and average precision are threshold-independent and evaluate ranking. The Microsoft documentation advises using threshold-independent metrics to compare models across thresholds and threshold-specific metrics to select an operating point.

Choosing the right metric

No single metric captures all aspects of model performance. The selection should be based on the specific task, the cost of different errors, and the class balance. For balanced datasets with equal error costs, accuracy may suffice. For imbalanced datasets or when one error type is more costly, precision, recall, or F1 are more informative.

The Google classification metrics guide notes that for imbalanced datasets, recall is more meaningful than accuracy because it measures the ability to identify positive instances. Similarly, precision is useful when false positives are costly, such as in spam filtering.

Practitioners often evaluate multiple metrics together. For example, a confusion matrix provides a detailed view of all four outcomes, while precision-recall curves show the tradeoff across thresholds. The scikit-learn guide emphasizes using a strictly consistent scoring function that aligns with the prediction goal, whether it is classification accuracy or probability estimation.

Frequently asked questions

Why is accuracy not enough for imbalanced datasets?

Accuracy can be high even if the model fails to identify any positive cases, because the majority class dominates. For example, a model that always predicts the majority class in a dataset with 1% positives would be 99% accurate but useless.

What is the difference between precision and recall?

Precision measures how many of the model's positive predictions are actually correct, while recall measures how many of the actual positives the model caught. Precision focuses on false positives, recall on false negatives.

When should the F1 score be used?

F1 score is useful when you need a single metric that balances precision and recall, especially for imbalanced datasets where both false positives and false negatives matter. It is the harmonic mean, so it penalizes extreme values.

How does the decision threshold affect these metrics?

Precision, recall, and F1 are calculated at a fixed threshold. Changing the threshold changes the tradeoff: a lower threshold increases recall but may decrease precision. Threshold-independent metrics like ROC AUC evaluate ranking across all thresholds.

What is the confusion matrix?

The confusion matrix is a 2x2 table that shows the counts of true positives, true negatives, false positives, and false negatives. It provides a detailed breakdown of a model's errors and is the basis for calculating accuracy, precision, recall, and F1.