OAI Labs — classification-model evaluation | oai-labs.com

OAI Labs — classification-model evaluation | oai-labs.com

Precision, recall and F1

Derive every metric from one fully worked numerical example so the formulas and the numbers stay linked.

Using TP=8, FP=2, FN=4 and TN=8 over 20 cases, precision is 8 / (8 + 2) = 80%. The denominator is the number of predicted positives, so precision answers how often a positive prediction was correct. Recall is 8 / (8 + 4) = 66.67%, with the denominator being the number of actual positives, so recall answers how much of the positive class was found.

F1 is the harmonic mean of the two: 2 × 0.80 × 0.6667 / (0.80 + 0.6667) ≈ 0.7273, or 72.73%. Accuracy is (8 + 8) / 20 = 80%. Note that accuracy and precision coincide here only because of these particular counts; they measure different things and will diverge as the class balance changes.

The signature mark of this site, drawn as a plate

On narrow screens, swipe or scroll the plate sideways.

A useful sanity check is to compare F1 with the arithmetic mean of precision and recall, which is about 73.33% here. F1 sits below it because the harmonic mean is pulled toward the smaller value, 66.67%. If the gap between F1 and the arithmetic mean is large, precision and recall are far apart and the single F1 number is hiding a trade-off worth reporting.

When a denominator is zero the metric is undefined. Precision is undefined when nothing was predicted positive; recall is undefined when there are no actual positives. Scikit-learn documents a zero_division parameter for controlling behaviour when a metric denominator is zero, and whatever policy you adopt should be recorded with the results so later comparisons use the same rule.

Report the cells next to the metrics. A reader who sees only 72.73% cannot tell whether the model misses positives or over-predicts them, and those two failures suggest different remedies.

  • Precision denominator: predicted positives (TP + FP).
  • Recall denominator: actual positives (TP + FN).
  • F1: harmonic mean, pulled toward the smaller of the two.
  • Accuracy: counts true negatives, so it answers a different question.

Further reading