Precision and Recall

This blog is to introduce some important classifier metrics: precision and recall. The precision of the classifier is the accuracy of the positive predictions.

where TP is the number of true positives, and FP is the number of false positives. Another metric, recall, also called sensitivity or the true positive rate (TPR), is the ratio of positive instances that are correctly detected by the classifier.

where FN is the number of false negatives. To compare binary classifiers, it is convenient to use the F1 score, which is the harmonic mean of precision and recall.

Whereas the regular mean treats all values equally, the harmonic mean gives much more weight to low values. As a result, the classifier will only get a high F1 score if both recall and precision are high. The F1 score favors classifiers that have similar precision and recall. However, in some contexts you mostly care about precision, and in other contexts you really care about recall. We can’t have it both ways: increasing precision reduces recall, and vice versa. This is called the precision/recall trade-off.

For each instance, classifer method computes a score based on a decision function. If that score is greater than a threshold, it assigns the instance to the positive class; otherwise it assigns it to the negative class. The higher the threshold, the lower the recall, but the higher the precision. In other words, raising the threshold decreases recall. Note that a high-precision classifier is not very useful if its recall is too low.