Precision or recall: which matters more, and why is accuracy misleading?

Definitions (positive = the class you are trying to detect)

• Precision = TP / (TP + FP): when the model says positive, how often is it right?
• Recall = TP / (TP + FN): of all the real positives, how many did it find?
• F1 = the harmonic mean of the two. Use Fβ to weight recall β times as much as precision.

Choose by the cost of each kind of mistake

• (a) Cancer screening: favour recall. A missed cancer (FN) can be fatal, while a false alarm (FP) leads to a follow-up test. Screening is tuned to catch nearly everything, and a more specific test then confirms.
• (b) Spam filter: favour precision. Losing an important e-mail (an interview call letter) in the spam folder (FP) is worse than seeing an occasional spam message (FN).
• (c) Fraud: usually recall at an acceptable precision. Missed fraud costs money, but too many false alerts swamp the review team and annoy customers. Teams often fix a review budget (for example, "we can check 1000 alerts a day") and maximise recall within it, or report precision at k.

Why accuracy fails for (c). With 0.5% fraud, a model that always says "legitimate" is 99.5% accurate and catches nothing. On imbalanced data, report:

• the confusion matrix;
• precision, recall and F1 for the minority class;
• PR-AUC (area under the precision–recall curve), which is more informative than ROC-AUC when positives are rare;
• balanced accuracy or the Matthews correlation coefficient (MCC).

The threshold is a choice. Most classifiers output a score, and moving the threshold trades precision against recall. Pick it on validation data according to the business cost, not by defaulting to 0.5. Handling imbalance (class weights, resampling) changes the scores, but you still have to choose the threshold.