Macro F1 and Macro F1 (original) (raw)
Abstract:The 'macro F1' metric is frequently used to evaluate binary, multi-class and multi-label classification problems. Yet, we find that there exist two different formulas to calculate this quantity. In this note, we show that only under rare circumstances the two computations can be considered equivalent. More specifically, one formula well 'rewards' classifiers which produce a skewed error type distribution. In fact, the difference in outcome of the two computations can be as high as 0.5. The two computations may not only diverge in their scalar result but can also lead to different classifier rankings.
Submission history
From: Juri Opitz [view email]
[v1] Fri, 8 Nov 2019 16:13:54 UTC (598 KB)
[v2] Wed, 20 Nov 2019 09:24:02 UTC (1,086 KB)
[v3] Mon, 8 Feb 2021 18:20:55 UTC (1,086 KB)