Semantic Mismapping in Generalized Category Discovery: Evaluation and Mitigation
Abstract Overview
Generalized category discovery aims to recognize known categories and discover new ones in the same unlabeled dataset. The paper identifies semantic mismapping: global Hungarian matching can reassign supervised old-category outputs and count prediction errors as correct. It proposes Semantics-Preserving Evaluation (SPE), which fixes old-output meanings and matches only new outputs, and Mismapping-Mitigating Optimization (MMO), which regularizes old–new prediction balance and new-prototype diversity. Experiments show substantial evaluation gaps on fine-grained benchmarks and generally improved new-category accuracy with MMO, though improvements are not universal.
Novelty
The work adapts evaluation to the partially exchangeable outputs of parametric category-discovery models and characterizes score gains from mismapping through permutation cycles. It also introduces two auxiliary training losses motivated by this evaluation issue, along with extensions of SPE to non-parametric methods and estimated category counts.
Results
On Herbarium 19, global matching reported new-category accuracy 17.40–25.63 percentage points higher than SPE across seven configurations and altered method rankings. MMO improved new-category accuracy in 23 of 28 fine-grained configurations under both protocols, although some settings exhibited old–new accuracy trade-offs.
Key Points
- SPE preserves supervised old-category labels, matches new outputs locally, and counts cross-region predictions as errors.
- MMO combines a region-balance loss based on estimated old–new sample proportions with a loss encouraging diversity among new-category prototypes.
- Experiments also demonstrate SPE with estimated category numbers and with non-parametric clusters calibrated using labeled data.