Toward Robust In-Context Segmentation via Concept Guidance
Abstract Overview
This paper studies in-context segmentation, where a model segments a target region in a query image using only a few reference images and masks without updating model parameters. The authors propose Concept-Guided In-Context Segmentation (CG-ICS), which performs segmentation by extracting high-level semantic concepts from the references. The method is motivated by the need for greater robustness in the face of varying reference choices. Experiments on standard in-context segmentation benchmarks indicate state-of-the-art accuracy and remarkably stronger robustness.
Novelty
The distinctive idea is to guide in-context segmentation with high-level semantic concepts extracted from reference examples rather than relying only on lower-level matching cues. The work is fundamentally positioned around robustness, specifically reducing sensitivity and performance variation depending on which reference examples are selected.
Results
On standard in-context segmentation benchmarks, CG-ICS is reported to achieve state-of-the-art accuracy. The paper also reports substantial robustness gains and a clear reduction in performance variation across diverse reference selections, resulting in a more reliable in-context segmentation system.
Key Points
- CG-ICS segments query images by extracting and leveraging high-level semantic concepts from a small set of reference images and masks.
- The method specifically targets robustness in in-context segmentation, aiming to reduce sensitivity to the varying choice of reference examples.
- Experiments demonstrate state-of-the-art accuracy together with improved robustness and significantly lower performance variation across diverse reference selections.