DishSeg24k: A Large-Scale Benchmark for Food Segmentation with Stochastic Expert Decoding

• URL: http://arxiv.org/abs/2607.23070v1
• Date: Sat, 25 Jul 2026 06:40:25 GMT
• Title: DishSeg24k: A Large-Scale Benchmark for Food Segmentation with Stochastic Expert Decoding
• Abstract summary: We introduce textbfDishSeg24k, a large-scale dish-level segmentation benchmark with 24,096 images, 112,281 instances, and 278 fine-grained categories in real-world dining environments.<n>Based on DishSeg24k, we propose textbfFood Expert-Adaptive Transformers (FEAST) to address these challenges.<n>Experiments on DishSeg24k demonstrate the state-of-the-art performance of FEAST, which outperforms previous methods by +3.21% mIoU, +3.68
• Score: 49.56343480718948
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Food segmentation is essential for applications such as intelligent catering, dietary assessment, and recommendation. However, existing benchmarks fail to capture the complexity of real-world dining scenes. The challenges of dense inter-dish overlap, fine-grained class similarity, and extreme long-tail class distributions exceed the fidelity of current datasets. To fill this gap, we introduce \textbf{DishSeg24k}, a large-scale dish-level segmentation benchmark with 24,096 images, 112,281 instances, and 278 fine-grained categories in real-world dining environments. Based on DishSeg24k, we further propose \textbf{Food Expert-Adaptive Segmentation Transformers (FEAST)} to address these challenges. FEAST models query-based decoding as a Markov Decision Process (MDP), where each decoder layer update is treated as a sequential decision step that explores uncertainty along dish boundaries. We further redesign the decoder with a reinforcement learning (RL)-guided Mixture-of-Experts (MoE) module, in which a dual-critic decoupled optimization scheme separates task-oriented query refinement from structure-aware expert routing. This design promotes expert specialization and prevents expert collapse under long-tail category distributions. Finally, extensive experiments on DishSeg24k demonstrate the state-of-the-art performance of FEAST, which outperforms previous methods by {+3.21\%} mIoU, {+3.68\%} mDice, and {+4.00\%} mAcc, respectively. We further validate the effectiveness of FEAST on FoodSeg103. The dataset and code will be publicly released.

Related papers

• Technical Report for ICRA 2026 GOOSE 2D Fine-Grained Semantic Segmentation Challenge: Exploring Query-Based Segmentation and Increased Spatial Context for Outdoor Scene Understanding [2.685567215235468]
  We present our submission to the GOOSE 2D Fine-Grained Semantic Challenge, organized as part of the Workshop on Field Robotics at ICRA 2026.<n>The challenge combines data from the GOOSE and GOOSE-Ex datasets, annotated using a hierarchical taxonomy of 56 fine-grained classes and 11 broader categories.<n>Our final submission achieves an mIoU of 69.6% on the challenge test set, providing a strong baseline for fine-grained semantic segmentation in outdoor environments.
  arXiv  Detail & Related papers  (2026-06-19T14:13:04Z)
• AnchorMoE: Interpretable Time Series Classification via Anchor-Routed MoE [51.994752158953084]
  AnchorMoE is an interpretable-by-construction classification framework.<n>It encodes multi-view representations of local patches and routes them to specialized experts.<n>Experiments on real-world and synthetic benchmarks demonstrate that AnchorMoE achieves highly competitive classification performance.
  arXiv  Detail & Related papers  (2026-06-02T13:30:54Z)
• ModuSeg: Decoupling Object Discovery and Semantic Retrieval for Training-Free Weakly Supervised Segmentation [8.671473711194741]
  Weakly supervised semantic segmentation aims to achieve pixel-level predictions using image-level labels.<n>We present ModuSeg, a training-free weakly supervised semantic segmentation framework centered on explicitly decoupling object discovery and semantic assignment.
  arXiv  Detail & Related papers  (2026-04-08T12:38:07Z)
• BenchSeg: A Large-Scale Dataset and Benchmark for Multi-View Food Video Segmentation [25.750204283738054]
  We introduce BenchSeg, a novel multi-view food video segmentation dataset and benchmark.<n> BenchSeg aggregates 55 dish scenes with 25,284 meticulously annotated frames, capturing each dish under free 360 camera motion.<n>We evaluate a diverse set of 20 state-of-the-art segmentation models on the existing FoodSeg103 dataset and evaluate them on BenchSeg.
  arXiv  Detail & Related papers  (2026-01-12T14:32:51Z)
• LLMs-based Augmentation for Domain Adaptation in Long-tailed Food Datasets [54.527878056610156]
  We present a framework empowered with large language models (LLMs) to address these challenges in food recognition.<n>We first leverage LLMs to parse food images to generate food titles and ingredients.<n>Then, we project the generated texts and food images from different domains to a shared embedding space to maximize the pair similarities.
  arXiv  Detail & Related papers  (2025-11-20T04:38:56Z)
• OMH: Structured Sparsity via Optimally Matched Hierarchy for Unsupervised Semantic Segmentation [69.37484603556307]
  Un Semantic segmenting (USS) involves segmenting images without relying on predefined labels. We introduce a novel approach called Optimally Matched Hierarchy (OMH) to simultaneously address the above issues. Our OMH yields better unsupervised segmentation performance compared to existing USS methods.
  arXiv  Detail & Related papers  (2024-03-11T09:46:41Z)
• Hierarchical Audio-Visual Information Fusion with Multi-label Joint Decoding for MER 2023 [51.95161901441527]
  In this paper, we propose a novel framework for recognizing both discrete and dimensional emotions. Deep features extracted from foundation models are used as robust acoustic and visual representations of raw video. Our final system achieves state-of-the-art performance and ranks third on the leaderboard on MER-MULTI sub-challenge.
  arXiv  Detail & Related papers  (2023-09-11T03:19:10Z)
• MEDOE: A Multi-Expert Decoder and Output Ensemble Framework for Long-tailed Semantic Segmentation [36.03023287593103]
  Long-tailed distribution of semantic categories causes unsatisfactory performance in semantic segmentation on tail categories. We propose MEDOE, a novel framework for long-tailed semantic segmentation via contextual information ensemble-and-grouping. Experimental results show that the proposed framework outperforms the current methods on both Cityscapes and ADE20K datasets.
  arXiv  Detail & Related papers  (2023-08-16T08:30:44Z)
• Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation [94.16816278191477]
  We present a framework for semi-adaptive and domain-supervised semantic segmentation. It is enhanced by self-supervised monocular depth estimation trained only on unlabeled image sequences. We validate the proposed model on the Cityscapes dataset.
  arXiv  Detail & Related papers  (2021-08-28T01:33:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.