Dual-Imbalance Continual Learning for Real-World Food Recognition

• URL: http://arxiv.org/abs/2603.29133v1
• Date: Tue, 31 Mar 2026 01:32:51 GMT
• Title: Dual-Imbalance Continual Learning for Real-World Food Recognition
• Abstract summary: We introduce DIME, a Dual-Imbalance-aware Adapter Merging framework for continual food recognition.<n>Experiments on realistic long-tailed food benchmarks under our step-imbalanced setup show that the proposed method consistently improves by more than 3% over the strongest existing continual learning baselines.
• Score: 15.73921281192878
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual food recognition in real-world dietary logging scenarios naturally exhibits severe data imbalance, where a small number of food categories appear frequently while many others occur rarely, resulting in long-tailed class distributions. In practice, food recognition systems often operate in a continual learning setting, where new categories are introduced sequentially over time. However, existing studies typically assume that each incremental step introduces a similar number of new food classes, which rarely happens in real world where the number of newly observed categories can vary significantly across steps, leading to highly uneven learning dynamics. As a result, continual food recognition exhibits a dual imbalance: imbalanced samples within each food class and imbalanced numbers of new food classes to learn at each incremental learning step. In this work, we introduce DIME, a Dual-Imbalance-aware Adapter Merging framework for continual food recognition. DIME learns lightweight adapters for each task using parameter-efficient fine-tuning and progressively integrates them through a class-count guided spectral merging strategy. A rank-wise threshold modulation mechanism further stabilizes the merging process by preserving dominant knowledge while allowing adaptive updates. The resulting model maintains a single merged adapter for inference, enabling efficient deployment without accumulating task-specific modules. Experiments on realistic long-tailed food benchmarks under our step-imbalanced setup show that the proposed method consistently improves by more than 3% over the strongest existing continual learning baselines. Code is available at https://github.com/xiaoyanzhang1/DIME.

Related papers

• Continual Learning for Food Category Classification Dataset: Enhancing Model Adaptability and Performance [0.9480662172227129]
  We propose a continual learning framework for text-guided food classification.<n>Our method enables incremental updates, allowing new categories to be integrated without degrading prior knowledge.<n>This design shows promise for adaptive food recognition, with applications in dietary monitoring and personalized nutrition planning.
  arXiv  Detail & Related papers  (2026-03-20T04:03:53Z)
• Bagging-Based Model Merging for Robust General Text Embeddings [73.51674133699196]
  General-purpose text embedding models underpin a wide range of NLP and information retrieval applications.<n>We present a systematic study of multi-task training for text embeddings from two perspectives: data scheduling and model merging.<n>We propose Bagging-based rObust mOdel Merging (BOOM), which trains multiple embedding models on sampled subsets and merges them into a single model.
  arXiv  Detail & Related papers  (2026-02-05T15:45:08Z)
• LLMs-based Augmentation for Domain Adaptation in Long-tailed Food Datasets [54.527878056610156]
  We present a framework empowered with large language models (LLMs) to address these challenges in food recognition.<n>We first leverage LLMs to parse food images to generate food titles and ingredients.<n>Then, we project the generated texts and food images from different domains to a shared embedding space to maximize the pair similarities.
  arXiv  Detail & Related papers  (2025-11-20T04:38:56Z)
• DRL: Discriminative Representation Learning with Parallel Adapters for Class Incremental Learning [63.65467569295623]
  We propose the Discriminative Representation Learning (DRL) framework to specifically address these challenges.<n>To conduct incremental learning effectively and yet efficiently, the DRL's network is built upon a PTM.<n>Our DRL consistently outperforms other state-of-the-art methods throughout the entire CIL period.
  arXiv  Detail & Related papers  (2025-10-14T03:19:15Z)
• Classifier-guided Gradient Modulation for Enhanced Multimodal Learning [50.7008456698935]
  Gradient-Guided Modulation (CGGM) is a novel method to balance multimodal learning with gradients. We conduct extensive experiments on four multimodal datasets: UPMC-Food 101, CMU-MOSI, IEMOCAP and BraTS. CGGM outperforms all the baselines and other state-of-the-art methods consistently.
  arXiv  Detail & Related papers  (2024-11-03T02:38:43Z)
• Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach [102.0769560460338]
  We develop a simple logits approach (LORT) without the requirement of prior knowledge of the number of samples per class. Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
  arXiv  Detail & Related papers  (2024-03-01T03:27:08Z)
• Food Image Classification and Segmentation with Attention-based Multiple Instance Learning [51.279800092581844]
  The paper presents a weakly supervised methodology for training food image classification and semantic segmentation models. The proposed methodology is based on a multiple instance learning approach in combination with an attention-based mechanism. We conduct experiments on two meta-classes within the FoodSeg103 data set to verify the feasibility of the proposed approach.
  arXiv  Detail & Related papers  (2023-08-22T13:59:47Z)
• Long-Tailed Continual Learning For Visual Food Recognition [8.246111678466717]
  We introduce a dataset that encompasses 186 American foods along with comprehensive annotations.<n>We propose a novel end-to-end framework that improves the generalization ability for instance-rare food classes.<n>Our method, evaluated on Food101-LT, VFN-LT, VFN186-LT, VFN186-INSULIN, and VFN186-T2DM, shows significant improvements over existing methods.
  arXiv  Detail & Related papers  (2023-07-01T00:55:05Z)
• Online Class-Incremental Learning For Real-World Food Image Classification [8.438092346233054]
  Real-world food consumption patterns, shaped by cultural, economic, and personal influences, involve dynamic and evolving data. Online Class Incremental Learning (OCIL) addresses the challenge of learning continuously from a single-pass data stream. We present an attachable Dynamic Model Update (DMU) module designed for existing ER methods, which enables the selection of relevant images for model training.
  arXiv  Detail & Related papers  (2023-01-12T19:00:27Z)
• Long-tailed Food Classification [5.874935571318868]
  We introduce two new benchmark datasets for long-tailed food classification including Food101-LT and VFN-LT. We propose a novel 2-Phase framework to address the problem of class-imbalance by (1) under the head classes to remove redundant samples along with maintaining the learned information through knowledge distillation. We show the effectiveness of our method by comparing with existing state-of-the-art long-tailed classification methods and show improved performance on both Food101-LT and VFN-LT benchmarks.
  arXiv  Detail & Related papers  (2022-10-26T14:29:30Z)
• Online Continual Learning For Visual Food Classification [7.704949298975352]
  Existing methods require static datasets for training and are not capable of learning from sequentially available new food images. We introduce a novel clustering based exemplar selection algorithm to store the most representative data belonging to each learned food. Our results show significant improvements compared with existing state-of-the-art online continual learning methods.
  arXiv  Detail & Related papers  (2021-08-15T17:48:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.