Long-Tailed Continual Learning For Visual Food Recognition

• URL: http://arxiv.org/abs/2307.00183v1
• Date: Sat, 1 Jul 2023 00:55:05 GMT
• Title: Long-Tailed Continual Learning For Visual Food Recognition
• Authors: Jiangpeng He and Luotao Lin and Jack Ma and Heather A. Eicher-Miller and Fengqing Zhu
• Abstract summary: The distribution of food images in real life is usually long-tailed as a small number of popular food types are consumed more frequently than others. We propose a novel end-to-end framework for long-tailed continual learning, which effectively addresses the catastrophic forgetting. We also introduce a novel data augmentation technique by integrating class-activation-map (CAM) and CutMix.
• Score: 5.377869029561348
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning based food recognition has achieved remarkable progress in predicting food types given an eating occasion image. However, there are two major obstacles that hinder deployment in real world scenario. First, as new foods appear sequentially overtime, a trained model needs to learn the new classes continuously without causing catastrophic forgetting for already learned knowledge of existing food types. Second, the distribution of food images in real life is usually long-tailed as a small number of popular food types are consumed more frequently than others, which can vary in different populations. This requires the food recognition method to learn from class-imbalanced data by improving the generalization ability on instance-rare food classes. In this work, we focus on long-tailed continual learning and aim to address both aforementioned challenges. As existing long-tailed food image datasets only consider healthy people population, we introduce two new benchmark food image datasets, VFN-INSULIN and VFN-T2D, which exhibits on the real world food consumption for insulin takers and individuals with type 2 diabetes without taking insulin, respectively. We propose a novel end-to-end framework for long-tailed continual learning, which effectively addresses the catastrophic forgetting by applying an additional predictor for knowledge distillation to avoid misalignment of representation during continual learning. We also introduce a novel data augmentation technique by integrating class-activation-map (CAM) and CutMix, which significantly improves the generalization ability for instance-rare food classes to address the class-imbalance issue. The proposed method show promising performance with large margin improvements compared with existing methods.

Related papers

• NutritionVerse-Direct: Exploring Deep Neural Networks for Multitask Nutrition Prediction from Food Images [63.314702537010355]
  Self-reporting methods are often inaccurate and suffer from substantial bias. Recent work has explored using computer vision prediction systems to predict nutritional information from food images. This paper aims to enhance the efficacy of dietary intake estimation by leveraging various neural network architectures.
  arXiv  Detail & Related papers  (2024-05-13T14:56:55Z)
• Learning to Classify New Foods Incrementally Via Compressed Exemplars [8.277136664415513]
  Food image classification systems play a crucial role in health monitoring and diet tracking through image-based dietary assessment techniques. Existing food recognition systems rely on static datasets characterized by a pre-defined fixed number of food classes. We introduce the concept of continuously learning a neural compression model to adaptively improve the quality of compressed data.
  arXiv  Detail & Related papers  (2024-04-11T06:55:44Z)
• From Canteen Food to Daily Meals: Generalizing Food Recognition to More Practical Scenarios [92.58097090916166]
  We present two new benchmarks, namely DailyFood-172 and DailyFood-16, designed to curate food images from everyday meals. These two datasets are used to evaluate the transferability of approaches from the well-curated food image domain to the everyday-life food image domain.
  arXiv  Detail & Related papers  (2024-03-12T08:32:23Z)
• NutritionVerse: Empirical Study of Various Dietary Intake Estimation Approaches [59.38343165508926]
  Accurate dietary intake estimation is critical for informing policies and programs to support healthy eating. Recent work has focused on using computer vision and machine learning to automatically estimate dietary intake from food images. We introduce NutritionVerse- Synth, the first large-scale dataset of 84,984 synthetic 2D food images with associated dietary information. We also collect a real image dataset, NutritionVerse-Real, containing 889 images of 251 dishes to evaluate realism.
  arXiv  Detail & Related papers  (2023-09-14T13:29:41Z)
• Food Image Classification and Segmentation with Attention-based Multiple Instance Learning [51.279800092581844]
  The paper presents a weakly supervised methodology for training food image classification and semantic segmentation models. The proposed methodology is based on a multiple instance learning approach in combination with an attention-based mechanism. We conduct experiments on two meta-classes within the FoodSeg103 data set to verify the feasibility of the proposed approach.
  arXiv  Detail & Related papers  (2023-08-22T13:59:47Z)
• Single-Stage Heavy-Tailed Food Classification [7.800379384628357]
  We introduce a novel single-stage heavy-tailed food classification framework. Our method is evaluated on two heavy-tailed food benchmark datasets, Food101-LT and VFN-LT.
  arXiv  Detail & Related papers  (2023-07-01T00:45:35Z)
• Transferring Knowledge for Food Image Segmentation using Transformers and Convolutions [65.50975507723827]
  Food image segmentation is an important task that has ubiquitous applications, such as estimating the nutritional value of a plate of food. One challenge is that food items can overlap and mix, making them difficult to distinguish. Two models are trained and compared, one based on convolutional neural networks and the other on Bidirectional representation for Image Transformers (BEiT) The BEiT model outperforms the previous state-of-the-art model by achieving a mean intersection over union of 49.4 on FoodSeg103.
  arXiv  Detail & Related papers  (2023-06-15T15:38:10Z)
• Long-tailed Food Classification [5.874935571318868]
  We introduce two new benchmark datasets for long-tailed food classification including Food101-LT and VFN-LT. We propose a novel 2-Phase framework to address the problem of class-imbalance by (1) under the head classes to remove redundant samples along with maintaining the learned information through knowledge distillation. We show the effectiveness of our method by comparing with existing state-of-the-art long-tailed classification methods and show improved performance on both Food101-LT and VFN-LT benchmarks.
  arXiv  Detail & Related papers  (2022-10-26T14:29:30Z)
• Online Continual Learning For Visual Food Classification [7.704949298975352]
  Existing methods require static datasets for training and are not capable of learning from sequentially available new food images. We introduce a novel clustering based exemplar selection algorithm to store the most representative data belonging to each learned food. Our results show significant improvements compared with existing state-of-the-art online continual learning methods.
  arXiv  Detail & Related papers  (2021-08-15T17:48:03Z)
• Vision-Based Food Analysis for Automatic Dietary Assessment [49.32348549508578]
  This review presents one unified Vision-Based Dietary Assessment (VBDA) framework, which generally consists of three stages: food image analysis, volume estimation and nutrient derivation. Deep learning makes VBDA gradually move to an end-to-end implementation, which applies food images to a single network to directly estimate the nutrition.
  arXiv  Detail & Related papers  (2021-08-06T05:46:01Z)
• Large Scale Visual Food Recognition [43.43598316339732]
  We introduce Food2K, which is the largest food recognition dataset with 2,000 categories and over 1 million images. Food2K bypasses them in both categories and images by one order of magnitude. We propose a deep progressive region enhancement network for food recognition.
  arXiv  Detail & Related papers  (2021-03-30T06:41:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.