Long-tailed Food Classification
- URL: http://arxiv.org/abs/2210.14748v1
- Date: Wed, 26 Oct 2022 14:29:30 GMT
- Title: Long-tailed Food Classification
- Authors: Jiangpeng He, Luotao Lin, Heather Eicher-Miller, Fengqing Zhu
- Abstract summary: We introduce two new benchmark datasets for long-tailed food classification including Food101-LT and VFN-LT.
We propose a novel 2-Phase framework to address the problem of class-imbalance by (1) under the head classes to remove redundant samples along with maintaining the learned information through knowledge distillation.
We show the effectiveness of our method by comparing with existing state-of-the-art long-tailed classification methods and show improved performance on both Food101-LT and VFN-LT benchmarks.
- Score: 5.874935571318868
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Food classification serves as the basic step of image-based dietary
assessment to predict the types of foods in each input image. However, food
image predictions in a real world scenario are usually long-tail distributed
among different food classes, which cause heavy class-imbalance problems and a
restricted performance. In addition, none of the existing long-tailed
classification methods focus on food data, which can be more challenging due to
the lower inter-class and higher intra-class similarity among foods. In this
work, we first introduce two new benchmark datasets for long-tailed food
classification including Food101-LT and VFN-LT where the number of samples in
VFN-LT exhibits the real world long-tailed food distribution. Then we propose a
novel 2-Phase framework to address the problem of class-imbalance by (1)
undersampling the head classes to remove redundant samples along with
maintaining the learned information through knowledge distillation, and (2)
oversampling the tail classes by performing visual-aware data augmentation. We
show the effectiveness of our method by comparing with existing
state-of-the-art long-tailed classification methods and show improved
performance on both Food101-LT and VFN-LT benchmarks. The results demonstrate
the potential to apply our method to related real life applications.
Related papers
- From Canteen Food to Daily Meals: Generalizing Food Recognition to More
Practical Scenarios [92.58097090916166]
We present two new benchmarks, namely DailyFood-172 and DailyFood-16, designed to curate food images from everyday meals.
These two datasets are used to evaluate the transferability of approaches from the well-curated food image domain to the everyday-life food image domain.
arXiv Detail & Related papers (2024-03-12T08:32:23Z) - Food Image Classification and Segmentation with Attention-based Multiple
Instance Learning [51.279800092581844]
The paper presents a weakly supervised methodology for training food image classification and semantic segmentation models.
The proposed methodology is based on a multiple instance learning approach in combination with an attention-based mechanism.
We conduct experiments on two meta-classes within the FoodSeg103 data set to verify the feasibility of the proposed approach.
arXiv Detail & Related papers (2023-08-22T13:59:47Z) - Long-Tailed Continual Learning For Visual Food Recognition [5.377869029561348]
The distribution of food images in real life is usually long-tailed as a small number of popular food types are consumed more frequently than others.
We propose a novel end-to-end framework for long-tailed continual learning, which effectively addresses the catastrophic forgetting.
We also introduce a novel data augmentation technique by integrating class-activation-map (CAM) and CutMix.
arXiv Detail & Related papers (2023-07-01T00:55:05Z) - Single-Stage Heavy-Tailed Food Classification [7.800379384628357]
We introduce a novel single-stage heavy-tailed food classification framework.
Our method is evaluated on two heavy-tailed food benchmark datasets, Food101-LT and VFN-LT.
arXiv Detail & Related papers (2023-07-01T00:45:35Z) - Transferring Knowledge for Food Image Segmentation using Transformers
and Convolutions [65.50975507723827]
Food image segmentation is an important task that has ubiquitous applications, such as estimating the nutritional value of a plate of food.
One challenge is that food items can overlap and mix, making them difficult to distinguish.
Two models are trained and compared, one based on convolutional neural networks and the other on Bidirectional representation for Image Transformers (BEiT)
The BEiT model outperforms the previous state-of-the-art model by achieving a mean intersection over union of 49.4 on FoodSeg103.
arXiv Detail & Related papers (2023-06-15T15:38:10Z) - Online Class-Incremental Learning For Real-World Food Image
Classification [8.438092346233054]
Real-world food consumption patterns, shaped by cultural, economic, and personal influences, involve dynamic and evolving data.
Online Class Incremental Learning (OCIL) addresses the challenge of learning continuously from a single-pass data stream.
We present an attachable Dynamic Model Update (DMU) module designed for existing ER methods, which enables the selection of relevant images for model training.
arXiv Detail & Related papers (2023-01-12T19:00:27Z) - Online Continual Learning For Visual Food Classification [7.704949298975352]
Existing methods require static datasets for training and are not capable of learning from sequentially available new food images.
We introduce a novel clustering based exemplar selection algorithm to store the most representative data belonging to each learned food.
Our results show significant improvements compared with existing state-of-the-art online continual learning methods.
arXiv Detail & Related papers (2021-08-15T17:48:03Z) - Class-Balanced Distillation for Long-Tailed Visual Recognition [100.10293372607222]
Real-world imagery is often characterized by a significant imbalance of the number of images per class, leading to long-tailed distributions.
In this work, we introduce a new framework, by making the key observation that a feature representation learned with instance sampling is far from optimal in a long-tailed setting.
Our main contribution is a new training method, that leverages knowledge distillation to enhance feature representations.
arXiv Detail & Related papers (2021-04-12T08:21:03Z) - ISIA Food-500: A Dataset for Large-Scale Food Recognition via Stacked
Global-Local Attention Network [50.7720194859196]
We introduce the dataset ISIA Food- 500 with 500 categories from the list in the Wikipedia and 399,726 images.
This dataset surpasses existing popular benchmark datasets by category coverage and data volume.
We propose a stacked global-local attention network, which consists of two sub-networks for food recognition.
arXiv Detail & Related papers (2020-08-13T02:48:27Z) - Multi-Task Image-Based Dietary Assessment for Food Recognition and
Portion Size Estimation [6.603050343996914]
We propose an end-to-end multi-task framework that can achieve both food classification and food portion size estimation.
Our results outperforms the baseline methods for both classification accuracy and mean absolute error for portion estimation.
arXiv Detail & Related papers (2020-04-27T21:35:07Z) - Cross-Modal Food Retrieval: Learning a Joint Embedding of Food Images
and Recipes with Semantic Consistency and Attention Mechanism [70.85894675131624]
We learn an embedding of images and recipes in a common feature space, such that the corresponding image-recipe embeddings lie close to one another.
We propose Semantic-Consistent and Attention-based Networks (SCAN), which regularize the embeddings of the two modalities through aligning output semantic probabilities.
We show that we can outperform several state-of-the-art cross-modal retrieval strategies for food images and cooking recipes by a significant margin.
arXiv Detail & Related papers (2020-03-09T07:41:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.