Personalized Class Incremental Context-Aware Food Classification for Food Intake Monitoring Systems

• URL: http://arxiv.org/abs/2503.06647v1
• Date: Sun, 09 Mar 2025 14:50:56 GMT
• Title: Personalized Class Incremental Context-Aware Food Classification for Food Intake Monitoring Systems
• Authors: Hassan Kazemi Tehrani, Jun Cai, Abbas Yekanlou, Sylvia Santosa,
• Abstract summary: Existing class-incremental food classification models have low accuracy for the new classes and lack personalization.<n>This paper introduces a personalized, class-incremental food classification model designed to overcome these challenges.<n>Our approach adapts itself to the new array of food classes, maintaining applicability and accuracy, both for new and existing classes by using personalization.
• Score: 3.8767314375943918
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate food intake monitoring is crucial for maintaining a healthy diet and preventing nutrition-related diseases. With the diverse range of foods consumed across various cultures, classic food classification models have limitations due to their reliance on fixed-sized food datasets. Studies show that people consume only a small range of foods across the existing ones, each consuming a unique set of foods. Existing class-incremental models have low accuracy for the new classes and lack personalization. This paper introduces a personalized, class-incremental food classification model designed to overcome these challenges and improve the performance of food intake monitoring systems. Our approach adapts itself to the new array of food classes, maintaining applicability and accuracy, both for new and existing classes by using personalization. Our model's primary focus is personalization, which improves classification accuracy by prioritizing a subset of foods based on an individual's eating habits, including meal frequency, times, and locations. A modified version of DSN is utilized to expand on the appearance of new food classes. Additionally, we propose a comprehensive framework that integrates this model into a food intake monitoring system. This system analyzes meal images provided by users, makes use of a smart scale to estimate food weight, utilizes a nutrient content database to calculate the amount of each macro-nutrient, and creates a dietary user profile through a mobile application. Finally, experimental evaluations on two new benchmark datasets FOOD101-Personal and VFN-Personal, personalized versions of well-known datasets for food classification, are conducted to demonstrate the effectiveness of our model in improving the classification accuracy of both new and existing classes, addressing the limitations of both conventional and class-incremental food classification models.

Related papers

• NutritionVerse-Direct: Exploring Deep Neural Networks for Multitask Nutrition Prediction from Food Images [63.314702537010355]
  Self-reporting methods are often inaccurate and suffer from substantial bias. Recent work has explored using computer vision prediction systems to predict nutritional information from food images. This paper aims to enhance the efficacy of dietary intake estimation by leveraging various neural network architectures.
  arXiv  Detail & Related papers  (2024-05-13T14:56:55Z)
• NutritionVerse-Real: An Open Access Manually Collected 2D Food Scene Dataset for Dietary Intake Estimation [68.49526750115429]
  We introduce NutritionVerse-Real, an open access manually collected 2D food scene dataset for dietary intake estimation. The NutritionVerse-Real dataset was created by manually collecting images of food scenes in real life, measuring the weight of every ingredient and computing the associated dietary content of each dish.
  arXiv  Detail & Related papers  (2023-11-20T11:05:20Z)
• Personalized Food Image Classification: Benchmark Datasets and New Baseline [8.019925729254178]
  We propose a new framework for personalized food image classification by leveraging self-supervised learning and temporal image feature information. Our method is evaluated on both benchmark datasets and shows improved performance compared to existing works.
  arXiv  Detail & Related papers  (2023-09-15T20:11:07Z)
• NutritionVerse: Empirical Study of Various Dietary Intake Estimation Approaches [59.38343165508926]
  Accurate dietary intake estimation is critical for informing policies and programs to support healthy eating. Recent work has focused on using computer vision and machine learning to automatically estimate dietary intake from food images. We introduce NutritionVerse- Synth, the first large-scale dataset of 84,984 synthetic 2D food images with associated dietary information. We also collect a real image dataset, NutritionVerse-Real, containing 889 images of 251 dishes to evaluate realism.
  arXiv  Detail & Related papers  (2023-09-14T13:29:41Z)
• Food Image Classification and Segmentation with Attention-based Multiple Instance Learning [51.279800092581844]
  The paper presents a weakly supervised methodology for training food image classification and semantic segmentation models. The proposed methodology is based on a multiple instance learning approach in combination with an attention-based mechanism. We conduct experiments on two meta-classes within the FoodSeg103 data set to verify the feasibility of the proposed approach.
  arXiv  Detail & Related papers  (2023-08-22T13:59:47Z)
• Long-Tailed Continual Learning For Visual Food Recognition [5.377869029561348]
  The distribution of food images in real life is usually long-tailed as a small number of popular food types are consumed more frequently than others. We propose a novel end-to-end framework for long-tailed continual learning, which effectively addresses the catastrophic forgetting. We also introduce a novel data augmentation technique by integrating class-activation-map (CAM) and CutMix.
  arXiv  Detail & Related papers  (2023-07-01T00:55:05Z)
• UMDFood: Vision-language models boost food composition compilation [26.5694236976957]
  We propose a novel vision-language model, UMDFood-VL, using front-of-package labeling and product images to accurately estimate food composition profiles. Up to 82.2% of selected products' estimated error between chemical analysis results and model estimation results are less than 10%. This performance sheds light on generalization towards other food and nutrition-related data compilation and catalyzation.
  arXiv  Detail & Related papers  (2023-05-18T03:18:12Z)
• Towards the Creation of a Nutrition and Food Group Based Image Database [58.429385707376554]
  We propose a framework to create a nutrition and food group based image database. We design a protocol for linking food group based food codes in the U.S. Department of Agriculture's (USDA) Food and Nutrient Database for Dietary Studies (FNDDS) Our proposed method is used to build a nutrition and food group based image database including 16,114 food datasets.
  arXiv  Detail & Related papers  (2022-06-05T02:41:44Z)
• A Mobile Food Recognition System for Dietary Assessment [6.982738885923204]
  We focus on developing a mobile friendly, Middle Eastern cuisine focused food recognition application for assisted living purposes. Using Mobilenet-v2 architecture for this task is beneficial in terms of both accuracy and the memory usage. The developed mobile application has potential to serve the visually impaired in automatic food recognition via images.
  arXiv  Detail & Related papers  (2022-04-20T12:49:36Z)
• Improving Dietary Assessment Via Integrated Hierarchy Food Classification [7.398060062678395]
  We introduce a new food classification framework to improve the quality of predictions by integrating the information from multiple domains. Our method is validated on the modified VIPER-FoodNet (VFN) food image dataset by including associated energy and nutrient information.
  arXiv  Detail & Related papers  (2021-09-06T20:59:58Z)
• Towards Building a Food Knowledge Graph for Internet of Food [66.57235827087092]
  We review the evolution of food knowledge organization, from food classification to food to food knowledge graphs. Food knowledge graphs play an important role in food search and Question Answering (QA), personalized dietary recommendation, food analysis and visualization. Future directions for food knowledge graphs cover several fields such as multimodal food knowledge graphs and food intelligence.
  arXiv  Detail & Related papers  (2021-07-13T06:26:53Z)
• MyFood: A Food Segmentation and Classification System to Aid Nutritional Monitoring [1.5469452301122173]
  The absence of food monitoring has contributed significantly to the increase in the population's weight. Some solutions have been proposed in computer vision to recognize food images, but few are specialized in nutritional monitoring. This work presents the development of an intelligent system that classifies and segments food presented in images to help the automatic monitoring of user diet and nutritional intake.
  arXiv  Detail & Related papers  (2020-12-05T17:40:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.