Video Domain Incremental Learning for Human Action Recognition in Home Environments

• URL: http://arxiv.org/abs/2412.16946v1
• Date: Sun, 22 Dec 2024 09:40:48 GMT
• Title: Video Domain Incremental Learning for Human Action Recognition in Home Environments
• Authors: Yuanda Hu, Xing Liu, Meiying Li, Yate Ge, Xiaohua Sun, Weiwei Guo,
• Abstract summary: We formalize the problem of Video Domain Incremental Learning (VDIL) VDIL enables models to learn continually from different domains while maintaining a fixed set of action classes. In this work, we introduce a novel benchmark of domain incremental human action recognition for unconstrained home environments.
• Score: 5.8984266500487
• License:
• Abstract: It is significantly challenging to recognize daily human actions in homes due to the diversity and dynamic changes in unconstrained home environments. It spurs the need to continually adapt to various users and scenes. Fine-tuning current video understanding models on newly encountered domains often leads to catastrophic forgetting, where the models lose their ability to perform well on previously learned scenarios. To address this issue, we formalize the problem of Video Domain Incremental Learning (VDIL), which enables models to learn continually from different domains while maintaining a fixed set of action classes. Existing continual learning research primarily focuses on class-incremental learning, while the domain incremental learning has been largely overlooked in video understanding. In this work, we introduce a novel benchmark of domain incremental human action recognition for unconstrained home environments. We design three domain split types (user, scene, hybrid) to systematically assess the challenges posed by domain shifts in real-world home settings. Furthermore, we propose a baseline learning strategy based on replay and reservoir sampling techniques without domain labels to handle scenarios with limited memory and task agnosticism. Extensive experimental results demonstrate that our simple sampling and replay strategy outperforms most existing continual learning methods across the three proposed benchmarks.

Related papers

• Feature Based Methods in Domain Adaptation for Object Detection: A Review Paper [0.6437284704257459]
  Domain adaptation aims to enhance the performance of machine learning models when deployed in target domains with distinct data distributions. This review delves into advanced methodologies for domain adaptation, including adversarial learning, discrepancy-based, multi-domain, teacher-student, ensemble, and Vision Language Models. Special attention is given to strategies that minimize the reliance on extensive labeled data, particularly in scenarios involving synthetic-to-real domain shifts.
  arXiv  Detail & Related papers  (2024-12-23T06:34:23Z)
• Evaluating the Effectiveness of Video Anomaly Detection in the Wild: Online Learning and Inference for Real-world Deployment [2.1374208474242815]
  Video Anomaly Detection (VAD) identifies unusual activities in video streams, a key technology with broad applications ranging from surveillance to healthcare. Tackling VAD in real-life settings poses significant challenges due to the dynamic nature of human actions, environmental variations, and domain shifts. Online learning is a potential strategy to mitigate this issue by allowing models to adapt to new information continuously.
  arXiv  Detail & Related papers  (2024-04-29T14:47:32Z)
• Normalization Perturbation: A Simple Domain Generalization Method for Real-World Domain Shifts [133.99270341855728]
  Real-world domain styles can vary substantially due to environment changes and sensor noises. Deep models only know the training domain style. We propose Normalization Perturbation to overcome this domain style overfitting problem.
  arXiv  Detail & Related papers  (2022-11-08T17:36:49Z)
• Forget Less, Count Better: A Domain-Incremental Self-Distillation Learning Benchmark for Lifelong Crowd Counting [51.44987756859706]
  Off-the-shelf methods have some drawbacks to handle multiple domains. Lifelong Crowd Counting aims at alleviating the catastrophic forgetting and improving the generalization ability.
  arXiv  Detail & Related papers  (2022-05-06T15:37:56Z)
• On Generalizing Beyond Domains in Cross-Domain Continual Learning [91.56748415975683]
  Deep neural networks often suffer from catastrophic forgetting of previously learned knowledge after learning a new task. Our proposed approach learns new tasks under domain shift with accuracy boosts up to 10% on challenging datasets such as DomainNet and OfficeHome.
  arXiv  Detail & Related papers  (2022-03-08T09:57:48Z)
• Cross-domain Imitation from Observations [50.669343548588294]
  Imitation learning seeks to circumvent the difficulty in designing proper reward functions for training agents by utilizing expert behavior. In this paper, we study the problem of how to imitate tasks when there exist discrepancies between the expert and agent MDP. We present a novel framework to learn correspondences across such domains.
  arXiv  Detail & Related papers  (2021-05-20T21:08:25Z)
• Lifelong Person Re-Identification via Adaptive Knowledge Accumulation [18.4671957106297]
  Lifelong person re-identification (LReID) enables to learn continuously across multiple domains. We design an Adaptive Knowledge Accumulation framework that is endowed with two crucial abilities: knowledge representation and knowledge operation. Our method alleviates catastrophic forgetting on seen domains and demonstrates the ability to generalize to unseen domains.
  arXiv  Detail & Related papers  (2021-03-23T11:30:38Z)
• Gradient Regularized Contrastive Learning for Continual Domain Adaptation [86.02012896014095]
  We study the problem of continual domain adaptation, where the model is presented with a labelled source domain and a sequence of unlabelled target domains. We propose Gradient Regularized Contrastive Learning (GRCL) to solve the obstacles. Experiments on Digits, DomainNet and Office-Caltech benchmarks demonstrate the strong performance of our approach.
  arXiv  Detail & Related papers  (2021-03-23T04:10:42Z)
• A Review of Single-Source Deep Unsupervised Visual Domain Adaptation [81.07994783143533]
  Large-scale labeled training datasets have enabled deep neural networks to excel across a wide range of benchmark vision tasks. In many applications, it is prohibitively expensive and time-consuming to obtain large quantities of labeled data. To cope with limited labeled training data, many have attempted to directly apply models trained on a large-scale labeled source domain to another sparsely labeled or unlabeled target domain.
  arXiv  Detail & Related papers  (2020-09-01T00:06:50Z)
• ADAIL: Adaptive Adversarial Imitation Learning [11.270858993502705]
  We present the ADaptive Adversarial Imitation Learning (ADAIL) algorithm for learning adaptive policies that can be transferred between environments of varying dynamics. This is an important problem in robotic learning because in real world scenarios 1) reward functions are hard to obtain, 2) learned policies from one domain are difficult to deploy in another due to varying source to target domain statistics, and 3) collecting expert demonstrations in multiple environments where the dynamics are known and controlled is often infeasible.
  arXiv  Detail & Related papers  (2020-08-23T06:11:00Z)
• Latent Domain Learning with Dynamic Residual Adapters [26.018759356470767]
  A practical shortcoming of deep neural networks is their specialization to a single task and domain. Here we focus on a less explored, but more realistic case: learning from data from multiple domains, without access to domain annotations. We address this limitation via dynamic residual adapters, an adaptive gating mechanism that helps account for latent domains.
  arXiv  Detail & Related papers  (2020-06-01T15:00:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.