Using Language Model to Bootstrap Human Activity Recognition Ambient Sensors Based in Smart Homes

• URL: http://arxiv.org/abs/2111.12158v1
• Date: Tue, 23 Nov 2021 21:21:14 GMT
• Title: Using Language Model to Bootstrap Human Activity Recognition Ambient Sensors Based in Smart Homes
• Authors: Damien Bouchabou, Sao Mai Nguyen, Christophe Lohr, Benoit Leduc, Ioannis Kanellos
• Abstract summary: We propose two Natural Language Processing embedding methods to enhance LSTM-based structures in activity-sequences classification tasks. Results indicate that this approach provides useful information, such as a sensor organization map. Our tests show that the embeddings can be pretrained on different datasets than the target one, enabling transfer learning.
• Score: 2.336163487623381
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long Short Term Memory LSTM-based structures have demonstrated their efficiency for daily living recognition activities in smart homes by capturing the order of sensor activations and their temporal dependencies. Nevertheless, they still fail in dealing with the semantics and the context of the sensors. More than isolated id and their ordered activation values, sensors also carry meaning. Indeed, their nature and type of activation can translate various activities. Their logs are correlated with each other, creating a global context. We propose to use and compare two Natural Language Processing embedding methods to enhance LSTM-based structures in activity-sequences classification tasks: Word2Vec, a static semantic embedding, and ELMo, a contextualized embedding. Results, on real smart homes datasets, indicate that this approach provides useful information, such as a sensor organization map, and makes less confusion between daily activity classes. It helps to better perform on datasets with competing activities of other residents or pets. Our tests show also that the embeddings can be pretrained on different datasets than the target one, enabling transfer learning. We thus demonstrate that taking into account the context of the sensors and their semantics increases the classification performances and enables transfer learning.

Related papers

• Language-centered Human Activity Recognition [8.925867647929088]
  Human Activity Recognition (HAR) using Inertial Measurement Unit (IMU) sensors is critical for applications in healthcare, safety, and industrial production. variation in activity patterns, device types, and sensor placements create distribution gaps across datasets. We propose LanHAR, a novel system that generates semantic interpretations of sensor readings and activity labels for cross-dataset HAR.
  arXiv  Detail & Related papers  (2024-09-12T22:57:29Z)
• Consistency Based Weakly Self-Supervised Learning for Human Activity Recognition with Wearables [1.565361244756411]
  We describe a weakly self-supervised approach for recognizing human activities from sensor-based data. We show that our approach can help the clustering algorithm achieve comparable performance in identifying and categorizing the underlying human activities.
  arXiv  Detail & Related papers  (2024-07-29T06:29:21Z)
• Layout Agnostic Human Activity Recognition in Smart Homes through Textual Descriptions Of Sensor Triggers (TDOST) [0.22354214294493352]
  We develop a layout-agnostic modeling approach for human activity recognition (HAR) systems in smart homes. We generate Textual Descriptions Of Sensor Triggers (TDOST) that encapsulate the surrounding trigger conditions. We demonstrate the effectiveness of TDOST-based models in unseen smart homes through experiments on benchmarked CASAS datasets.
  arXiv  Detail & Related papers  (2024-05-20T20:37:44Z)
• Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge [62.981429762309226]
  The ability to actively ground task instructions from an egocentric view is crucial for AI agents to accomplish tasks or assist humans virtually. We propose to improve phrase grounding models' ability on localizing the active objects by: learning the role of objects undergoing change and extracting them accurately from the instructions. We evaluate our framework on Ego4D and Epic-Kitchens datasets.
  arXiv  Detail & Related papers  (2023-10-23T16:14:05Z)
• Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task [50.72283841720014]
  We propose a novel learning strategy that can improve reasoning about the effects of actions. We demonstrate the effectiveness of our proposed approach and discuss its advantages over previous baselines in terms of performance, data efficiency, and generalization capability.
  arXiv  Detail & Related papers  (2022-12-07T05:41:58Z)
• CHARM: A Hierarchical Deep Learning Model for Classification of Complex Human Activities Using Motion Sensors [0.9594432031144714]
  CHARM is a hierarchical deep learning model for classification of complex human activities using motion sensors. It outperforms state-of-the-art supervised learning approaches for high-level activity recognition in terms of average accuracy and F1 scores. The ability to learn low-level user activities when trained using only high-level activity labels may pave the way to semi-supervised learning of HAR tasks.
  arXiv  Detail & Related papers  (2022-07-16T01:36:54Z)
• ALLSH: Active Learning Guided by Local Sensitivity and Hardness [98.61023158378407]
  We propose to retrieve unlabeled samples with a local sensitivity and hardness-aware acquisition function. Our method achieves consistent gains over the commonly used active learning strategies in various classification tasks.
  arXiv  Detail & Related papers  (2022-05-10T15:39:11Z)
• Human Activity Recognition using Attribute-Based Neural Networks and Context Information [61.67246055629366]
  We consider human activity recognition (HAR) from wearable sensor data in manual-work processes. We show how context information can be integrated systematically into a deep neural network-based HAR system. We empirically show that our proposed architecture increases HAR performance, compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-28T06:08:25Z)
• Learning Asynchronous and Sparse Human-Object Interaction in Videos [56.73059840294019]
  Asynchronous-Sparse Interaction Graph Networks (ASSIGN) is able to automatically detect the structure of interaction events associated with entities in a video scene. ASSIGN is tested on human-object interaction recognition and shows superior performance in segmenting and labeling of human sub-activities and object affordances from raw videos.
  arXiv  Detail & Related papers  (2021-03-03T23:43:55Z)
• Learning Invariant Representations for Reinforcement Learning without Reconstruction [98.33235415273562]
  We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction. Bisimulation metrics quantify behavioral similarity between states in continuous MDPs. We demonstrate the effectiveness of our method at disregarding task-irrelevant information using modified visual MuJoCo tasks.
  arXiv  Detail & Related papers  (2020-06-18T17:59:35Z)
• ActiLabel: A Combinatorial Transfer Learning Framework for Activity Recognition [14.605223647792862]
  ActiLabel is a framework that learns structural similarities among events in an arbitrary domain and those of a different domain. Experiments based on three public datasets demonstrate the superiority of ActiLabel over state-of-the-art transfer learning and deep learning methods.
  arXiv  Detail & Related papers  (2020-03-16T19:19:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.