Unimodal and Multimodal Sensor Fusion for Wearable Activity Recognition

• URL: http://arxiv.org/abs/2404.16005v1
• Date: Wed, 24 Apr 2024 17:35:29 GMT
• Title: Unimodal and Multimodal Sensor Fusion for Wearable Activity Recognition
• Authors: Hymalai Bello,
• Abstract summary: Human activity recognition (HAR) benefits from combining redundant and complementary information. This Ph.D. work employs sensing modalities such as inertial, pressure (audio and atmospheric pressure), and textile capacitive sensing for HAR. The selected wearable devices and sensing modalities are fully integrated with machine learning-based algorithms.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Combining different sensing modalities with multiple positions helps form a unified perception and understanding of complex situations such as human behavior. Hence, human activity recognition (HAR) benefits from combining redundant and complementary information (Unimodal/Multimodal). Even so, it is not an easy task. It requires a multidisciplinary approach, including expertise in sensor technologies, signal processing, data fusion algorithms, and domain-specific knowledge. This Ph.D. work employs sensing modalities such as inertial, pressure (audio and atmospheric pressure), and textile capacitive sensing for HAR. The scenarios explored are gesture and hand position tracking, facial and head pattern recognition, and body posture and gesture recognition. The selected wearable devices and sensing modalities are fully integrated with machine learning-based algorithms, some of which are implemented in the embedded device, on the edge, and tested in real-time.

Related papers

• Unified Framework with Consistency across Modalities for Human Activity Recognition [14.639249548669756]
  We propose a comprehensive framework for robust video-based human activity recognition. Key contribution is the introduction of a novel query machine, called COMPUTER. Our approach demonstrates superior performance when compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-09-04T02:25:10Z)
• Apprenticeship-Inspired Elegance: Synergistic Knowledge Distillation Empowers Spiking Neural Networks for Efficient Single-Eye Emotion Recognition [53.359383163184425]
  We introduce a novel multimodality synergistic knowledge distillation scheme tailored for efficient single-eye motion recognition tasks. This method allows a lightweight, unimodal student spiking neural network (SNN) to extract rich knowledge from an event-frame multimodal teacher network.
  arXiv  Detail & Related papers  (2024-06-20T07:24:47Z)
• SoK: Behind the Accuracy of Complex Human Activity Recognition Using Deep Learning [4.580983642743026]
  Human Activity Recognition (HAR) is a well-studied field with research dating back to the 1980s. In this paper, we comprehensively systematise factors leading to inaccuracy in complex HAR, such as data variety and model capacity.
  arXiv  Detail & Related papers  (2024-04-25T10:07:56Z)
• FedOpenHAR: Federated Multi-Task Transfer Learning for Sensor-Based Human Activity Recognition [0.0]
  This paper explores Federated Transfer Learning in a Multi-Task manner for both sensor-based human activity recognition and device position identification tasks. The OpenHAR framework is used to train the models, which contains ten smaller datasets. By utilizing transfer learning and training a task-specific and personalized federated model, we obtained a similar accuracy with training each client individually and higher accuracy than a fully centralized approach.
  arXiv  Detail & Related papers  (2023-11-13T21:31:07Z)
• MultiIoT: Benchmarking Machine Learning for the Internet of Things [70.74131118309967]
  The next generation of machine learning systems must be adept at perceiving and interacting with the physical world. sensory data from motion, thermal, geolocation, depth, wireless signals, video, and audio are increasingly used to model the states of physical environments. Existing efforts are often specialized to a single sensory modality or prediction task. This paper proposes MultiIoT, the most expansive and unified IoT benchmark to date, encompassing over 1.15 million samples from 12 modalities and 8 real-world tasks.
  arXiv  Detail & Related papers  (2023-11-10T18:13:08Z)
• Investigating Deep Neural Network Architecture and Feature Extraction Designs for Sensor-based Human Activity Recognition [0.0]
  In light of deep learning's proven effectiveness across various domains, numerous deep methods have been explored to tackle the challenges in activity recognition. We investigate the performance of common deep learning and machine learning approaches as well as different training mechanisms. Various feature representations extracted from the sensor time-series data and measure their effectiveness for the human activity recognition task.
  arXiv  Detail & Related papers  (2023-09-26T14:55:32Z)
• Agile gesture recognition for capacitive sensing devices: adapting on-the-job [55.40855017016652]
  We demonstrate a hand gesture recognition system that uses signals from capacitive sensors embedded into the etee hand controller. The controller generates real-time signals from each of the wearer five fingers. We use a machine learning technique to analyse the time series signals and identify three features that can represent 5 fingers within 500 ms.
  arXiv  Detail & Related papers  (2023-05-12T17:24:02Z)
• Vision+X: A Survey on Multimodal Learning in the Light of Data [64.03266872103835]
  multimodal machine learning that incorporates data from various sources has become an increasingly popular research area. We analyze the commonness and uniqueness of each data format mainly ranging from vision, audio, text, and motions. We investigate the existing literature on multimodal learning from both the representation learning and downstream application levels.
  arXiv  Detail & Related papers  (2022-10-05T13:14:57Z)
• TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensors [15.418884994244996]
  We focus on the process of guiding tactile exploration, and its interplay with task-related decision making. We propose TANDEM, an architecture to learn efficient exploration strategies in conjunction with decision making. We demonstrate this method on a tactile object recognition task, where a robot equipped with a touch sensor must explore and identify an object from a known set based on tactile feedback alone.
  arXiv  Detail & Related papers  (2022-03-01T23:55:09Z)
• Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition [131.6328804788164]
  We propose a framework, named Semantics-aware Adaptive Knowledge Distillation Networks (SAKDN), to enhance action recognition in vision-sensor modality (videos) The SAKDN uses multiple wearable-sensors as teacher modalities and uses RGB videos as student modality.
  arXiv  Detail & Related papers  (2020-09-01T03:38:31Z)
• Continuous Emotion Recognition via Deep Convolutional Autoencoder and Support Vector Regressor [70.2226417364135]
  It is crucial that the machine should be able to recognize the emotional state of the user with high accuracy. Deep neural networks have been used with great success in recognizing emotions. We present a new model for continuous emotion recognition based on facial expression recognition.
  arXiv  Detail & Related papers  (2020-01-31T17:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.