CROMOSim: A Deep Learning-based Cross-modality Inertial Measurement
Simulator
- URL: http://arxiv.org/abs/2202.10562v1
- Date: Mon, 21 Feb 2022 22:30:43 GMT
- Title: CROMOSim: A Deep Learning-based Cross-modality Inertial Measurement
Simulator
- Authors: Yujiao Hao, Boyu Wang, Rong Zheng
- Abstract summary: Inertial measurement unit (IMU) data has been utilized in monitoring and assessment of human mobility.
To mitigate the data scarcity problem, we design CROMOSim, a cross-modality sensor simulator.
It simulates high fidelity virtual IMU sensor data from motion capture systems or monocular RGB cameras.
- Score: 7.50015216403068
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: With the prevalence of wearable devices, inertial measurement unit (IMU) data
has been utilized in monitoring and assessment of human mobility such as human
activity recognition (HAR). Training deep neural network (DNN) models for these
tasks require a large amount of labeled data, which are hard to acquire in
uncontrolled environments. To mitigate the data scarcity problem, we design
CROMOSim, a cross-modality sensor simulator that simulates high fidelity
virtual IMU sensor data from motion capture systems or monocular RGB cameras.
It utilizes a skinned multi-person linear model (SMPL) for 3D body pose and
shape representations, to enable simulation from arbitrary on-body positions. A
DNN model is trained to learn the functional mapping from imperfect trajectory
estimations in a 3D SMPL body tri-mesh due to measurement noise, calibration
errors, occlusion and other modeling artifacts, to IMU data. We evaluate the
fidelity of CROMOSim simulated data and its utility in data augmentation on
various HAR datasets. Extensive experiment results show that the proposed model
achieves a 6.7% improvement over baseline methods in a HAR task.
Related papers
- Bridging the Sim-to-Real Gap with Bayesian Inference [53.61496586090384]
We present SIM-FSVGD for learning robot dynamics from data.
We use low-fidelity physical priors to regularize the training of neural network models.
We demonstrate the effectiveness of SIM-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system.
arXiv Detail & Related papers (2024-03-25T11:29:32Z) - Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous
Driving Research [76.93956925360638]
Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes.
It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training.
We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
arXiv Detail & Related papers (2023-10-12T20:49:15Z) - Turbulence in Focus: Benchmarking Scaling Behavior of 3D Volumetric
Super-Resolution with BLASTNet 2.0 Data [4.293221567339693]
Analysis of compressible turbulent flows is essential for applications related to propulsion, energy generation, and the environment.
We present a 2.2 TB network-of-datasets containing 744 full-domain samples from 34 high-fidelity direct numerical simulations.
We benchmark a total of 49 variations of five deep learning approaches for 3D super-resolution.
arXiv Detail & Related papers (2023-09-23T18:57:02Z) - On Transferability of Driver Observation Models from Simulated to Real
Environments in Autonomous Cars [23.514129229090987]
This paper investigates the viability of transferring video-based driver observation models from simulation to real-world scenarios in autonomous vehicles.
We record a dataset featuring actual autonomous driving conditions and involving seven participants engaged in highly distracting secondary activities.
Our dataset was designed in accordance with an existing large-scale simulator dataset used as the training source.
arXiv Detail & Related papers (2023-07-31T10:18:49Z) - Towards Multimodal Multitask Scene Understanding Models for Indoor
Mobile Agents [49.904531485843464]
In this paper, we discuss the main challenge: insufficient, or even no, labeled data for real-world indoor environments.
We describe MMISM (Multi-modality input Multi-task output Indoor Scene understanding Model) to tackle the above challenges.
MMISM considers RGB images as well as sparse Lidar points as inputs and 3D object detection, depth completion, human pose estimation, and semantic segmentation as output tasks.
We show that MMISM performs on par or even better than single-task models.
arXiv Detail & Related papers (2022-09-27T04:49:19Z) - Learning to Simulate Realistic LiDARs [66.7519667383175]
We introduce a pipeline for data-driven simulation of a realistic LiDAR sensor.
We show that our model can learn to encode realistic effects such as dropped points on transparent surfaces.
We use our technique to learn models of two distinct LiDAR sensors and use them to improve simulated LiDAR data accordingly.
arXiv Detail & Related papers (2022-09-22T13:12:54Z) - Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse
Data using a Learning-based Unscented Kalman Filter [65.93205328894608]
We learn the residual errors between a dynamic and/or simulator model and the real robot.
We show that with the learned residual errors, we can further close the reality gap between dynamic models, simulations, and actual hardware.
arXiv Detail & Related papers (2022-09-07T15:15:12Z) - Mixed Effects Neural ODE: A Variational Approximation for Analyzing the
Dynamics of Panel Data [50.23363975709122]
We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data.
We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem.
We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
arXiv Detail & Related papers (2022-02-18T22:41:51Z) - Learning Similarity Metrics for Volumetric Simulations with Multiscale
CNNs [25.253880881581956]
We propose a similarity model based on entropy, which allows for the creation of physically meaningful ground truth distances.
We create collections of fields from numerical PDE solvers and existing simulation data repositories.
A multiscale CNN architecture that computes a volumetric similarity metric (VolSiM) is proposed.
arXiv Detail & Related papers (2022-02-08T19:19:08Z) - Forward and Inverse models in HCI:Physical simulation and deep learning
for inferring 3D finger pose [2.8952292379640636]
We use machine learning to develop data-driven models to infer position, pose and sensor readings.
We combine a Conditional Variational Autoencoder with domain expertise/models experimentally collected data.
arXiv Detail & Related papers (2021-09-07T23:11:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.