FlexLoc: Conditional Neural Networks for Zero-Shot Sensor Perspective Invariance in Object Localization with Distributed Multimodal Sensors

• URL: http://arxiv.org/abs/2406.06796v1
• Date: Mon, 10 Jun 2024 21:02:53 GMT
• Title: FlexLoc: Conditional Neural Networks for Zero-Shot Sensor Perspective Invariance in Object Localization with Distributed Multimodal Sensors
• Authors: Jason Wu, Ziqi Wang, Xiaomin Ouyang, Ho Lyun Jeong, Colin Samplawski, Lance Kaplan, Benjamin Marlin, Mani Srivastava,
• Abstract summary: We introduce FlexLoc, which employs conditional neural networks to inject node perspective information to adapt the localization pipeline. Our evaluations on a multimodal, multiview indoor tracking dataset showcase that FlexLoc improves the localization accuracy by almost 50% in the zero-shot case.
• Score: 6.676517041445593
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Localization is a critical technology for various applications ranging from navigation and surveillance to assisted living. Localization systems typically fuse information from sensors viewing the scene from different perspectives to estimate the target location while also employing multiple modalities for enhanced robustness and accuracy. Recently, such systems have employed end-to-end deep neural models trained on large datasets due to their superior performance and ability to handle data from diverse sensor modalities. However, such neural models are often trained on data collected from a particular set of sensor poses (i.e., locations and orientations). During real-world deployments, slight deviations from these sensor poses can result in extreme inaccuracies. To address this challenge, we introduce FlexLoc, which employs conditional neural networks to inject node perspective information to adapt the localization pipeline. Specifically, a small subset of model weights are derived from node poses at run time, enabling accurate generalization to unseen perspectives with minimal additional overhead. Our evaluations on a multimodal, multiview indoor tracking dataset showcase that FlexLoc improves the localization accuracy by almost 50% in the zero-shot case (no calibration data available) compared to the baselines. The source code of FlexLoc is available at https://github.com/nesl/FlexLoc.

Related papers

• DBA-Fusion: Tightly Integrating Deep Dense Visual Bundle Adjustment with Multiple Sensors for Large-Scale Localization and Mapping [3.5047603107971397]
  We tightly integrate the trainable deep dense bundle adjustment (DBA) with multi-sensor information through a factor graph. A pipeline for visual-inertial integration is firstly developed, which provides the minimum ability of metric-scale localization and mapping. The results validate the superior localization performance of our approach, which enables real-time dense mapping in large-scale environments.
  arXiv  Detail & Related papers  (2024-03-20T16:20:54Z)
• Advancing Location-Invariant and Device-Agnostic Motion Activity Recognition on Wearable Devices [6.557453686071467]
  We conduct a comprehensive evaluation of the generalizability of motion models across sensor locations. Our analysis highlights this challenge and identifies key on-body locations for building location-invariant models. We present deployable on-device motion models reaching 91.41% frame-level F1-score from a single model irrespective of sensor placements.
  arXiv  Detail & Related papers  (2024-02-06T05:10:00Z)
• Diffusion-based Data Augmentation for Object Counting Problems [62.63346162144445]
  We develop a pipeline that utilizes a diffusion model to generate extensive training data. We are the first to generate images conditioned on a location dot map with a diffusion model. Our proposed counting loss for the diffusion model effectively minimizes the discrepancies between the location dot map and the crowd images generated.
  arXiv  Detail & Related papers  (2024-01-25T07:28:22Z)
• Physical-Layer Semantic-Aware Network for Zero-Shot Wireless Sensing [74.12670841657038]
  Device-free wireless sensing has recently attracted significant interest due to its potential to support a wide range of immersive human-machine interactive applications. Data heterogeneity in wireless signals and data privacy regulation of distributed sensing have been considered as the major challenges that hinder the wide applications of wireless sensing in large area networking systems. We propose a novel zero-shot wireless sensing solution that allows models constructed in one or a limited number of locations to be directly transferred to other locations without any labeled data.
  arXiv  Detail & Related papers  (2023-12-08T13:50:30Z)
• Leveraging arbitrary mobile sensor trajectories with shallow recurrent decoder networks for full-state reconstruction [4.243926243206826]
  We show that a sequence-to-vector model, such as an LSTM (long, short-term memory) network, with a decoder network, dynamic information can be mapped to full state-space estimates. The exceptional performance of the network architecture is demonstrated on three applications.
  arXiv  Detail & Related papers  (2023-07-20T21:42:01Z)
• UnLoc: A Universal Localization Method for Autonomous Vehicles using LiDAR, Radar and/or Camera Input [51.150605800173366]
  UnLoc is a novel unified neural modeling approach for localization with multi-sensor input in all weather conditions. Our method is extensively evaluated on Oxford Radar RobotCar, ApolloSouthBay and Perth-WA datasets.
  arXiv  Detail & Related papers  (2023-07-03T04:10:55Z)
• LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensors [38.60837840737258]
  We propose a multi-sensor fusion method for capturing 3D human motions with accurate consecutive local poses and global trajectories in large-scale scenarios. We design a two-stage pose estimator in a coarse-to-fine manner, where point clouds provide the coarse body shape and IMU measurements optimize the local actions. We collect a LiDAR-IMU multi-modal mocap dataset, LIPD, with diverse human actions in long-range scenarios.
  arXiv  Detail & Related papers  (2022-05-30T20:15:11Z)
• FuNNscope: Visual microscope for interactively exploring the loss landscape of fully connected neural networks [77.34726150561087]
  We show how to explore high-dimensional landscape characteristics of neural networks. We generalize observations on small neural networks to more complex systems. An interactive dashboard opens up a number of possible application networks.
  arXiv  Detail & Related papers  (2022-04-09T16:41:53Z)
• Zero-Shot Multi-View Indoor Localization via Graph Location Networks [66.05980368549928]
  indoor localization is a fundamental problem in location-based applications. We propose a novel neural network based architecture Graph Location Networks (GLN) to perform infrastructure-free, multi-view image based indoor localization. GLN makes location predictions based on robust location representations extracted from images through message-passing networks. We introduce a novel zero-shot indoor localization setting and tackle it by extending the proposed GLN to a dedicated zero-shot version.
  arXiv  Detail & Related papers  (2020-08-06T07:36:55Z)
• Localized convolutional neural networks for geospatial wind forecasting [0.0]
  Convolutional Neural Networks (CNN) possess positive qualities when it comes to many spatial data. In this work, we propose localized convolutional neural networks that enable CNNs to learn local features in addition to the global ones. They can be added to any convolutional layers, easily end-to-end trained, introduce minimal additional complexity, and let CNNs retain most of their benefits to the extent that they are needed.
  arXiv  Detail & Related papers  (2020-05-12T17:14:49Z)
• Deep Soft Procrustes for Markerless Volumetric Sensor Alignment [81.13055566952221]
  In this work, we improve markerless data-driven correspondence estimation to achieve more robust multi-sensor spatial alignment. We incorporate geometric constraints in an end-to-end manner into a typical segmentation based model and bridge the intermediate dense classification task with the targeted pose estimation one. Our model is experimentally shown to achieve similar results with marker-based methods and outperform the markerless ones, while also being robust to the pose variations of the calibration structure.
  arXiv  Detail & Related papers  (2020-03-23T10:51:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.