DeFlowSLAM: Self-Supervised Scene Motion Decomposition for Dynamic Dense
SLAM
- URL: http://arxiv.org/abs/2207.08794v1
- Date: Mon, 18 Jul 2022 17:47:39 GMT
- Title: DeFlowSLAM: Self-Supervised Scene Motion Decomposition for Dynamic Dense
SLAM
- Authors: Weicai Ye, Xingyuan Yu, Xinyue Lan, Yuhang Ming, Jinyu Li, Hujun Bao,
Zhaopeng Cui and Guofeng Zhang
- Abstract summary: We present a dynamic SLAM, dubbed DeFlowSLAM, that exploits both static and dynamic pixels in the images to solve the camera poses.
DeFlowSLAM generalizes well to both static and dynamic scenes as it exhibits comparable performance to the state-of-the-art DROID-SLAM in static and less dynamic scenes.
- Score: 41.76744945068124
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: We present a novel dual-flow representation of scene motion that decomposes
the optical flow into a static flow field caused by the camera motion and
another dynamic flow field caused by the objects' movements in the scene. Based
on this representation, we present a dynamic SLAM, dubbed DeFlowSLAM, that
exploits both static and dynamic pixels in the images to solve the camera
poses, rather than simply using static background pixels as other dynamic SLAM
systems do. We propose a dynamic update module to train our DeFlowSLAM in a
self-supervised manner, where a dense bundle adjustment layer takes in
estimated static flow fields and the weights controlled by the dynamic mask and
outputs the residual of the optimized static flow fields, camera poses, and
inverse depths. The static and dynamic flow fields are estimated by warping the
current image to the neighboring images, and the optical flow can be obtained
by summing the two fields. Extensive experiments demonstrate that DeFlowSLAM
generalizes well to both static and dynamic scenes as it exhibits comparable
performance to the state-of-the-art DROID-SLAM in static and less dynamic
scenes while significantly outperforming DROID-SLAM in highly dynamic
environments. Code and data are available on the project webpage: \urlstyle{tt}
\textcolor{url_color}{\url{https://zju3dv.github.io/deflowslam/}}.
Related papers
- NGD-SLAM: Towards Real-Time SLAM for Dynamic Environments without GPU [4.970590187498935]
This paper proposes a novel visual SLAM system for dynamic environments that obtains real-time performance on CPU by incorporating a mask prediction mechanism.
It further introduces a dual-stage optical flow tracking approach and employs a hybrid usage of optical flow and ORB features.
Compared with state-of-the-art methods, this system maintains high localization accuracy in dynamic environments while achieving a tracking frame rate of 56 fps on a single laptop CPU without any hardware acceleration.
arXiv Detail & Related papers (2024-05-12T23:00:53Z) - EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via
Self-Supervision [85.17951804790515]
EmerNeRF is a simple yet powerful approach for learning spatial-temporal representations of dynamic driving scenes.
It simultaneously captures scene geometry, appearance, motion, and semantics via self-bootstrapping.
Our method achieves state-of-the-art performance in sensor simulation.
arXiv Detail & Related papers (2023-11-03T17:59:55Z) - Forward Flow for Novel View Synthesis of Dynamic Scenes [97.97012116793964]
We propose a neural radiance field (NeRF) approach for novel view synthesis of dynamic scenes using forward warping.
Our method outperforms existing methods in both novel view rendering and motion modeling.
arXiv Detail & Related papers (2023-09-29T16:51:06Z) - DynaMoN: Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields [71.94156412354054]
We propose DynaMoN to handle dynamic content for initial camera pose estimation and statics-focused ray sampling for fast and accurate novel-view synthesis.
Our novel iterative learning scheme switches between training the NeRF and updating the pose parameters for an improved reconstruction and trajectory estimation quality.
arXiv Detail & Related papers (2023-09-16T08:46:59Z) - Alignment-free HDR Deghosting with Semantics Consistent Transformer [76.91669741684173]
High dynamic range imaging aims to retrieve information from multiple low-dynamic range inputs to generate realistic output.
Existing methods often focus on the spatial misalignment across input frames caused by the foreground and/or camera motion.
We propose a novel alignment-free network with a Semantics Consistent Transformer (SCTNet) with both spatial and channel attention modules.
arXiv Detail & Related papers (2023-05-29T15:03:23Z) - MOTSLAM: MOT-assisted monocular dynamic SLAM using single-view depth
estimation [5.33931801679129]
MOTSLAM is a dynamic visual SLAM system with the monocular configuration that tracks both poses and bounding boxes of dynamic objects.
Our experiments on the KITTI dataset demonstrate that our system has reached best performance on both camera ego-motion and object tracking on monocular dynamic SLAM.
arXiv Detail & Related papers (2022-10-05T06:07:10Z) - Dynamic Dense RGB-D SLAM using Learning-based Visual Odometry [0.8029049649310211]
We propose a dense dynamic RGB-D SLAM pipeline based on a learning-based visual odometry, TartanVO.
Our pipeline resolves dynamic/static segmentation by leveraging the optical flow output, and only fuse static points into the map.
arXiv Detail & Related papers (2022-05-12T07:11:41Z) - DOT: Dynamic Object Tracking for Visual SLAM [83.69544718120167]
DOT combines instance segmentation and multi-view geometry to generate masks for dynamic objects.
To determine which objects are actually moving, DOT segments first instances of potentially dynamic objects and then, with the estimated camera motion, tracks such objects by minimizing the photometric reprojection error.
Our results show that our approach improves significantly the accuracy and robustness of ORB-SLAM 2, especially in highly dynamic scenes.
arXiv Detail & Related papers (2020-09-30T18:36:28Z) - FlowFusion: Dynamic Dense RGB-D SLAM Based on Optical Flow [17.040818114071833]
We present a novel dense RGB-D SLAM solution that simultaneously accomplishes the dynamic/static segmentation and camera ego-motion estimation.
Our novelty is using optical flow residuals to highlight the dynamic semantics in the RGB-D point clouds.
arXiv Detail & Related papers (2020-03-11T04:00:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.