GaussTwin: Unified Simulation and Correction with Gaussian Splatting for Robotic Digital Twins

• URL: http://arxiv.org/abs/2603.05108v1
• Date: Thu, 05 Mar 2026 12:27:05 GMT
• Title: GaussTwin: Unified Simulation and Correction with Gaussian Splatting for Robotic Digital Twins
• Abstract summary: GaussTwin is a real-time digital twin that combines position-based dynamics with discrete Cosserat rod formulations for simulation.<n>We show that GaussTwin consistently improves tracking accuracy and robustness compared to shape-matching and rigid-only baselines.
• Score: 15.345154779819302
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Digital twins promise to enhance robotic manipulation by maintaining a consistent link between real-world perception and simulation. However, most existing systems struggle with the lack of a unified model, complex dynamic interactions, and the real-to-sim gap, which limits downstream applications such as model predictive control. Thus, we propose GaussTwin, a real-time digital twin that combines position-based dynamics with discrete Cosserat rod formulations for physically grounded simulation, and Gaussian splatting for efficient rendering and visual correction. By anchoring Gaussians to physical primitives and enforcing coherent SE(3) updates driven by photometric error and segmentation masks, GaussTwin achieves stable prediction-correction while preserving physical fidelity. Through experiments in both simulation and on a Franka Research 3 platform, we show that GaussTwin consistently improves tracking accuracy and robustness compared to shape-matching and rigid-only baselines, while also enabling downstream tasks such as push-based planning. These results highlight GaussTwin as a step toward unified, physically meaningful digital twins that can support closed-loop robotic interaction and learning.

Related papers

• DecoupleGS: Interactive 3D Gaussian Splatting for End-to-End Autonomous Driving Testing [37.82194951319459]
  End-to-end (E2E) autonomous driving algorithms require rigorous closed-loop validation in simulation environments.<n>Existing approaches, from game engines to static neural rendering, inherently trade off these requirements.<n>We propose a novel decoupled 3D Gaussian Splatting (3DGS) framework tailored for large-scale E2E evaluation.
  arXiv  Detail & Related papers  (2026-08-03T06:32:00Z)
• ArtiTwinSplat: Interactable Digital Twin Reconstruction via Gaussian Splatting from RGB-D videos [47.4010297818512]
  ArtiTwinSplat is a framework that automatically constructs articulated, photo-realistic digital twins of objects from RGB-D videos.<n>Our method is built on 3D Gaussian Splatting that preserve geometric fidelity and photometric realism.
  arXiv  Detail & Related papers  (2026-06-23T14:24:07Z)
• ManiSplat: Manipulation Trajectory Synthesis from Monocular Video via Decoupled 3D Gaussian Splatting [28.256286705954846]
  Reconstructing dynamic and interactive 3D scenes from real-world observations remains a fundamental challenge in computer vision and robotics.<n>We introduce ManiSplat, a unified framework that reconstructs controllable and decoupled Gaussian digital twins directly from robotic videos.<n>Our approach reconstructs interaction-driven dynamic scenes with high fidelity and controllability, effectively supporting downstream robotic tasks and policy learning.
  arXiv  Detail & Related papers  (2026-06-09T09:55:58Z)
• DOC-GS: Dual-Domain Observation and Calibration for Reliable Sparse-View Gaussian Splatting [80.43237927269575]
  In this paper, we revisit sparse-view 3DGS reconstruction from a new perspective.<n>We identify the core challenge as the unobservability of Gaussian primitive reliability.<n>Motivated by this observation, we propose a unified Dual-domain Observation and geometrics in rendered images framework.
  arXiv  Detail & Related papers  (2026-04-08T07:01:24Z)
• GaussFly: Contrastive Reinforcement Learning for Visuomotor Policies in 3D Gaussian Fields [10.10713417247828]
  GaussFly is a novel framework that explicitly decouples representation learning from policy optimization.<n>It enables robust and zero-shot policy transfer to unseen real-world environments with complex textures.
  arXiv  Detail & Related papers  (2026-04-06T18:10:52Z)
• DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancer [62.18680935878919]
  We introduce DiffusionHarmonizer, an online generative enhancement framework that transforms renderings into temporally consistent outputs.<n>At its core is a single-step temporally-conditioned enhancer capable of running in online simulators on a single GPU.
  arXiv  Detail & Related papers  (2026-02-27T15:35:30Z)
• GaussianFluent: Gaussian Simulation for Dynamic Scenes with Mixed Materials [34.530526668727056]
  GaussianFluent is a framework for realistic simulation and rendering of dynamic object states.<n>It synthesizes interiors by densifying internal Gaussians guided by generative models.<n>It integrates an optimized Continuum Damage Material Point Method (CD-MPM) to enable brittle fracture simulation at remarkably high speed.
  arXiv  Detail & Related papers  (2026-01-14T07:59:52Z)
• A High-Fidelity Digital Twin for Robotic Manipulation Based on 3D Gaussian Splatting [0.0]
  We present a framework that constructs high-quality digital twins within minutes from sparse RGB inputs.<n>Our system employs 3D Gaussian Splatting (3DGS) for fast, photorealistic reconstruction as a unified scene representation.<n>In experiments with a Franka Emika Panda robot, we demonstrate that this enhanced geometric accuracy effectively supports robust manipulation in real-world trials.
  arXiv  Detail & Related papers  (2026-01-06T17:29:10Z)
• CaricatureGS: Exaggerating 3D Gaussian Splatting Faces With Gaussian Curvature [13.47263744740423]
  A controllable and controllable 3D caricaturization framework for faces is introduced.<n>We resort to 3D Gaussian Splatting (3DGS), which has recently been shown to produce realistic free-viewpoint avatars.
  arXiv  Detail & Related papers  (2026-01-06T13:56:28Z)
• Gaussian Swaying: Surface-Based Framework for Aerodynamic Simulation with 3D Gaussians [35.72395384675807]
  Gaussian Swaying is a surface-based framework for aerodynamic simulation using 3D Gaussians.<n>Our framework unifies simulation and rendering on the same representation: Gaussian patches.<n>Our framework achieves state-of-the-art performance and efficiency, offering a scalable approach for realistic aerodynamic scene simulation.
  arXiv  Detail & Related papers  (2025-12-01T06:03:47Z)
• RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS [79.15416002879239]
  3D Gaussian Splatting has gained significant attention for its real-time, photo-realistic rendering in novel-view synthesis and 3D modeling.<n>Existing methods struggle with accurately modeling scenes affected by transient objects, leading to artifacts in the rendered images.<n>We propose RobustSplat, a robust solution based on two critical designs.
  arXiv  Detail & Related papers  (2025-06-03T11:13:48Z)
• GausSim: Foreseeing Reality by Gaussian Simulator for Elastic Objects [55.02281855589641]
  GausSim is a novel neural network-based simulator designed to capture the dynamic behaviors of real-world elastic objects represented through Gaussian kernels.<n>We leverage continuum mechanics and treat each kernel as a Center of Mass System (CMS) that represents continuous piece of matter.<n>In addition, GausSim incorporates explicit physics constraints, such as mass and momentum conservation, ensuring interpretable results and robust, physically plausible simulations.
  arXiv  Detail & Related papers  (2024-12-23T18:58:17Z)
• DeSiRe-GS: 4D Street Gaussians for Static-Dynamic Decomposition and Surface Reconstruction for Urban Driving Scenes [71.61083731844282]
  We present DeSiRe-GS, a self-supervised gaussian splatting representation.<n>It enables effective static-dynamic decomposition and high-fidelity surface reconstruction in complex driving scenarios.
  arXiv  Detail & Related papers  (2024-11-18T05:49:16Z)
• Differentiable DG with Neural Operator Source Term Correction [0.0]
  We introduce an end-to-end differentiable framework for solving the compressible Navier-Stokes equations.<n>This integrated approach combines a differentiable discontinuous Galerkin solver with a neural network source term.<n>We demonstrate the performance of the proposed framework through two examples.
  arXiv  Detail & Related papers  (2023-10-29T04:26:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.