Accurate Open-Loop Control of a Soft Continuum Robot Through Visually Learned Latent Representations

• URL: http://arxiv.org/abs/2603.19655v1
• Date: Fri, 20 Mar 2026 05:43:26 GMT
• Title: Accurate Open-Loop Control of a Soft Continuum Robot Through Visually Learned Latent Representations
• Abstract summary: This work addresses open-loop control of a soft continuum robot (SCR) from video-learned latent dynamics.<n>ABCD-based models consistently reduce image-space tracking error.<n>Several architecture choices and training settings contribute to the open-loop control performance.
• Score: 4.857795247230421
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work addresses open-loop control of a soft continuum robot (SCR) from video-learned latent dynamics. Visual Oscillator Networks (VONs) from previous work are used, that provide mechanistically interpretable 2D oscillator latents through an attention broadcast decoder (ABCD). Open-loop, single-shooting optimal control is performed in latent space to track image-specified waypoints without camera feedback. An interactive SCR live simulator enables design of static, dynamic, and extrapolated targets and maps them to model-specific latent waypoints. On a two-segment pneumatic SCR, Koopman, MLP, and oscillator dynamics, each with and without ABCD, are evaluated on setpoint and dynamic trajectories. ABCD-based models consistently reduce image-space tracking error. The VON and ABCD-based Koopman models attains the lowest MSEs. Using an ablation study, we demonstrate that several architecture choices and training settings contribute to the open-loop control performance. Simulation stress tests further confirm static holding, stable extrapolated equilibria, and plausible relaxation to the rest state. To the best of our knowledge, this is the first demonstration that interpretable, video-learned latent dynamics enable reliable long-horizon open-loop control of an SCR.

Related papers

• Dynamic-Robust Photometric-Semantic Reconstruction for Open-Vocabulary 3D Scene Understanding [63.599342750566485]
  SPAR is a novel joint semantic-geometric encoding architecture that explicitly isolates transient dynamic noise prior to latent space aggregation.<n>Our end-to-end approach achieves exceptional novel view synthesis quality, yielding a PSNR of 22.15 dB and 23.33 dB given only 3 and 4 input views respectively.
  arXiv  Detail & Related papers  (2026-08-29T09:58:24Z)
• A Task-Space Receding Horizon Controller for Fast Collision Avoidance [0.08594140167290099]
  Real-time collision avoidance for robotic manipulators requires fast reactions to unexpected obstacle motion.<n>Full model predictive control can provide this foresight, but its online cost may grow quickly with horizon length, model fidelity, and the number of active geometric constraints.<n>We present a task-space receding-horizon controller that uses a short contact-consistent rollout to generate a terminal kinematic reference.
  arXiv  Detail & Related papers  (2026-07-17T08:13:12Z)
• Native Video-Action Pretraining for Generalizable Robot Control [77.59831077077176]
  LingBot-VA 2.0 is a video-action foundation model built from the ground up for embodiment.<n>To meet the demands of high-frequency inference, our model employs a sparse MoE backbone.<n>Real-world deployment validates LingBot-VA 2.0 as a robust foundation model.
  arXiv  Detail & Related papers  (2026-07-09T16:15:43Z)
• Cortical Policy: A Dual-Stream View Transformer for Robotic Manipulation [57.28703268044067]
  We propose Cortical Policy, a novel dual-stream view transformer for robotic manipulation.<n>Our framework offers a fresh perspective for robotic manipulation and holds potential for broader application in vision-based robot control.
  arXiv  Detail & Related papers  (2026-03-22T04:18:54Z)
• From Frames to Sequences: Temporally Consistent Human-Centric Dense Prediction [22.291273919939957]
  We develop a scalable synthetic data pipeline that generates human frames and motion-aligned sequences with pixel-accurate depth, normals, and masks.<n>We train a unified ViT-based dense predictor that injects an explicit geometric human prior via CSE embeddings.<n>Our two-stage training strategy, combining static pretraining with dynamic sequence supervision, enables the model first to acquire robust spatial representations and then refine temporal consistency across motion-aligned sequences.
  arXiv  Detail & Related papers  (2026-02-02T05:28:58Z)
• DRAW2ACT: Turning Depth-Encoded Trajectories into Robotic Demonstration Videos [24.681248200255975]
  Video models provide powerful real-world simulators for embodied AI but remain limited in controllability for robotic manipulation.<n>We present DRAW2ACT, a trajectory-conditioned video generation framework that extracts multiple representations from the input trajectory.<n>We show that DRAW2ACT achieves superior visual fidelity and consistency while yielding higher manipulation success rates compared to existing baselines.
  arXiv  Detail & Related papers  (2025-12-16T09:11:36Z)
• Learning Visually Interpretable Oscillator Networks for Soft Continuum Robots from Video [4.857795247230421]
  We introduce Attention Broadcast Decoder (ABCD), a plug-and-play module for autoencoder-based latent dynamics learning.<n>ABCD generates pixel-accurate attention maps localizing each latent dimension's contribution while filtering static backgrounds.<n>We validate our approach on single- and double-segment SCRs, demonstrating that ABCD-based models significantly improve multi-step prediction accuracy.
  arXiv  Detail & Related papers  (2025-11-23T07:27:39Z)
• RobotDancing: Residual-Action Reinforcement Learning Enables Robust Long-Horizon Humanoid Motion Tracking [50.200035833530876]
  RobotDancing is a simple, scalable framework that predicts residual joint targets to explicitly correct dynamics discrepancies.<n>It can track multi-minute, high-energy behaviors (jumps, spins, cartwheels) and deploys zero-shot to hardware with high motion tracking quality.
  arXiv  Detail & Related papers  (2025-09-25T03:30:34Z)
• Bidirectional Feature-aligned Motion Transformation for Efficient Dynamic Point Cloud Compression [97.66080040613726]
  We propose a Bidirectional Feature-aligned Motion Transformation (Bi-FMT) framework that implicitly models motion in the feature space.<n>Bi-FMT aligns features across both past and future frames to produce temporally consistent latent representations.<n>We show Bi-FMT surpasses D-DPCC and AdaDPCC in both compression efficiency and runtime.
  arXiv  Detail & Related papers  (2025-09-18T03:51:06Z)
• Tracking the Unstable: Appearance-Guided Motion Modeling for Robust Multi-Object Tracking in UAV-Captured Videos [58.156141601478794]
  Multi-object tracking (UAVT) aims to track multiple objects while maintaining consistent identities across frames of a given video.<n>Existing methods typically model motion cues and appearance separately, overlooking their interplay and resulting in suboptimal tracking performance.<n>We propose AMOT, which exploits appearance and motion cues through two key components: an Appearance-Motion Consistency (AMC) matrix and a Motion-aware Track Continuation (MTC) module.
  arXiv  Detail & Related papers  (2025-08-03T12:06:47Z)
• Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Model [63.336123527432136]
  We introduce Bench2Drive-R, a generative framework that enables reactive closed-loop evaluation.<n>Unlike existing video generative models for autonomous driving, the proposed designs are tailored for interactive simulation.<n>We compare the generation quality of Bench2Drive-R with existing generative models and achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-12-11T06:35:18Z)
• DATAP-SfM: Dynamic-Aware Tracking Any Point for Robust Structure from Motion in the Wild [85.03973683867797]
  This paper proposes a concise, elegant, and robust pipeline to estimate smooth camera trajectories and obtain dense point clouds for casual videos in the wild. We show that the proposed method achieves state-of-the-art performance in terms of camera pose estimation even in complex dynamic challenge scenes.
  arXiv  Detail & Related papers  (2024-11-20T13:01:16Z)
• KFD-NeRF: Rethinking Dynamic NeRF with Kalman Filter [49.85369344101118]
  We introduce KFD-NeRF, a novel dynamic neural radiance field integrated with an efficient and high-quality motion reconstruction framework based on Kalman filtering. Our key idea is to model the dynamic radiance field as a dynamic system whose temporally varying states are estimated based on two sources of knowledge: observations and predictions. Our KFD-NeRF demonstrates similar or even superior performance within comparable computational time and state-of-the-art view synthesis performance with thorough training.
  arXiv  Detail & Related papers  (2024-07-18T05:48:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.