AI-Enabled Image-Based Hybrid Vision/Force Control of Tendon-Driven Aerial Continuum Manipulators

• URL: http://arxiv.org/abs/2604.18961v1
• Date: Tue, 21 Apr 2026 01:19:11 GMT
• Title: AI-Enabled Image-Based Hybrid Vision/Force Control of Tendon-Driven Aerial Continuum Manipulators
• Abstract summary: This paper presents an AI-enabled cascaded hybrid vision/force control framework for tendon-driven aerial manipulators.<n>The proposed controller is designed to enable autonomous, physical interaction with a static environment while stabilizing the image feature error.
• Score: 0.6999740786886535
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents an AI-enabled cascaded hybrid vision/force control framework for tendon-driven aerial continuum manipulators based on constant-strain modeling in $SE(3)$ as a coupled system. The proposed controller is designed to enable autonomous, physical interaction with a static environment while stabilizing the image feature error. The developed strategy combines the cascaded fast fixed-time sliding mode control and a radial basis function neural network to cope with the uncertainties in the image acquired by the eye-in-hand monocular camera and the measurements from the force sensing apparatus. This ensures rapid, online learning of the vision- and force-related uncertainties without requiring offline training. Furthermore, the features are extracted via a state-of-the-art graph neural network architecture employed by a visual servoing framework using line features, rather than relying on heuristic geometric line extractors, to concurrently contribute to tracking the desired normal interaction force during contact and regulating the image feature error. A comparative study benchmarks the proposed controller against established rigid-arm aerial manipulation methods, evaluating robustness across diverse scenarios and feature extraction strategies. The simulation and experimental results showcase the effectiveness of the proposed methodology under various initial conditions and demonstrate robust performance in executing manipulation tasks.

Related papers

• Partial Skeleton Visibility for Action Recognition: A Constrained Field-of-View Approach [60.596944437968524]
  PartialVisGraph is a novel framework for robust skeleton-based action recognition under constrained field-of-view.<n>It consistently achieves state-of-the-art accuracy under partial visibility, with gains of up to 68.8% on subsets with severe FoV restrictions.<n>Our approach offers a principled and practical pathway toward deployable skeleton-based action understanding in unconstrained environments.
  arXiv  Detail & Related papers  (2026-07-01T10:03:11Z)
• From Articulated Kinematics to Routed Visual Control for Action-Conditioned Surgical Video Generation [43.4347713269043]
  Action-conditioned surgical video generation is a critical yet highly challenging problem for robotic surgery.<n>We propose a kinematic-to-visual lifting paradigm that converts articulated kinematics into a unified set of five image-aligned control modalities.<n>Our method consistently improves action faithfulness, visual fidelity, and cross-domain generalization over diverse baselines.
  arXiv  Detail & Related papers  (2026-05-09T05:48:51Z)
• Strain-Parameterized Coupled Dynamics and Dual-Camera Visual Servoing for Aerial Continuum Manipulators [0.8594140167290097]
  Tendon-driven continuum aerial manipulators (TD-ACMs) combine the maneuverability of uncrewed aerial vehicles (UAVs) with the compliance of lightweight robots (CRs)<n>Existing continuum coupled dynamic modeling approaches for TD-ACMs incur high computational costs.<n>This paper presents a generalized dynamic formulation of a coupled TD-ACM with an underactuated base.
  arXiv  Detail & Related papers  (2026-03-24T15:33:22Z)
• From Vision to Decision: Neuromorphic Control for Autonomous Navigation and Tracking [10.144707192962068]
  We propose a parsimonious neuromorphic control framework for vision-guided navigation and tracking.<n>Image pixels from an onboard camera are encoded as inputs to dynamic neuronal populations.<n>Inspired by recently proposed mechanistic models of animal cognition and opinion dynamics, the neuromorphic controller provides real-time autonomy with a minimal computational burden.
  arXiv  Detail & Related papers  (2026-02-05T14:09:09Z)
• Exploring Conditions for Diffusion models in Robotic Control [70.27711404291573]
  We explore leveraging pre-trained text-to-image diffusion models to obtain task-adaptive visual representations for robotic control.<n>We find that naively applying textual conditions yields minimal or even negative gains in control tasks.<n>We propose ORCA, which introduces learnable task prompts that adapt to the control environment and visual prompts that capture fine-grained, frame-specific details.
  arXiv  Detail & Related papers  (2025-10-17T10:24:14Z)
• ScaleWeaver: Weaving Efficient Controllable T2I Generation with Multi-Scale Reference Attention [86.93601565563954]
  ScaleWeaver is a framework designed to achieve high-fidelity, controllable generation upon advanced visual autoregressive( VAR) models.<n>The proposed Reference Attention module discards the unnecessary attention from image$rightarrow$condition, reducing computational cost.<n>Experiments show that ScaleWeaver delivers high-quality generation and precise control while attaining superior efficiency over diffusion-based methods.
  arXiv  Detail & Related papers  (2025-10-16T17:00:59Z)
• Verification of Visual Controllers via Compositional Geometric Transformations [49.81690518952909]
  We introduce a novel verification framework for perception-based controllers that can generate outer-approximations of reachable sets.<n>We provide theoretical guarantees on the soundness of our method and demonstrate its effectiveness across benchmark control environments.
  arXiv  Detail & Related papers  (2025-07-06T20:22:58Z)
• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• ControlVAR: Exploring Controllable Visual Autoregressive Modeling [48.66209303617063]
  Conditional visual generation has witnessed remarkable progress with the advent of diffusion models (DMs) Challenges such as expensive computational cost, high inference latency, and difficulties of integration with large language models (LLMs) have necessitated exploring alternatives to DMs. This paper introduces Controlmore, a novel framework that explores pixel-level controls in visual autoregressive modeling for flexible and efficient conditional generation.
  arXiv  Detail & Related papers  (2024-06-14T06:35:33Z)
• Visual Servoing for Pose Control of Soft Continuum Arm in a Structured Environment [9.599112149531557]
  This letter presents a deep neural network-based method to perform smooth and robust 3D positioning tasks on a soft arm by visual servoing. A convolutional neural network is trained to predict the actuations required to achieve the desired pose. A proportional control law is implemented to reduce the error between the desired and current image as seen by the camera.
  arXiv  Detail & Related papers  (2022-02-10T18:03:28Z)
• Control Design of Autonomous Drone Using Deep Learning Based Image Understanding Techniques [1.0953917735844645]
  This paper presents a new framework to use images as the inputs for the controller to have autonomous flight, considering the noisy indoor environment and uncertainties. A new Proportional-Integral-Derivative-Accelerated (PIDA) control with a derivative filter is proposed to improve drone/quadcopter flight stability within a noisy environment.
  arXiv  Detail & Related papers  (2020-04-27T15:50:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.