Robot-DIFT: Distilling Diffusion Features for Geometrically Consistent Visuomotor Control

• URL: http://arxiv.org/abs/2602.11934v1
• Date: Thu, 12 Feb 2026 13:30:24 GMT
• Title: Robot-DIFT: Distilling Diffusion Features for Geometrically Consistent Visuomotor Control
• Abstract summary: A key bottleneck in generalizable robot manipulation is a structural mismatch between current visual backbones and the physical requirements of closed-loop control.<n>We propose Robot-DIFT, a framework that decouples the source of geometric information from the process of inference via Manifold Distillation.<n>By distilling a frozen diffusion teacher into a deterministic Spatial-Semantic Feature Pyramid Network (S2-FPN), we retain the rich geometric priors of the generative model while ensuring temporal stability, real-time execution, and robustness against drift.
• Score: 31.523644271297616
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We hypothesize that a key bottleneck in generalizable robot manipulation is not solely data scale or policy capacity, but a structural mismatch between current visual backbones and the physical requirements of closed-loop control. While state-of-the-art vision encoders (including those used in VLAs) optimize for semantic invariance to stabilize classification, manipulation typically demands geometric sensitivity the ability to map millimeter-level pose shifts to predictable feature changes. Their discriminative objective creates a "blind spot" for fine-grained control, whereas generative diffusion models inherently encode geometric dependencies within their latent manifolds, encouraging the preservation of dense multi-scale spatial structure. However, directly deploying stochastic diffusion features for control is hindered by stochastic instability, inference latency, and representation drift during fine-tuning. To bridge this gap, we propose Robot-DIFT, a framework that decouples the source of geometric information from the process of inference via Manifold Distillation. By distilling a frozen diffusion teacher into a deterministic Spatial-Semantic Feature Pyramid Network (S2-FPN), we retain the rich geometric priors of the generative model while ensuring temporal stability, real-time execution, and robustness against drift. Pretrained on the large-scale DROID dataset, Robot-DIFT demonstrates superior geometric consistency and control performance compared to leading discriminative baselines, supporting the view that how a model learns to see dictates how well it can learn to act.

Related papers

• DyG$^2$T: Modeling Object Dynamics with 3D Gaussian Temporal-Spatial Particle Graph Transformer [86.4618122245946]
  DyG$2$T is a dynamics modeling framework that infers object motion trajectories by spatially completing and temporally discriminating Key Point representations.<n>Experiments on both synthetic and real-world datasets demonstrate that DyG$2$T achieves accurate dynamics modeling and reasoning.
  arXiv  Detail & Related papers  (2026-08-19T03:44:28Z)
• Deformable Gaussian Occupancy: Decoupling Rigid and Nonrigid Motion with Factorized Distillation [81.44955493339835]
  DeGO is a deformable Gaussian occupancy framework that unifies decoupled Gaussian deformation with factorized 4D foundation-model distillation.<n> Experiments on the Occ3D-NuScenes benchmark demonstrate that our method achieves state-of-the-art performance under weak supervision.
  arXiv  Detail & Related papers  (2026-05-27T15:09:55Z)
• Stabilizing Streaming Video Geometry via Dynamic Feature Normalization [45.91399406203578]
  Dynamic Feature Normalization (DyFN) is a causal recurrent module that modulates feature statistics to maintain stable geometry over time.<n>DyFN achieves state-of-the-art temporal stability, improving over prior streaming methods by up to 14% and even outperforming heavier non-causal video baselines.
  arXiv  Detail & Related papers  (2026-05-25T00:13:15Z)
• ViPS: Video-informed Pose Spaces for Auto-Rigged Meshes [55.32681167870698]
  Video-informed Pose Spaces (ViPS) is a feed-forward framework that discovers the latent distribution of valid articulations for auto-rigged meshes.<n>ViPS transfers generative video priors into a universal distribution over a given rig parameterization.<n>Our evaluations show that ViPS matches the performance of state-of-the-art methods trained on synthetic artist-created 4D data in both plausibility and diversity.
  arXiv  Detail & Related papers  (2026-04-19T21:21:11Z)
• Sparse Autoencoders as a Steering Basis for Phase Synchronization in Graph-Based CFD Surrogates [6.396157607535812]
  Graph-based surrogate models provide fast alternatives to high-fidelity CFD solvers, but their opaque latent spaces and limited controllability restrict use in safety-critical settings.<n>We ask whether phase drift can instead be corrected post hoc by manipulating the latent space of a frozen surrogate.<n>We propose a phase-steering framework for pretrained graph-based CFD models that combines the right representation with the right intervention mechanism.
  arXiv  Detail & Related papers  (2026-03-28T07:03:44Z)
• Domain Elastic Transform: Bayesian Function Registration for High-Dimensional Scientific Data [8.674184859651135]
  We propose Domain Elastic Transform, a grid-free probabilistic framework that unifies geometric and functional alignment.<n>We formulate the problem within a rigorous Bayesian framework, modeling domain deformation as an elastic motion guided by a joint spatial-functional likelihood.<n>We demonstrate that DET achieves 92% topological preservation on MERFISH data where state-of-the-art optimal transport methods struggle.
  arXiv  Detail & Related papers  (2026-03-22T13:47:29Z)
• Adaptive Diffusion Posterior Sampling for Data and Model Fusion of Complex Nonlinear Dynamical Systems [1.9800280967139285]
  High-fidelity numerical simulations of chaotic, high dimensional nonlinear dynamical systems are computationally expensive.<n>Most surrogate models for such systems are deterministic, for example when neural operators are involved.<n>This work presents a surrogate modeling formulation that leverages generative machine learning.
  arXiv  Detail & Related papers  (2026-03-13T04:11:37Z)
• Shape Control of a Planar Hyper-Redundant Robot via Hybrid Kinematics-Informed and Learning-based Approach [42.55947704321053]
  Hyper-redundant robots offer high dexterity, making them good at operating in confined and unstructured environments.<n>To extend the reachable workspace, we built a multi-segment flexible rack actuated planar robot.<n>We propose a hybrid kinematics-informed and learning-based shape control method, named SpatioCoupledNet.
  arXiv  Detail & Related papers  (2026-03-11T04:35:06Z)
• ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learning [31.000965640377128]
  ABot-M0 is a framework that builds a systematic data curation pipeline.<n>It enables end-to-end transformation of heterogeneous raw data into unified, efficient representations.<n>ABot-M0 supports modular perception via a dual-stream mechanism.
  arXiv  Detail & Related papers  (2026-02-11T16:47:01Z)
• TRACER: Texture-Robust Affordance Chain-of-Thought for Deformable-Object Refinement [37.77903164878976]
  TRACER is a Texture-Robust Affordance Chain-of-thought with dEformable-object Refinement framework.<n>It provides a cross-hierarchical mapping from semantic reasoning to appearance-robust and physically consistent functional region refinement.<n>Experiments conducted on the Fine-AGDDO15 dataset and a real-world robotic platform demonstrate that TRACER significantly improves affordance grounding precision.
  arXiv  Detail & Related papers  (2026-01-28T03:12:18Z)
• SourceNet: Interpretable Sim-to-Real Inference on Variable-Geometry Sensor Arrays for Earthquake Source Inversion [7.936803576241796]
  SourceNet is a Transformer-based framework that treats the sensor array as a flexible set to model arbitrary geometries.<n>By pre-training on 100,000 synthetic events and fine-tuning on 2,000 real world events, SourceNet achieves state-of-the-art precision on held-out real data.
  arXiv  Detail & Related papers  (2026-01-09T21:42:28Z)
• Simulating Distribution Dynamics: Liquid Temporal Feature Evolution for Single-Domain Generalized Object Detection [58.25418970608328]
  Single-Domain Generalized Object Detection (Single-DGOD) aims to transfer a detector trained on one source domain to multiple unknown domains.<n>Existing methods for Single-DGOD typically rely on discrete data augmentation or static perturbation methods to expand data diversity.<n>We propose a new method, which simulates the progressive evolution of features from the source domain to simulated latent distributions.
  arXiv  Detail & Related papers  (2025-11-13T03:10:39Z)
• Registration is a Powerful Rotation-Invariance Learner for 3D Anomaly Detection [64.0168648353038]
  3D anomaly detection in point-cloud data is critical for industrial quality control, aiming to identify structural defects with high reliability.<n>Current memory bank-based methods often suffer from inconsistent feature transformations and limited discriminative capacity.<n>We propose a registration-induced, rotation-invariant feature extraction framework that integrates the objectives of point-cloud registration and memory-based anomaly detection.
  arXiv  Detail & Related papers  (2025-10-19T14:56:38Z)
• PointNSP: Autoregressive 3D Point Cloud Generation with Next-Scale Level-of-Detail Prediction [87.33016661440202]
  Autoregressive point cloud generation has long lagged behind diffusion-based approaches in quality.<n>We propose PointNSP, a coarse-to-fine generative framework that preserves global shape structure at low resolutions.<n> Experiments on ShapeNet show that PointNSP establishes state-of-the-art (SOTA) generation quality for the first time within the autoregressive paradigm.
  arXiv  Detail & Related papers  (2025-10-07T06:31:02Z)
• FreSca: Scaling in Frequency Space Enhances Diffusion Models [55.75504192166779]
  This paper explores frequency-based control within latent diffusion models.<n>We introduce FreSca, a novel framework that decomposes noise difference into low- and high-frequency components.<n>FreSca operates without any model retraining or architectural change, offering model- and task-agnostic control.
  arXiv  Detail & Related papers  (2025-04-02T22:03:11Z)
• Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers [59.0181939916084]
  Traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries. We propose a novel Priors Distillation (RPD) method to extract priors from the well-trained transformers on massive images. Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification.
  arXiv  Detail & Related papers  (2024-07-26T06:29:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.