More Structure, Not More Capacity: Object-Centric Representations for Visuomotor Imitation Learning

• URL: http://arxiv.org/abs/2607.09825v1
• Date: Fri, 10 Jul 2026 10:35:24 GMT
• Title: More Structure, Not More Capacity: Object-Centric Representations for Visuomotor Imitation Learning
• Abstract summary: We show that object-centric slot representations are a structured alternative to pre-trained vision models.<n>A dense patch grid with 16x the tokens performs no better than the global feature.<n>An explicit 2D spatial goal and native-resolution rendering raises the full system to 68.7$pm$4.2%, just below a privileged 3D-oracle upper bound.
• Score: 54.65906330923846
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robotic manipulation policies rely on pre-trained vision models that give either a global scene embedding or a dense patch grid. Both mix task-relevant and task-irrelevant features. Object-centric slot representations are a structured alternative: they group features into a few per-object slots. We test what this structure buys on ManiSkill3 PickCube-v1, with a frozen encoder and a held-out-seed evaluation. Holding the policy, goal token, rendering, and calibration fixed and changing only the encoder, a frozen object-centric SPOT representation (DINO ViT-B/16 + Slot Attention) reaches 55.0$\pm$2.9% success, 22.4% above a dense DINO global-feature baseline (32.6 $\pm$ 1.5%), with the same trainable policy and no encoder fine-tuning. More tokens alone do not help: a dense patch grid with 16x the tokens performs no better than the global feature. Adding an explicit 2D spatial goal and native-resolution rendering raises the full system to 68.7$\pm$4.2%, just below a privileged 3D-oracle upper bound (71.7$\pm$4.1%). An automated kinematic failure taxonomy then separates spatial-precision (Near-Miss) failures from object-tracking (No-Grasp) failures: spatial grounding reduces Near-Miss while leaving No- Grasp unchanged. The same taxonomy transfers to the harder StackCube-v1 and points to occlusion as the main bottleneck.

Related papers

• CoVeR: Coverage-Based Token Pruning for Multi-View 3D Reasoning in VLMs [58.22241532542095]
  Representing a 3D scene as multi-view images allows 2D VLMs to reason in 3D by reusing priors from pre-training, sidestepping the scarcity of annotated 3D data.<n>Existing visual token pruners fall into two families, each limited in the 3D multi-view setting.<n>We show that spatial coverage is associated with 3D reasoning performance and introduce CoVeR, a deterministic, training-free selector that uses only token coordinates, with no learned signals.
  arXiv  Detail & Related papers  (2026-09-08T07:16:37Z)
• Graph Machine: Towards Better Pretraining via Edges [0.0]
  Graph Machine (GM) architecture maintains a $O(n)$-sized state and accesses it through sparse, dynamic routing.<n>We replace 75% of the dense Transformer layers in Qwen3-0.6B with GM sparse layers and pretrain from scratch on 15.7B tokens.<n>With only 2 of 4,096 tokens retrieved per KV head in each sparse layer, loss degrades only slightly; with 4, the best model marginally improves loss.
  arXiv  Detail & Related papers  (2026-09-02T17:56:41Z)
• Putting Registers to Work: Task Registers for Token Pruning in Vision Transformers [5.410304374735817]
  Token-pruning policies are usually designed for a single recognition pipeline, but pretrained Vision Transformers are reused across tasks with different spatial demands.<n>We ask which parts of a pruning policy transfer across image classification, semantic segmentation, and object detection.<n>For each pipeline, controlled probes freeze the no-pruning checkpoint and apply a series of parameter-free reduction criteria at one eligible layer at a time without retraining.<n>The probes reveal three differences: segmentation and detection rank the criteria differently, classification is especially sensitive to attention-based pruning in the earliest layers, and the dense tasks prefer opposite recovery endpoints.
  arXiv  Detail & Related papers  (2026-08-11T14:43:52Z)
• Write-Protected Discrete Bottlenecks for Language-Grounded World Models: A Structural Limitation and Sufficient Fix [0.0]
  We characterize a sufficient set of three constraints that prevent the failure of a Gumbel-softmax-based discrete symbol bottleneck.<n>We demonstrate zero symbol collapse in all 32 seeds, with the blackboard achieving 79-100% semantic binding across three encoder architectures.
  arXiv  Detail & Related papers  (2026-07-09T09:55:17Z)
• Level-Crossing Density as a Mesh-Free High-Frequency Auxiliary Loss for Implicit Neural Representations [0.0]
  We derive smooth Monte-Carlo estimators for a continuous neural field using no grid, no complex, no persistence.<n>The estimator is accurate to 1-3% against exact topology in 2D and 3D, and costs about 3 ms per iteration.<n>In 3D neural-SDF fitting, however, a failure mode we believe general to any sampled soft topology objective appears.
  arXiv  Detail & Related papers  (2026-07-07T04:23:23Z)
• Sequential Planning via Anchored Robotic Keypoints [3.176338968032152]
  We present Sequential Planning via Anchored Robotic Keypoints, SPARK, a training-free neurosymbolic manipulation system.<n>A single Gemini call composes the plan as a typed behavior tree (BT) of composable kinematic primitives.<n>A second Gemini call proposes three alternative text prompts per object, SAM3 evaluates each, and we keep the prompt$to$label pair with the most confident detection.<n>The alternative prompts add +27.7 points on the spatial suite and +10.0 on the object suite, with the recovery loop adding +5.0 overall.
  arXiv  Detail & Related papers  (2026-06-29T17:48:01Z)
• Label-Free Cross-Task LoRA Merging with Null-Space Compression [50.63908869296697]
  We introduce Null-Space Compression (NSC) Merging, a label-free, output-agnostic method that sets merge weights from adapter geometry.<n>NSC achieves state-of-the-art performance across twenty heterogeneous vision tasks with balanced gains where prior methods overfit subsets of tasks.
  arXiv  Detail & Related papers  (2026-03-27T11:34:41Z)
• Learning Accurate Segmentation Purely from Self-Supervision [87.78965637247107]
  Selfment is a fully self-supervised framework that segments objects directly from raw images without human labels.<n>Selfment sets new state-of-the-art (SoTA) results across multiple benchmarks.
  arXiv  Detail & Related papers  (2026-02-27T07:36:32Z)
• $I^{2}$-World: Intra-Inter Tokenization for Efficient Dynamic 4D Scene Forecasting [2.722128680610171]
  $I2$-World is an efficient framework for 4D occupancy forecasting.<n>Our method decouples scene tokenization into intra-scene and inter-scene tokenizers.<n>$I2$-World achieves state-of-the-art performance, outperforming existing methods by 25.1% in mIoU and 36.9% in IoU for 4D occupancy forecasting.
  arXiv  Detail & Related papers  (2025-07-12T05:14:39Z)
• VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion [129.5975573092919]
  VoxFormer is a Transformer-based semantic scene completion framework. It can output complete 3D semantics from only 2D images. Our framework outperforms the state of the art with a relative improvement of 20.0% in geometry and 18.1% in semantics.
  arXiv  Detail & Related papers  (2023-02-23T18:59:36Z)
• UniFormer: Unifying Convolution and Self-attention for Visual Recognition [69.68907941116127]
  Convolution neural networks (CNNs) and vision transformers (ViTs) have been two dominant frameworks in the past few years. We propose a novel Unified transFormer (UniFormer) which seamlessly integrates the merits of convolution and self-attention in a concise transformer format. Our UniFormer achieves 86.3 top-1 accuracy on ImageNet-1K classification.
  arXiv  Detail & Related papers  (2022-01-24T04:39:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.