SPARC: Reliable Spatial Annotations from Robot Demonstrations at Scale

• URL: http://arxiv.org/abs/2606.13497v1
• Date: Thu, 11 Jun 2026 15:46:28 GMT
• Title: SPARC: Reliable Spatial Annotations from Robot Demonstrations at Scale
• Abstract summary: This work introduces spatial temporals from Robot Demonstrations with Reliability (SPARC), a risk-aware framework that automatically labels robot demonstrations with structured annotations.<n> Structured spatial annotations, such as bounding boxes, object trajectories, and manipulation phase labels, benefit a broad range of robotics applications.
• Score: 7.084784996380857
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work introduces Spatial Annotations from Robot Demonstrations with Reliability Calibration (SPARC), a risk-aware framework that automatically labels robot demonstrations with structured spatial annotations and assigns each annotation a reliability score. Structured spatial annotations, such as bounding boxes, object trajectories, and manipulation phase labels, benefit a broad range of robotics applications from training grounded robot policies and embodied foundation models to motion planning and hierarchical task composition. Existing automated pipelines generate such annotations at scale but provide no reliable quality signal: detector confidence is poorly calibrated for annotation correctness, forcing a choice between accepting noisy labels or discarding useful samples. In contrast to existing automated pipelines, SPARC leverages the spatio-temporal structure inherent to robot tasks to generate a reliability signal, reducing noisy labels and retaining more useful samples. We further introduce Interaction-Aware Bench (IA-Bench), a benchmark that measures model accuracy in grounding the locations of interacted objects in robot demonstrations. On 1.7k human-annotated demonstrations spanning diverse embodiments and scenarios, SPARC significantly outperforms detection-only baselines in localization accuracy while retaining three times more samples at high-precision operating points. Our experiments demonstrate that models finetuned on our annotations achieve state-of-the-art results on object-grounding and pointing benchmarks among similarly sized models, while remaining competitive on broader spatial-reasoning suites without manually verified or annotated training data. Furthermore, policies trained on SPARC-generated annotations outperform baselines in cluttered, visually ambiguous real-world scenes. Code, data, and models are available at intuitive-robots.github.io/sparc-labeling.

Related papers

• Learning Multi-Modal Trajectory Policies for Data-Efficient Robotic Manipulation [21.99077052779724]
  MATE (Multi-Modal MoE) is a novel trajectory prediction framework built upon MoE.<n>We introduce a Multi-Modal MoE architecture to achieve fine-grained sub-modal cosine, and design a cross-modal router for stable and scale-invariant expert assignment.<n>Experiments on robotic ping-pong show that our MATE consistently outperforms prior work under data scarcity.
  arXiv  Detail & Related papers  (2026-05-31T06:35:10Z)
• TapSampling: Inference-Time Sampling with a Task-Progress-Understanding Verifier for Robotic Manipulation [61.35569005726248]
  Existing embodied control research demonstrates remarkable performance improvements by scaling training data and model size.<n>Non-deterministic generative models, such as diffusion and autoregressive models, have been widely adopted in the field of embodied control.<n>We propose textbfTapSampling, a plug-and-play framework for inference-time sampling.
  arXiv  Detail & Related papers  (2026-05-25T08:03:31Z)
• Instance-Guided Unsupervised Domain Adaptation for Robotic Semantic Segmentation [4.556831252263135]
  Unsupervised Domain Adaptation (UDA) adapts the network to the robot's target environment without external supervision.<n>We propose a method that starts from a 3D map to generate multi-view consistent pseudo-labels.<n>We refine these labels using the zero-shot instance segmentation capabilities of a foundation model, enforcing instance-level coherence.
  arXiv  Detail & Related papers  (2026-02-01T18:49:03Z)
• About an Automating Annotation Method for Robot Markers [0.0]
  This paper proposes an automated annotation method for training deep-learning models on ArUco marker images.<n>A YOLO-based model is trained using the automatically annotated dataset, and its performance is evaluated under various conditions.
  arXiv  Detail & Related papers  (2026-01-30T13:44:56Z)
• Enhancing Egocentric Object Detection in Static Environments using Graph-based Spatial Anomaly Detection and Correction [0.0]
  We propose a graph-based post-processing pipeline that explicitly models the spatial relationships between objects to correct detection anomalies in egocentric frames.<n>Using a graph neural network (GNN) trained on manually annotated data, our model identifies invalid object class labels and predicts corrected class labels based on their neighbourhood context.<n>Experiments demonstrate that incorporating this spatial reasoning significantly improves detection performance, with mAP@50 gains of up to 4%.
  arXiv  Detail & Related papers  (2025-08-11T05:08:02Z)
• Self-supervised Feature Extraction for Enhanced Ball Detection on Soccer Robots [2.111102681327218]
  We present a self-supervised learning framework for domain-adaptive feature extraction to enhance ball detection performance.<n>The proposed approach leverages a general-purpose pretrained model to generate pseudo-labels, which are then used in a suite of self-supervised pretext tasks.<n> Experimental results demonstrate that the proposed pipeline outperforms baseline models in terms of accuracy, F1 score, and IoU, while also exhibiting faster convergence.
  arXiv  Detail & Related papers  (2025-06-20T08:21:34Z)
• PPT: Pretraining with Pseudo-Labeled Trajectories for Motion Forecasting [90.47748423913369]
  State-of-the-art motion forecasting models rely on large curated datasets with manually annotated or heavily post-processed trajectories.<n>PWT is a simple and scalable alternative that uses unprocessed and diverse trajectories automatically generated from off-the-shelf 3D detectors and tracking.<n>It achieves strong performance across standard benchmarks particularly in low-data regimes, and in cross-domain, end-to-end and multi-class settings.
  arXiv  Detail & Related papers  (2024-12-09T13:48:15Z)
• Keypoint Abstraction using Large Models for Object-Relative Imitation Learning [78.92043196054071]
  Generalization to novel object configurations and instances across diverse tasks and environments is a critical challenge in robotics. Keypoint-based representations have been proven effective as a succinct representation for essential object capturing features. We propose KALM, a framework that leverages large pre-trained vision-language models to automatically generate task-relevant and cross-instance consistent keypoints.
  arXiv  Detail & Related papers  (2024-10-30T17:37:31Z)
• Why Sample Space Matters: Keyframe Sampling Optimization for LiDAR-based Place Recognition [6.468510459310326]
  We introduce the concept of sample space and propose a novel sampling approach for LiDAR-based place recognition.<n>Our approach demonstrates robust performance across diverse datasets, with the ability to adapt seamlessly from indoor to outdoor scenarios.
  arXiv  Detail & Related papers  (2024-10-03T16:29:47Z)
• Downstream-Pretext Domain Knowledge Traceback for Active Learning [138.02530777915362]
  We propose a downstream-pretext domain knowledge traceback (DOKT) method that traces the data interactions of downstream knowledge and pre-training guidance. DOKT consists of a traceback diversity indicator and a domain-based uncertainty estimator. Experiments conducted on ten datasets show that our model outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-20T01:34:13Z)
• Semi-supervised Open-World Object Detection [74.95267079505145]
  We introduce a more realistic formulation, named semi-supervised open-world detection (SS-OWOD) We demonstrate that the performance of the state-of-the-art OWOD detector dramatically deteriorates in the proposed SS-OWOD setting. Our experiments on 4 datasets including MS COCO, PASCAL, Objects365 and DOTA demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-02-25T07:12:51Z)
• Improving a Named Entity Recognizer Trained on Noisy Data with a Few Clean Instances [55.37242480995541]
  We propose to denoise noisy NER data with guidance from a small set of clean instances. Along with the main NER model we train a discriminator model and use its outputs to recalibrate the sample weights. Results on public crowdsourcing and distant supervision datasets show that the proposed method can consistently improve performance with a small guidance set.
  arXiv  Detail & Related papers  (2023-10-25T17:23:37Z)
• Be Your Own Neighborhood: Detecting Adversarial Example by the Neighborhood Relations Built on Self-Supervised Learning [64.78972193105443]
  This paper presents a novel AE detection framework, named trustworthy for predictions. performs the detection by distinguishing the AE's abnormal relation with its augmented versions. An off-the-shelf Self-Supervised Learning (SSL) model is used to extract the representation and predict the label.
  arXiv  Detail & Related papers  (2022-08-31T08:18:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.