Artificial Foveated Perception for Mitigating Shortcut Learning in Robotic Foundation Models

• URL: http://arxiv.org/abs/2607.10655v1
• Date: Sun, 12 Jul 2026 08:42:36 GMT
• Title: Artificial Foveated Perception for Mitigating Shortcut Learning in Robotic Foundation Models
• Abstract summary: We propose Artificial Foveated Perception (AFP), a lightweight, policy-agnostic module for robotic foundation models.<n>AFP predicts task-conditioned masks over relevant objects, the robot, and other action-critical regions.<n>We show AFP reduces fine-tuning time, suppresses overfitting, and improves generalization under environmental perturbations.
• Score: 28.23835152537988
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robotic foundation models have recently made substantial progress in multi-task capability, cross-embodiment transfer, and language-conditioned control. Yet robust deployment across diverse real-world settings remains difficult, in part because policies often fail to distinguish causally relevant visual structure from spurious scene-level correlations. We identify this failure mode as shortcut learning: the tendency to exploit predictive but non-causal correlations in the training distribution rather than the task-relevant visual evidence that determines successful action. Although shortcut learning has been extensively studied in computer vision and broader machine learning, its role in robotic foundation models remains comparatively underexplored. We propose Artificial Foveated Perception (AFP), a lightweight, policy-agnostic module that takes the same vision and language inputs as Vision-Language-Action and World Action Model pipelines and predicts task-conditioned masks over relevant objects, the robot, and other action-critical regions. We use these masks primarily as an auxiliary grounding signal during fine-tuning, aligning policy attention with task-relevant regions while leaving the core architecture unchanged. After fine-tuning, the policy executes on the original observation stream without requiring AFP in the control loop. We evaluate AFP across state-of-the-art robotic foundation models and show that foveated perception reduces fine-tuning time, suppresses overfitting, and improves generalization under environmental perturbations. Ablations over mask quality and grounding-loss design further show that these gains arise from directing policy learning toward task-relevant visual evidence. These results suggest that task-conditioned foveated perception is a practical mechanism for making robotic foundation models more robust, data-efficient, and scalable.

Related papers

• AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding [55.535374902204296]
  AffordanceVLA is a unified framework that introduces structured affordance forecasting as a task-oriented intermediate representation.<n>We show that AffordanceVLA achieves strong performance across diverse manipulation scenarios.
  arXiv  Detail & Related papers  (2026-06-04T13:28:51Z)
• PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations [66.94988600664574]
  We present textbfPRTS (textbfPrimitive textbfReasoning and textbfTasking textbfSystem), a VLA foundation model that reformulates pretraining through Goal-Conditioned Reinforcement Learning.
  arXiv  Detail & Related papers  (2026-04-30T06:14:02Z)
• Gaze-Regularized Vision-Language-Action Models for Robotic Manipulation [7.281396624646809]
  We introduce a gaze-regularized training framework that aligns VLA models' internal attention with human visual patterns.<n>When integrated into existing VLA architectures, our approach yields 4-12% improvements across manipulation benchmarks.
  arXiv  Detail & Related papers  (2026-03-24T13:50:28Z)
• Closed-Loop Verbal Reinforcement Learning for Task-Level Robotic Planning [1.6641413441634143]
  We propose a new Verbal Reinforcement Learning framework for interpretable task-level planning in mobile robotic systems.<n>The framework follows a closed-loop architecture that enables iterative policy improvement through interaction with the physical environment.<n>We validate the proposed framework on a real mobile robot performing a multi-stage manipulation and navigation task under execution uncertainty.
  arXiv  Detail & Related papers  (2026-03-23T16:28:36Z)
• ICLR: In-Context Imitation Learning with Visual Reasoning [7.622880558476133]
  In-context imitation learning enables robots to adapt to new tasks from a small number of demonstrations without additional training.<n>We present In-Context Imitation Learning with Visual Reasoning (ICLR), a novel framework that augments demonstration prompts with structured visual reasoning traces.
  arXiv  Detail & Related papers  (2026-03-08T08:40:05Z)
• Act, Think or Abstain: Complexity-Aware Adaptive Inference for Vision-Language-Action Models [7.802379200026965]
  We propose an adaptive framework that dynamically routes VLA execution based on the complexity of the perceived state.<n>Our approach transforms the VLA's vision-language backbone into an active detection tool by projecting latent embeddings into an ensemble of parametric and non-parametric estimators.
  arXiv  Detail & Related papers  (2026-03-05T13:14:41Z)
• I-Perceive: A Foundation Model for Active Perception with Language Instructions [41.67607728608853]
  We propose I-Perceive, a foundation model for active perception conditioned on natural language instructions.<n>I-Perceive predicts camera views that follow open-ended language instructions, based on image-based scene contexts.<n> Experiments demonstrate that I-Perceive significantly outperforms state-of-the-art VLMs in both prediction accuracy and instruction following of generated camera views.
  arXiv  Detail & Related papers  (2026-02-28T11:38:56Z)
• Reshaping Action Error Distributions for Reliable Vision-Language-Action Models [69.38615670891038]
  In robotic manipulation, vision-language-action (VLA) models have emerged as a promising paradigm for learning generalizable and scalable robot policies.<n>We focus on continuous-action VLA models and move beyond conventional MSE-based regression by reshaping action error distributions during training.<n>We evaluate our approaches across standard, few-shot, and noisy settings on multiple representative VLA architectures.
  arXiv  Detail & Related papers  (2026-02-04T05:37:09Z)
• Exploring Conditions for Diffusion models in Robotic Control [70.27711404291573]
  We explore leveraging pre-trained text-to-image diffusion models to obtain task-adaptive visual representations for robotic control.<n>We find that naively applying textual conditions yields minimal or even negative gains in control tasks.<n>We propose ORCA, which introduces learnable task prompts that adapt to the control environment and visual prompts that capture fine-grained, frame-specific details.
  arXiv  Detail & Related papers  (2025-10-17T10:24:14Z)
• Executable Analytic Concepts as the Missing Link Between VLM Insight and Precise Manipulation [70.8381970762877]
  Vision-Language Models (VLMs) have demonstrated remarkable capabilities in semantic reasoning and task planning.<n>We introduce GRACE, a novel framework that grounds VLM-based reasoning through executable analytic concepts.<n>G GRACE provides a unified and interpretable interface between high-level instruction understanding and low-level robot control.
  arXiv  Detail & Related papers  (2025-10-09T09:08:33Z)
• Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
  We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models.<n>Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning.<n>Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
  arXiv  Detail & Related papers  (2023-06-09T07:22:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.