A4-Agent: An Agentic Framework for Zero-Shot Affordance Reasoning

• URL: http://arxiv.org/abs/2512.14442v1
• Date: Tue, 16 Dec 2025 14:27:47 GMT
• Title: A4-Agent: An Agentic Framework for Zero-Shot Affordance Reasoning
• Authors: Zixin Zhang, Kanghao Chen, Hanqing Wang, Hongfei Zhang, Harold Haodong Chen, Chenfei Liao, Litao Guo, Ying-Cong Chen,
• Abstract summary: Affordance prediction, which identifies interaction regions on objects based on language instructions, is critical for embodied AI.<n>We propose A4-Agent, a training-free agentic framework that decouples affordance prediction into a three-stage pipeline.<n>Our framework significantly outperforms state-of-the-art supervised methods across multiple benchmarks.
• Score: 29.682282730123234
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Affordance prediction, which identifies interaction regions on objects based on language instructions, is critical for embodied AI. Prevailing end-to-end models couple high-level reasoning and low-level grounding into a single monolithic pipeline and rely on training over annotated datasets, which leads to poor generalization on novel objects and unseen environments. In this paper, we move beyond this paradigm by proposing A4-Agent, a training-free agentic framework that decouples affordance prediction into a three-stage pipeline. Our framework coordinates specialized foundation models at test time: (1) a $\textbf{Dreamer}$ that employs generative models to visualize $\textit{how}$ an interaction would look; (2) a $\textbf{Thinker}$ that utilizes large vision-language models to decide $\textit{what}$ object part to interact with; and (3) a $\textbf{Spotter}$ that orchestrates vision foundation models to precisely locate $\textit{where}$ the interaction area is. By leveraging the complementary strengths of pre-trained models without any task-specific fine-tuning, our zero-shot framework significantly outperforms state-of-the-art supervised methods across multiple benchmarks and demonstrates robust generalization to real-world settings.

Related papers

• One Model to Critique Them All: Rewarding Agentic Tool-Use via Efficient Reasoning [54.580646706013965]
  Reward models (RMs) play a critical role in aligning large language models with human preferences.<n>We introduce ToolRM, a family of lightweight generative RMs tailored for general tool-use scenarios.<n>To build these models, we propose a novel pipeline that constructs pairwise preference data using rule-based scoring and multidimensional sampling.
  arXiv  Detail & Related papers  (2025-10-30T06:08:27Z)
• VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents [130.70999337445468]
  Key challenge in training Vision-Language Model (VLM) agents, compared to Language Model (LLM) agents, is shift from textual states to complex visual observations.<n>We ask: Can VLM agents construct internal world models through explicit visual state reasoning?<n>We architecturally enforce and reward the agent's reasoning process via reinforcement learning (RL)<n>We find that the agent's reasoning into State Estimation and Transition Modeling is critical for success.
  arXiv  Detail & Related papers  (2025-10-19T16:05:07Z)
• FLARE: Robot Learning with Implicit World Modeling [87.81846091038676]
  $textbfFLARE$ integrates predictive latent world modeling into robot policy learning.<n>$textbfFLARE$ achieves state-of-the-art performance, outperforming prior policy learning baselines by up to 26%.<n>Our results establish $textbfFLARE$ as a general and scalable approach for combining implicit world modeling with high-frequency robotic control.
  arXiv  Detail & Related papers  (2025-05-21T15:33:27Z)
• GADS: A Super Lightweight Model for Head Pose Estimation [0.0]
  Grouped Attention Deep Sets (GADS) is a novel architecture based on the Deep Set framework.<n>By grouping landmarks into regions, we reduce computational complexity.<n>Our model is $7.5times$ smaller and executes $25times$ faster than the current lightest state-of-the-art model.
  arXiv  Detail & Related papers  (2025-04-22T09:53:25Z)
• IAAO: Interactive Affordance Learning for Articulated Objects in 3D Environments [56.85804719947]
  We present IAAO, a framework that builds an explicit 3D model for intelligent agents to gain understanding of articulated objects in their environment through interaction.<n>We first build hierarchical features and label fields for each object state using 3D Gaussian Splatting (3DGS) by distilling mask features and view-consistent labels from multi-view images.<n>We then perform object- and part-level queries on the 3D Gaussian primitives to identify static and articulated elements, estimating global transformations and local articulation parameters along with affordances.
  arXiv  Detail & Related papers  (2025-04-09T12:36:48Z)
• UniF$^2$ace: A Unified Fine-grained Face Understanding and Generation Model [62.66515621965686]
  We introduce a novel theoretical framework with a Dual Discrete Diffusion (D3Diff) loss, unifying masked generative models with discrete score matching diffusion.<n>This D3Diff significantly enhances the model's ability to synthesize high-fidelity facial details aligned with text input.<n>We construct UniF$2$aceD-1M, a large-scale dataset comprising 130K fine-grained image-caption pairs and 1M visual question-answering pairs.
  arXiv  Detail & Related papers  (2025-03-11T07:34:59Z)
• A Modern Take on Visual Relationship Reasoning for Grasp Planning [10.543168383800532]
  We present a modern take on visual relational reasoning for grasp planning.<n>We introduce D3GD, a novel testbed that includes bin picking scenes with up to 35 objects from 97 distinct categories.<n>We also propose D3G, a new end-to-end transformer-based dependency graph generation model.
  arXiv  Detail & Related papers  (2024-09-03T16:30:48Z)
• Robot Instance Segmentation with Few Annotations for Grasping [10.005879464111915]
  We propose a novel framework that combines Semi-Supervised Learning (SSL) with Learning Through Interaction (LTI)<n>Our approach exploits partially annotated data through self-supervision and incorporates temporal context using pseudo-sequences generated from unlabeled still images.<n>We validate our method on two common benchmarks, ARMBench mix-object-tote and OCID, where it achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-07-01T13:58:32Z)
• SM$^3$: Self-Supervised Multi-task Modeling with Multi-view 2D Images for Articulated Objects [24.737865259695006]
  We propose a self-supervised interaction perception method, referred to as SM$3$, to model articulated objects. By constructing 3D geometries and textures from the captured 2D images, SM$3$ achieves integrated optimization of movable part and joint parameters. Evaluations demonstrate that SM$3$ surpasses existing benchmarks across various categories and objects, while its adaptability in real-world scenarios has been thoroughly validated.
  arXiv  Detail & Related papers  (2024-01-17T11:15:09Z)
• FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects [55.77542145604758]
  FoundationPose is a unified foundation model for 6D object pose estimation and tracking. Our approach can be instantly applied at test-time to a novel object without fine-tuning.
  arXiv  Detail & Related papers  (2023-12-13T18:28:09Z)
• One-Shot Open Affordance Learning with Foundation Models [54.15857111929812]
  We introduce One-shot Open Affordance Learning (OOAL), where a model is trained with just one example per base object category. We propose a vision-language framework with simple and effective designs that boost the alignment between visual features and affordance text embeddings. Experiments on two affordance segmentation benchmarks show that the proposed method outperforms state-of-the-art models with less than 1% of the full training data.
  arXiv  Detail & Related papers  (2023-11-29T16:23:06Z)
• Unifying Language Learning Paradigms [96.35981503087567]
  We present a unified framework for pre-training models that are universally effective across datasets and setups. We show how different pre-training objectives can be cast as one another and how interpolating between different objectives can be effective. Our model also achieve strong results at in-context learning, outperforming 175B GPT-3 on zero-shot SuperGLUE and tripling the performance of T5-XXL on one-shot summarization.
  arXiv  Detail & Related papers  (2022-05-10T19:32:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.