Bidirectional Semantic Complementary Tool Retrieval for Remote Sensing Agents

• URL: http://arxiv.org/abs/2606.07538v1
• Date: Wed, 29 Apr 2026 14:07:19 GMT
• Title: Bidirectional Semantic Complementary Tool Retrieval for Remote Sensing Agents
• Abstract summary: Large language model (LLM)-based agents provide a novel paradigm for the automated processing of remote sensing(RS) data.<n>Existing tool retrieval methods face "semantic asymmetry"<n>This paper proposes a bidirectional semantic complementary tool retrieval method.
• Score: 14.603837941754483
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM)-based agents provide a novel paradigm for the automated processing of remote sensing(RS) data. Their success in complex RS tasks rely on extensive specialized tool libraries. However, tool documentation often exceeds the context window limits of LLMs, making precise tool retrieval essential for agentic workflows. Existing tool retrieval methods face "semantic asymmetry" bottleneck: natural language queries typically express macro-level intentions lacking tool-specific semantics, while tool documentation provides fine-grained technical descriptions lacking operational context for workflows. To bridge this semantic gap, this paper proposes a bidirectional semantic complementary tool retrieval method. First, on the query side, we introduce a planning-based query enhancement mechanism that leverages the reasoning capabilities of agents to decompose abstract intentions into logical subtasks, thereby actively supplementing the query with missing functional semantics. Second, on the tool side, addressing the strong coupling characteristics of RS tool chains, we construct a dynamic tool dependency graph with continual learning capabilities. By employing a neighborhood information aggregation mechanism, contextual information from precursor tools is explicitly injected into the current node representation, enriching tool descriptions with contextual semantics. Experimental results on the RS dataset GeoPlan-bench and the general-purpose dataset API- Bank demonstrate that the proposed method not only significantly improves tool retrieval accuracy for complex RS tasks but also exhibits robust extensibility for transfer to general-domain tasks. The source code and dataset are available at https://github.com/geox-lab/BSCTR.

Related papers

• SING: Synthetic Intention Graph for Scalable Active Tool Discovery in LLM Agents [64.59100414726556]
  Large language model (LLM) agents increasingly rely on harnesses that manage context, tools, and multi-turn execution.<n>Retrieval-augmented tool selection offers a natural alternative, but existing one-shot retrieval methods fail to align isolated tool descriptions with the agent's true task intention.<n>We propose SING, an intention-aware active tool discovery framework that builds an intention-tool graph linking user intentions, tool capabilities, and tool collaboration patterns.
  arXiv  Detail & Related papers  (2026-06-15T11:37:37Z)
• RS-Claw: Progressive Active Tool Exploration via Hierarchical Skill Trees for Remote Sensing Agents [14.606770347521008]
  This paper argues that agents should act as an active explorer within the tool space.<n>By leveraging Skill encapsulation technology at the tool end, we propose RS-Claw.<n>This architecture hierarchically structures tool descriptions, enabling the agent to execute on-demand sequential decision-making.
  arXiv  Detail & Related papers  (2026-05-13T11:49:18Z)
• RaTA-Tool: Retrieval-based Tool Selection with Multimodal Large Language Models [57.15854852525046]
  We introduce RaTA-Tool, a novel framework for open-world multimodal tool selection.<n>Our approach enables an MLLM to convert a multimodal query into a structured task description and subsequently retrieve the most appropriate tool.<n>To further improve alignment between task descriptions and tool selection, we incorporate a preference-based optimization stage.
  arXiv  Detail & Related papers  (2026-04-16T12:47:09Z)
• ToolFlood: Beyond Selection -- Hiding Valid Tools from LLM Agents via Semantic Covering [2.6928305857508974]
  This paper introduces ToolFlood, a retrieval-layer attack on tool-augmented Large Language Model (LLM) agents.<n>Rather than altering which tool is chosen after retrieval, ToolFlood overwhelms retrieval itself by injecting a few attacker-controlled tools.<n>ToolFlood achieves up to a 95% attack success rate with a low injection rate.
  arXiv  Detail & Related papers  (2026-03-14T13:54:49Z)
• ToolLibGen: Scalable Automatic Tool Creation and Aggregation for LLM Reasoning [80.10274552177096]
  Large Language Models (LLMs) equipped with external tools have demonstrated enhanced performance on complex reasoning tasks.<n>The widespread adoption of this tool-augmented reasoning is hindered by the scarcity of domain-specific tools.<n>We propose a systematic approach to automatically an unstructured collection of tools into a structured tool library.
  arXiv  Detail & Related papers  (2025-10-09T04:11:16Z)
• MassTool: A Multi-Task Search-Based Tool Retrieval Framework for Large Language Models [45.63804847907601]
  MassTool is a multi-task search-based framework designed to enhance both query representation and tool retrieval accuracy.<n>It employs a two-tower architecture: a tool usage detection tower that predicts the need for function calls, and a tool retrieval tower that leverages a query-centric graph convolution network (QC-GCN) for effective query-tool matching.<n>By jointly optimizing tool usage detection loss, list-wise retrieval loss, and contrastive regularization loss, MassTool establishes a robust dual-step sequential decision-making pipeline for precise query understanding.
  arXiv  Detail & Related papers  (2025-07-01T07:02:26Z)
• ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks [64.86209459039313]
  ThinkGeo is an agentic benchmark designed to evaluate tool-augmented agents on remote sensing tasks via structured tool use and multi-step planning.<n>We implement a ReAct-style interaction loop and evaluate both open and closed-source LLMs on 486 structured agentic tasks with 1,773 expert-verified reasoning steps.<n>Our analysis reveals notable disparities in tool accuracy and planning consistency across models.
  arXiv  Detail & Related papers  (2025-05-29T17:59:38Z)
• Improving Tool Retrieval by Leveraging Large Language Models for Query Generation [16.7926347207647]
  In-context learning can provide a short list of relevant tools in the prompt.<n>We propose leveraging Large Language Models (LLMs) to generate a retrieval query.<n>The generated query is embedded and used to find the most relevant tools via a nearest-neighbor search.
  arXiv  Detail & Related papers  (2024-11-17T03:02:09Z)
• Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrieval [47.81307125613145]
  Re-Invoke is an unsupervised tool retrieval method designed to scale effectively to large toolsets without training. We employ a novel multi-view similarity ranking strategy based on intents to pinpoint the most relevant tools for each query. Our evaluation demonstrates that Re-Invoke significantly outperforms state-of-the-art alternatives in both single-tool and multi-tool scenarios.
  arXiv  Detail & Related papers  (2024-08-03T22:49:27Z)
• Towards Completeness-Oriented Tool Retrieval for Large Language Models [60.733557487886635]
  Real-world systems often incorporate a wide array of tools, making it impractical to input all tools into Large Language Models. Existing tool retrieval methods primarily focus on semantic matching between user queries and tool descriptions. We propose a novel modelagnostic COllaborative Learning-based Tool Retrieval approach, COLT, which captures not only the semantic similarities between user queries and tool descriptions but also takes into account the collaborative information of tools.
  arXiv  Detail & Related papers  (2024-05-25T06:41:23Z)
• EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction [56.02100384015907]
  EasyTool is a framework transforming diverse and lengthy tool documentation into a unified and concise tool instruction. It can significantly reduce token consumption and improve the performance of tool utilization in real-world scenarios.
  arXiv  Detail & Related papers  (2024-01-11T15:45:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.