Beyond Visual Grasping: Benchmarking Complex Grasping from Detection to Execution

• URL: http://arxiv.org/abs/2607.14341v1
• Date: Wed, 15 Jul 2026 20:10:09 GMT
• Title: Beyond Visual Grasping: Benchmarking Complex Grasping from Detection to Execution
• Abstract summary: We propose GCA-Bench, a benchmark featuring challenging textitgrasping with complex action scenarios.<n>We implement a diverse set of baselines, ranging from traditional grasp detection pipelines to end-to-end learning methods.<n>In addition, we propose new evaluation metrics, analyze critical failure models, and provide insights to guide the development of more robust and generalizable grasping strategies.
• Score: 20.75638062019513
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robust robotic grasping remains a fundamental challenge for complex real-world applications. Recent advances in large-scale models demonstrate promising capabilities for reasoning in robotic tasks. However, existing benchmarks for grasping primarily focus on isolated, visual-based grasp pose detection, failing to capture the complexity of grasping tasks that require multi-step reasoning and semantic understanding during execution. To address this gap, we propose GCA-Bench, a benchmark featuring challenging \textit{grasping with complex action} scenarios that involve both scene-level reasoning and semantic constraints. GCA-Bench enables the evaluation of recent large foundation models under the same settings. To demonstrate the effectiveness of our new benchmark, we implement a diverse set of baselines, ranging from traditional grasp detection pipelines to end-to-end learning methods. Empirical studies achieve success rates below 70\% on complex grasping scenarios, underscoring critical limitations. In addition, we propose new evaluation metrics, analyze critical failure models, and provide insights to guide the development of more robust and generalizable grasping strategies.

Related papers

• StAR: Segment Anything Reasoner [11.958150552719296]
  We present Segment Anything Reasoner (StAR), a comprehensive framework that refines the design space from multiple perspectives.<n>StAR achieves significant gains over its base counterparts across extensive benchmarks.
  arXiv  Detail & Related papers  (2026-03-15T13:43:34Z)
• The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearning [54.67958855362658]
  We propose a dynamic framework that stress tests unlearning using complex structured queries.<n>Our approach first elicits knowledge from the target model (pre-unlearning) and constructs targeted probes, ranging from simple queries to multi-hop chains.<n>Our framework enables practical and scalable evaluation of unlearning methods without the need for manual construction of forget test sets.
  arXiv  Detail & Related papers  (2026-03-11T19:51:33Z)
• Is This Tracker On? A Benchmark Protocol for Dynamic Tracking [6.23176842962524]
  ITTO is a new benchmark suite for evaluating and diagnosing the capabilities and limitations of point tracking methods.<n>We conduct a rigorous analysis of state-of-the-art tracking methods on ITTO, breaking down performance along key axes of motion complexity.
  arXiv  Detail & Related papers  (2025-10-22T17:53:56Z)
• Demystifying deep search: a holistic evaluation with hint-free multi-hop questions and factorised metrics [89.1999907891494]
  We present WebDetective, a benchmark of hint-free multi-hop questions paired with a controlled Wikipedia sandbox.<n>Our evaluation of 25 state-of-the-art models reveals systematic weaknesses across all architectures.<n>We develop an agentic workflow, EvidenceLoop, that explicitly targets the challenges our benchmark identifies.
  arXiv  Detail & Related papers  (2025-10-01T07:59:03Z)
• Overcoming Over-Fitting in Constraint Acquisition via Query-Driven Interactive Refinement [0.7874708385247353]
  Constraint Acquisition (CA) aims to automate manual modeling in Constraint Programming.<n> passive CA methods are prone to over-fitting, often learning models that include spurious global constraints when trained on limited data.<n>We introduce a hybrid CA framework specifically designed to address the challenge of over-fitting in CA.
  arXiv  Detail & Related papers  (2025-09-29T09:02:16Z)
• Rethinking Evaluation of Infrared Small Target Detection [105.59753496831739]
  This paper introduces a hybrid-level metric incorporating pixel- and target-level performance, proposing a systematic error analysis method, and emphasizing the importance of cross-dataset evaluation.<n>An open-source toolkit has be released to facilitate standardized benchmarking.
  arXiv  Detail & Related papers  (2025-09-21T02:45:07Z)
• GRADE: Generating multi-hop QA and fine-gRAined Difficulty matrix for RAG Evaluation [5.002953635224383]
  Retrieval-Augmented Generation (RAG) systems are widely adopted in knowledge-intensive NLP tasks.<n>Current evaluations often overlook the structural complexity and multi-step reasoning required in real-world scenarios.<n>We propose textscGRADE, a novel evaluation framework that models task difficulty along two dimensions.
  arXiv  Detail & Related papers  (2025-08-23T11:26:41Z)
• The Devil is in Fine-tuning and Long-tailed Problems:A New Benchmark for Scene Text Detection [2.4829769683482437]
  Scene text detection methods often fail to replicate such success in real-world scenarios.<n>We uncover two key factors contributing to this discrepancy through extensive experiments.<n>We propose a Long-Tailed Benchmark (LTB) to evaluate ability to handle a diverse range of long-tailed challenges.
  arXiv  Detail & Related papers  (2025-05-21T15:26:46Z)
• Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space [92.6187727249868]
  We introduce LatentSeek, a framework that enhances reasoning through Test-Time Instance-level Adaptation (TTIA) within the model's latent space.<n>LatentSeek is evaluated on a range of reasoning benchmarks, including GSM8K, MATH-500, and AIME2024.<n>Results show that LatentSeek consistently outperforms strong baselines.
  arXiv  Detail & Related papers  (2025-05-19T16:26:02Z)
• Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute [61.00662702026523]
  We propose a unified Test-Time Compute scaling framework that leverages increased inference-time instead of larger models.<n>Our framework incorporates two complementary strategies: internal TTC and external TTC.<n>We demonstrate our textbf32B model achieves a 46% issue resolution rate, surpassing significantly larger models such as DeepSeek R1 671B and OpenAI o1.
  arXiv  Detail & Related papers  (2025-03-31T07:31:32Z)
• Causality can systematically address the monsters under the bench(marks) [64.36592889550431]
  Benchmarks are plagued by various biases, artifacts, or leakage.<n>Models may behave unreliably due to poorly explored failure modes.<n> causality offers an ideal framework to systematically address these challenges.
  arXiv  Detail & Related papers  (2025-02-07T17:01:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.