FIKA-Bench: From Fine-grained Recognition to Fine-Grained Knowledge Acquisition

• URL: http://arxiv.org/abs/2605.13193v2
• Date: Tue, 19 May 2026 13:19:30 GMT
• Title: FIKA-Bench: From Fine-grained Recognition to Fine-Grained Knowledge Acquisition
• Abstract summary: Fine-grained recognition in everyday life is often not a closed-book classification problem.<n>Existing benchmarks primarily evaluate visually recognition, leaving this active external knowledge acquisition ability underexplored.<n>We study fine-grained knowledge acquisition, where a system must seek, verify, and use external evidence to answer open-ended fine-grained recognition questions.
• Score: 54.31138496553705
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fine-grained recognition in everyday life is often not a closed-book classification problem: when encountering unfamiliar objects, humans actively search, compare visual details, and verify evidence before deciding. Existing benchmarks primarily evaluate visually recognition, leaving this active external knowledge acquisition ability underexplored. We study fine-grained knowledge acquisition, where a system must seek, verify, and use external evidence to answer open-ended fine-grained recognition questions. We introduce FIKA-Bench, a leakage-aware and evidence-grounded collection of 311 public-source and real-life instances. To ensure high quality, every example is filtered against frontier closed-book models to remove memorized cases and audited to eliminate image-answer leakage, retaining only samples supported by verified evidence. Our evaluation of latest Large Multimodal Models (LMMs) and agents reveals that the task remains a formidable challenge: the best system reaches only 25.1% accuracy, with no model exceeding 30%. Crucially, we find that merely equipping models with tools is insufficient to bridge this gap; agent failures are predominantly driven by wrong entity retrieval and poor visual judgement. These results show that reliable knowledge acquisition needs better agent designs that focus on fine-grained recognition.

Related papers

• Does It Fail to See or Fail to Know? Attributing Errors in Vision-Language Models [33.564453692157095]
  Uncertainty should not only indicate whether the model is likely to fail but also diagnose why it is uncertain.<n>We propose a unified framework for disentangling failure modes and investigate whether pre-generation signals can predict these failure sources.<n>Our main finding is that recognition-related failures are best captured by visual-token representations, while failures that remain after recognition are better captured by prompt-conditioned hidden states.
  arXiv  Detail & Related papers  (2026-07-06T05:11:45Z)
• Advancing Creative Physical Intelligence in Large Multimodal Models [62.56522271769017]
  We introduce MM-CreativityBench, a benchmark for affordance-grounded creative tool use in visually rich, physically constrained environments.<n>Our experiments show that current LMMs often fall short, not due to lack of generative capability, but because they do not sustain grounded exploration.<n>Motivated by this failure mode, we propose affordance-grounded alignment, which casts creative tool use as a preference learning problem.
  arXiv  Detail & Related papers  (2026-05-25T23:59:02Z)
• Seamless Deception: Larger Language Models Are Better Knowledge Concealers [26.38326875955904]
  Language Models (LMs) may acquire harmful knowledge, and feign ignorance of these topics when under audit.<n>Inspired by the recent discovery of deception-related behaviour patterns in LMs, we aim to train classifiers that detect when a LM is actively concealing knowledge.
  arXiv  Detail & Related papers  (2026-03-15T23:55:21Z)
• Demystifying deep search: a holistic evaluation with hint-free multi-hop questions and factorised metrics [89.1999907891494]
  We present WebDetective, a benchmark of hint-free multi-hop questions paired with a controlled Wikipedia sandbox.<n>Our evaluation of 25 state-of-the-art models reveals systematic weaknesses across all architectures.<n>We develop an agentic workflow, EvidenceLoop, that explicitly targets the challenges our benchmark identifies.
  arXiv  Detail & Related papers  (2025-10-01T07:59:03Z)
• ADSeeker: A Knowledge-Infused Framework for Anomaly Detection and Reasoning [17.249025173985697]
  We propose ADSeeker, an anomaly task assistant designed to enhance inspection performance through knowledge-grounded reasoning.<n>To tackle the challenge of limited in industry anomaly detection (IAD) data, we introduce the largest-scale AD dataset, Multi-type Anomaly (MulA)<n>Our plug-and-play framework, ADSeeker, achieves state-of-the-art zero-shot performance on several benchmark datasets.
  arXiv  Detail & Related papers  (2025-08-05T05:05:06Z)
• Forensics-Bench: A Comprehensive Forgery Detection Benchmark Suite for Large Vision Language Models [53.55128042938329]
  Forensics-Bench is a new forgery detection evaluation benchmark suite.<n>It comprises 63,292 meticulously curated multi-choice visual questions, covering 112 unique forgery detection types.<n>We conduct thorough evaluations on 22 open-sourced LVLMs and 3 proprietary models GPT-4o, Gemini 1.5 Pro, and Claude 3.5 Sonnet.
  arXiv  Detail & Related papers  (2025-03-19T09:21:44Z)
• AttributionBench: How Hard is Automatic Attribution Evaluation? [19.872081697282002]
  We present AttributionBench, a comprehensive benchmark compiled from various existing attribution datasets. Our experiments show that even a fine-tuned GPT-3.5 only achieves around 80% macro-F1 under a binary classification formulation. A detailed analysis of more than 300 error cases indicates that a majority of failures stem from the model's inability to process nuanced information.
  arXiv  Detail & Related papers  (2024-02-23T04:23:33Z)
• Free Lunch for Generating Effective Outlier Supervision [46.37464572099351]
  We propose an ultra-effective method to generate near-realistic outlier supervision. Our proposed textttBayesAug significantly reduces the false positive rate over 12.50% compared with the previous schemes.
  arXiv  Detail & Related papers  (2023-01-17T01:46:45Z)
• Evidence-based Match-status-Aware Gait Recognition for Out-of-Gallery Gait Identification [31.287312613939203]
  We propose an Evidence-based Match-status-Aware Gait Recognition framework. Inspired by Evidential Deep Learning (EDL), EMA-GR is designed to quantify the uncertainty associated with the match status of recognition. Our method surpasses existing state-of-the-art methods by a substantial margin.
  arXiv  Detail & Related papers  (2022-11-15T09:42:07Z)
• A Unified End-to-End Retriever-Reader Framework for Knowledge-based VQA [67.75989848202343]
  This paper presents a unified end-to-end retriever-reader framework towards knowledge-based VQA. We shed light on the multi-modal implicit knowledge from vision-language pre-training models to mine its potential in knowledge reasoning. Our scheme is able to not only provide guidance for knowledge retrieval, but also drop these instances potentially error-prone towards question answering.
  arXiv  Detail & Related papers  (2022-06-30T02:35:04Z)
• The Familiarity Hypothesis: Explaining the Behavior of Deep Open Set Methods [86.39044549664189]
  Anomaly detection algorithms for feature-vector data identify anomalies as outliers, but outlier detection has not worked well in deep learning. This paper proposes the Familiarity Hypothesis that these methods succeed because they are detecting the absence of familiar learned features rather than the presence of novelty. The paper concludes with a discussion of whether familiarity detection is an inevitable consequence of representation learning.
  arXiv  Detail & Related papers  (2022-03-04T18:32:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.