CArtBench: Evaluating Vision-Language Models on Chinese Art Understanding, Interpretation, and Authenticity

• URL: http://arxiv.org/abs/2604.11632v1
• Date: Mon, 13 Apr 2026 15:44:02 GMT
• Title: CArtBench: Evaluating Vision-Language Models on Chinese Art Understanding, Interpretation, and Authenticity
• Abstract summary: We introduce CARTBENCH, a museum-grounded benchmark for evaluating vision-language models (VLMs) on Chinese artworks.<n>CARTBENCH comprises four subtasks: CURATORQA for evidence-grounded recognition and reasoning, CATALOGCAPTION for structured four-section expert-style appreciation, RE INTERPRET for defensible reinterpretation with expert ratings, and CONNOISSEURPAIRS for diagnostic authenticity discrimination under visually similar confounds.
• Score: 46.5769105684013
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce CARTBENCH, a museum-grounded benchmark for evaluating vision-language models (VLMs) on Chinese artworks beyond short-form recognition and QA. CARTBENCH comprises four subtasks: CURATORQA for evidence-grounded recognition and reasoning, CATALOGCAPTION for structured four-section expert-style appreciation, REINTERPRET for defensible reinterpretation with expert ratings, and CONNOISSEURPAIRS for diagnostic authenticity discrimination under visually similar confounds. CARTBENCH is built by aligning image-bearing Palace Museum objects from Wikidata with authoritative catalog pages, spanning five art categories across multiple dynasties. Across nine representative VLMs, we find that high overall CURATORQA accuracy can mask sharp drops on hard evidence linking and style-to-period inference; long-form appreciation remains far from expert references; and authenticity-oriented diagnostic discrimination stays near chance, underscoring the difficulty of connoisseur-level reasoning for current models.

Related papers

• JieZi: A Large-Scale Expert-Audited Dataset and Benchmark for Ancient Chinese Character Exegesis [43.26094253985753]
  Ancient Chinese Character Exegesis (ACCE) is a vision-language question answering task that models the scholarly exegesis process.<n>ACCE is organized into four progressive levels: basic character identification, glyph-form analysis, meaning exegesis, and diachronic evolution analysis.<n>JieZi-Dataset is the first large-scale, expert-audited VQA training dataset for ACCE, comprising over 500K QA pairs.<n>JieZi-Bench is an evaluation benchmark aligned with the exegesis process, constructed and verified by human experts.
  arXiv  Detail & Related papers  (2026-08-12T07:30:00Z)
• CiQi-Agent: Aligning Vision, Tools and Aesthetics in Multimodal Agent for Cultural Reasoning on Chinese Porcelains [57.25643971175172]
  We introduce CiQi-Agent -- a domain-specific Porcelain Connoisseurship Agent for intelligent analysis of antique Chinese porcelain.<n>CiQi-Agent supports multi-image porcelain inputs and enables vision tool invocation and multimodal retrieval-augmented generation.<n>We construct a large-scale, expert-annotated dataset CiQi-VQA, comprising 29,596 porcelain specimens, 51,553 images, and 557,940 visual question--answering pairs.
  arXiv  Detail & Related papers  (2026-03-30T14:13:47Z)
• HanMoVLM: Large Vision-Language Models for Professional Artistic Painting Evaluation [58.199846050357074]
  We transform Large Vision-Language Models (VLMs) into experts capable of professional-grade painting evaluation in the Chinese Artistic Domain.<n>We introduce HanMo-Bench, a new dataset that features authentic auction-grade masterpieces and AI-generated works.<n>We show that the proposed HanMoVLM effectively bridges the gap, achieving a high consistency with professional experts and significantly improving the quality of Chinese Painting generation.
  arXiv  Detail & Related papers  (2026-03-11T14:21:59Z)
• DepthCharge: A Domain-Agnostic Framework for Measuring Depth-Dependent Knowledge in Large Language Models [51.56484100374058]
  Large Language Models appear competent when answering general questions but often fail when pushed into domain-specific details.<n>We present DepthCharge, a domain-agnostic framework that measures knowledge depth through three innovations.<n> adaptive probing that generates follow-up questions based on concepts the model actually mentions, on-demand fact verification from authoritative sources, and survival statistics with constant sample sizes at every depth level.
  arXiv  Detail & Related papers  (2026-03-05T20:49:11Z)
• Beyond Accuracy: Evaluating Grounded Visual Evidence in Thinking with Images [34.324634481264034]
  We propose ViEBench, a process-verifiable benchmark designed to evaluate faithful visual reasoning.<n>Comprising 200 multi-scenario high-resolution images with expert-annotated visual evidence, ViEBench categorizes tasks by difficulty into perception and reasoning dimensions.<n>Our experiments yield several interesting observations: (1) VLMs can sometimes produce correct final answers despite grounding on irrelevant regions, and (2) they may successfully locate the correct evidence but still fail to utilize it to reach accurate conclusions.
  arXiv  Detail & Related papers  (2026-01-14T07:25:15Z)
• VQArt-Bench: A semantically rich VQA Benchmark for Art and Cultural Heritage [0.0]
  VQArt-Bench is a large-scale Visual Question Answering benchmark for the cultural heritage domain.<n>It is constructed using a novel multi-agent pipeline where specialized agents collaborate to generate nuanced, validated, and linguistically diverse questions.<n>Our evaluation of 14 state-of-the-art MLLMs on this benchmark reveals significant limitations in current models.
  arXiv  Detail & Related papers  (2025-10-14T17:29:52Z)
• Accuracy is Not Agreement: Expert-Aligned Evaluation of Crash Narrative Classification Models [2.1797343876622097]
  This study investigates the relationship between deep learning (DL) model accuracy and expert agreement in classifying crash narratives.<n>We evaluate five DL models against expert labels and narratives, and extend the analysis to four large language models (LLMs)<n>Our findings reveal an inverse relationship: models with higher technical accuracy often show lower agreement with human experts, while LLMs demonstrate stronger expert alignment despite lower accuracy.
  arXiv  Detail & Related papers  (2025-04-17T16:29:08Z)
• Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editing [84.16442052968615]
  We introduce RISEBench, the first benchmark for evaluating Reasoning-Informed viSual Editing (RISE)<n>RISEBench focuses on four key reasoning categories: Temporal, Causal, Spatial, and Logical Reasoning.<n>We conduct experiments evaluating nine prominent visual editing models, comprising both open-source and proprietary models.
  arXiv  Detail & Related papers  (2025-04-03T17:59:56Z)
• Understanding Museum Exhibits using Vision-Language Reasoning [52.35301212718003]
  Museums serve as repositories of cultural heritage and historical artifacts from diverse epochs, civilizations, and regions.<n> domain-specific models are essential for interactive query resolution and gaining historical insights.<n>We collect and curate a large-scale dataset of 65M images and 200M question-answer pairs for exhibits from all around the world.
  arXiv  Detail & Related papers  (2024-12-02T10:54:31Z)
• A multi-modal vision-language model for generalizable annotation-free pathology localization [38.22221410663355]
  Existing deep learning models for defining pathology from clinical imaging data rely on expert annotations and lack capabilities in open clinical environments.<n>Here, we present a generalizable vision-language model for generalization-Free pathology localization (AFLoc)<n>The core strength of AFLoc is extensive multi-level semantic structure-based contrastive learning, which aligns medical concepts with abundant image features to adapt to the diverse expressions of pathologies without the reliance on expert image annotations.
  arXiv  Detail & Related papers  (2024-01-04T03:09:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.