ArtAnno: Annotating Implicit Semantics in Artworks through LLM Agent-Driven Bidirectional Human-AI Augmentation

• URL: http://arxiv.org/abs/2608.05026v1
• Date: Wed, 05 Aug 2026 16:30:12 GMT
• Title: ArtAnno: Annotating Implicit Semantics in Artworks through LLM Agent-Driven Bidirectional Human-AI Augmentation
• Abstract summary: Current AI-assisted annotation tools often lack assistance or rely on one-way calibrations to improve AI models.<n>We propose Bi Human-AI Augmentation, a closed-loop framework in which skills and domain knowledge base evolve through real-time interaction and bidirectional HAI augmentation.<n>We implement this framework in ArtAnno, an artwork annotation system driven by a multi-agent architecture.
• Score: 11.3983038688347
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-quality annotation of artworks is essential for computational art research, yet extracting implicit semantics remains challenging due to the reliance on culturally grounded meanings and deep contextual knowledge behind the images. Current AI-assisted annotation tools often lack assistance or rely on one-way workflows where experts have to perform extra manual calibrations to improve AI models, resulting in limited efficiency. To address this, we propose Bidirectional Human-AI Augmentation(BiHAA), a closed-loop framework in which skills and domain knowledge base evolve through real-time interaction and bidirectional HAI augmentation. Informed by a formative study with 20 artwork annotators from different backgrounds, we implement this framework in ArtAnno, an artwork annotation system driven by a multi-agent architecture. The system includes a Proactive Agentic Support Module, where AI augments humans through semantic mining and label suggestion, and an Interaction-Driven Evolution Module, where human expertise continuously enhances the AI through distilling annotation trajectories into reusable experience. Evaluation through a user study and two case studies demonstrates that our framework and system improve annotation efficiency, enable knowledge accumulation, and reduce the effort of information seeking and verification for annotators with limited domain expertise. We conclude by discussing broader implications and future directions.

Related papers

• Intentmaking and Sensemaking: Human Interaction with AI-Guided Mathematical Discovery [72.49593499512092]
  We identify and characterize a distinct workflow we term intentmaking.<n>We frame this as a natural extension to sensemaking.<n>Our documentation of these themes suggests an approach to designing AI tools for scientific discovery.
  arXiv  Detail & Related papers  (2026-05-07T09:30:25Z)
• AI to Learn 2.0: A Deliverable-Oriented Governance Framework and Maturity Rubric for Opaque AI in Learning-Intensive Domains [0.0]
  Generative AI is entering research, education, and professional work faster than current governance frameworks can specify how AI-assisted outputs should be judged.<n>This paper proposes AI to Learn 2.0, a deliverable-oriented governance framework for AI-assisted work.
  arXiv  Detail & Related papers  (2026-03-16T11:44:43Z)
• Artificial Intelligence Agents in Music Analysis: An Integrative Perspective Based on Two Use Cases [0.0]
  This paper presents an integrative review and experimental validation of artificial intelligence (AI) agents applied to music analysis and education.<n>We synthesize the historical evolution from rule-based models to contemporary approaches involving deep learning, multi-agent architectures, and retrieval-augmented generation frameworks.<n> Experimental results demonstrate that AI agents effectively enhance musical pattern recognition, compositional parameterization, and educational feedback.
  arXiv  Detail & Related papers  (2025-11-17T23:46:47Z)
• Executable Knowledge Graphs for Replicating AI Research [65.41207324831583]
  Executable Knowledge Graphs (xKG) is a modular and pluggable knowledge base that automatically integrates technical insights, code snippets, and domain-specific knowledge extracted from scientific literature.<n>Code will released at https://github.com/zjunlp/xKG.
  arXiv  Detail & Related papers  (2025-10-20T17:53:23Z)
• AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Models [78.08374249341514]
  The rapid development of AI-generated content (AIGC) has led to the misuse of AI-generated images (AIGI) in spreading misinformation.<n>We introduce a large-scale and comprehensive dataset, Holmes-Set, which includes an instruction-tuning dataset with explanations on whether images are AI-generated.<n>Our work introduces an efficient data annotation method called the Multi-Expert Jury, enhancing data generation through structured MLLM explanations and quality control.<n>In addition, we propose Holmes Pipeline, a meticulously designed three-stage training framework comprising visual expert pre-training, supervised fine-tuning, and direct preference optimization
  arXiv  Detail & Related papers  (2025-07-03T14:26:31Z)
• IXAII: An Interactive Explainable Artificial Intelligence Interface for Decision Support Systems [0.13654846342364302]
  IXAII offers explanations from four explainable AI methods: LIME, SHAP, Anchors, and DiCE.<n>Our prototype provides tailored views for five user groups and gives users agency over the explanations' content and format.
  arXiv  Detail & Related papers  (2025-06-26T14:28:13Z)
• When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration [79.69935257008467]
  We introduce Knowledge Integration and Transfer Evaluation (KITE), a conceptual and experimental framework for Human-AI knowledge transfer capabilities.<n>We conduct the first large-scale human study (N=118) explicitly designed to measure it.<n>In our two-phase setup, humans first ideate with an AI on problem-solving strategies, then independently implement solutions, isolating model explanations' influence on human understanding.
  arXiv  Detail & Related papers  (2025-06-05T20:48:16Z)
• Advancing AI Research Assistants with Expert-Involved Learning [84.30323604785646]
  Large language models (LLMs) and large multimodal models (LMMs) promise to accelerate biomedical discovery, yet their reliability remains unclear.<n>We introduce ARIEL (AI Research Assistant for Expert-in-the-Loop Learning), an open-source evaluation and optimization framework.<n>We find that state-of-the-art models generate fluent but incomplete summaries, whereas LMMs struggle with detailed visual reasoning.
  arXiv  Detail & Related papers  (2025-05-03T14:21:48Z)
• AI Readiness in Healthcare through Storytelling XAI [0.5120567378386615]
  We develop an approach that combines multi-task distillation with interpretability techniques to enable audience-centric explainability.<n>Our methods increase the trust of both the domain experts and the machine learning experts to enable a responsible AI.
  arXiv  Detail & Related papers  (2024-10-24T13:30:18Z)
• PADTHAI-MM: Principles-based Approach for Designing Trustworthy, Human-centered AI using MAST Methodology [5.215782336985273]
  The Multisource AI Scorecard Table (MAST) was designed to bridge the gap by offering a systematic, tradecraft-centered approach to evaluating AI-enabled decision support systems.<n>We introduce an iterative design framework called textitPrinciples-based Approach for Designing Trustworthy, Human-centered AI.<n>We demonstrate this framework in our development of the Reporting Assistant for Defense and Intelligence Tasks (READIT)
  arXiv  Detail & Related papers  (2024-01-24T23:15:44Z)
• Exploration with Principles for Diverse AI Supervision [88.61687950039662]
  Training large transformers using next-token prediction has given rise to groundbreaking advancements in AI. While this generative AI approach has produced impressive results, it heavily leans on human supervision. This strong reliance on human oversight poses a significant hurdle to the advancement of AI innovation. We propose a novel paradigm termed Exploratory AI (EAI) aimed at autonomously generating high-quality training data.
  arXiv  Detail & Related papers  (2023-10-13T07:03:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.