Catalogue Grounded Multimodal Attribution for Museum Video under Resource and Regulatory Constraints

• URL: http://arxiv.org/abs/2603.11147v1
• Date: Wed, 11 Mar 2026 17:30:30 GMT
• Title: Catalogue Grounded Multimodal Attribution for Museum Video under Resource and Regulatory Constraints
• Abstract summary: We propose a catalogue-grounded multimodal attribution for museum AV content using an open, locally deployable video language model.<n>Early deployments on a painting catalogue suggest that this framework can improve AV archive discoverability while respecting resource constraints, data sovereignty, and emerging regulation.
• Score: 23.69662034253901
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Audiovisual (AV) archives in museums and galleries are growing rapidly, but much of this material remains effectively locked away because it lacks consistent, searchable metadata. Existing method for archiving requires extensive manual effort. We address this by automating the most labour intensive part of the workflow: catalogue style metadata curation for in gallery video, grounded in an existing collection database. Concretely, we propose catalogue-grounded multimodal attribution for museum AV content using an open, locally deployable video language model. We design a multi pass pipeline that (i) summarises artworks in a video, (ii) generates catalogue style descriptions and genre labels, and (iii) attempts to attribute title and artist via conservative similarity matching to the structured catalogue. Early deployments on a painting catalogue suggest that this framework can improve AV archive discoverability while respecting resource constraints, data sovereignty, and emerging regulation, offering a transferable template for application-driven machine learning in other high-stakes domains.

Related papers

• Lot Machine: Multimodal Lot Extraction from Auction Catalogs [3.5918066453136697]
  We propose a pipeline to automatically extract structured lot-level metadata from historical auction catalogs.<n>We evaluate Vision-Language Models (VLMs) under varying prompt strategies and constrained decoding frameworks.<n>This work demonstrates that a VLM-based pipeline can successfully unlock historical auction catalogs for large-scale automated analysis.
  arXiv  Detail & Related papers  (2026-08-31T09:44:15Z)
• LAION-BVD: A 10-Million-Hour Open Video Dataset for Multimodal Pre-training [71.18006624888662]
  We present LAION-BVD, a large-scale open video dataset for multimodal learning.<n>It contains 1.3B platform-specific video URLs collected from CommonCrawl.<n>We download 80M videos with a total duration of 10 million hours.
  arXiv  Detail & Related papers  (2026-08-25T17:34:19Z)
• ViLL-E: Video LLM Embeddings for Retrieval [55.66444095205989]
  ViLL-E (Video-LLM-Embed) is a unified VideoLLM architecture endowed with a novel embedding generation mechanism.<n>We train this model with a three-stage training methodology combining generative and contrastive learning.<n>Our model significantly improves temporal localization (on avg. 7% over other VideoLLMs) and video retrieval (up to 4% over dual encoder models)
  arXiv  Detail & Related papers  (2026-04-13T23:54:58Z)
• From Show Programmes to Data: Designing a Workflow to Make Performing Arts Ephemera Accessible Through Language Models [0.3331620034375478]
  We show how vision-language models can accurately parse and transcribe born-digital and digitised programmes.<n>We train a reasoning model (POntAvignon) using reinforcement learning with both formal and semantic rewards.<n>This approach enables automated RDF triple generation and supports alignment with existing knowledge graphs.
  arXiv  Detail & Related papers  (2025-12-08T11:27:10Z)
• A Reproducible Workflow for Scraping, Structuring, and Segmenting Legacy Archaeological Artifact Images [0.0]
  The case study focuses on the Lower Palaeolithic hand axe and biface collection curated by the Archaeology Data Service (ADS)<n>To address this, two open source tools were developed: a web scraping script that retrieves all record pages, extracts associated metadata, and downloads the available images while respecting ADS Terms of Use and ethical scraping guidelines.<n>The original images are not redistributed, and only derived products such as masks, outlines, and annotations are shared.
  arXiv  Detail & Related papers  (2025-11-27T14:29:05Z)
• WikiVideo: Article Generation from Multiple Videos [82.00241010200368]
  We introduce the task of creating a Wikipedia-style article from multiple videos about real-world events.<n>To close this gap, we introduce WikiVideo, a benchmark consisting of expert-written articles and densely annotated videos.<n>We propose Collaborative Article Generation (CAG), a novel interactive method for article creation from multiple videos.
  arXiv  Detail & Related papers  (2025-04-01T16:22:15Z)
• An archaeological Catalog Collection Method Based on Large Vision-Language Models [9.177297031425859]
  Archaeological catalogs, containing key elements such as artifact images, morphological descriptions, and excavation information, are essential for studying artifact evolution and cultural inheritance.<n>Existing Large Vision-Language Models and their derivative data collection methods face challenges in accurate image detection and modal matching.<n>We propose a novel archaeological catalog collection method based on Large Vision-Language Models that follows an approach comprising three modules: document localization, block comprehension and block matching.
  arXiv  Detail & Related papers  (2024-12-28T09:10:41Z)
• Openstory++: A Large-scale Dataset and Benchmark for Instance-aware Open-domain Visual Storytelling [81.69474860607542]
  We present Openstory++, a large-scale dataset combining additional instance-level annotations with both images and text. We also present Cohere-Bench, a pioneering benchmark framework for evaluating the image generation tasks when long multimodal context is provided.
  arXiv  Detail & Related papers  (2024-08-07T11:20:37Z)
• EUFCC-340K: A Faceted Hierarchical Dataset for Metadata Annotation in GLAM Collections [6.723689308768857]
  The EUFCC340K dataset is organized across multiple facets: Materials, Object Types, Disciplines, and Subjects, following a hierarchical structure based on the Art & Architecture Thesaurus (AAT) Our experiments to evaluate model robustness and generalization capabilities in two different test scenarios demonstrate the utility of the dataset in improving multi-label classification tools.
  arXiv  Detail & Related papers  (2024-06-04T14:57:56Z)
• VidLA: Video-Language Alignment at Scale [48.665918882615195]
  We propose VidLA, an approach for video-language alignment at scale. Our proposed approach surpasses state-of-the-art methods on multiple retrieval benchmarks.
  arXiv  Detail & Related papers  (2024-03-21T22:36:24Z)
• UnLoc: A Unified Framework for Video Localization Tasks [82.59118972890262]
  UnLoc is a new approach for temporal localization in untrimmed videos. It uses pretrained image and text towers, and feeds tokens to a video-text fusion model. We achieve state of the art results on all three different localization tasks with a unified approach.
  arXiv  Detail & Related papers  (2023-08-21T22:15:20Z)
• DocumentNet: Bridging the Data Gap in Document Pre-Training [78.01647768018485]
  We propose a method to collect massive-scale and weakly labeled data from the web to benefit the training of VDER models. The collected dataset, named DocumentNet, does not depend on specific document types or entity sets. Experiments on a set of broadly adopted VDER tasks show significant improvements when DocumentNet is incorporated into the pre-training.
  arXiv  Detail & Related papers  (2023-06-15T08:21:15Z)
• On-Device Document Classification using multimodal features [3.180188886143297]
  We show that a single modality is insufficient for classification and present a novel pipeline to classify documents on-device. We integrate an open-source library for Optical Character Recognition (OCR) and our novel model architecture in the pipeline. We benchmark our classification model with a standard multimodal dataset FOOD-101 and showcase competitive results with the previous State of the Art with 30% model compression.
  arXiv  Detail & Related papers  (2021-01-06T05:36:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.