VideoAVE: A Multi-Attribute Video-to-Text Attribute Value Extraction Dataset and Benchmark Models

• URL: http://arxiv.org/abs/2508.11801v1
• Date: Fri, 15 Aug 2025 20:58:47 GMT
• Title: VideoAVE: A Multi-Attribute Video-to-Text Attribute Value Extraction Dataset and Benchmark Models
• Abstract summary: VideoAVE is the first publicly available video-to-text e-commerce AVE dataset across 14 different domains and covering 172 unique attributes.<n>To ensure data quality, we propose a post-hoc CLIP-based Mixture of Experts filtering system (CLIP-MoE)<n>Our results analysis reveals that video-to-text AVE remains a challenging problem, particularly in open settings.
• Score: 7.632869516436505
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Attribute Value Extraction (AVE) is important for structuring product information in e-commerce. However, existing AVE datasets are primarily limited to text-to-text or image-to-text settings, lacking support for product videos, diverse attribute coverage, and public availability. To address these gaps, we introduce VideoAVE, the first publicly available video-to-text e-commerce AVE dataset across 14 different domains and covering 172 unique attributes. To ensure data quality, we propose a post-hoc CLIP-based Mixture of Experts filtering system (CLIP-MoE) to remove the mismatched video-product pairs, resulting in a refined dataset of 224k training data and 25k evaluation data. In order to evaluate the usability of the dataset, we further establish a comprehensive benchmark by evaluating several state-of-the-art video vision language models (VLMs) under both attribute-conditioned value prediction and open attribute-value pair extraction tasks. Our results analysis reveals that video-to-text AVE remains a challenging problem, particularly in open settings, and there is still room for developing more advanced VLMs capable of leveraging effective temporal information. The dataset and benchmark code for VideoAVE are available at: https://github.com/gjiaying/VideoAVE

Related papers

• Visual Search Augmented Chain-of-Thought Reasoning for Attribute Value Extraction from Product Videos [14.43540077402288]
  We propose visual search augmented chain-of-thought reasoning (ViS-CoT) for visual attribute value extraction.<n>ViS-CoT can be easily applied to any open-source video VLM for video-to-text AVE in e-Commerce.
  arXiv  Detail & Related papers  (2026-09-06T06:01:57Z)
• ViMix-14M: A Curated Multi-Source Video-Text Dataset with Long-Form, High-Quality Captions and Crawl-Free Access [16.89068730775312]
  ViMix-14M is a curated multi-source video-text dataset of around 14 million pairs.<n> ViMix-14M is built by merging diverse open video sources, followed by unified de-duplication and quality filtering.<n>We evaluate the dataset by multimodal retrieval, text-to-video generation, and video question answering tasks.
  arXiv  Detail & Related papers  (2025-11-23T10:19:56Z)
• DocVideoQA: Towards Comprehensive Understanding of Document-Centric Videos through Question Answering [13.466266412068475]
  We introduce the DocVideoQA task and dataset for the first time, comprising 1454 videos across 23 categories with a total duration of about 828 hours.<n>The dataset is annotated with 154k question-answer pairs generated manually and via GPT, assessing models' comprehension, temporal awareness, and modality integration capabilities.<n>Our method enhances unimodal feature extraction with diverse instruction-tuning data and employs contrastive learning to strengthen modality integration.
  arXiv  Detail & Related papers  (2025-03-20T06:21:25Z)
• Leveraging Vision-Language Models for Open-Vocabulary Instance Segmentation and Tracking [15.551049337773962]
  Vision-language models (VLMs) excel in visual understanding but often lack reliable grounding capabilities and actionable inference rates.<n>We utilize VLM-generated structured descriptions to identify visible object instances, collect application-relevant attributes, and inform an open-vocabulary detector to extract corresponding bounding boxes.<n>Tracks can be updated online as needed by generating new structured descriptions and detections.
  arXiv  Detail & Related papers  (2025-03-18T20:18:42Z)
• LLaVA-Video: Video Instruction Tuning With Synthetic Data [84.64519990333406]
  We create a high-quality synthetic dataset specifically for video instruction-following, namely LLaVA-Video-178K.<n>This dataset includes key tasks such as detailed captioning, open-ended question-answering (QA), and multiple-choice QA.<n>By training on this dataset, in combination with existing visual instruction tuning data, we introduce LLaVA-Video, a new video LMM.
  arXiv  Detail & Related papers  (2024-10-03T17:36:49Z)
• Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs [20.168429351519055]
  Video understanding is a crucial next step for multimodal large language models (LMLMs)<n>We propose VideoNIAH (Video Needle In A Haystack), a benchmark construction framework through synthetic video generation.<n>We conduct a comprehensive evaluation of both proprietary and open-source models, uncovering significant differences in their video understanding capabilities.
  arXiv  Detail & Related papers  (2024-06-13T17:50:05Z)
• Spatio-Temporal Side Tuning Pre-trained Foundation Models for Video-based Pedestrian Attribute Recognition [58.79807861739438]
  Existing pedestrian recognition (PAR) algorithms are mainly developed based on a static image. We propose to understand human attributes using video frames that can fully use temporal information.
  arXiv  Detail & Related papers  (2024-04-27T14:43:32Z)
• EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLM [52.016009472409166]
  EIVEN is a data- and parameter-efficient generative framework for implicit attribute value extraction. We introduce a novel Learning-by-Comparison technique to reduce model confusion. Our experiments reveal that EIVEN significantly outperforms existing methods in extracting implicit attribute values.
  arXiv  Detail & Related papers  (2024-04-13T03:15:56Z)
• JPAVE: A Generation and Classification-based Model for Joint Product Attribute Prediction and Value Extraction [59.94977231327573]
  We propose a multi-task learning model with value generation/classification and attribute prediction called JPAVE. Two variants of our model are designed for open-world and closed-world scenarios. Experimental results on a public dataset demonstrate the superiority of our model compared with strong baselines.
  arXiv  Detail & Related papers  (2023-11-07T18:36:16Z)
• InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation [90.71796406228265]
  InternVid is a large-scale video-centric multimodal dataset that enables learning powerful and transferable video-text representations. The InternVid dataset contains over 7 million videos lasting nearly 760K hours, yielding 234M video clips accompanied by detailed descriptions of total 4.1B words.
  arXiv  Detail & Related papers  (2023-07-13T17:58:32Z)
• MAVE: A Product Dataset for Multi-source Attribute Value Extraction [10.429320377835241]
  We introduce MAVE, a new dataset to better facilitate research on product attribute value extraction. MAVE is composed of a curated set of 2.2 million products from Amazon pages, with 3 million attribute-value annotations across 1257 unique categories. We propose a novel approach that effectively extracts the attribute value from the multi-source product information.
  arXiv  Detail & Related papers  (2021-12-16T06:48:31Z)
• VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation [124.02278735049235]
  VALUE benchmark aims to cover a broad range of video genres, video lengths, data volumes, and task difficulty levels. We evaluate various baseline methods with and without large-scale VidL pre-training. The significant gap between our best model and human performance calls for future study for advanced VidL models.
  arXiv  Detail & Related papers  (2021-06-08T18:34:21Z)
• Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions [75.77044856100349]
  We present the Spoken Moments dataset of 500k spoken captions each attributed to a unique short video depicting a broad range of different events. We show that our AMM approach consistently improves our results and that models trained on our Spoken Moments dataset generalize better than those trained on other video-caption datasets.
  arXiv  Detail & Related papers  (2021-05-10T16:30:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.