Visual Search Augmented Chain-of-Thought Reasoning for Attribute Value Extraction from Product Videos

• URL: http://arxiv.org/abs/2609.06410v1
• Date: Sun, 06 Sep 2026 06:01:57 GMT
• Title: Visual Search Augmented Chain-of-Thought Reasoning for Attribute Value Extraction from Product Videos
• Abstract summary: We propose visual search augmented chain-of-thought reasoning (ViS-CoT) for visual attribute value extraction.<n>ViS-CoT can be easily applied to any open-source video VLM for video-to-text AVE in e-Commerce.
• Score: 14.43540077402288
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing approaches to visual attribute value extraction (AVE) primarily rely on static product images, failing to capture temporal cues, multi-angle views and fine-grained visual details. Directly applying video vision-language models (VLMs) to product AVE results in limited performance due to the lack of domain knowledge, and fine-tuning them requires extensive high-quality data and substantial computational resources. Thus, we propose visual search augmented chain-of-thought reasoning (ViS-CoT), a training-free, plug-and-play pipeline that can be easily applied to any open-source video VLM for video-to-text AVE in e-Commerce. Specifically, ViS-CoT employs visual clustering to identify representative frames, followed by visual search to retrieve semantically similar product knowledge that can enrich attribute cues. Next, an interleaved CoT reasoning module iteratively refines reasoning through visually-aligned auxiliary texts derived from captioning and automatic speech recognition. Finally, the integrated information guides the model toward accurate and fine-grained attribute predictions. Extensive experiments across 14 product categories on the VideoAVE dataset show that ViS-CoT consistently enhances multiple state-of-the-art video VLMs, achieving an average improvement of 17.91 percentage points in micro-F1.

Related papers

• Text-Guided Visual Representation Learning for Robust Multimodal E-Commerce Recommendation [51.52908699287466]
  Text-Guided Q-Former is a text-guided visual representation learning framework.<n>It disentangles metadata-anchored and exploratory visual streams, and introduces a lightweight reliability-aware dual-gated vector modulation module.<n>Experiments on large-scale, real-world e-commerce datasets with full-pool retrieval show that TGQ-Former consistently outperforms strong connector baselines and end-to-end MLLMs.
  arXiv  Detail & Related papers  (2026-05-17T10:20:23Z)
• Act2See: Emergent Active Visual Perception for Video Reasoning [70.52373240573856]
  Vision-Language Models (VLMs) typically rely on static initial frames for video reasoning.<n>We introduce Act-to-See (Act2See), a novel framework that enables active visual perception.
  arXiv  Detail & Related papers  (2026-05-03T00:52:51Z)
• VideoPerceiver: Enhancing Fine-Grained Temporal Perception in Video Multimodal Large Language Models [9.896951371033229]
  VideoPerceiver is a video multimodal large language model (VMLLM) that enhances fine-grained perception in video understanding.<n>We construct "key-information-missing" videos by extracting event-action keywords from captions, identifying corresponding key frames, and replacing them with adjacent frames.<n> Experiments show VideoPerceiver substantially outperforms state-of-the-art VMLLMs on fine-grained action understanding and rare event captioning benchmarks.
  arXiv  Detail & Related papers  (2025-11-24T06:57:26Z)
• GMFVAD: Using Grained Multi-modal Feature to Improve Video Anomaly Detection [63.16754542429089]
  We propose Grained Multi-modal Feature for Video Anomaly Detection (GMFVAD)<n>We generate more grained multi-modal feature based on the video snippet, which summarizes the main content.<n> Experiments show that GMFVAD achieves state-of-the-art performance on four mainly datasets.
  arXiv  Detail & Related papers  (2025-10-23T06:52:53Z)
• VideoAVE: A Multi-Attribute Video-to-Text Attribute Value Extraction Dataset and Benchmark Models [7.632869516436505]
  VideoAVE is the first publicly available video-to-text e-commerce AVE dataset across 14 different domains and covering 172 unique attributes.<n>To ensure data quality, we propose a post-hoc CLIP-based Mixture of Experts filtering system (CLIP-MoE)<n>Our results analysis reveals that video-to-text AVE remains a challenging problem, particularly in open settings.
  arXiv  Detail & Related papers  (2025-08-15T20:58:47Z)
• OpenVidVRD: Open-Vocabulary Video Visual Relation Detection via Prompt-Driven Semantic Space Alignment [5.215417164787923]
  Visual language models (VLMs) help explore open-vocabulary visual relation detection, yet often overlook the connections between various visual regions and their relations.<n>We propose a novel open-vocabulary VidVRD framework, termed OpenVidVRD, which improves VidVRD tasks through prompt learning.
  arXiv  Detail & Related papers  (2025-03-12T14:13:17Z)
• Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models [81.71651422951074]
  Chain-of-Spot (CoS) method is a novel approach that enhances feature extraction by focusing on key regions of interest. This technique allows LVLMs to access more detailed visual information without altering the original image resolution. Our empirical findings demonstrate a significant improvement in LVLMs' ability to understand and reason about visual content.
  arXiv  Detail & Related papers  (2024-03-19T17:59:52Z)
• VaQuitA: Enhancing Alignment in LLM-Assisted Video Understanding [63.075626670943116]
  We introduce a cutting-edge framework, VaQuitA, designed to refine the synergy between video and textual information. At the data level, instead of sampling frames uniformly, we implement a sampling method guided by CLIP-score rankings. At the feature level, we integrate a trainable Video Perceiver alongside a Visual-Query Transformer.
  arXiv  Detail & Related papers  (2023-12-04T19:48:02Z)
• Visual Commonsense-aware Representation Network for Video Captioning [84.67432867555044]
  We propose a simple yet effective method, called Visual Commonsense-aware Representation Network (VCRN) for video captioning. Our method reaches state-of-the-art performance, indicating the effectiveness of our method.
  arXiv  Detail & Related papers  (2022-11-17T11:27:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.