BiMa: Towards Biases Mitigation for Text-Video Retrieval via Scene Element Guidance

• URL: http://arxiv.org/abs/2506.03589v3
• Date: Mon, 07 Jul 2025 09:47:46 GMT
• Title: BiMa: Towards Biases Mitigation for Text-Video Retrieval via Scene Element Guidance
• Authors: Huy Le, Nhat Chung, Tung Kieu, Anh Nguyen, Ngan Le,
• Abstract summary: BiMa is a novel framework designed to mitigate biases in both visual and textual representations.<n>For visual debiasing, we integrate these scene elements into the video embeddings, enhancing them to emphasize fine-grained and salient details.<n>For textual debiasing, we introduce a mechanism to disentangle text features into content and bias components, enabling the model to focus on meaningful content.
• Score: 10.268638578607977
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-video retrieval (TVR) systems often suffer from visual-linguistic biases present in datasets, which cause pre-trained vision-language models to overlook key details. To address this, we propose BiMa, a novel framework designed to mitigate biases in both visual and textual representations. Our approach begins by generating scene elements that characterize each video by identifying relevant entities/objects and activities. For visual debiasing, we integrate these scene elements into the video embeddings, enhancing them to emphasize fine-grained and salient details. For textual debiasing, we introduce a mechanism to disentangle text features into content and bias components, enabling the model to focus on meaningful content while separately handling biased information. Extensive experiments and ablation studies across five major TVR benchmarks (i.e., MSR-VTT, MSVD, LSMDC, ActivityNet, and DiDeMo) demonstrate the competitive performance of BiMa. Additionally, the model's bias mitigation capability is consistently validated by its strong results on out-of-distribution retrieval tasks.

Related papers

• VidText: Towards Comprehensive Evaluation for Video Text Understanding [54.15328647518558]
  VidText is a benchmark for comprehensive and in-depth evaluation of video text understanding.<n>It covers a wide range of real-world scenarios and supports multilingual content.<n>It introduces a hierarchical evaluation framework with video-level, clip-level, and instance-level tasks.
  arXiv  Detail & Related papers  (2025-05-28T19:39:35Z)
• Video Summarization with Large Language Models [41.51242348081083]
  We propose a new video summarization framework that leverages the capabilities of recent Large Language Models (LLMs)<n>Our method, dubbed LLM-based Video Summarization (LLMVS), translates video frames into a sequence of captions using a Muti-modal Large Language Model (MLLM)<n>Our experimental results demonstrate the superiority of the proposed method over existing ones in standard benchmarks.
  arXiv  Detail & Related papers  (2025-04-15T13:56:14Z)
• Multi-Modal interpretable automatic video captioning [1.9874264019909988]
  We introduce a novel video captioning method trained with multi-modal contrastive loss. Our approach is designed to capture the dependency between these modalities, resulting in more accurate, thus pertinent captions.
  arXiv  Detail & Related papers  (2024-11-11T11:12:23Z)
• Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approach [56.610806615527885]
  A key challenge in text-video retrieval (TVR) is the information asymmetry between video and text.<n>This paper introduces a data-centric framework to bridge this gap by enriching textual representations to better match the richness of video content.<n>We propose a query selection mechanism that identifies the most relevant and diverse queries, reducing computational cost while improving accuracy.
  arXiv  Detail & Related papers  (2024-08-14T01:24:09Z)
• Vamos: Versatile Action Models for Video Understanding [23.631145570126268]
  We propose versatile action models (Vamos), a learning framework powered by a large language model as the reasoner'' We evaluate Vamos on five benchmarks, Ego4D, NeXT-QA, IntentQA, Spacewalk-18, and Ego on its capability to model temporal dynamics, encode visual history, and perform reasoning.
  arXiv  Detail & Related papers  (2023-11-22T17:44:24Z)
• Zero-Shot Video Moment Retrieval from Frozen Vision-Language Models [58.17315970207874]
  We propose a zero-shot method for adapting generalisable visual-textual priors from arbitrary VLM to facilitate moment-text alignment. Experiments conducted on three VMR benchmark datasets demonstrate the notable performance advantages of our zero-shot algorithm.
  arXiv  Detail & Related papers  (2023-09-01T13:06:50Z)
• MVMR: A New Framework for Evaluating Faithfulness of Video Moment Retrieval against Multiple Distractors [24.858928681280634]
  We propose the MVMR (Massive Videos Moment Retrieval for Faithfulness Evaluation) task. It aims to retrieve video moments within a massive video set, including multiple distractors, to evaluate the faithfulness of VMR models. For this task, we suggest an automated massive video pool construction framework to categorize negative (distractors) and positive (false-negative) video sets.
  arXiv  Detail & Related papers  (2023-08-15T17:38:55Z)
• A Video Is Worth 4096 Tokens: Verbalize Videos To Understand Them In Zero Shot [67.00455874279383]
  We propose verbalizing long videos to generate descriptions in natural language, then performing video-understanding tasks on the generated story as opposed to the original video. Our method, despite being zero-shot, achieves significantly better results than supervised baselines for video understanding. To alleviate a lack of story understanding benchmarks, we publicly release the first dataset on a crucial task in computational social science on persuasion strategy identification.
  arXiv  Detail & Related papers  (2023-05-16T19:13:11Z)
• Towards Generalisable Video Moment Retrieval: Visual-Dynamic Injection to Image-Text Pre-Training [70.83385449872495]
  The correlation between the vision and text is essential for video moment retrieval (VMR) Existing methods rely on separate pre-training feature extractors for visual and textual understanding. We propose a generic method, referred to as Visual-Dynamic Injection (VDI), to empower the model's understanding of video moments.
  arXiv  Detail & Related papers  (2023-02-28T19:29:05Z)
• Mitigating Representation Bias in Action Recognition: Algorithms and Benchmarks [76.35271072704384]
  Deep learning models perform poorly when applied to videos with rare scenes or objects. We tackle this problem from two different angles: algorithm and dataset. We show that the debiased representation can generalize better when transferred to other datasets and tasks.
  arXiv  Detail & Related papers  (2022-09-20T00:30:35Z)
• Video Understanding as Machine Translation [53.59298393079866]
  We tackle a wide variety of downstream video understanding tasks by means of a single unified framework. We report performance gains over the state-of-the-art on several downstream tasks including video classification (EPIC-Kitchens), question answering (TVQA), captioning (TVC, YouCook2, and MSR-VTT)
  arXiv  Detail & Related papers  (2020-06-12T14:07:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.