ReSpec: Relevance and Specificity Grounded Online Filtering for Learning on Video-Text Data Streams

• URL: http://arxiv.org/abs/2504.14875v1
• Date: Mon, 21 Apr 2025 06:02:03 GMT
• Title: ReSpec: Relevance and Specificity Grounded Online Filtering for Learning on Video-Text Data Streams
• Authors: Chris Dongjoo Kim, Jihwan Moon, Sangwoo Moon, Heeseung Yun, Sihaeng Lee, Aniruddha Kembhavi, Soonyoung Lee, Gunhee Kim, Sangho Lee, Christopher Clark,
• Abstract summary: Video-text data presents challenges in storage and computation during training.<n>We propose Relevance and Specificity-based online filtering framework (ReSpec)<n>By establishing reference points from target task data, ReSpec filters incoming data in real-time, eliminating the need for extensive storage and compute.
• Score: 57.080448177724264
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid growth of video-text data presents challenges in storage and computation during training. Online learning, which processes streaming data in real-time, offers a promising solution to these issues while also allowing swift adaptations in scenarios demanding real-time responsiveness. One strategy to enhance the efficiency and effectiveness of learning involves identifying and prioritizing data that enhances performance on target downstream tasks. We propose Relevance and Specificity-based online filtering framework (ReSpec) that selects data based on four criteria: (i) modality alignment for clean data, (ii) task relevance for target focused data, (iii) specificity for informative and detailed data, and (iv) efficiency for low-latency processing. Relevance is determined by the probabilistic alignment of incoming data with downstream tasks, while specificity employs the distance to a root embedding representing the least specific data as an efficient proxy for informativeness. By establishing reference points from target task data, ReSpec filters incoming data in real-time, eliminating the need for extensive storage and compute. Evaluating on large-scale datasets WebVid2M and VideoCC3M, ReSpec attains state-of-the-art performance on five zeroshot video retrieval tasks, using as little as 5% of the data while incurring minimal compute. The source code is available at https://github.com/cdjkim/ReSpec.

Related papers

• ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment [13.303577263109858]
  ZIP-FIT is a data selection framework that uses gzip compression to measure alignment between potential training data and the target task distribution.<n>Models trained on ZIP-FIT-selected data achieve their lowest cross-entropy loss up to 85.1% faster than baselines.<n>Z ZIP-FIT shows that smaller, well-aligned datasets often outperform larger but less targeted ones.
  arXiv  Detail & Related papers  (2024-10-23T18:01:06Z)
• From Data Deluge to Data Curation: A Filtering-WoRA Paradigm for Efficient Text-based Person Search [30.88999109835329]
  In text-based person search endeavors, data generation has emerged as a prevailing practice, addressing concerns over privacy preservation and the arduous task of manual annotation. We observe that only a subset of the data in constructed datasets plays a decisive role. We introduce a new Filtering-WoRA paradigm, which contains a filtering algorithm to identify this crucial data subset and WoRA learning strategy for light fine-tuning.
  arXiv  Detail & Related papers  (2024-04-16T05:29:14Z)
• Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnostic [99.3682210827572]
  Vision-language models (VLMs) are trained for thousands of GPU hours on carefully curated web datasets. Data curation strategies are typically developed agnostic of the available compute for training. We introduce neural scaling laws that account for the non-homogeneous nature of web data.
  arXiv  Detail & Related papers  (2024-04-10T17:27:54Z)
• LESS: Selecting Influential Data for Targeted Instruction Tuning [64.78894228923619]
  We propose LESS, an efficient algorithm to estimate data influences and perform Low-rank gradiEnt Similarity Search for instruction data selection. We show that training on a LESS-selected 5% of the data can often outperform training on the full dataset across diverse downstream tasks. Our method goes beyond surface form cues to identify data that the necessary reasoning skills for the intended downstream application.
  arXiv  Detail & Related papers  (2024-02-06T19:18:04Z)
• The Devil is in the Details: A Deep Dive into the Rabbit Hole of Data Filtering [23.68112988933411]
  This paper describes our learning and solution when participating in the DataComp challenge. Our filtering strategy includes three stages: single-modality filtering, cross-modality filtering, and data distribution alignment. Our approach outperforms the best method from the DataComp paper by over 4% on the average performance of 38 tasks and by over 2% on ImageNet.
  arXiv  Detail & Related papers  (2023-09-27T19:10:43Z)
• Data-Juicer: A One-Stop Data Processing System for Large Language Models [73.27731037450995]
  A data recipe is a mixture of data from different sources for training Large Language Models (LLMs) We build a new system named Data-Juicer, with which we can efficiently generate diverse data recipes. The data recipes derived with Data-Juicer gain notable improvements on state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2023-09-05T08:22:07Z)
• Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity [72.85248553787538]
  This paper focuses on performance analysis and optimization for wireless FL, considering data heterogeneity, combined with wireless resource allocation. We formulate the loss function minimization problem, under constraints on long-term energy consumption and latency, and jointly optimize client scheduling, resource allocation, and the number of local training epochs (CRE) Experiments on real-world datasets demonstrate that the proposed algorithm outperforms other benchmarks in terms of the learning accuracy and energy consumption.
  arXiv  Detail & Related papers  (2023-08-04T04:18:01Z)
• STREAMLINE: Streaming Active Learning for Realistic Multi-Distributional Settings [2.580765958706854]
  STREAMLINE is a novel streaming active learning framework that mitigates scenario-driven slice imbalance in working labeled data. We evaluate STREAMLINE on real-world streaming scenarios for image classification and object detection tasks.
  arXiv  Detail & Related papers  (2023-05-18T02:01:45Z)
• METAM: Goal-Oriented Data Discovery [9.73435089036831]
  METAM is a goal-oriented framework that queries the downstream task with a candidate dataset, forming a feedback loop that automatically steers the discovery and augmentation process. We show METAM's theoretical guarantees and demonstrate those empirically on a broad set of tasks.
  arXiv  Detail & Related papers  (2023-04-18T15:42:25Z)
• Data Selection for Language Models via Importance Resampling [90.9263039747723]
  We formalize the problem of selecting a subset of a large raw unlabeled dataset to match a desired target distribution. We extend the classic importance resampling approach used in low-dimensions for LM data selection. We instantiate the DSIR framework with hashed n-gram features for efficiency, enabling the selection of 100M documents in 4.5 hours.
  arXiv  Detail & Related papers  (2023-02-06T23:57:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.