A Comprehensive Ecosystem for Open-Domain Customized Video Generation

• URL: http://arxiv.org/abs/2606.11783v1
• Date: Wed, 10 Jun 2026 08:15:52 GMT
• Title: A Comprehensive Ecosystem for Open-Domain Customized Video Generation
• Abstract summary: PexelsCustom-1M is the first publicly available million-scale dataset for identity-preserving video generation.<n>CustoMDiT is a parameter-efficient framework that adapts a pretrained multimodal Diffusion Transformer into a customized video generator.
• Score: 43.16057607613357
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent progress in video generation has shown impressive visual synthesis capabilities. However, open-domain customized video generation remains limited by the lack of large-scale, annotated datasets capturing diverse identity-specific attributes. To address this, we introduce PexelsCustom-1M, the first publicly available million-scale dataset for identity-preserving video generation, containing one million curated <identity, text, video> triplets across 8,000+ categories. Leveraging this, we propose CustoMDiT, a parameter-efficient framework that adapts a pretrained multimodal Diffusion Transformer into a customized video generator with only 8% additional learnable parameters. Our method surpasses prior state-of-the-art. However, benchmarks such as DreamBooth cover only 100 classes, which is insufficient for real-world applications. To overcome this, we construct OpenCustom, a new benchmark with 1,000+ categories, created via cross-dataset knowledge fusion from ImageNet and MS-COCO. Extensive experiments confirm the advantages of both our dataset and model. We will open-source the entire ecosystem--including dataset, pipeline, benchmark, and implementations--to support further research.

Related papers

• OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding [55.29748680163419]
  Video Temporal Grounding (VTG) struggles in open-world settings due to limited dataset scale and semantic diversity.<n>We introduce OmniVTG, a new large-scale dataset for open-world VTG.<n>We train the MLLM to first predict, then use its understanding capabilities to reflect on and refine its own predictions.
  arXiv  Detail & Related papers  (2026-04-28T06:34:19Z)
• SceneScribe-1M: A Large-Scale Video Dataset with Comprehensive Geometric and Semantic Annotations [88.8747004592363]
  SceneScribe-1M is a new large-scale, multi-temporal video dataset.<n>It comprises one million in-the-wild videos, each meticulously annotated with detailed textual descriptions, precise parameters, dense depth maps, and consistent 3D point tracks.<n>We demonstrate the versatility and value of SceneScribe-1M by establishing benchmarks across a wide array of downstream tasks, including monocular depth estimation, scene reconstruction dynamic point tracking, as well as generative tasks such as text-to-video synthesis, with or without camera control.
  arXiv  Detail & Related papers  (2026-04-09T08:59:33Z)
• Scaling Audio-Visual Quality Assessment Dataset via Crowdsourcing [62.250874651622574]
  We propose a practical approach for AVQA dataset construction.<n>We design a crowdsourced subjective experiment framework for AVQA, breaks the constraints of in-lab settings, and achieves reliable annotation across varied environments.<n>We validate this approach through YT-NTU-AVQ, the largest and most diverse AVQA dataset to date, consisting of 1,620 user-generated audio and video sequences.
  arXiv  Detail & Related papers  (2026-02-26T06:18:11Z)
• EA-Swin: An Embedding-Agnostic Swin Transformer for AI-Generated Video Detection [2.1627654218138237]
  We propose EA-Swin, an Embedding-Agnostic Swin model that integrates dependencies directly on video embeddings via factorized windowed attention.<n>We show that EA-Swin 0.97-0.99 accuracy across major generators, outperforming prior SoTA methods (typically 0.80.9-20%) by a margin of 5-20%.
  arXiv  Detail & Related papers  (2026-02-19T11:04:20Z)
• Generative Video Matting [57.186684844156595]
  Video matting has traditionally been limited by the lack of high-quality ground-truth data.<n>Most existing video matting datasets provide only human-annotated imperfect alpha and foreground annotations.<n>We introduce a novel video matting approach that can effectively leverage the rich priors from pre-trained video diffusion models.
  arXiv  Detail & Related papers  (2025-08-11T12:18:55Z)
• Tiger200K: Manually Curated High Visual Quality Video Dataset from UGC Platform [0.0]
  Tiger200K is a manually curated high visual quality video dataset sourced from User-Generated Content (UGC) platforms.<n>By prioritizing visual fidelity and aesthetic quality, Tiger200K underscores the critical role of human expertise in data curation.<n>The dataset will undergo ongoing expansion and be released as an open-source initiative to advance research and applications in video generative models.
  arXiv  Detail & Related papers  (2025-04-21T15:44:06Z)
• Pre-training for Action Recognition with Automatically Generated Fractal Datasets [23.686476742398973]
  We present methods to automatically produce large-scale datasets of short synthetic video clips. The generated video clips are characterized by notable variety, stemmed by the innate ability of fractals to generate complex multi-scale structures. Compared to standard Kinetics pre-training, our reported results come close and are even superior on a portion of downstream datasets.
  arXiv  Detail & Related papers  (2024-11-26T16:51:11Z)
• Pedestrian Attribute Recognition: A New Benchmark Dataset and A Large Language Model Augmented Framework [15.991114464911844]
  In the past five years, no large-scale dataset has been opened to the public. This paper proposes a new large-scale, cross-domain pedestrian attribute recognition dataset, MSP60K. It consists of 60,122 images and 57 attribute annotations across eight scenarios.
  arXiv  Detail & Related papers  (2024-08-19T06:19:31Z)
• OmniDataComposer: A Unified Data Structure for Multimodal Data Fusion and Infinite Data Generation [8.149870655785955]
  OmniDataComposer is an innovative approach for multimodal data fusion and unlimited data generation. It is capable of identifying over 6400 categories of objects, substantially broadening the spectrum of visual information. It amalgamates diverse modalities, promoting reciprocal enhancement among modalities and facilitating cross-modal data correction.
  arXiv  Detail & Related papers  (2023-08-08T08:30:16Z)
• MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos [106.06278332186106]
  Multimodal summarization with multimodal output (MSMO) has emerged as a promising research direction. Numerous limitations exist within existing public MSMO datasets. We have meticulously curated the textbfMMSum dataset.
  arXiv  Detail & Related papers  (2023-06-07T07:43:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.