TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generation

• URL: http://arxiv.org/abs/2604.19473v1
• Date: Tue, 21 Apr 2026 13:56:36 GMT
• Title: TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generation
• Abstract summary: We propose a novel, training-free attention mechanism, Temporal-wise Separable Attention (TS-Attn)<n>TS-Attn rearranges attention distribution to ensure temporal awareness and global coherence in multi-event scenarios.<n>It boosts StoryEval-Bench scores by 33.5% and 16.4% on Wan2.1-T2V-14B and Wan2.2-T2V-A14B with only a 2% increase in inference time.
• Score: 61.27832169507143
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generating high-quality videos from complex temporal descriptions that contain multiple sequential actions is a key unsolved problem. Existing methods are constrained by an inherent trade-off: using multiple short prompts fed sequentially into the model improves action fidelity but compromises temporal consistency, while a single complex prompt preserves consistency at the cost of prompt-following capability. We attribute this problem to two primary causes: 1) temporal misalignment between video content and the prompt, and 2) conflicting attention coupling between motion-related visual objects and their associated text conditions. To address these challenges, we propose a novel, training-free attention mechanism, Temporal-wise Separable Attention (TS-Attn), which dynamically rearranges attention distribution to ensure temporal awareness and global coherence in multi-event scenarios. TS-Attn can be seamlessly integrated into various pre-trained text-to-video models, boosting StoryEval-Bench scores by 33.5% and 16.4% on Wan2.1-T2V-14B and Wan2.2-T2V-A14B with only a 2% increase in inference time. It also supports plug-and-play usage across models for multi-event image-to-video generation. The source code and project page are available at https://github.com/Hong-yu-Zhang/TS-Attn.

Related papers

• TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs [36.2604844032041]
  Video multimodal large language models (MLLMs) can describe what happens in a video, but rarely identify when the supporting evidence occurs.<n>We study generalist video temporal grounding, in which one model predicts a variable-cardinality set of evidence intervals across video lengths, domains, query forms, and viewpoints.<n>TimeLens2 treats temporal evidence as an interval set throughout supervision and optimization.
  arXiv  Detail & Related papers  (2026-07-19T22:04:33Z)
• TunerDiT: Training-free Progressive Steering of Diffusion Transformer for Multi-Event Video Generation [61.04423890085284]
  We present TunerDiT, a progressive steering method that requires no additional training for multi-event generation.<n>TunerDiT achieves state-of-the-art performance across 8 metrics and offers a tunable trade-off between video consistency and event separation.
  arXiv  Detail & Related papers  (2026-05-29T17:56:09Z)
• Prompt Relay: Inference-Time Temporal Control for Multi-Event Video Generation [40.694968116482315]
  We propose Prompt Relay, an inference-time, plug-and-play method to enable fine-grained temporal control in video generation.<n> Prompt Relay introduces a penalty into the cross-attention mechanism, so that each temporal segment attends only to its assigned prompt.
  arXiv  Detail & Related papers  (2026-04-11T04:59:06Z)
• SwitchCraft: Training-Free Multi-Event Video Generation with Attention Controls [21.71988638522276]
  We present SwitchCraft, a training-free framework for multi-event video generation.<n>SwitchCraft steers frame-level attention to align with relevant event prompts.<n>Experiments demonstrate that SwitchCraft substantially improves prompt alignment, event clarity, and scene consistency.
  arXiv  Detail & Related papers  (2026-02-27T11:59:06Z)
• AlcheMinT: Fine-grained Temporal Control for Multi-Reference Consistent Video Generation [58.844504598618094]
  We propose AlcheMinT, a unified framework that introduces explicit timestamps conditioning for subject-driven video generation.<n>Our approach introduces a novel positional encoding mechanism that unlocks the encoding of temporal intervals, associated in our case with subject identities.<n>We incorporate subject-descriptive text tokens to strengthen binding between visual identity and video captions, mitigating ambiguity during generation.
  arXiv  Detail & Related papers  (2025-12-11T18:59:34Z)
• When and Where do Events Switch in Multi-Event Video Generation? [59.9025263656304]
  Existing methods that extend to multi-event generation omit an inspection of the intrinsic factor in event shifting.<n>This work introduces MEve, a self-curated prompt suite for evaluating multi-event text-to-video (T2V) generation.
  arXiv  Detail & Related papers  (2025-10-03T14:31:56Z)
• FullDiT2: Efficient In-Context Conditioning for Video Diffusion Transformers [63.788600404496115]
  FullDiT2 is an efficient in-context conditioning framework for general controllability in both video generation and editing tasks.<n>FullDiT2 achieves significant computation reduction and 2-3 times speedup in averaged time cost per diffusion step.
  arXiv  Detail & Related papers  (2025-06-04T17:57:09Z)
• Video Flow as Time Series: Discovering Temporal Consistency and Variability for VideoQA [41.61905821058282]
  We introduce the Temporal Trio Transformer (T3T), a novel architecture that models time consistency and time variability.<n>The efficacy of the T3T is demonstrated through extensive testing on multiple VideoQA benchmark datasets.
  arXiv  Detail & Related papers  (2025-04-08T08:08:03Z)
• VideoFusion: A Spatio-Temporal Collaborative Network for Multi-modal Video Fusion and Restoration [26.59510171451438]
  Existing multi-sensor fusion research predominantly integrates complementary from multiple images rather than videos.<n>VideoFusion exploits cross-modal complementarity and temporal dynamics to generate context-temporally coherent videos.<n>Extensive experiments reveal that VideoFusion outperforms existing image-oriented fusion paradigms in sequential scenarios.
  arXiv  Detail & Related papers  (2025-03-30T08:27:18Z)
• Mind the Time: Temporally-Controlled Multi-Event Video Generation [65.05423863685866]
  We present MinT, a multi-event video generator with temporal control.<n>Our key insight is to bind each event to a specific period in the generated video, which allows the model to focus on one event at a time.<n>For the first time in the literature, our model offers control over the timing of events in generated videos.
  arXiv  Detail & Related papers  (2024-12-06T18:52:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.