TLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streaming

• URL: http://arxiv.org/abs/2608.20958v1
• Date: Fri, 21 Aug 2026 10:25:38 GMT
• Title: TLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streaming
• Abstract summary: E-commerce live streaming requires omni-modal understanding of noisy, temporally extended streams.<n>We present TLive- Omni, an omni-modal understanding model tailored to live-commerce scenarios.
• Score: 11.909536558149837
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: E-commerce live streaming requires omni-modal understanding of noisy, temporally extended streams, where product facts are distributed across speech, video frames, product images, overlaid text, and user queries. We present TLive-Omni, an omni-modal understanding model tailored to live-commerce scenarios. It maps image, video, audio, and text inputs into a unified representation space. For long-form live streaming analysis, we introduce Per-vGrid, a timestamped token organization that groups each video grid with its temporally corresponding audio within explicit boundary tokens to facilitate temporal alignment. We design a three-stage supervised training recipe that progressively develops live-commerce understanding, from omni-modal perception to instruction-following responses. We then propose Faithful-RFT, a reinforcement fine-tuning stage that further improves answer faithfulness and expression quality while meeting real-time demands, scoring final responses directly with task-verifiable feedback rather than optimizing for reasoning-style exploration during rollout. Moreover, TLive-Omni is supported by a scenario-oriented atomic capability taxonomy and a compact data production engine that converts live-commerce audio, image, and video streams into training signals for speech recognition, speaker analysis, product visual grounding, text recognition, temporal grounding, video dense caption, and omni-modal QA, etc. For scalable training, a synchronized length-grouped sampler reduces padding while preserving comparable workloads across workers, while a lightweight dynamic sampling strategy regenerates rollout groups with near-zero reward variance to maintain meaningful relative advantages for GRPO. Experiments on e-commerce live streaming benchmarks demonstrate strong performance across live-commerce domain tasks, together with excellent generalization on general benchmarks.

Related papers

• InteracVid: Building a Real Interactive Audio-Visual Response Dataset from Live-Chat Videos [7.003662856458999]
  We introduce textbfInteracVid, emphthe firstopen-source large-scale dataset that addresses this missing supervision<n>Everysample couples a preceding audio-visual context and an external stimulus with the realinteractive response that follows.<n>A ten-rater human study confirms that the extracted interactions are causal,natural, and temporally complete for both genuine and reconstructed queries.
  arXiv  Detail & Related papers  (2026-08-02T11:28:13Z)
• Empowering Long-form Omni-modal Understanding with Robust Audio Perception [48.23485436432239]
  AVDC (Audio-Visual Decoupled Captions) is a large-scale dataset designed to disentangle visual and auditory semantics.<n>AVDC captures both modality-specific nuances and complex cross-modalinteractions.<n>AVDC-QA-CoT is a Chain-of-Thought augmented question-answering dataset to foster audio-visualreasoning.
  arXiv  Detail & Related papers  (2026-07-11T13:04:24Z)
• Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding [69.296913137409]
  We introduce a novel paradigm for online video understanding: Streaming Video-Language Synchrony (SVLS)<n>We present LyraV, a live streaming assistant built upon a hierarchical control framework with two core innovations.<n>First, the Frame-Driven Transition Controller (FDTC), makes high-level semantic decisions on when to continue speaking, start a new response, or stay silent.<n>Second, the Streaming Token Pacer (SToP), a plug-and-play lightweight predictive module, dynamically adapts the language generation rate to match the pace of the visual content.
  arXiv  Detail & Related papers  (2026-06-05T07:29:20Z)
• AURA: Always-On Understanding and Real-Time Assistance via Video Streams [37.80002196035961]
  AURA (Always-On Understanding and Real-Time Assistance) is an end-to-end streaming visual interaction framework.<n>It enables a unified VideoLLM to continuously process video streams and support both real-time question answering and proactive responses.<n>AURA achieves state-of-the-art performance on streaming benchmarks and supports a real-time demo system with ASR and TTS running at 2 FPS on two 80G accelerators.
  arXiv  Detail & Related papers  (2026-04-05T16:53:46Z)
• Complementary and Contrastive Learning for Audio-Visual Segmentation [74.11434759171199]
  We present Complementary and Contrastive Transformer (CCFormer), a novel framework adept at processing both local and global information.<n>Our method sets new state-of-the-art benchmarks across the S4, MS3 and AVSS datasets.
  arXiv  Detail & Related papers  (2025-10-11T06:36:59Z)
• Harnessing Synthetic Preference Data for Enhancing Temporal Understanding of Video-LLMs [54.502280390499756]
  We propose TimeWarp to create a targeted synthetic temporal dataset to fine-tune the model's responses to encourage it to focus on the given input video.<n>We demonstrate that when our method is applied to existing models, it significantly improves performance on temporal understanding benchmarks.
  arXiv  Detail & Related papers  (2025-10-04T21:48:40Z)
• ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shorts [56.75723197779384]
  ARC-Hunyuan-Video is a multimodal model that processes visual, audio, and textual signals end-to-end for structured comprehension.<n>Our model is capable of multi-granularity timestamped video captioning and summarization, open-ended video question answering, temporal video grounding, and video reasoning.
  arXiv  Detail & Related papers  (2025-07-28T15:52:36Z)
• LoViC: Efficient Long Video Generation with Context Compression [68.22069741704158]
  We introduce LoViC, a DiT-based framework trained on million-scale open-domain videos.<n>At the core of our approach is FlexFormer, an expressive autoencoder that jointly compresses video and text into unified latent representations.
  arXiv  Detail & Related papers  (2025-07-17T09:46:43Z)
• OmniResponse: Online Multimodal Conversational Response Generation in Dyadic Interactions [62.19092662469285]
  Online Multimodal Conversational Response Generation (OMCRG) is a novel task designed to produce synchronized verbal and non-verbal listener feedback online.<n>We propose OmniResponse, a Multimodal Large Language Model (MLLM) that autoregressively generates accurate multimodal listener responses.<n>We offer ResponseNet, a dataset of 696 detailed dyadic interactions featuring synchronized split-screen videos, multichannel audio, transcripts, and annotated facial behaviors.
  arXiv  Detail & Related papers  (2025-05-27T20:12:46Z)
• Everything Can Be Described in Words: A Simple Unified Multi-Modal Framework with Semantic and Temporal Alignment [0.0]
  We propose UMaT, a framework that unifies visual and auditory inputs as structured text for large language models.<n>It significantly improves state-of-the-art Long Video Question Answering accuracy.
  arXiv  Detail & Related papers  (2025-03-12T05:28:24Z)
• OMCAT: Omni Context Aware Transformer [27.674943980306423]
  OCTAV is a novel dataset designed to capture event transitions across audio and video. OMCAT is a powerful model that leverages RoTE to enhance temporal grounding and computational efficiency in time-anchored tasks. Our model demonstrates state-of-the-art performance on Audio-Visual Question Answering (AVQA) tasks and the OCTAV benchmark, showcasing significant gains in temporal reasoning and cross-modal alignment.
  arXiv  Detail & Related papers  (2024-10-15T23:16:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.