PromptMobile: Efficient Promptus for Low Bandwidth Mobile Video Streaming

• URL: http://arxiv.org/abs/2503.16112v1
• Date: Thu, 20 Mar 2025 13:00:36 GMT
• Title: PromptMobile: Efficient Promptus for Low Bandwidth Mobile Video Streaming
• Abstract summary: This paper presents PromptMobile, an efficient acceleration framework tailored for on-device Promptus.<n>We propose a two-stage efficient generation framework to reduce computational cost by 8.1x, (2) a fine-grained inter-frame caching to reduce redundant computations by 16.6%, (3) system-level optimizations to further enhance efficiency.<n>Compared with other streaming methods, PromptMobile achives an average LPIPS improvement of 0.016 (compared with H.265), reducing 60% of severely distorted frames (compared to VQGAN)
• Score: 18.32444457609669
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional video compression algorithms exhibit significant quality degradation at extremely low bitrates. Promptus emerges as a new paradigm for video streaming, substantially cutting down the bandwidth essential for video streaming. However, Promptus is computationally intensive and can not run in real-time on mobile devices. This paper presents PromptMobile, an efficient acceleration framework tailored for on-device Promptus. Specifically, we propose (1) a two-stage efficient generation framework to reduce computational cost by 8.1x, (2) a fine-grained inter-frame caching to reduce redundant computations by 16.6\%, (3) system-level optimizations to further enhance efficiency. The evaluations demonstrate that compared with the original Promptus, PromptMobile achieves a 13.6x increase in image generation speed. Compared with other streaming methods, PromptMobile achives an average LPIPS improvement of 0.016 (compared with H.265), reducing 60\% of severely distorted frames (compared to VQGAN).

Related papers

• ScalablePromptus: Scalable and High-Fidelity Prompt-Based Video Streaming [51.48008871921326]
  Prompt-based video streaming transmits compact semantic prompts instead of pixel-level content for generative reconstruction.<n>We propose ScalablePromptus, which enhances Promptus with semantic and color-aware prompt inversion, spherical linear for intermediate frames, and--most critically--a dropout training strategy that produces rank-ordered prompt representations.
  arXiv  Detail & Related papers  (2026-07-28T08:59:39Z)
• CineMobile: On-Device Image-to-Video Diffusion for Cinematic Camera Motion Generation [17.654969014220033]
  CineMobile generates 49-frame 480p videos with a per-step denoising latency of 0.6s on an NVIDIA H200 GPU and 20s on the MediaTek Dimensity 8400 Ultimate 5G platform.<n>CineMobile achieves a 40x speedup in generation while maintaining comparable visual quality.
  arXiv  Detail & Related papers  (2026-07-04T10:24:35Z)
• Streaming of rendered content with adaptive frame rate and resolution [6.34507331016388]
  Existing solutions render content on a server at a fixed frame rate, typically 30 or 60 frames per second, and reduce resolution when bandwidth is restricted.<n>We exploit limits of human visual system to improve perceived quality while reducing rendering costs by adaptively adjusting both frame rate and resolution based on scene content and motion.<n>We propose a system in which a lightweight neural network predicts the optimal combination of frame rate and resolution for a given transmission bandwidth, content and motion velocity.
  arXiv  Detail & Related papers  (2026-05-09T14:48:10Z)
• Less is Enough: Training-Free Video Diffusion Acceleration via Runtime-Adaptive Caching [57.7533917467934]
  EasyCache is a training-free acceleration framework for video diffusion models.<n>We conduct comprehensive studies on various large-scale video generation models, including OpenSora, Wan2.1, and HunyuanVideo.<n>Our method achieves leading acceleration performance, reducing inference time by up to 2.1-3.3$times$ compared to the original baselines.
  arXiv  Detail & Related papers  (2025-07-03T17:59:54Z)
• Astraea: A Token-wise Acceleration Framework for Video Diffusion Transformers [29.130090574300635]
  Video diffusion transformers (vDiTs) have made tremendous progress in text-to-video generation, but their compute demands pose a major challenge for practical deployment.<n>We introduce Astraea, a framework that searches for near-optimal configurations for vDiT-based video generation under a performance target.
  arXiv  Detail & Related papers  (2025-06-05T14:41:38Z)
• VORTA: Efficient Video Diffusion via Routing Sparse Attention [54.84294780326206]
  VORTA is an acceleration framework with two novel components.<n>It achieves an end-to-end speedup $1.76times$ without loss of quality on VBench.<n>It can seamlessly integrate with various other acceleration methods, such as model caching and step distillation, reaching up to speedup $14.41times$ with negligible performance degradation.
  arXiv  Detail & Related papers  (2025-05-24T17:46:47Z)
• QuickVideo: Real-Time Long Video Understanding with System Algorithm Co-Design [54.38970077613728]
  Long-video understanding has emerged as a crucial capability in real-world applications such as video surveillance, meeting summarization, educational lecture analysis, and sports broadcasting.<n>We propose QuickVideo, a system-algorithm co-design that substantially accelerates long-video understanding to support real-time downstream applications.
  arXiv  Detail & Related papers  (2025-05-22T03:26:50Z)
• Mobile Video Diffusion [11.568267488097401]
  Video models have achieved impressive realism and controllability but are limited by computational demands.<n>This paper introduces the first mobile-optimized video diffusion model.<n>Our model, coined as MobileVD, is 523x more efficient (181 vs. 4.34 TFLOs) with a slight quality drop.
  arXiv  Detail & Related papers  (2024-12-10T15:19:10Z)
• REDUCIO! Generating 1024$\times$1024 Video within 16 Seconds using Extremely Compressed Motion Latents [110.41795676048835]
  One crucial obstacle for large-scale applications is the expensive training and inference cost. In this paper, we argue that videos contain much more redundant information than images, thus can be encoded by very few motion latents. We train Reducio-DiT in around 3.2K training hours in total and generate a 16-frame 1024*1024 video clip within 15.5 seconds on a single A100 GPU.
  arXiv  Detail & Related papers  (2024-11-20T18:59:52Z)
• Adaptive Caching for Faster Video Generation with Diffusion Transformers [52.73348147077075]
  Diffusion Transformers (DiTs) rely on larger models and heavier attention mechanisms, resulting in slower inference speeds. We introduce a training-free method to accelerate video DiTs, termed Adaptive Caching (AdaCache) We also introduce a Motion Regularization (MoReg) scheme to utilize video information within AdaCache, controlling the compute allocation based on motion content.
  arXiv  Detail & Related papers  (2024-11-04T18:59:44Z)
• Fast Encoding and Decoding for Implicit Video Representation [88.43612845776265]
  We introduce NeRV-Enc, a transformer-based hyper-network for fast encoding; and NeRV-Dec, a parallel decoder for efficient video loading. NeRV-Enc achieves an impressive speed-up of $mathbf104times$ by eliminating gradient-based optimization. NeRV-Dec simplifies video decoding, outperforming conventional codecs with a loading speed $mathbf11times$ faster.
  arXiv  Detail & Related papers  (2024-09-28T18:21:52Z)
• PatchEX: High-Quality Real-Time Temporal Supersampling through Patch-based Parallel Extrapolation [0.4143603294943439]
  This paper introduces PatchEX, a novel frame extrapolation method that aims to provide the quality of at the speed of extrapolation. PatchEX achieves a 65.29% and 48.46% improvement in PSNR over the latest extrapolation methods ExtraNet and ExtraSS, respectively.
  arXiv  Detail & Related papers  (2024-07-05T13:59:05Z)
• Promptus: Can Prompts Streaming Replace Video Streaming with Stable Diffusion [6.093768205619601]
  We propose Promptus, a system that streaming prompts instead of video content with Stable Diffusion. We show that Promptus can enhance perceptual quality by 0.111 and 0.092 (in LPIPS) compared to VAE and H.265, respectively. Our work opens up a new paradigm for efficient video communication beyond the Shannon limit.
  arXiv  Detail & Related papers  (2024-05-30T13:16:48Z)
• Binarized Low-light Raw Video Enhancement [49.65466843856074]
  Deep neural networks have achieved excellent performance on low-light raw video enhancement. In this paper, we explore the feasibility of applying the extremely compact binary neural network (BNN) to low-light raw video enhancement.
  arXiv  Detail & Related papers  (2024-03-29T02:55:07Z)
• ReBotNet: Fast Real-time Video Enhancement [59.08038313427057]
  Most restoration networks are slow, have high computational bottleneck, and can't be used for real-time video enhancement. In this work, we design an efficient and fast framework to perform real-time enhancement for practical use-cases like live video calls and video streams. To evaluate our method, we emulate two new datasets that real-world video call and streaming scenarios, and show extensive results on multiple datasets where ReBotNet outperforms existing approaches with lower computations, reduced memory requirements, and faster inference time.
  arXiv  Detail & Related papers  (2023-03-23T17:58:05Z)
• Prediction-assistant Frame Super-Resolution for Video Streaming [40.60863957681011]
  We propose to enhance video quality using lossy frames in two situations. For the first case, we propose a small yet effective video frame prediction network. For the second case, we improve the video prediction network to associate current frames as well as previous frames to restore high-quality images.
  arXiv  Detail & Related papers  (2021-03-17T06:05:27Z)
• Subjective and Objective Quality Assessment of High Frame Rate Videos [60.970191379802095]
  High frame rate (HFR) videos are becoming increasingly common with the tremendous popularity of live, high-action streaming content such as sports. Live-YT-HFR dataset is comprised of 480 videos having 6 different frame rates, obtained from 16 diverse contents. To obtain subjective labels on the videos, we conducted a human study yielding 19,000 human quality ratings obtained from a pool of 85 human subjects.
  arXiv  Detail & Related papers  (2020-07-22T19:11:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.