PanoLora: Bridging Perspective and Panoramic Video Generation with LoRA Adaptation
- URL: http://arxiv.org/abs/2509.11092v1
- Date: Sun, 14 Sep 2025 05:05:27 GMT
- Title: PanoLora: Bridging Perspective and Panoramic Video Generation with LoRA Adaptation
- Authors: Zeyu Dong, Yuyang Yin, Yuqi Li, Eric Li, Hao-Xiang Guo, Yikai Wang,
- Abstract summary: Panoramic videos rely on a single viewpoint with a limited field of view, making it difficult for standard video generation models to adapt.<n>Existing solutions often introduce complex architectures or large-scale training, leading to inefficiency and suboptimal results.<n>We propose treating panoramic video generation as an adaptation problem from perspective views.<n>Our approach efficiently fine-tunes a pretrained video diffusion model using only approximately 1,000 videos while achieving high-quality panoramic generation.
- Score: 17.498427118787045
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Generating high-quality 360{\deg} panoramic videos remains a significant challenge due to the fundamental differences between panoramic and traditional perspective-view projections. While perspective videos rely on a single viewpoint with a limited field of view, panoramic content requires rendering the full surrounding environment, making it difficult for standard video generation models to adapt. Existing solutions often introduce complex architectures or large-scale training, leading to inefficiency and suboptimal results. Motivated by the success of Low-Rank Adaptation (LoRA) in style transfer tasks, we propose treating panoramic video generation as an adaptation problem from perspective views. Through theoretical analysis, we demonstrate that LoRA can effectively model the transformation between these projections when its rank exceeds the degrees of freedom in the task. Our approach efficiently fine-tunes a pretrained video diffusion model using only approximately 1,000 videos while achieving high-quality panoramic generation. Experimental results demonstrate that our method maintains proper projection geometry and surpasses previous state-of-the-art approaches in visual quality, left-right consistency, and motion diversity.
Related papers
- Plenoptic Video Generation [80.3116444692858]
We introduce PlenopticDreamer, a framework that synchronizes generative hallucinations to maintain synchronization-temporal memory.<n>The core idea is to train a multi-in-out video-conditioned model in an autoregressive manner.<n>Our training incorporates context-scaling to improve convergence, self-conditioning to hallucinations caused by error accumulation, and a long-video conditioning mechanism to support extended video generation.
arXiv Detail & Related papers (2026-01-08T18:58:32Z) - VividFace: High-Quality and Efficient One-Step Diffusion For Video Face Enhancement [51.83206132052461]
Video Face Enhancement (VFE) seeks to reconstruct high-quality facial regions from degraded video sequences.<n>Current methods that rely on video super-resolution and generative frameworks face three fundamental challenges.<n>We propose VividFace, a novel and efficient one-step diffusion framework for video face enhancement.
arXiv Detail & Related papers (2025-09-28T02:39:48Z) - ANYPORTAL: Zero-Shot Consistent Video Background Replacement [8.690698677022992]
ANYPORTAL is a zero-shot framework for video background replacement.<n>It integrates the temporal prior of video diffusion models with the relighting capabilities of image diffusion models in a zero-shot setting.<n>It overcomes the challenges of achieving foreground consistency and temporally coherent relighting.
arXiv Detail & Related papers (2025-09-09T07:50:53Z) - One Flight Over the Gap: A Survey from Perspective to Panoramic Vision [117.80970697177025]
This survey reviews recent panoramic vision techniques with a particular emphasis on the perspective-to-panorama adaptation.<n>We first revisit the panoramic imaging pipeline and projection methods to build the prior knowledge required for analyzing the structural disparities.<n>Building on this, we cover 20+ representative tasks drawn from more than 300 research papers in two dimensions.
arXiv Detail & Related papers (2025-09-04T17:59:10Z) - ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models [52.87334248847314]
We propose a novel framework utilizing pretrained perspective video models for generating panoramic videos.<n>Specifically, we design a novel panorama representation named ViewPoint map, which possesses global spatial continuity and fine-grained visual details simultaneously.<n>Our method can synthesize highly dynamic and spatially consistent panoramic videos, achieving state-of-the-art performance and surpassing previous methods.
arXiv Detail & Related papers (2025-06-30T04:33:34Z) - Frame Guidance: Training-Free Guidance for Frame-Level Control in Video Diffusion Models [59.62564091684881]
We present Frame Guidance, a training-free guidance for controllable video generation based on frame-level signals.<n>For practical training-free guidance, we propose a simple latent processing method that dramatically reduces memory usage.<n>We apply a novel latent optimization strategy designed for globally coherent video generation.
arXiv Detail & Related papers (2025-06-08T14:54:41Z) - PanoWan: Lifting Diffusion Video Generation Models to 360° with Latitude/Longitude-aware Mechanisms [41.92179513409301]
Existing panoramic video generation models struggle to leverage pre-trained generative priors from conventional text-to-video models for high-quality panoramic videos.<n>In this paper, we introduce PanoWan to effectively lift pre-trained text-to-video models to the panoramic domain, equipped with minimal modules.<n>To provide sufficient panoramic videos for learning these lifted representations, we contribute PanoVid, a high-quality panoramic video dataset with captions and diverse scenarios.
arXiv Detail & Related papers (2025-05-28T06:24:21Z) - Beyond the Frame: Generating 360° Panoramic Videos from Perspective Videos [64.10180665546237]
360deg videos offer a more complete perspective of our surroundings.<n>Existing video models excel at producing standard videos, but their ability to generate full panoramic videos remains elusive.<n>We develop a high-quality data filtering pipeline to curate pairwise training data and improve the quality of 360deg video generation.<n> Experimental results demonstrate that our model can generate realistic and coherent 360deg videos from in-the-wild perspective video.
arXiv Detail & Related papers (2025-04-10T17:51:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.