AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video

• URL: http://arxiv.org/abs/2609.14462v1
• Date: Sun, 13 Sep 2026 12:13:51 GMT
• Title: AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video
• Abstract summary: We present AlayaVista, a camera-controllable streaming video world model that decouples panoramic world evolution from perspective observation.<n>To support efficient streaming, we adapt the panoramic generator to chunk-autoregressive generation and distill both panoramic generation and perspective refinement into few-step processes.<n>We construct MUGEN, a large-scale real-world panoramic video dataset containing 1,318 hours of videos at resolutions of at least 4K, together with rich semantic and geometric annotations.
• Score: 34.83810293078634
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Interactive video world models must maintain broad scene context under camera motion while producing high-fidelity observations with low latency. Existing approaches face a representation trade-off: perspective models operate on local views and must preserve off-screen content over long rollouts, whereas broader spatial coverage is typically obtained by synthesizing full-sphere videos or constructing explicit 3D representations. Motivated by the complementary roles of global context and selective local acuity in visual perception, we present AlayaVista, a camera-controllable streaming video world model that decouples panoramic world evolution from perspective observation synthesis. Given a single perspective image, AlayaVista constructs a 360-degree scene prior using a pretrained panorama expansion model and then evolves the scene as a camera-conditioned panoramic latent state. A latent viewport renderer maps this state to the requested perspective video latents, while a perspective refiner restores details, suppresses artifacts, and performs super-resolution. To support efficient streaming, we adapt the panoramic generator to chunk-autoregressive generation and distill both panoramic generation and perspective refinement into few-step processes. To provide the supervision required by this design, we construct MUGEN, a large-scale real-world panoramic video dataset containing 1,318 hours of videos at resolutions of at least 4K, together with rich semantic and geometric annotations.

Related papers

• Pano2World: End-to-End 3D Generation via Unified Multi-View Sequences [9.89397063309918]
  We present Pano2World, which takes a single indoor panorama as input and directly outputs a persistent, explorable 3D Gaussian scene.<n> Experiments demonstrate that Pano2World significantly outperforms existing methods on the multi-position panoramic novel-view synthesis benchmark.
  arXiv  Detail & Related papers  (2026-07-01T11:54:02Z)
• OmniRoam: World Wandering via Long-Horizon Panoramic Video Generation [42.159343032593014]
  We propose OmniRoam, a controllable panoramic video generation framework.<n>Our framework exploits the rich per-frame scene coverage and inherent long-term spatial and temporal consistency of panoramic representation.<n> Experiments show that our framework consistently outperforms state-of-the-art methods in terms of visual quality, controllability, and long-term scene consistency.
  arXiv  Detail & Related papers  (2026-03-31T17:59:33Z)
• Stepper: Stepwise Immersive Scene Generation with Multiview Panoramas [101.58587238272462]
  We present Stepper, a unified framework for text-driven immersive 3D scene synthesis.<n>Stepper is trained on a new large-scale, multi-view panorama dataset.<n>It achieves state-of-the-art fidelity and structural consistency, outperforming prior approaches.
  arXiv  Detail & Related papers  (2026-03-30T20:26:28Z)
• PanoWorld-X: Generating Explorable Panoramic Worlds via Sphere-Aware Video Diffusion [87.13016347332943]
  PanoWorld-X is a novel framework for high-fidelity and controllable panoramic video generation with diverse camera trajectories.<n>Our experiments demonstrate superior performance in various aspects, including motion range, control precision, and visual quality.
  arXiv  Detail & Related papers  (2025-09-29T16:22:00Z)
• TiP4GEN: Text to Immersive Panorama 4D Scene Generation [82.8444414014506]
  TiP4GEN is a text-to-dynamic panorama scene generation framework.<n>It enables fine-grained content control and synthesizes motion-rich, geometry-consistent panoramic 4D scenes.<n> TiP4GEN integrates panorama video generation and dynamic scene reconstruction to create 360-degree immersive virtual environments.
  arXiv  Detail & Related papers  (2025-08-17T16:02:24Z)
• ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models [52.87334248847314]
  We propose a novel framework utilizing pretrained perspective video models for generating panoramic videos.<n>Specifically, we design a novel panorama representation named ViewPoint map, which possesses global spatial continuity and fine-grained visual details simultaneously.<n>Our method can synthesize highly dynamic and spatially consistent panoramic videos, achieving state-of-the-art performance and surpassing previous methods.
  arXiv  Detail & Related papers  (2025-06-30T04:33:34Z)
• PanoWan: Lifting Diffusion Video Generation Models to 360° with Latitude/Longitude-aware Mechanisms [41.92179513409301]
  Existing panoramic video generation models struggle to leverage pre-trained generative priors from conventional text-to-video models for high-quality panoramic videos.<n>In this paper, we introduce PanoWan to effectively lift pre-trained text-to-video models to the panoramic domain, equipped with minimal modules.<n>To provide sufficient panoramic videos for learning these lifted representations, we contribute PanoVid, a high-quality panoramic video dataset with captions and diverse scenarios.
  arXiv  Detail & Related papers  (2025-05-28T06:24:21Z)
• Beyond the Frame: Generating 360° Panoramic Videos from Perspective Videos [64.10180665546237]
  360deg videos offer a more complete perspective of our surroundings.<n>Existing video models excel at producing standard videos, but their ability to generate full panoramic videos remains elusive.<n>We develop a high-quality data filtering pipeline to curate pairwise training data and improve the quality of 360deg video generation.<n> Experimental results demonstrate that our model can generate realistic and coherent 360deg videos from in-the-wild perspective video.
  arXiv  Detail & Related papers  (2025-04-10T17:51:38Z)
• ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis [63.169364481672915]
  We propose textbfViewCrafter, a novel method for synthesizing high-fidelity novel views of generic scenes from single or sparse images. Our method takes advantage of the powerful generation capabilities of video diffusion model and the coarse 3D clues offered by point-based representation to generate high-quality video frames.
  arXiv  Detail & Related papers  (2024-09-03T16:53:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.