360° Image Perception with MLLMs: A Comprehensive Benchmark and a Training-Free Method

• URL: http://arxiv.org/abs/2603.16179v2
• Date: Thu, 26 Mar 2026 02:43:38 GMT
• Title: 360° Image Perception with MLLMs: A Comprehensive Benchmark and a Training-Free Method
• Abstract summary: We develop Free360, a training-free scene-graph-based framework for high-resolution 360 VQA.<n>Free360 decomposes the reasoning process into modular steps, applies adaptive spherical image transformations to 360 images tailored to each step, and seamlessly integrates the resulting information into a unified graph representation for answer generation.<n>Experiments show that Free360 consistently improves its base MLLM and provides a strong training-free solution for 360 VQA tasks.
• Score: 13.535148142390817
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal Large Language Models (MLLMs) have shown impressive abilities in understanding and reasoning over conventional images. However, their perception of 360° images remains largely underexplored. Unlike conventional images, 360° images capture the entire surrounding environment, enabling holistic spatial reasoning but introducing challenges such as geometric distortion and complex spatial relations. To comprehensively assess MLLMs' capabilities to perceive 360° images, we introduce 360Bench, a Visual Question Answering (VQA) benchmark featuring 7K-resolution 360° images, seven representative (sub)tasks with annotations carefully curated by human annotators. Using 360Bench, we systematically evaluate seven MLLMs and six enhancement methods, revealing their shortcomings in 360° image perception. To address these challenges, we propose Free360, a training-free scene-graph-based framework for high-resolution 360° VQA. Free360 decomposes the reasoning process into modular steps, applies adaptive spherical image transformations to 360° images tailored to each step, and seamlessly integrates the resulting information into a unified graph representation for answer generation. Experiments show that Free360 consistently improves its base MLLM and provides a strong training-free solution for 360° VQA tasks. The source code and dataset will be publicly released upon acceptance.

Related papers

• OMEGA-Avatar: One-shot Modeling of 360° Gaussian Avatars [54.688420347927725]
  OMEGA-Avatar is the first framework that simultaneously generates a generalizable, 360-complete, and animatable 3D Gaussian head from a single image.<n>We show that OMEGA-Avatar achieves state-of-the-art performance, significantly outperforming existing baselines in 360 full-head completeness.
  arXiv  Detail & Related papers  (2026-02-12T08:16:38Z)
• 360Anything: Geometry-Free Lifting of Images and Videos to 360° [51.50120114305155]
  Existing approaches rely on explicit geometric alignment between the perspective and the equirectangular projection space.<n>We propose 360Anything, a geometry-free framework built upon pre-trained diffusion transformers.<n>Our approach achieves state-of-the-art performance on both image and video perspective-to-360 generation.
  arXiv  Detail & Related papers  (2026-01-22T18:45:59Z)
• DiT360: High-Fidelity Panoramic Image Generation via Hybrid Training [76.82789568988557]
  DiT360 is a DiT-based framework that performs hybrid training on perspective and panoramic data for panoramic image generation.<n>Our method achieves better boundary consistency and image fidelity across eleven quantitative metrics.
  arXiv  Detail & Related papers  (2025-10-13T17:59:15Z)
• Dream360: Diverse and Immersive Outdoor Virtual Scene Creation via Transformer-Based 360 Image Outpainting [33.95741744421632]
  We propose a transformer-based 360 image outpainting framework called Dream360. It can generate diverse, high-fidelity, and high-resolution panoramas from user-selected viewports. Our Dream360 achieves significantly lower Frechet Inception Distance (FID) scores and better visual fidelity than existing methods.
  arXiv  Detail & Related papers  (2024-01-19T09:01:20Z)
• Perceptual Quality Assessment of 360$^\circ$ Images Based on Generative Scanpath Representation [40.00063797833765]
  We introduce a unique generative scanpath representation (GSR) for effective quality inference of 360$circ$ images. GSR aggregates varied perceptual experiences of multi-hypothesis users under a predefined viewing condition. We then propose an efficient OIQA computational framework by learning the quality maps GSR.
  arXiv  Detail & Related papers  (2023-09-07T04:10:30Z)
• NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes [59.15910989235392]
  We introduce NeO 360, Neural fields for sparse view synthesis of outdoor scenes. NeO 360 is a generalizable method that reconstructs 360deg scenes from a single or a few posed RGB images. Our representation combines the best of both voxel-based and bird's-eye-view (BEV) representations.
  arXiv  Detail & Related papers  (2023-08-24T17:59:50Z)
• NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360{\deg} Views [77.93662205673297]
  In this work, we study the challenging task of lifting a single image to a 3D object. We demonstrate the ability to generate a plausible 3D object with 360deg views that correspond well with a given reference image. We propose a novel framework, dubbed NeuralLift-360, that utilizes a depth-aware radiance representation.
  arXiv  Detail & Related papers  (2022-11-29T17:59:06Z)
• 360{\deg} Optical Flow using Tangent Images [18.146747748702513]
  equirectangular projection (ERP) is the most common format for storing, processing and visualising 360deg images. We propose a 360deg optical flow method based on tangent images.
  arXiv  Detail & Related papers  (2021-12-28T23:50:46Z)
• Visual Question Answering on 360{\deg} Images [96.00046925811515]
  VQA 360 is a novel task of visual question answering on 360 images. We collect the first VQA 360 dataset, containing around 17,000 real-world image-question-answer triplets for a variety of question types.
  arXiv  Detail & Related papers  (2020-01-10T08:18:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.