An Evaluation Framework for Generating Multi-View Images of a Person in a Scene

• URL: http://arxiv.org/abs/2609.04603v1
• Date: Fri, 04 Sep 2026 01:02:12 GMT
• Title: An Evaluation Framework for Generating Multi-View Images of a Person in a Scene
• Abstract summary: A primary bottleneck in training foundation models to execute free-form, promptable camera angle changes is the lack of specialized training data.<n>We propose the Head Scene Rotation Difference (HSRD) metric to quantitatively evaluate camera movements around a person.<n>HSRD provides the pipeline necessary to evaluate 3D spatial parallax for a person in a scene, paving the way to reliably construct high-quality multi-view synthetic datasets.
• Score: 8.826327162415287
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent generative image-editing Diffusion Transformers (DiTs) demonstrate impressive semantic editing capabilities but still struggle with spatially consistent camera angle changes. A primary bottleneck in training foundation models to execute free-form, promptable camera angle changes is the lack of specialized training data. While multi-view datasets exist for generic 3D environments and objects, there remains an absence of paired, multi-view datasets featuring human subjects at fixed locations in natural scenes, including frontal and side-profile views. Capturing such multi-camera data in unconstrained environments is logistically challenging and unscalable. In this paper, we first experiment with multiple state-of-the-art image editing models to create this data synthetically, but find that the outputs are frequently prone to hallucinations involving how much the subject's head turns relative to the background, often producing inconsistent environments. To address this issue, we propose the Head Scene Rotation Difference (HSRD) metric to quantitatively evaluate camera movements around a person. The proposed metric operates by decoupling camera movement from localized head pose manipulation. As demonstrated by the extensive experimentation, HSRD provides the pipeline necessary to evaluate 3D spatial parallax for a person in a scene, paving the way to reliably construct high-quality multi-view synthetic datasets.

Related papers

• CalibAnyView: Beyond Single-View Camera Calibration in the Wild [59.66873936532375]
  Camera calibration is a fundamental prerequisite for reliable geometric perception.<n>Recent learning-based methods have shown promising results for single-view calibration, but inherently neglect geometric consistency across multiple views.<n>We introduce CalibAnyView, a unified formulation that supports an arbitrary number of input views.
  arXiv  Detail & Related papers  (2026-05-14T09:32:12Z)
• Tri-Prompting: Video Diffusion with Unified Control over Scene, Subject, and Motion [69.64308172688368]
  Tri-Prompting is a unified framework that integrates scene composition, multi-view subject consistency, and motion control.<n>Results demonstrate that Tri-Prompting significantly outperforms specialized baselines such as Phantom and DaS in multi-view subject identity, 3D consistency, and motion accuracy.
  arXiv  Detail & Related papers  (2026-03-16T17:59:05Z)
• ViewMorpher3D: A 3D-aware Diffusion Framework for Multi-Camera Novel View Synthesis in Autonomous Driving [20.935790354765604]
  We introduce ViewMorpher3D, a multi-view image enhancement framework based on image diffusion models.<n>Unlike single-view approaches, ViewMorpher3D jointly processes a set of rendered views conditioned on camera poses, 3D geometric priors, and temporally adjacent or spatially overlapping reference views.<n>Our framework accommodates variable numbers of cameras and flexible reference/target view configurations, making it adaptable to diverse sensor setups.
  arXiv  Detail & Related papers  (2026-01-12T13:44:14Z)
• MultiViPerFrOG: A Globally Optimized Multi-Viewpoint Perception Framework for Camera Motion and Tissue Deformation [18.261678529996104]
  We propose a framework that can flexibly integrate the output of low-level perception modules with kinematic and scene-modeling priors. Overall, our method shows robustness to combined noisy input measures and can process hundreds of points in a few milliseconds.
  arXiv  Detail & Related papers  (2024-08-08T10:55:55Z)
• Learning Robust Multi-Scale Representation for Neural Radiance Fields from Unposed Images [65.41966114373373]
  We present an improved solution to the neural image-based rendering problem in computer vision. The proposed approach could synthesize a realistic image of the scene from a novel viewpoint at test time.
  arXiv  Detail & Related papers  (2023-11-08T08:18:23Z)
• Decaf: Monocular Deformation Capture for Face and Hand Interactions [77.75726740605748]
  This paper introduces the first method that allows tracking human hands interacting with human faces in 3D from single monocular RGB videos. We model hands as articulated objects inducing non-rigid face deformations during an active interaction. Our method relies on a new hand-face motion and interaction capture dataset with realistic face deformations acquired with a markerless multi-view camera system.
  arXiv  Detail & Related papers  (2023-09-28T17:59:51Z)
• Cross-View Cross-Scene Multi-View Crowd Counting [56.83882084112913]
  Multi-view crowd counting has been previously proposed to utilize multi-cameras to extend the field-of-view of a single camera. We propose a cross-view cross-scene (CVCS) multi-view crowd counting paradigm, where the training and testing occur on different scenes with arbitrary camera layouts.
  arXiv  Detail & Related papers  (2022-05-03T15:03:44Z)
• Self-supervised Human Detection and Segmentation via Multi-view Consensus [116.92405645348185]
  We propose a multi-camera framework in which geometric constraints are embedded in the form of multi-view consistency during training. We show that our approach outperforms state-of-the-art self-supervised person detection and segmentation techniques on images that visually depart from those of standard benchmarks.
  arXiv  Detail & Related papers  (2020-12-09T15:47:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.