Roll Your Eyes: Gaze Redirection via Explicit 3D Eyeball Rotation

• URL: http://arxiv.org/abs/2508.06136v2
• Date: Thu, 18 Sep 2025 03:50:55 GMT
• Title: Roll Your Eyes: Gaze Redirection via Explicit 3D Eyeball Rotation
• Authors: YoungChan Choi, HengFei Wang, YiHua Cheng, Boeun Kim, Hyung Jin Chang, YoungGeun Choi, Sang-Il Choi,
• Abstract summary: We propose a novel 3D gaze redirection framework that leverages an explicit 3D eyeball structure.<n>Our method generates photorealistic images that faithfully reproduce the desired gaze direction by explicitly rotating and translating the 3D eyeball structure.
• Score: 27.442858956985344
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel 3D gaze redirection framework that leverages an explicit 3D eyeball structure. Existing gaze redirection methods are typically based on neural radiance fields, which employ implicit neural representations via volume rendering. Unlike these NeRF-based approaches, where the rotation and translation of 3D representations are not explicitly modeled, we introduce a dedicated 3D eyeball structure to represent the eyeballs with 3D Gaussian Splatting (3DGS). Our method generates photorealistic images that faithfully reproduce the desired gaze direction by explicitly rotating and translating the 3D eyeball structure. In addition, we propose an adaptive deformation module that enables the replication of subtle muscle movements around the eyes. Through experiments conducted on the ETH-XGaze dataset, we demonstrate that our framework is capable of generating diverse novel gaze images, achieving superior image quality and gaze estimation accuracy compared to previous state-of-the-art methods.

Related papers

• EVolSplat: Efficient Volume-based Gaussian Splatting for Urban View Synthesis [61.1662426227688]
  Existing NeRF and 3DGS-based methods show promising results in achieving photorealistic renderings but require slow, per-scene optimization.<n>We introduce EVolSplat, an efficient 3D Gaussian Splatting model for urban scenes that works in a feed-forward manner.
  arXiv  Detail & Related papers  (2025-03-26T02:47:27Z)
• Neural Radiance and Gaze Fields for Visual Attention Modeling in 3D Environments [6.311952721757901]
  We introduce Neural Radiance and Gaze Fields (NeRGs) as a novel approach for representing visual attention patterns in 3D scenes.<n>Our system renders a 2D view of a 3D scene with a pre-trained Neural Radiance Field (NeRF) and visualizes the gaze field for arbitrary observer positions.
  arXiv  Detail & Related papers  (2025-03-10T20:18:42Z)
• NovelGS: Consistent Novel-view Denoising via Large Gaussian Reconstruction Model [57.92709692193132]
  NovelGS is a diffusion model for Gaussian Splatting given sparse-view images. We leverage the novel view denoising through a transformer-based network to generate 3D Gaussians.
  arXiv  Detail & Related papers  (2024-11-25T07:57:17Z)
• Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and Editing [53.05069432989608]
  We present a novel framework for generating 3D human heads with remarkable flexibility. Our method facilitates the creation of diverse and realistic 3D human heads with fine-grained editing over facial features and expressions.
  arXiv  Detail & Related papers  (2023-12-05T19:05:58Z)
• Text2Control3D: Controllable 3D Avatar Generation in Neural Radiance Fields using Geometry-Guided Text-to-Image Diffusion Model [39.64952340472541]
  We propose a controllable text-to-3D avatar generation method whose facial expression is controllable. Our main strategy is to construct the 3D avatar in Neural Radiance Fields (NeRF) optimized with a set of controlled viewpoint-aware images. We demonstrate the empirical results and discuss the effectiveness of our method.
  arXiv  Detail & Related papers  (2023-09-07T08:14:46Z)
• Accurate Gaze Estimation using an Active-gaze Morphable Model [9.192482716410511]
  Rather than regressing gaze direction directly from images, we show that adding a 3D shape model can improve gaze estimation accuracy. We equip this with a geometric vergence model of gaze to give an active-gaze 3DMM' Our method can learn with only the ground truth gaze target point and the camera parameters, without access to the ground truth gaze origin points.
  arXiv  Detail & Related papers  (2023-01-30T18:51:14Z)
• GazeNeRF: 3D-Aware Gaze Redirection with Neural Radiance Fields [100.53114092627577]
  Existing gaze redirection methods operate on 2D images and struggle to generate 3D consistent results. We build on the intuition that the face region and eyeballs are separate 3D structures that move in a coordinated yet independent fashion.
  arXiv  Detail & Related papers  (2022-12-08T13:19:11Z)
• Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatars [36.4402388864691]
  3D-aware generative adversarial networks (GANs) synthesize high-fidelity and multi-view-consistent facial images using only collections of single-view 2D imagery. Recent efforts incorporate 3D Morphable Face Model (3DMM) to describe deformation in generative radiance fields either explicitly or implicitly. We propose a novel 3D GAN framework for unsupervised learning of generative, high-quality and 3D-consistent facial avatars from unstructured 2D images.
  arXiv  Detail & Related papers  (2022-11-21T06:40:46Z)
• Free-HeadGAN: Neural Talking Head Synthesis with Explicit Gaze Control [54.079327030892244]
  Free-HeadGAN is a person-generic neural talking head synthesis system. We show that modeling faces with sparse 3D facial landmarks are sufficient for achieving state-of-the-art generative performance.
  arXiv  Detail & Related papers  (2022-08-03T16:46:08Z)
• Disentangled3D: Learning a 3D Generative Model with Disentangled Geometry and Appearance from Monocular Images [94.49117671450531]
  State-of-the-art 3D generative models are GANs which use neural 3D volumetric representations for synthesis. In this paper, we design a 3D GAN which can learn a disentangled model of objects, just from monocular observations.
  arXiv  Detail & Related papers  (2022-03-29T22:03:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.