VGGT-HPE: Reframing Head Pose Estimation as Relative Pose Prediction

• URL: http://arxiv.org/abs/2604.10106v1
• Date: Sat, 11 Apr 2026 08:55:15 GMT
• Title: VGGT-HPE: Reframing Head Pose Estimation as Relative Pose Prediction
• Abstract summary: VGGT-HPE is a relative head pose estimator built upon a general-purpose geometry foundation model.<n>Our method sidesteps the need for an implicit anchor by reducing the problem to estimating a geometric displacement from an explicitly provided anchor with a known pose.<n>Despite zero real-world training data, VGGT-HPE achieves state-of-the-art results on the BIWI benchmark.
• Score: 39.937064557481555
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Monocular head pose estimation is traditionally formulated as direct regression from a single image to an absolute pose. This paradigm forces the network to implicitly internalize a dataset-specific canonical reference frame. In this work, we argue that predicting the relative rigid transformation between two observed head configurations is a fundamentally easier and more robust formulation. We introduce VGGT-HPE, a relative head pose estimator built upon a general-purpose geometry foundation model. Finetuned exclusively on synthetic facial renderings, our method sidesteps the need for an implicit anchor by reducing the problem to estimating a geometric displacement from an explicitly provided anchor with a known pose. As a practical benefit, the relative formulation also allows the anchor to be chosen at test time - for instance, a near-neutral frame or a temporally adjacent one - so that the prediction difficulty can be controlled by the application. Despite zero real-world training data, VGGT-HPE achieves state-of-the-art results on the BIWI benchmark, outperforming established absolute regression methods trained on mixed and real datasets. Through controlled easy- and hard-pair benchmarks, we also systematically validate our core hypothesis: relative prediction is intrinsically more accurate than absolute regression, with the advantage scaling alongside the difficulty of the target pose. Project page and code: https://vasilikivas.github.io/VGGT-HPE

Related papers

• Gravity-aware partially calibrated absolute pose estimation from affine- or rotation-covariant features [8.4929502957148]
  Inertial measurement units (IMUs) are now standard in most consumer devices, such as smartphones, drones, and extended reality (XR) headsets.<n>By fusing visual and inertial data, localization systems gain significantly in speed and robustness compared to vision-only or IMU-only approaches.<n>Traditional pose estimation methods fail to utilize the local geometric information embedded in feature descriptors like SIFT.<n>Recent work has proved the advantages of leveraging this information for relative and absolute pose estimation, but its application to partially calibrated absolute pose estimation remains unexplored.
  arXiv  Detail & Related papers  (2026-08-20T13:52:54Z)
• STEP: Score-Based Temporal Energy for Human Pose Video Anomaly Detection [13.87765205634944]
  Skeleton-based Video Anomaly Detection (VAD) offers a robust, privacy-preserving solution for identifying abnormal behaviors.<n>Recent methods train Energy-Based Models (EBMs) via Denoising Score Matching (DSM)<n>However, directly injecting noise, required for training, into raw joint coordinates creates physically impossible poses.<n>We introduce STEP, a simple framework that utilizes Principal Component Analysis (PCA) to project pose sequences into a compact, whitened PC-space.
  arXiv  Detail & Related papers  (2026-08-20T13:02:19Z)
• Pose Anything Anywhere:Model-free Object Poses from Arbitrary References [64.58017857473774]
  We present PANY, a model-free framework that seamlessly supports both RGB and RGB-D inputs.<n>It operates on one or sparse pose-free reference views, and generalizes effectively to novel objects.<n>It achieves state-of-the-art performance across multiple benchmarks, substantially outperforming existing model-free methods.
  arXiv  Detail & Related papers  (2026-06-22T17:23:57Z)
• Object Pose Transformer: Unifying Unseen Object Pose Estimation [54.20344997573707]
  Learning model-free object pose estimation for unseen instances remains a fundamental challenge in 3D vision.<n>ours jointly predicts depth, point maps, camera parameters, and normalized object coordinates from RGB inputs.<n>ours is camera-agnostic, learning camera intrinsics on-the-fly and supporting optional depth input for metric-scale recovery.
  arXiv  Detail & Related papers  (2026-03-24T16:04:16Z)
• ER-Pose: Rethinking Keypoint-Driven Representation Learning for Real-Time Human Pose Estimation [2.2248409468073143]
  Single-stage multi-person pose estimation aims to jointly perform human localization and keypoint prediction.<n>Box-driven single-stage pose estimation is implicitly constrained by bounding-box supervision during training.<n>We propose a keypoint-driven learning paradigm that elevates pose estimation to a primary prediction objective.<n>On MS COCO and CrowdPose, ER-Pose-n achieves AP improvements of 3.2/6.7 without pre-training and 7.4/4.9 with pre-training respectively.
  arXiv  Detail & Related papers  (2026-03-09T17:49:46Z)
• Score-Guided Proximal Projection: A Unified Geometric Framework for Rectified Flow Editing [1.0312968200748118]
  Rectified Flow models achieve state-of-the-art generation quality, yet controlling them for precise tasks remains a challenge.<n>Current approaches bifurcate into inversion-based guidance, which suffers from "geometric locking"<n>We propose Score-Guided Proximal Projection, a unified framework that bridges the gap between deterministic optimization and contraction sampling.
  arXiv  Detail & Related papers  (2026-03-05T23:44:45Z)
• UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Image [86.7128543480229]
  Unseen object pose estimation methods often rely on CAD models or multiple reference views.<n>To simplify reference acquisition, we aim to estimate the unseen object's pose through a single unposed RGB-D reference image.<n>We present a novel approach and benchmark, termed UNOPose, for unseen one-reference-based object pose estimation.
  arXiv  Detail & Related papers  (2024-11-25T05:36:00Z)
• Bidirectional Regression for Monocular 6DoF Head Pose Estimation and Reference System Alignment [23.65735794927899]
  TRGv2 is a lightweight extension of our Translation, Rotation, and Geometry (TRG) network.<n>It infers facial landmarks and 6DoF pose through an iterative refinement loop with landmark-to-image projection.<n>To improve generalization to out-of-distribution data, TRGv2 regresses correction parameters instead of directly predicting translation.<n>We identify a previously overlooked source of bias in cross-dataset evaluations due to inconsistent head center definitions.
  arXiv  Detail & Related papers  (2024-07-19T09:05:49Z)
• Poseur: Direct Human Pose Regression with Transformers [119.79232258661995]
  We propose a direct, regression-based approach to 2D human pose estimation from single images. Our framework is end-to-end differentiable, and naturally learns to exploit the dependencies between keypoints. Ours is the first regression-based approach to perform favorably compared to the best heatmap-based pose estimation methods.
  arXiv  Detail & Related papers  (2022-01-19T04:31:57Z)
• TFPose: Direct Human Pose Estimation with Transformers [83.03424247905869]
  We formulate the pose estimation task into a sequence prediction problem that can effectively be solved by transformers. Our framework is simple and direct, bypassing the drawbacks of the heatmap-based pose estimation. Experiments on the MS-COCO and MPII datasets demonstrate that our method can significantly improve the state-of-the-art of regression-based pose estimation.
  arXiv  Detail & Related papers  (2021-03-29T04:18:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.