OnlineHMR: Video-based Online World-Grounded Human Mesh Recovery

• URL: http://arxiv.org/abs/2603.17355v1
• Date: Wed, 18 Mar 2026 04:33:01 GMT
• Title: OnlineHMR: Video-based Online World-Grounded Human Mesh Recovery
• Abstract summary: OnlineHMR is a fully online framework that satisfies four essential criteria of online processing.<n>OnlineHMR enables streaming inference via a causal key-value cache design and a curated sliding-window learning strategy.<n> Experimental results show that our method achieves performance comparable to existing chunk-based approaches.
• Score: 15.300739456656906
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human mesh recovery (HMR) models 3D human body from monocular videos, with recent works extending it to world-coordinate human trajectory and motion reconstruction. However, most existing methods remain offline, relying on future frames or global optimization, which limits their applicability in interactive feedback and perception-action loop scenarios such as AR/VR and telepresence. To address this, we propose OnlineHMR, a fully online framework that jointly satisfies four essential criteria of online processing, including system-level causality, faithfulness, temporal consistency, and efficiency. Built upon a two-branch architecture, OnlineHMR enables streaming inference via a causal key-value cache design and a curated sliding-window learning strategy. Meanwhile, a human-centric incremental SLAM provides online world-grounded alignment under physically plausible trajectory correction. Experimental results show that our method achieves performance comparable to existing chunk-based approaches on the standard EMDB benchmark and highly dynamic custom videos, while uniquely supporting online processing. Page and code are available at https://tsukasane.github.io/Video-OnlineHMR/.

Related papers

• OARS: Process-Aware Online Alignment for Generative Real-World Image Super-Resolution [47.35253230699505]
  OARS is a process-aware online alignment framework that evaluates the LR to SR transition by jointly modeling fidelity preservation and perceptual gain.<n>We show consistent perceptual improvements while maintaining fidelity, achieving state-of-the-art performance on Real-ISR benchmarks.
  arXiv  Detail & Related papers  (2026-03-13T09:12:46Z)
• OnlineX: Unified Online 3D Reconstruction and Understanding with Active-to-Stable State Evolution [34.8105632078785]
  We introduce OnlineX, a feed-forward framework that reconstructs both 3D visual appearance and language fields in an online manner using only streaming images.<n>Our framework decouples the memory state into a dedicated active state and a persistent stable state, and then cohesively fuses the information from the former into the latter to achieve both fidelity and stability.
  arXiv  Detail & Related papers  (2026-03-02T17:52:02Z)
• UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Pass [83.7071371474926]
  UniSH is a unified, feed-forward framework for joint metric-scale 3D scene and human reconstruction.<n>Our framework bridges strong, disparate priors from scene reconstruction and HMR.<n>Our model achieves state-of-the-art performance on human-centric scene reconstruction.
  arXiv  Detail & Related papers  (2026-01-03T16:06:27Z)
• Integrating Offline Pre-Training with Online Fine-Tuning: A Reinforcement Learning Approach for Robot Social Navigation [3.5801655940143413]
  This paper proposes a novel offline-to-online fine-tuning algorithm for robot social navigation by integrating Return-to-Go (RTG)<n>Our algorithm features a Transformer-poral fusion model designed to precisely estimate RTG values in real-time by jointly encoding temporal pedestrian motion patterns and spatial crowd dynamics.<n>Experiments in simulated social navigation environments demonstrate that our method achieves a higher success rate and lower collision rate compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-10-01T03:37:02Z)
• Track-On2: Enhancing Online Point Tracking with Memory [57.820749134569574]
  We extend our prior model Track-On into Track-On2, a simple and efficient transformer-based model for online long-term tracking.<n>Track-On2 improves both performance and efficiency through architectural refinements, more effective use of memory, and improved synthetic training strategies.
  arXiv  Detail & Related papers  (2025-09-23T15:00:18Z)
• UI-S1: Advancing GUI Automation via Semi-online Reinforcement Learning [78.86567400365392]
  We present Semi-online Reinforcement Learning, a novel paradigm that simulates online RL on offline trajectories.<n>To capture long-term training signals, Semi-online RL introduces discounted future returns into the reward computation.<n>Experiments show that ours Semi-online RL achieves SOTA performance among 7B models across four dynamic benchmarks.
  arXiv  Detail & Related papers  (2025-09-15T03:24:08Z)
• Video-Enhanced Offline Reinforcement Learning: A Model-Based Approach [55.76249793590689]
  Video-Enhanced Offline RL (VeoRL) is a model-based method that constructs an interactive world model from diverse, unlabeled video data readily available online.<n>VeoRL achieves substantial performance gains across visual control tasks in robotic manipulation, autonomous driving, and open-world video games.
  arXiv  Detail & Related papers  (2025-05-10T00:54:12Z)
• Endo3R: Unified Online Reconstruction from Dynamic Monocular Endoscopic Video [35.241054116681426]
  Endo3R is a unified 3D foundation model for online scale-consistent reconstruction from monocular surgical video.<n>Our model unifies the tasks by predicting globally aligned pointmaps, scale-consistent video depths, and camera parameters without any offline optimization.
  arXiv  Detail & Related papers  (2025-04-04T06:05:22Z)
• Online hand gesture recognition using Continual Graph Transformers [1.3927943269211591]
  We propose a novel online recognition system designed for real-time skeleton sequence streaming.<n>Our approach achieves state-of-the-art accuracy and significantly reduces false positive rates, making it a compelling solution for real-time applications.<n>The proposed system can be seamlessly integrated into various domains, including human-robot collaboration and assistive technologies.
  arXiv  Detail & Related papers  (2025-02-20T17:27:55Z)
• Online Video Understanding: OVBench and VideoChat-Online [22.814813541695997]
  Multimodal Large Language Models (MLLMs) have significantly progressed in offline video understanding.<n>Applying these models to real-world scenarios, such as autonomous driving and human-computer interaction, presents unique challenges.<n>This paper presents systematic efforts from three perspectives: evaluation benchmark, model architecture, and training strategy.
  arXiv  Detail & Related papers  (2024-12-31T18:17:05Z)
• OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentation [75.07460026246582]
  Referring video object segmentation (RVOS) aims at segmenting an object in a video following human instruction. Current state-of-the-art methods fall into an offline pattern, in which each clip independently interacts with text embedding. We propose a simple yet effective online model using explicit query propagation, named OnlineRefer.
  arXiv  Detail & Related papers  (2023-07-18T15:43:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.