A Structure-aware and Motion-adaptive Framework for 3D Human Pose Estimation with Mamba

• URL: http://arxiv.org/abs/2507.19852v1
• Date: Sat, 26 Jul 2025 07:59:52 GMT
• Title: A Structure-aware and Motion-adaptive Framework for 3D Human Pose Estimation with Mamba
• Abstract summary: We propose a structure-aware and motion-adaptive framework to capture spatial joint topology.<n>Through the above key modules, our algorithm enables structure-aware and motion-adaptive pose lifting.
• Score: 18.376143217023934
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent Mamba-based methods for the pose-lifting task tend to model joint dependencies by 2D-to-1D mapping with diverse scanning strategies. Though effective, they struggle to model intricate joint connections and uniformly process all joint motion trajectories while neglecting the intrinsic differences across motion characteristics. In this work, we propose a structure-aware and motion-adaptive framework to capture spatial joint topology along with diverse motion dynamics independently, named as SAMA. Specifically, SAMA consists of a Structure-aware State Integrator (SSI) and a Motion-adaptive State Modulator (MSM). The Structure-aware State Integrator is tasked with leveraging dynamic joint relationships to fuse information at both the joint feature and state levels in the state space, based on pose topology rather than sequential state transitions. The Motion-adaptive State Modulator is responsible for joint-specific motion characteristics recognition, thus applying tailored adjustments to diverse motion patterns across different joints. Through the above key modules, our algorithm enables structure-aware and motion-adaptive pose lifting. Extensive experiments across multiple benchmarks demonstrate that our algorithm achieves advanced results with fewer computational costs.

Related papers

• Following Motion for Sequential Modeling in Video Frame Interpolation [22.71397762551783]
  State Space Models (SSMs) have surfaced as a promising architecture in Video Frame Interpolation (VFI)<n>We propose Motion-Guided Mamba for Video Frame Interpolation (MGMVFI), an adaptation of the selective state space model tailored explicitly for VFI.
  arXiv  Detail & Related papers  (2026-08-24T06:43:48Z)
• AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling [60.13338774249153]
  We introduce OmniHuMo, a dataset of over 5,000 hours of motion and 3.2 million sequences with precisely aligned multimodal annotations.<n>We propose AnyMo, a unified multimodal framework combining a Residual FSQ-based motion tokenizer with a scalable masked modeling transformer.<n>Experiments show that AnyMo achieves high-fidelity synthesis while offering flexible control over both spatial and stylistic attributes.
  arXiv  Detail & Related papers  (2026-05-28T07:15:19Z)
• Interactive State Space Model with Cross-Modal Local Scanning for Depth Super-Resolution [26.94624454006905]
  Guided depth super-resolution (GDSR) reconstructs HR depth maps from LR inputs with HR RGB guidance.<n>Existing methods either model each modality independently or rely on computationally expensive attention mechanisms with quadratic complexity.<n>We propose a novel GDSR framework centered around the Interactive State Space Model.
  arXiv  Detail & Related papers  (2026-05-12T10:45:36Z)
• MOSA: Motion-Guided Semantic Alignment for Dynamic Scene Graph Generation [27.7774056956553]
  This paper proposes a motion-guided semantic alignment method for Dynamic Scene Graph Generation (DSGG)<n>MoSA encodes object-pair motion attributes such as distance, velocity, motion persistence, and directional consistency.<n>Cross-modal Action Semantic Matching mechanism aligns visual relationship features with text embeddings of relationship categories.
  arXiv  Detail & Related papers  (2026-04-21T16:18:26Z)
• Hypergraph-State Collaborative Reasoning for Multi-Object Tracking [63.32950991964095]
  We propose a collaborative reasoning framework that enhances motion estimation through joint inference among multiple correlated objects.<n>By allowing objects with similar motion states to mutually constrain and refine each other, our framework stabilizes noisy trajectories and infers plausible motion continuity even when target is occluded.
  arXiv  Detail & Related papers  (2026-04-14T12:37:39Z)
• Training-free Motion Factorization for Compositional Video Generation [57.819757612370374]
  We propose a motion factorization framework that decomposes complex motion into three primary categories: motionlessness, rigid motion, and non-rigid motion.<n>Our framework achieves impressive performance in motion synthesis on real-world benchmarks.
  arXiv  Detail & Related papers  (2026-03-10T02:27:48Z)
• From Perception to Action: An Interactive Benchmark for Vision Reasoning [51.11355591375073]
  Causal Hierarchy of Actions and Interactions (CHAIN) benchmark designed to evaluate whether models can understand, plan, and execute structured action sequences grounded in physical constraints.<n> CHAIN shifts evaluation from passive perception to active problem solving, spanning tasks such as interlocking mechanical puzzles and 3D stacking and packing.<n>Our results show that top-performing models still struggle to internalize physical structure and causal constraints, often failing to produce reliable long-horizon plans and cannot robustly translate perceived structure into effective actions.
  arXiv  Detail & Related papers  (2026-02-24T15:33:02Z)
• UniMotion: A Unified Motion Framework for Simulation, Prediction and Planning [60.55873455475112]
  We propose UniMotion, a unified motion framework that captures shared structures across motion tasks.<n>UniMotion employs dedicated interaction modes and tailored training strategies to simultaneously support these motion tasks.<n>With further fine-tuning, UniMotion achieves state-of-the-art performance across a range of motion tasks, establishing it as a versatile and scalable solution for autonomous driving.
  arXiv  Detail & Related papers  (2026-01-31T07:12:26Z)
• A3D: Adaptive Affordance Assembly with Dual-Arm Manipulation [25.57693108471214]
  We propose A3D, a framework which learns adaptive affordances to identify optimal support and stabilization locations on furniture parts.<n>To handle evolving assembly states, we introduce an adaptive module that uses interaction feedback to adjust dynamically support strategies.<n>Experiments demonstrate that our framework generalizes effectively to diverse part geometries and furniture categories in both simulation and real-world settings.
  arXiv  Detail & Related papers  (2026-01-16T08:21:42Z)
• MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editing [53.98607267063729]
  MotionVerse is a framework to comprehend, generate, and edit human motion in both single-person and multi-person scenarios.<n>We employ a motion tokenizer with residual quantization, which converts continuous motion sequences into multi-stream discrete tokens.<n>We also introduce a textitDelay Parallel Modeling strategy, which temporally staggers the encoding of residual token streams.
  arXiv  Detail & Related papers  (2025-09-28T04:20:56Z)
• DMSC: Dynamic Multi-Scale Coordination Framework for Time Series Forecasting [14.176801586961286]
  Time Series Forecasting (TSF) faces persistent challenges in modeling intricate temporal dependencies across different scales.<n>We propose a novel Dynamic Multi-Scale Coordination Framework (DMSC) with Multi-Scale Patch Decomposition block (EMPD), Triad Interaction Block (TIB) and Adaptive Scale Routing MoE block (ASR-MoE)<n>EMPD is designed as a built-in component to dynamically segment sequences into hierarchical patches with exponentially scaled granularities.<n>TIB then jointly models intra-patch, inter-patch, and cross-variable dependencies within each layer's decomposed representations.
  arXiv  Detail & Related papers  (2025-08-03T13:11:52Z)
• Structural Similarity-Inspired Unfolding for Lightweight Image Super-Resolution [88.20464308588889]
  We propose a Structural Similarity-Inspired Unfolding (SSIU) method for efficient image SR.<n>This method is designed through unfolding an SR optimization function constrained by structural similarity.<n>Our model outperforms current state-of-the-art models, boasting lower parameter counts and reduced memory consumption.
  arXiv  Detail & Related papers  (2025-06-13T14:29:40Z)
• GENMO: A GENeralist Model for Human MOtion [64.16188966024542]
  We present GENMO, a unified Generalist Model for Human Motion that bridges motion estimation and generation in a single framework.<n>Our key insight is to reformulate motion estimation as constrained motion generation, where the output motion must precisely satisfy observed conditioning signals.<n>Our novel architecture handles variable-length motions and mixed multimodal conditions (text, audio, video) at different time intervals, offering flexible control.
  arXiv  Detail & Related papers  (2025-05-02T17:59:55Z)
• ReCoM: Realistic Co-Speech Motion Generation with Recurrent Embedded Transformer [58.49950218437718]
  We present ReCoM, an efficient framework for generating high-fidelity and generalizable human body motions synchronized with speech.<n>The core innovation lies in the Recurrent Embedded Transformer (RET), which integrates Dynamic Embedding Regularization (DER) into a Vision Transformer (ViT) core architecture.<n>To enhance model robustness, we incorporate the proposed DER strategy, which equips the model with dual capabilities of noise resistance and cross-domain generalization.
  arXiv  Detail & Related papers  (2025-03-27T16:39:40Z)
• A Decoupled Spatio-Temporal Framework for Skeleton-based Action Segmentation [89.86345494602642]
  Existing methods are limited in weak-temporal modeling capability. We propose a Decoupled Scoupled Framework (DeST) to address the issues. DeST significantly outperforms current state-of-the-art methods with less computational complexity.
  arXiv  Detail & Related papers  (2023-12-10T09:11:39Z)
• HiH: A Multi-modal Hierarchy in Hierarchy Network for Unconstrained Gait Recognition [3.431054404120758]
  We present a multi-modal Hierarchy in Hierarchy network (HiH) that integrates silhouette and pose sequences for robust gait recognition. HiH features a main branch that utilizes Hierarchical Gait Decomposer modules for depth-wise and intra-module hierarchical examination of general gait patterns from silhouette data. An auxiliary branch, based on 2D joint sequences, enriches the spatial and temporal aspects of gait analysis.
  arXiv  Detail & Related papers  (2023-11-19T03:25:14Z)
• Hierarchical Spatio-Temporal Representation Learning for Gait Recognition [6.877671230651998]
  Gait recognition is a biometric technique that identifies individuals by their unique walking styles. We propose a hierarchical-temporal representation learning framework for extracting gait features from coarse to fine. Our method outperforms the state-of-the-art while maintaining a reasonable balance between model accuracy and complexity.
  arXiv  Detail & Related papers  (2023-07-19T09:30:00Z)
• GraMMaR: Ground-aware Motion Model for 3D Human Motion Reconstruction [61.833152949826946]
  We propose a novel Ground-aware Motion Model for 3D Human Motion Reconstruction, named GraMMaR. GraMMaR learns the distribution of transitions in both pose and interaction between every joint and ground plane at each time step of a motion sequence. It is trained to explicitly promote consistency between the motion and distance change towards the ground.
  arXiv  Detail & Related papers  (2023-06-29T07:22:20Z)
• Interactive Character Control with Auto-Regressive Motion Diffusion Models [18.727066177880708]
  We propose A-MDM (Auto-regressive Motion Diffusion Model) for real-time motion synthesis. Our conditional diffusion model takes an initial pose as input, and auto-regressively generates successive motion frames conditioned on previous frame. We introduce a suite of techniques for incorporating interactive controls into A-MDM, such as task-oriented sampling, in-painting, and hierarchical reinforcement learning.
  arXiv  Detail & Related papers  (2023-06-01T07:48:34Z)
• SoMoFormer: Social-Aware Motion Transformer for Multi-Person Motion Prediction [10.496276090281825]
  We propose a novel Social-Aware Motion Transformer (SoMoFormer) to model individual motion and social interactions in a joint manner. SoMoFormer extracts motion features from sub-sequences in displacement trajectory space to learn both local and global pose dynamics for each individual. In addition, we devise a novel social-aware motion attention mechanism in SoMoFormer to further optimize dynamics representations and capture interaction dependencies simultaneously.
  arXiv  Detail & Related papers  (2022-08-19T08:57:34Z)
• MoDi: Unconditional Motion Synthesis from Diverse Data [51.676055380546494]
  We present MoDi, an unconditional generative model that synthesizes diverse motions. Our model is trained in a completely unsupervised setting from a diverse, unstructured and unlabeled motion dataset. We show that despite the lack of any structure in the dataset, the latent space can be semantically clustered.
  arXiv  Detail & Related papers  (2022-06-16T09:06:25Z)
• EAN: Event Adaptive Network for Enhanced Action Recognition [66.81780707955852]
  We propose a unified action recognition framework to investigate the dynamic nature of video content. First, when extracting local cues, we generate the spatial-temporal kernels of dynamic-scale to adaptively fit the diverse events. Second, to accurately aggregate these cues into a global video representation, we propose to mine the interactions only among a few selected foreground objects by a Transformer.
  arXiv  Detail & Related papers  (2021-07-22T15:57:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.