Sapiens2

• URL: http://arxiv.org/abs/2604.21681v1
• Date: Thu, 23 Apr 2026 13:45:32 GMT
• Title: Sapiens2
• Abstract summary: We present Sapiens2, a model family of high-resolution transformers for human-centric vision.<n>Our model sizes range from 0.4 to 5 billion parameters, with native 1K resolution and hierarchical variants that support 4K.
• Score: 28.286198457494596
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Sapiens2, a model family of high-resolution transformers for human-centric vision focused on generalization, versatility, and high-fidelity outputs. Our model sizes range from 0.4 to 5 billion parameters, with native 1K resolution and hierarchical variants that support 4K. Sapiens2 substantially improves over its predecessor in both pretraining and post-training. First, to learn features that capture low-level details (for dense prediction) and high-level semantics (for zero-shot or few-label settings), we combine masked image reconstruction with self-distilled contrastive objectives. Our evaluations show that this unified pretraining objective is better suited for a wider range of downstream tasks. Second, along the data axis, we pretrain on a curated dataset of 1 billion high-quality human images and improve the quality and quantity of task annotations. Third, architecturally, we incorporate advances from frontier models that enable longer training schedules with improved stability. Our 4K models adopt windowed attention to reason over longer spatial context and are pretrained with 2K output resolution. Sapiens2 sets a new state-of-the-art and improves over the first generation on pose (+4 mAP), body-part segmentation (+24.3 mIoU), normal estimation (45.6% lower angular error) and extends to new tasks such as pointmap and albedo estimation. Code: https://github.com/facebookresearch/sapiens2

Related papers

• 4KLSDB: A Large-Scale Dataset for 4K Image Restoration and Generation [23.288582652884642]
  Current publicly available datasets lack both the native 4K resolution and the extensive scale necessary for training state-of-the-art models.<n>We introduce a 4K Large Scale dataset and Benchmark (4KLSDB) consisting of 129,484 carefully curated 4K resolution images spanning multiple categories.<n>We demonstrate 4KLSDB's effectiveness by training representative super-resolution and diffusion models, observing significant improvements on native 4K benchmarks.
  arXiv  Detail & Related papers  (2026-05-23T22:49:04Z)
• Towards Scalable Pre-training of Visual Tokenizers for Generation [41.785568766118594]
  We present a unified visual tokenizer pre-training framework, pioneering the joint optimization of image-text contrastive, self-supervised, and reconstruction losses.<n>After large-scale pre-training, our tokenizer delivers a competitive profile (78.2 zero-shot accuracy and 0.36 rFID on ImageNet) and 4.1 times faster convergence on generation compared to advanced distillation methods.
  arXiv  Detail & Related papers  (2025-12-15T18:59:54Z)
• Silhouette-based Gait Foundation Model [56.27974816297294]
  Building a unified gait foundation model requires addressing two longstanding barriers: Scalability and Generalization.<n>We introduce FoundationGait, the first scalable, self-supervised pretraining framework for gait understanding.
  arXiv  Detail & Related papers  (2025-11-30T01:53:41Z)
• PriVi: Towards A General-Purpose Video Model For Primate Behavior In The Wild [50.656578456979496]
  We introduce PriVi, a large-scale primate-centric video pretraining dataset.<n>We pretrain V-JEPA, a large-scale video model, on PriVi to learn primate-specific representations.<n>Results demonstrate that primate-centric pretraining substantially improves data efficiency and generalization.
  arXiv  Detail & Related papers  (2025-11-12T19:27:40Z)
• UP2You: Fast Reconstruction of Yourself from Unconstrained Photo Collections [21.55668740343458]
  UP2You is a tuning-free solution for reconstructing high-fidelity 3D portraits from unconstrained in-the-wild 2D photos.<n>Central to UP2You is a pose-correlated feature aggregation module.<n>Experiments on 4D-Dress, PuzzleIOI, and in-the-wild captures demonstrate that UP2You consistently surpasses previous methods in both geometric accuracy and texture fidelity.
  arXiv  Detail & Related papers  (2025-09-29T14:06:00Z)
• 4KAgent: Agentic Any Image to 4K Super-Resolution [62.99433518118836]
  We present 4KAgent, a super-resolution generalist system designed to upscale any image to 4K resolution (and even higher, if applied iteratively)<n>4KAgent comprises three core components: (1) Profiling, a module that customizes the 4KAgent pipeline based on bespoke use cases; (2) A Perception Agent, which leverages vision-language models alongside image quality assessment experts to analyze the input image and make a tailored restoration plan; and (3) A Restoration Agent, which executes the plan, following a quality-driven mixture-of-expert policy to select the optimal output for each step.<n>We rigorously evaluate our 4KAgent
  arXiv  Detail & Related papers  (2025-07-09T17:59:19Z)
• DICEPTION: A Generalist Diffusion Model for Visual Perceptual Tasks [61.16389024252561]
  We develop a robust generalist perception model capable of addressing multiple tasks under constraints of computational resources and limited training data.<n>We leverage text-to-image diffusion models pre-trained on billions of images and successfully introduce our DICEPTION, a visual generalist model.<n> Exhaustive evaluations demonstrate that DICEPTION effectively tackles diverse perception tasks, even achieving performance comparable to SOTA single-task specialist models.
  arXiv  Detail & Related papers  (2025-02-24T13:51:06Z)
• Sapiens: Foundation for Human Vision Models [14.72839332332364]
  We present Sapiens, a family of models for four fundamental human-centric vision tasks. Our models support 1K high-resolution inference and are easy to adapt for individual tasks. We observe that self-supervised pretraining on a curated dataset of human images significantly boosts the performance for a diverse set of human-centric tasks.
  arXiv  Detail & Related papers  (2024-08-22T17:37:27Z)
• Model Inversion Attacks Through Target-Specific Conditional Diffusion Models [54.69008212790426]
  Model inversion attacks (MIAs) aim to reconstruct private images from a target classifier's training set, thereby raising privacy concerns in AI applications. Previous GAN-based MIAs tend to suffer from inferior generative fidelity due to GAN's inherent flaws and biased optimization within latent space. We propose Diffusion-based Model Inversion (Diff-MI) attacks to alleviate these issues.
  arXiv  Detail & Related papers  (2024-07-16T06:38:49Z)
• HoloHisto: End-to-end Gigapixel WSI Segmentation with 4K Resolution Sequential Tokenization [21.1691961979094]
  In digital pathology, the traditional method for deep learning-based image segmentation typically involves a two-stage process. We propose the holistic histopathology (HoloHisto) segmentation method to achieve end-to-end segmentation on gigapixel WSIs. Under the HoloHisto platform, we unveil a random 4K sampler that transcends ultra-high resolution, delivering 31 and 10 times more pixels than standard 2D and 3D patches.
  arXiv  Detail & Related papers  (2024-07-03T17:49:31Z)
• Learning Affinity-Aware Upsampling for Deep Image Matting [83.02806488958399]
  We show that learning affinity in upsampling provides an effective and efficient approach to exploit pairwise interactions in deep networks. In particular, results on the Composition-1k matting dataset show that A2U achieves a 14% relative improvement in the SAD metric against a strong baseline. Compared with the state-of-the-art matting network, we achieve 8% higher performance with only 40% model complexity.
  arXiv  Detail & Related papers  (2020-11-29T05:09:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.