Self-Improving 4D Perception via Self-Distillation

• URL: http://arxiv.org/abs/2604.08532v1
• Date: Thu, 09 Apr 2026 17:59:04 GMT
• Title: Self-Improving 4D Perception via Self-Distillation
• Abstract summary: SelfEvo is a self-trained framework that continually improves pre-trained multiview reconstruction using untruth videos.<n>We systematically study design choices that make self-improvement effective, including loss signals, forms of asymmetry, and other training strategies.<n>Overall, SelfEvo achieves up to 36.5% relative improvement in video depth estimation and 20.1% in camera estimation, without using any labeled data.
• Score: 49.729154812786696
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large-scale multi-view reconstruction models have made remarkable progress, but most existing approaches still rely on fully supervised training with ground-truth 3D/4D annotations. Such annotations are expensive and particularly scarce for dynamic scenes, limiting scalability. We propose SelfEvo, a self-improving framework that continually improves pretrained multi-view reconstruction models using unlabeled videos. SelfEvo introduces a self-distillation scheme using spatiotemporal context asymmetry, enabling self-improvement for learning-based 4D perception without external annotations. We systematically study design choices that make self-improvement effective, including loss signals, forms of asymmetry, and other training strategies. Across eight benchmarks spanning diverse datasets and domains, SelfEvo consistently improves pretrained baselines and generalizes across base models (e.g. VGGT and $π^3$), with significant gains on dynamic scenes. Overall, SelfEvo achieves up to 36.5% relative improvement in video depth estimation and 20.1% in camera estimation, without using any labeled data. Project Page: https://self-evo.github.io/.

Related papers

• Vernata: Self-Supervised Learning of LiDAR Point Representations [47.91691222306441]
  We present a multi-teacher distillation framework for self-supervised learning on outdoor LiDAR point clouds.<n>Our experiments demonstrate a significant performance improvement over Sonata baselines.<n>We show that the self-supervised approach maintains strong performance even in reduced-modality settings.
  arXiv  Detail & Related papers  (2026-08-07T07:51:39Z)
• IVRA: Improving Visual-Token Relations for Robot Action Policy with Training-Free Hint-Based Guidance [35.90972175891154]
  Many Vision-Language-Action (VLA) models flatten image patches into a 1D token sequence, weakening the 2D spatial cues needed for precise manipulation.<n>We introduce a lightweight, training-free method that improves spatial understanding by exploiting affinity hints already available in the model's built-in vision.
  arXiv  Detail & Related papers  (2026-01-22T18:57:13Z)
• Developing Vision-Language-Action Model from Egocentric Videos [14.1517430035289]
  Egocentric videos capture how humans manipulate objects and tools, providing diverse motion cues for learning object manipulation.<n>Prior studies that leverage such videos for training robot policies typically rely on auxiliary annotations, such as detailed hand-pose recordings.<n>In this work, we address this challenge by leveraging EgoScaler, a framework that extracts 6DoF object manipulation trajectories from egocentric videos.
  arXiv  Detail & Related papers  (2025-09-26T07:09:33Z)
• Towards More Diverse and Challenging Pre-training for Point Cloud Learning: Self-Supervised Cross Reconstruction with Decoupled Views [58.45662356692379]
  We propose Point-PQAE, a cross-reconstruction generative paradigm that first generates two decoupled point clouds/views and then reconstructs one from the other.<n>The cross-reconstruction significantly increases the difficulty of pre-training compared to self-reconstruction, which enables our method to surpass previous single-modal self-reconstruction methods in 3D self-supervised learning.
  arXiv  Detail & Related papers  (2025-09-01T08:42:17Z)
• Easi3R: Estimating Disentangled Motion from DUSt3R Without Training [69.51086319339662]
  We introduce Easi3R, a simple yet efficient training-free method for 4D reconstruction.<n>Our approach applies attention adaptation during inference, eliminating the need for from-scratch pre-training or network fine-tuning.<n>Our experiments on real-world dynamic videos demonstrate that our lightweight attention adaptation significantly outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-03-31T17:59:58Z)
• Evolving LLMs' Self-Refinement Capability via Synergistic Training-Inference Optimization [53.93621974137829]
  Self-Refinement refers to a model's ability to revise its own responses to produce improved outputs.<n>EVOLVE is a framework for eliciting and tracking the evolution of Self-Refinement through iterative training.<n>We demonstrate the potential of leveraging Self-Refinement to achieve broader Self-Improvement of intrinsic model abilities.
  arXiv  Detail & Related papers  (2025-02-08T15:21:55Z)
• Self-Supervised Monocular 4D Scene Reconstruction for Egocentric Videos [25.41337525728398]
  EgoMono4D is a novel model that unifies the estimation of multiple variables necessary for Egocentric Monocular 4D reconstruction.<n>It achieves superior performance in dense pointclouds sequence reconstruction compared to all baselines.
  arXiv  Detail & Related papers  (2024-11-14T02:57:11Z)
• End-to-End Autonomous Driving without Costly Modularization and 3D Manual Annotation [34.070813293944944]
  We propose UAD, a method for vision-based end-to-end autonomous driving (E2EAD) Our motivation stems from the observation that current E2EAD models still mimic the modular architecture in typical driving stacks. Our UAD achieves 38.7% relative improvements over UniAD on the average collision rate in nuScenes and surpasses VAD for 41.32 points on the driving score in CARLA's Town05 Long benchmark.
  arXiv  Detail & Related papers  (2024-06-25T16:12:52Z)
• Auto-Rectify Network for Unsupervised Indoor Depth Estimation [119.82412041164372]
  We establish that the complex ego-motions exhibited in handheld settings are a critical obstacle for learning depth. We propose a data pre-processing method that rectifies training images by removing their relative rotations for effective learning. Our results outperform the previous unsupervised SOTA method by a large margin on the challenging NYUv2 dataset.
  arXiv  Detail & Related papers  (2020-06-04T08:59:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.