Rethinking the Good Enough Embedding for Easy Few-Shot Learning

• URL: http://arxiv.org/abs/2605.14145v1
• Date: Wed, 13 May 2026 21:52:05 GMT
• Title: Rethinking the Good Enough Embedding for Easy Few-Shot Learning
• Abstract summary: In this paper, we leverage the Platonic Representation Hypothesis to demonstrate that off-the-shelf embeddings are inherently "good enough" for complex tasks.<n>By utilizing a k-Nearest Neighbor on frozen DINOv2-L features, we conduct a layer-wise characterization to identify an optimal feature extraction.<n>Our results across four major benchmarks demonstrate that our approach consistently surpasses sophisticated meta-learning algorithms.
• Score: 4.941630596191806
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The field of deep visual recognition is undergoing a paradigm shift toward universal representations. The Platonic Representation Hypothesis suggests that diverse architectures trained on massive datasets are converging toward a shared, "ideal" latent space. This again raises a critical question: is a "Good Embedding All You Need?" In this paper, we leverage this convergence to demonstrate that off-the-shelf embeddings are inherently "good enough" for complex tasks, rendering intensive task-specific fine-tuning unnecessary. We explore this hypothesis within the few-shot learning framework, proposing a straightforward, non-parametric pipeline that entirely bypasses backpropagation. By utilizing a k-Nearest Neighbor classifier on frozen DINOv2-L features, we conduct a layer-wise characterization to identify an optimal feature extraction. We further demonstrate that manifold refinement via PCA and ICA provides a beneficial regularizing effect. Our results across four major benchmarks demonstrate that our approach consistently surpasses sophisticated meta-learning algorithms, achieving state-of-the-art performance.

Related papers

• Inverse Reinforcement Learning without an Optimal Demonstrator: A Feasible Reward Set Approach [7.857217531314837]
  Inverse reinforcement learning (IRL) typically assumes demonstrations from a single optimal demonstrator.<n>We study reward learning in this setting through a feasible-reward-set framework.
  arXiv  Detail & Related papers  (2026-05-29T06:38:10Z)
• Hyperbolic Enhanced Representation Learning for Incomplete Multi-view Clustering [57.38215918201251]
  We propose HERL, a Hyperbolic Enhanced Representation Learning framework for Incomplete Multi-View Clustering.<n> operating within the Poincaré ball, HERL constructs a structure-aware latent space to enhance representation learning.<n>We show that HERL consistently outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2026-04-18T10:50:46Z)
• StepVAR: Structure-Texture Guided Pruning for Visual Autoregressive Models [98.72926158261937]
  We propose a training-free token pruning framework for Visual AutoRegressive models.<n>We employ a lightweight high-pass filter to capture local texture details, while leveraging Principal Component Analysis (PCA) to preserve global structural information.<n>To maintain valid next-scale prediction under sparse tokens, we introduce a nearest neighbor feature propagation strategy.
  arXiv  Detail & Related papers  (2026-03-02T11:35:05Z)
• Learning Binarized Representations with Pseudo-positive Sample Enhancement for Efficient Graph Collaborative Filtering [35.82405808653398]
  We study the problem of graph representation binarization for efficient collaborative filtering.<n>Our findings indicate that explicitly mitigating information loss at various stages of embedding binarization has a significant positive impact on performance.<n>Compared to its predecessor BiGeaR, BiGeaR++ introduces a fine-grained inference distillation mechanism and an effective embedding sample synthesis approach.
  arXiv  Detail & Related papers  (2025-06-03T11:11:43Z)
• Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains [92.36624674516553]
  Reinforcement learning with verifiable rewards (RLVR) has demonstrated significant success in enhancing mathematical reasoning and coding performance of large language models (LLMs)<n>We investigate the effectiveness and scalability of RLVR across diverse real-world domains including medicine, chemistry, psychology, economics, and education.<n>We utilize a generative scoring technique that yields soft, model-based reward signals to overcome limitations posed by binary verifications.
  arXiv  Detail & Related papers  (2025-03-31T08:22:49Z)
• USDRL: Unified Skeleton-Based Dense Representation Learning with Multi-Grained Feature Decorrelation [24.90512145836643]
  We introduce a Unified Skeleton-based Dense Representation Learning framework based on feature decorrelation.<n>We show that our approach significantly outperforms the current state-of-the-art (SOTA) approaches.
  arXiv  Detail & Related papers  (2024-12-12T12:20:27Z)
• Joint Learning for Scattered Point Cloud Understanding with Hierarchical Self-Distillation [34.26170741722835]
  We propose an end-to-end architecture that compensates for and identifies partial point clouds on the fly.<n> hierarchical self-distillation (HSD) can be applied to any hierarchy-based point cloud methods.
  arXiv  Detail & Related papers  (2023-12-28T08:51:04Z)
• Disentangled Representation Learning with Transmitted Information Bottleneck [57.22757813140418]
  We present textbfDisTIB (textbfTransmitted textbfInformation textbfBottleneck for textbfDisd representation learning), a novel objective that navigates the balance between information compression and preservation.
  arXiv  Detail & Related papers  (2023-11-03T03:18:40Z)
• Enhancing Representation Learning on High-Dimensional, Small-Size Tabular Data: A Divide and Conquer Method with Ensembled VAEs [7.923088041693465]
  We present an ensemble of lightweight VAEs to learn posteriors over subsets of the feature-space, which get aggregated into a joint posterior in a novel divide-and-conquer approach. We show that our approach is robust to partial features at inference, exhibiting little performance degradation even with most features missing.
  arXiv  Detail & Related papers  (2023-06-27T17:55:31Z)
• Hyperbolic Representation Learning: Revisiting and Advancing [43.1661098138936]
  We introduce a position-tracking mechanism to scrutinize existing prevalent hlms, revealing that the learned representations are sub-optimal and unsatisfactory. We propose a simple yet effective method, hyperbolic informed embedding (HIE), by incorporating cost-free hierarchical information deduced from the hyperbolic distance of the node to origin. Our method achieves a remarkable improvement of up to 21.4% compared to the competing baselines.
  arXiv  Detail & Related papers  (2023-06-15T13:25:39Z)
• Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning [79.83792914684985]
  We prove a new identifiability result that provides conditions under which maximally sparse base-predictors yield disentangled representations. Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem.
  arXiv  Detail & Related papers  (2022-11-26T21:02:09Z)
• Embed to Control Partially Observed Systems: Representation Learning with Provable Sample Efficiency [105.17746223041954]
  Reinforcement learning in partially observed Markov decision processes (POMDPs) faces two challenges. It often takes the full history to predict the future, which induces a sample complexity that scales exponentially with the horizon. We propose a reinforcement learning algorithm named Embed to Control (ETC), which learns the representation at two levels while optimizing the policy.
  arXiv  Detail & Related papers  (2022-05-26T16:34:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.