Hypersolid: Emergent Vision Representations via Short-Range Repulsion

• URL: http://arxiv.org/abs/2601.21255v1
• Date: Thu, 29 Jan 2026 04:25:43 GMT
• Title: Hypersolid: Emergent Vision Representations via Short-Range Repulsion
• Abstract summary: We reinterpret representation learning as a discrete packing problem, where preserving information simplifies to maintaining injectivity.<n>We operationalize this in Hypersolid, a method using short-range hard-ball repulsion to prevent local collisions.<n>This constraint results in a high-separation geometric regime that preserves augmentation diversity, excelling on fine-grained and low-resolution classification tasks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A recurring challenge in self-supervised learning is preventing representation collapse. Existing solutions typically rely on global regularization, such as maximizing distances, decorrelating dimensions or enforcing certain distributions. We instead reinterpret representation learning as a discrete packing problem, where preserving information simplifies to maintaining injectivity. We operationalize this in Hypersolid, a method using short-range hard-ball repulsion to prevent local collisions. This constraint results in a high-separation geometric regime that preserves augmentation diversity, excelling on fine-grained and low-resolution classification tasks.

Related papers

• Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attention [58.484607892210015]
  Diffusion Language Models (DLMs) enable globally coherent, bidirectional, and controllable text generation.<n>Many existing block-sparse attention methods select blocks by fixed sampling patterns over the high-resolution attention space.<n>We propose the Block Approximate Sparse Attention framework (BA-Att) with block-wise pre-downsampled operation.
  arXiv  Detail & Related papers  (2026-05-19T12:01:13Z)
• Scalable Production Scheduling: Linear Complexity via Unified Homogeneous Graphs [0.1274452325287335]
  We introduce a unified graph framework that employs feature-based homogenization to project distinct node roles into a shared latent space.<n>This allows a standard homogeneous Graph Isomorphism Network to capture complex resource contention with linear complexity.<n>We show that our framework achieves state-of-the-art performance while exhibiting consistent zero-shot generalization.
  arXiv  Detail & Related papers  (2026-04-26T19:00:34Z)
• Pairing Regularization for Mitigating Many-to-One Collapse in GANs [11.064613023526745]
  We propose a pairing regularizer jointly optimized with the generator to mitigate the many-to-one collapse.<n>We show that the effect of pairing regularization depends on the dominant failure mode of training.
  arXiv  Detail & Related papers  (2026-04-22T02:57:30Z)
• Hyperbolic Enhanced Representation Learning for Incomplete Multi-view Clustering [57.38215918201251]
  We propose HERL, a Hyperbolic Enhanced Representation Learning framework for Incomplete Multi-View Clustering.<n> operating within the Poincaré ball, HERL constructs a structure-aware latent space to enhance representation learning.<n>We show that HERL consistently outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2026-04-18T10:50:46Z)
• Referring-Aware Visuomotor Policy Learning for Closed-Loop Manipulation [91.20850436220267]
  We introduce the Referring-Aware Visuomotor Policy (ReV)<n>ReV incorporates sparse referring points provided by a human or a high-level reasoning planner.<n>It is trained only by applying targeted perturbations to expert demonstrations.
  arXiv  Detail & Related papers  (2026-04-07T07:41:11Z)
• Unsupervised Behavioral Compression: Learning Low-Dimensional Policy Manifolds through State-Occupancy Matching [42.26405543045843]
  Action-based Policy Compression (APC) is widely recognized as sample-inefficient.<n>We introduce Occupancy-based Policy Compression (OPC), which enhances APC by shifting behavior representation from immediate action-matching to long-horizon state-space coverage.<n>These modifications force the generative model to organize the latent space around true functional similarity, promoting a latent representation that generalizes over a broad spectrum of behaviors.
  arXiv  Detail & Related papers  (2026-03-27T23:16:27Z)
• Rethinking Multi-Condition DiTs: Eliminating Redundant Attention via Position-Alignment and Keyword-Scoping [61.459927600301654]
  Multi-condition control is bottlenecked by the conventional concatenate-and-attend'' strategy.<n>Our analysis reveals that much of this cross-modal interaction is spatially or semantically redundant.<n>We propose Position-aligned and Keyword-scoped Attention (PKA), a highly efficient framework designed to eliminate these redundancies.
  arXiv  Detail & Related papers  (2026-02-06T16:39:10Z)
• Sparse Attention Post-Training for Mechanistic Interpretability [55.030850996535776]
  We introduce a simple post-training method that makes transformer attention sparse without sacrificing performance.<n>Applying a flexible sparsity regularisation under a constrained-loss objective, we show on models up to 1B parameters that it is possible to retain the original pretraining loss while reducing attention connectivity to $approx 0.3 % of its edges.
  arXiv  Detail & Related papers  (2025-12-05T16:40:08Z)
• Convergence and Generalization of Anti-Regularization for Parametric Models [0.0]
  Anti-regularization introduces a reward term with a reversed sign into the loss function.<n>We formalize spectral safety conditions and trust-region constraints.<n>We design a lightweight safeguard that combines a projection operator with gradient clipping to guarantee stable intervention.
  arXiv  Detail & Related papers  (2025-08-24T15:34:17Z)
• Towards the Semantic Weak Generalization Problem in Generative Zero-Shot Learning: Ante-hoc and Post-hoc [89.68803484284408]
  We present a simple and effective strategy lowering the previously unexplored factors that limit the performance ceiling of generative Zero-Shot Learning (ZSL) We begin by formally defining semantic generalization, then look into approaches for reducing the semantic weak generalization problem. In the ante-hoc phase, we augment the generator's semantic input, as well as relax the fitting target of the generator.
  arXiv  Detail & Related papers  (2022-04-24T13:54:42Z)
• Linear Adversarial Concept Erasure [108.37226654006153]
  We formulate the problem of identifying and erasing a linear subspace that corresponds to a given concept.<n>We show that the method is highly expressive, effectively mitigating bias in deep nonlinear classifiers while maintaining tractability and interpretability.
  arXiv  Detail & Related papers  (2022-01-28T13:00:17Z)
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning [43.96465407127458]
  We introduce VICReg, a method that explicitly avoids the collapse problem with a simple regularization term on the variance of the embeddings. VICReg achieves results on par with the state of the art on several downstream tasks.
  arXiv  Detail & Related papers  (2021-05-11T09:53:21Z)
• Online and Distribution-Free Robustness: Regression and Contextual Bandits with Huber Contamination [29.85468294601847]
  We revisit two classic high-dimensional online learning problems, namely linear regression and contextual bandits. We show that our algorithms succeed where conventional methods fail.
  arXiv  Detail & Related papers  (2020-10-08T17:59:05Z)
• Global Distance-distributions Separation for Unsupervised Person Re-identification [93.39253443415392]
  Existing unsupervised ReID approaches often fail in correctly identifying the positive samples and negative samples through the distance-based matching/ranking. We introduce a global distance-distributions separation constraint over the two distributions to encourage the clear separation of positive and negative samples from a global view. We show that our method leads to significant improvement over the baselines and achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-06-01T07:05:39Z)
• Adversarial Classification via Distributional Robustness with Wasserstein Ambiguity [12.576828231302134]
  Under Wasserstein ambiguity, the model aims to minimize the value-at-risk of misclassification. We show that, despite the non-marginity of this classification, standard descent methods appear to converger for this problem.
  arXiv  Detail & Related papers  (2020-05-28T07:28:47Z)
• Discrete Variational Attention Models for Language Generation [51.88612022940496]
  We propose a discrete variational attention model with categorical distribution over the attention mechanism owing to the discrete nature in languages. Thanks to the property of discreteness, the training of our proposed approach does not suffer from posterior collapse.
  arXiv  Detail & Related papers  (2020-04-21T05:49:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.