ARGENT: Adaptive Hierarchical Image-Text Representations

• URL: http://arxiv.org/abs/2603.23311v1
• Date: Tue, 24 Mar 2026 15:14:12 GMT
• Title: ARGENT: Adaptive Hierarchical Image-Text Representations
• Abstract summary: We introduce a stronger hyperbolic VLM baseline ARGENT, Adaptive hieRarchical imaGe-tExt represeNTation.<n>This paper introduces a stronger hyperbolic VLM baseline ARGENT, Adaptive hieRarchical imaGe-tExt represeNTation.
• Score: 12.724220731465392
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale Vision-Language Models (VLMs) such as CLIP learn powerful semantic representations but operate in Euclidean space, which fails to capture the inherent hierarchical structure of visual and linguistic concepts. Hyperbolic geometry, with its exponential volume growth, offers a principled alternative for embedding such hierarchies with low distortion. However, existing hyperbolic VLMs use entailment losses that are unstable: as parent embeddings contract toward the origin, their entailment cones widen toward a half-space, causing catastrophic cone collapse that destroys the intended hierarchy. Additionally, hierarchical evaluation of these models remains unreliable, being largely retrieval-based and correlation-based metrics and prone to taxonomy dependence and ambiguous negatives. To address these limitations, we propose an adaptive entailment loss paired with a norm regularizer that prevents cone collapse without heuristic aperture clipping. We further introduce an angle-based probabilistic entailment protocol (PEP) for evaluating hierarchical understanding, scored with AUC-ROC and Average Precision. This paper introduces a stronger hyperbolic VLM baseline ARGENT, Adaptive hieRarchical imaGe-tExt represeNTation. ARGENT improves the SOTA hyperbolic VLM by 0.7, 1.1, and 0.8 absolute points on image classification, text-to-image retrieval, and proposed hierarchical metrics, respectively.

Related papers

• Hierarchically Robust Zero-shot Vision-language Models [74.79155369056639]
  Vision-Language Models (VLMs) can perform zero-shot classification but are susceptible to adversarial attacks.<n>Existing approaches align fixed text embeddings with an image embedding, sacrificing natural performance and robustness.<n>We propose a novel adversarial fine-tuning framework based on hierarchical embeddings and adversarially robust alignment of image-text modalities.
  arXiv  Detail & Related papers  (2026-04-20T21:42:14Z)
• Why Advanced Encoders Lag on Sparse Retrieval? The Answer and an Approach to Bridging Vocabulary Gaps [5.182584489074576]
  Modern tokenizers utilize raw, case-sensitive vocabularies, which map single semantic units to redundant surface forms.<n>We propose textbfVocabulary Transfer (VT), a model-agnostic framework that migrates advanced encoders to sparse-friendly, normalized with minimal computational cost.<n>VT is universally effective: it enables ModernBERT to achieve state-of-the-art performance on the BEIR benchmark.
  arXiv  Detail & Related papers  (2026-04-20T05:52:25Z)
• Hyperbolic Enhanced Representation Learning for Incomplete Multi-view Clustering [57.38215918201251]
  We propose HERL, a Hyperbolic Enhanced Representation Learning framework for Incomplete Multi-View Clustering.<n> operating within the Poincaré ball, HERL constructs a structure-aware latent space to enhance representation learning.<n>We show that HERL consistently outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2026-04-18T10:50:46Z)
• Explaining, Verifying, and Aligning Semantic Hierarchies in Vision-Language Model Embeddings [10.299190830046872]
  We present a post-hoc framework to explain, verify, and align semantic hierarchies induced by a VLM over a given set of child classes.<n>We quantify plausibility by comparing the extracted tree against human retrieval using efficient tree-aware and edge-level consistency measures, and we evaluate utility via explainable hierarchical tree-traversal inference.
  arXiv  Detail & Related papers  (2026-03-26T00:38:38Z)
• Beyond Semantic Priors: Mitigating Optimization Collapse for Generalizable Visual Forensics [12.973951916863989]
  We identify a failure mode termed Optimization Collapse, where detectors degenerate to random guessing on non-semantic forgeries.<n>We propose a Contrastive Regional Injection Transformer (CoRIT) which integrates a computationally efficient Contrastive Gradient Proxy (CGP) with three training-free strategies.<n>CoRIT mitigates optimization collapse and achieves state-of-the-art generalization across cross-domain and universal forgery benchmarks.
  arXiv  Detail & Related papers  (2026-03-25T08:03:30Z)
• OCP: Orthogonal Constrained Projection for Sparse Scaling in Industrial Commodity Recommendation [9.265389086960656]
  A key challenge is that Item-Id vocabularies suffer from low-frequency information interference when subjected to sparse scaling.<n>We propose an Orthogonal Constrained Projection method to optimize embedding representation.
  arXiv  Detail & Related papers  (2026-03-19T09:57:26Z)
• Understanding Degradation with Vision Language Model [56.09241449206817]
  Understanding visual degradations is a critical yet challenging problem in computer vision.<n>We introduce DU-VLM, a multimodal chain-of-thought model trained with supervised fine-tuning and reinforcement learning.<n>We also introduce textbfDU-110k, a large-scale dataset comprising 110,000 clean-degraded pairs with grounded physical annotations.
  arXiv  Detail & Related papers  (2026-02-04T13:51:15Z)
• SAGE: Sequence-level Adaptive Gradient Evolution for Generative Recommendation [8.54123828673921]
  We propose a unified optimization framework tailored for list-wise generative recommendation.<n>Sequence-level Signal Decoupling: By combining a geometric mean importance ratio with decoupled multi-objective advantages, we eliminate token-level variance.<n>Asymmetric Adaptive Dynamics: We construct a dynamic gradient manifold that applies a "Boost Factor" to high-potential cold start items to achieve super-linear updates.
  arXiv  Detail & Related papers  (2026-01-29T09:30:13Z)
• Hierarchy-Aware Fine-Tuning of Vision-Language Models [18.244518940229202]
  Vision-Language Models learn powerful multimodal representations through large-scale image-text pretraining.<n>Standard approaches treat labels as flat categories and require full fine-tuning, which is expensive and produces inconsistent predictions.<n>We propose an efficient hierarchy-aware fine-tuning framework that updates a few parameters while enforcing structural consistency.
  arXiv  Detail & Related papers  (2025-12-25T06:44:33Z)
• Enhancing CLIP Robustness via Cross-Modality Alignment [54.01929554563447]
  We propose Cross-modality Alignment, an optimal transport-based framework for vision-language models.<n> COLA restores global image-text alignment and local structural consistency in the feature space.<n> COLA is training-free and compatible with existing fine-tuned models.
  arXiv  Detail & Related papers  (2025-10-28T03:47:44Z)
• IAR2: Improving Autoregressive Visual Generation with Semantic-Detail Associated Token Prediction [77.06211178777939]
  IAR2 is an advanced autoregressive framework that enables a hierarchical semantic-detail synthesis process.<n>We show that IAR2 sets a new state-of-the-art for autoregressive image generation, achieving a FID of 1.50 on ImageNet.
  arXiv  Detail & Related papers  (2025-10-08T12:08:21Z)
• Dense Semantic Matching with VGGT Prior [49.42199006453071]
  We propose an approach that retains VGGT's intrinsic strengths by reusing early feature stages, fine-tuning later ones, and adding a semantic head for bidirectional correspondences.<n>Our approach achieves superior geometry awareness, matching reliability, and manifold preservation, outperforming previous baselines.
  arXiv  Detail & Related papers  (2025-09-25T14:56:11Z)
• HyperTTA: Test-Time Adaptation for Hyperspectral Image Classification under Distribution Shifts [28.21559601586271]
  HyperTTA (Test-Time Adaptable Transformer for Hyperspectral Degradation) is a unified framework that enhances model robustness under diverse degradation conditions.<n>Test-time adaptation strategy, the Confidence-aware Entropy-minimized LayerNorm Adapter (CELA), dynamically updates only the affine parameters of LayerNorm layers.<n>Experiments on two benchmark datasets demonstrate that HyperTTA outperforms state-of-the-art baselines across a wide range of degradation scenarios.
  arXiv  Detail & Related papers  (2025-09-10T09:31:37Z)
• TRUST: Leveraging Text Robustness for Unsupervised Domain Adaptation [9.906359339999039]
  We introduce a novel UDA approach that exploits the robustness of the language modality to guide the adaptation of a vision model.<n>We propose a multimodal soft-contrastive learning loss that aligns the vision and language feature spaces.<n>Our approach outperforms previous methods, setting the new state-of-the-art on classical (DomainNet) and complex (GeoNet) domain shifts.
  arXiv  Detail & Related papers  (2025-08-08T16:51:44Z)
• Benchmarking Unified Face Attack Detection via Hierarchical Prompt Tuning [58.16354555208417]
  PAD and FFD are proposed to protect face data from physical media-based Presentation Attacks and digital editing-based DeepFakes, respectively.<n>The lack of a Unified Face Attack Detection model to simultaneously handle attacks in these two categories is mainly attributed to two factors.<n>We present a novel Visual-Language Model-based Hierarchical Prompt Tuning Framework that adaptively explores multiple classification criteria from different semantic spaces.
  arXiv  Detail & Related papers  (2025-05-19T16:35:45Z)
• High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification [84.43394420267794]
  We propose a novel framework by learning high-order relation and topology information for discriminative features and robust alignment. Our framework significantly outperforms state-of-the-art by6.5%mAP scores on Occluded-Duke dataset.
  arXiv  Detail & Related papers  (2020-03-18T12:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.