Contrastive-Collapsed Loss for Flexible and Geometrically Optimal Embeddings and Faster Convergence

• URL: http://arxiv.org/abs/2607.12916v1
• Date: Tue, 14 Jul 2026 15:51:41 GMT
• Title: Contrastive-Collapsed Loss for Flexible and Geometrically Optimal Embeddings and Faster Convergence
• Abstract summary: We introduce CoCo, a loss function aimed at learning normalized and well-structured representations.<n>We show that CoCo achieves competitive performance with state-of-the-art methods, including kernel SVM, Random Forest, dot regression, and cross-entropy-based neural networks.
• Score: 0.4205692673448205
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this work, we introduce CoCo, a loss function aimed at learning normalized and well-structured representations. The proposed loss encourages intra-class collapse and inter-class contrast while preserving sufficient flexibility for neural networks to approximate geometrically optimal embeddings with large angular separation between classes. We provide a theoretical analysis positioning CoCo with respect to related objectives such as dot regression and cross-entropy, showing that the new proposed loss benefits from closer initialization to the optimal configuration, more informative gradients, and stronger incentives for class-wise representation collapse. Extensive experiments on diverse tabular datasets from the OpenML-CC18 benchmark show that CoCo achieves competitive performance with state-of-the-art methods, including kernel SVM, Random Forest, dot regression, and cross-entropy-based neural networks. In addition, both theoretical arguments and empirical analyses demonstrate that the proposal promotes tighter class clustering and faster convergence. These results highlight CoCo loss as an effective objective for learning discriminative representations while maintaining competitive predictive performance.

Related papers

• Generalized Convexity and Smoothness via Conjugate Duality: Optimization Theory for Deep Neural Networks [0.0]
  Deep neural network (DNN) training with gradient descent (SGD) and its variants achieves strong empirical performance.<n>Traditional analyses rely on assumptions such as differentiability, convexity, or smoothness.<n>We introduce $mathcalH()$ smoothness and $mathcalH()$-smoothness, which unify convex non-smooth as well as smooth non-smooth objectives.
  arXiv  Detail & Related papers  (2026-08-10T12:24:29Z)
• Silhouette Loss: Differentiable Global Structure Learning for Deep Representations [0.0]
  We introduce Soft Silhouette Loss, a novel differentiable objective inspired by the classical silhouette coefficient from clustering analysis.<n>Our formulation evaluates each sample against all classes in the batch, providing a batch-level notion of global structure.<n>It can be seamlessly combined with cross-entropy, and is also complementary to supervised contrastive learning.
  arXiv  Detail & Related papers  (2026-03-27T12:11:47Z)
• Is Softmax Loss All You Need? A Principled Analysis of Softmax-family Loss [91.61796429377041]
  The Softmax loss is one of the most widely employed surrogate objectives for classification and ranking tasks.<n>We investigate whether different surrogates achieve consistency with classification and ranking metrics, and analyze their gradient dynamics to reveal distinct convergence behaviors.<n>Our results establish a principled foundation and offer practical guidance for loss selections in large-class machine learning applications.
  arXiv  Detail & Related papers  (2026-01-30T09:24:52Z)
• CoCo-Fed: A Unified Framework for Memory- and Communication-Efficient Federated Learning at the Wireless Edge [50.42067935605982]
  We propose a novel Compression and Combination-based Federated learning framework that unifies local memory efficiency and global communication reduction.<n>CoCo-Fed significantly outperforms state-of-the-art baselines in both memory and communication efficiency while maintaining robust convergence under non-IID settings.
  arXiv  Detail & Related papers  (2026-01-02T03:39:50Z)
• Latent Chain-of-Thought for Visual Reasoning [53.541579327424046]
  Chain-of-thought (CoT) reasoning is critical for improving the interpretability and reliability of Large Vision-Language Models (LVLMs)<n>We reformulate reasoning in LVLMs as posterior inference and propose a scalable training algorithm based on amortized variational inference.<n>We empirically demonstrate that the proposed method enhances the state-of-the-art LVLMs on seven reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-10-27T23:10:06Z)
• Distributed optimization: designed for federated learning [6.642087364775619]
  Federated learning (FL) is a distributed collaborative machine learning framework under privacy-preserving constraints.<n>This paper proposes a class of distributed optimization algorithms based on the augmented Lagrangian technique.<n> Numerical experiments demonstrate that the proposed algorithm exhibits strong performance in large-scale settings.
  arXiv  Detail & Related papers  (2025-08-12T03:39:07Z)
• COPO: Consistency-Aware Policy Optimization [17.328515578426227]
  Reinforcement learning has significantly enhanced the reasoning capabilities of Large Language Models (LLMs) in complex problem-solving tasks.<n>Recently, the introduction of DeepSeek R1 has inspired a surge of interest in leveraging rule-based rewards as a low-cost alternative for computing advantage functions and guiding policy optimization.<n>We propose a consistency-aware policy optimization framework that introduces a structured global reward based on outcome consistency.
  arXiv  Detail & Related papers  (2025-08-06T07:05:18Z)
• Consensus-based optimization for closed-box adversarial attacks and a connection to evolution strategies [4.813333335683418]
  Consensus-based optimization (CBO) has established itself as an efficient gradient-free optimization scheme.<n>Our study is as introduced by Riedl et al. and natural evolution strategies (NES) commonly applied in adversarial attacks.<n>Despite the conceptual similarities, CBO can outperform NES and other evolutionary strategies in certain scenarios.
  arXiv  Detail & Related papers  (2025-06-30T16:54:44Z)
• NDCG-Consistent Softmax Approximation with Accelerated Convergence [67.10365329542365]
  We propose novel loss formulations that align directly with ranking metrics.<n>We integrate the proposed RG losses with the highly efficient Alternating Least Squares (ALS) optimization method.<n> Empirical evaluations on real-world datasets demonstrate that our approach achieves comparable or superior ranking performance.
  arXiv  Detail & Related papers  (2025-06-11T06:59:17Z)
• Relaxed Contrastive Learning for Federated Learning [48.96253206661268]
  We propose a novel contrastive learning framework to address the challenges of data heterogeneity in federated learning. Our framework outperforms all existing federated learning approaches by huge margins on the standard benchmarks.
  arXiv  Detail & Related papers  (2024-01-10T04:55:24Z)
• Stochastic Unrolled Federated Learning [85.6993263983062]
  We introduce UnRolled Federated learning (SURF), a method that expands algorithm unrolling to federated learning. Our proposed method tackles two challenges of this expansion, namely the need to feed whole datasets to the unrolleds and the decentralized nature of federated learning.
  arXiv  Detail & Related papers  (2023-05-24T17:26:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.