Differentiable Sorting Networks for Scalable Sorting and Ranking Supervision

• URL: http://arxiv.org/abs/2105.04019v1
• Date: Sun, 9 May 2021 20:39:03 GMT
• Title: Differentiable Sorting Networks for Scalable Sorting and Ranking Supervision
• Authors: Felix Petersen, Christian Borgelt, Hilde Kuehne, Oliver Deussen
• Abstract summary: We propose differentiable sorting networks by relaxing their pairwise conditional swap operations. We show that bitonic sorting networks can achieve stable training on large input sets of up to 1024 elements.
• Score: 19.437400671428737
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sorting and ranking supervision is a method for training neural networks end-to-end based on ordering constraints. That is, the ground truth order of sets of samples is known, while their absolute values remain unsupervised. For that, we propose differentiable sorting networks by relaxing their pairwise conditional swap operations. To address the problems of vanishing gradients and extensive blurring that arise with larger numbers of layers, we propose mapping activations to regions with moderate gradients. We consider odd-even as well as bitonic sorting networks, which outperform existing relaxations of the sorting operation. We show that bitonic sorting networks can achieve stable training on large input sets of up to 1024 elements.

Related papers

• Generalized Neural Sorting Networks with Error-Free Differentiable Swap Functions [44.71975181739874]
  We consider sorting problems for more abstract yet expressive inputs, e.g., multi-digit images and image fragments, through a neural sorting network. To learn a mapping from a high-dimensional input to an ordinal variable, the differentiability of sorting networks needs to be guaranteed.
  arXiv  Detail & Related papers  (2023-10-11T03:47:34Z)
• Bipartite Ranking Fairness through a Model Agnostic Ordering Adjustment [54.179859639868646]
  We propose a model agnostic post-processing framework xOrder for achieving fairness in bipartite ranking. xOrder is compatible with various classification models and ranking fairness metrics, including supervised and unsupervised fairness metrics. We evaluate our proposed algorithm on four benchmark data sets and two real-world patient electronic health record repositories.
  arXiv  Detail & Related papers  (2023-07-27T07:42:44Z)
• Layer Ensembles [95.42181254494287]
  We introduce a method for uncertainty estimation that considers a set of independent categorical distributions for each layer of the network. We show that the method can be further improved by ranking samples, resulting in models that require less memory and time to run.
  arXiv  Detail & Related papers  (2022-10-10T17:52:47Z)
• Monotonic Differentiable Sorting Networks [29.75063301688965]
  Differentiable sorting algorithms allow training with sorting and ranking supervision, where only the ordering or ranking of samples is known. One problem of current differentiable sorting methods is that they are non-monotonic. We propose a novel relaxation of conditional swap operations that guarantees monotonicity in differentiable sorting networks.
  arXiv  Detail & Related papers  (2022-03-17T21:51:29Z)
• Learning to Hash Naturally Sorts [84.90210592082829]
  We introduce Naturally-Sorted Hashing (NSH) to train a deep hashing model with sorted results end-to-end. NSH sort the Hamming distances of samples' hash codes and accordingly gather their latent representations for self-supervised training. We describe a novel Sorted Noise-Contrastive Estimation (SortedNCE) loss that selectively picks positive and negative samples for contrastive learning.
  arXiv  Detail & Related papers  (2022-01-31T16:19:02Z)
• PiRank: Learning To Rank via Differentiable Sorting [85.28916333414145]
  We propose PiRank, a new class of differentiable surrogates for ranking. We show that PiRank exactly recovers the desired metrics in the limit of zero temperature.
  arXiv  Detail & Related papers  (2020-12-12T05:07:36Z)
• Embedding Propagation: Smoother Manifold for Few-Shot Classification [131.81692677836202]
  We propose to use embedding propagation as an unsupervised non-parametric regularizer for manifold smoothing in few-shot classification. We empirically show that embedding propagation yields a smoother embedding manifold. We show that embedding propagation consistently improves the accuracy of the models in multiple semi-supervised learning scenarios by up to 16% points.
  arXiv  Detail & Related papers  (2020-03-09T13:51:09Z)
• Learn to Predict Sets Using Feed-Forward Neural Networks [63.91494644881925]
  This paper addresses the task of set prediction using deep feed-forward neural networks. We present a novel approach for learning to predict sets with unknown permutation and cardinality. We demonstrate the validity of our set formulations on relevant vision problems.
  arXiv  Detail & Related papers  (2020-01-30T01:52:07Z)
• Deep Attentive Ranking Networks for Learning to Order Sentences [31.70737249775046]
  We present an attention-based ranking framework for learning to order sentences given a paragraph. Our framework is built on a bidirectional sentence encoder and a self-attention based transformer network. It allows seamless training using a variety of ranking based loss functions, such as pointwise, pairwise, and listwise ranking.
  arXiv  Detail & Related papers  (2019-12-31T19:54:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.