AGRaME: Any-Granularity Ranking with Multi-Vector Embeddings

• URL: http://arxiv.org/abs/2405.15028v1
• Date: Thu, 23 May 2024 20:04:54 GMT
• Title: AGRaME: Any-Granularity Ranking with Multi-Vector Embeddings
• Authors: Revanth Gangi Reddy, Omar Attia, Yunyao Li, Heng Ji, Saloni Potdar,
• Abstract summary: We introduce the idea of any-granularity ranking, which leverages multi-vector embeddings to rank at varying levels of granularity. We demonstrate the application of proposition-level ranking to post-hoc citation addition in retrieval-augmented generation.
• Score: 53.78802457488845
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ranking is a fundamental and popular problem in search. However, existing ranking algorithms usually restrict the granularity of ranking to full passages or require a specific dense index for each desired level of granularity. Such lack of flexibility in granularity negatively affects many applications that can benefit from more granular ranking, such as sentence-level ranking for open-domain question-answering, or proposition-level ranking for attribution. In this work, we introduce the idea of any-granularity ranking, which leverages multi-vector embeddings to rank at varying levels of granularity while maintaining encoding at a single (coarser) level of granularity. We propose a multi-granular contrastive loss for training multi-vector approaches, and validate its utility with both sentences and propositions as ranking units. Finally, we demonstrate the application of proposition-level ranking to post-hoc citation addition in retrieval-augmented generation, surpassing the performance of prompt-driven citation generation.

Related papers

• Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models [63.714662435555674]
  Large language models (LLMs) exhibit positional bias in how they use context. We propose permutation self-consistency, a form of self-consistency over ranking list outputs of black-box LLMs. Our approach improves scores from conventional inference by up to 7-18% for GPT-3.5 and 8-16% for LLaMA v2 (70B)
  arXiv  Detail & Related papers  (2023-10-11T17:59:02Z)
• Partitioned Saliency Ranking with Dense Pyramid Transformers [4.449304130658638]
  Saliency ranking has emerged as a challenging task focusing on assessing the degree of saliency at instance-level. Previous approaches undertake the saliency ranking by directly sorting the rank scores of salient instances, which have not explicitly resolved the inherent ambiguities. We propose the ranking by partition paradigm, which segments unordered salient instances into partitions and then ranks them based on the correlations among these partitions.
  arXiv  Detail & Related papers  (2023-08-01T02:33:10Z)
• Bipartite Ranking Fairness through a Model Agnostic Ordering Adjustment [54.179859639868646]
  We propose a model agnostic post-processing framework xOrder for achieving fairness in bipartite ranking. xOrder is compatible with various classification models and ranking fairness metrics, including supervised and unsupervised fairness metrics. We evaluate our proposed algorithm on four benchmark data sets and two real-world patient electronic health record repositories.
  arXiv  Detail & Related papers  (2023-07-27T07:42:44Z)
• RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank [54.854714257687334]
  We propose a novel approach, RankCSE, for unsupervised sentence representation learning. It incorporates ranking consistency and ranking distillation with contrastive learning into a unified framework. An extensive set of experiments are conducted on both semantic textual similarity (STS) and transfer (TR) tasks.
  arXiv  Detail & Related papers  (2023-05-26T08:27:07Z)
• Learning List-Level Domain-Invariant Representations for Ranking [59.3544317373004]
  We propose list-level alignment -- learning domain-invariant representations at the higher level of lists. The benefits are twofold: it leads to the first domain adaptation generalization bound for ranking, in turn providing theoretical support for the proposed method.
  arXiv  Detail & Related papers  (2022-12-21T04:49:55Z)
• Heuristic Search for Rank Aggregation with Application to Label Ranking [16.275063634853584]
  We propose an effective hybrid evolutionary ranking algorithm to solve the rank aggregation problem. The algorithm features a semantic crossover based on concordant pairs and a late acceptance local search reinforced by an efficient incremental evaluation technique. Experiments are conducted to assess the algorithm, indicating a highly competitive performance on benchmark instances.
  arXiv  Detail & Related papers  (2022-01-11T11:43:17Z)
• PiRank: Learning To Rank via Differentiable Sorting [85.28916333414145]
  We propose PiRank, a new class of differentiable surrogates for ranking. We show that PiRank exactly recovers the desired metrics in the limit of zero temperature.
  arXiv  Detail & Related papers  (2020-12-12T05:07:36Z)
• Rank over Class: The Untapped Potential of Ranking in Natural Language Processing [8.637110868126546]
  We argue that many tasks which are currently addressed using classification are in fact being shoehorned into a classification mould. We propose a novel end-to-end ranking approach consisting of a Transformer network responsible for producing representations for a pair of text sequences. In an experiment on a heavily-skewed sentiment analysis dataset, converting ranking results to classification labels yields an approximately 22% improvement over state-of-the-art text classification.
  arXiv  Detail & Related papers  (2020-09-10T22:18:57Z)
• Rough Set based Aggregate Rank Measure & its Application to Supervised Multi Document Summarization [0.0]
  The paper proposes a novel Rough Set based membership called Rank Measure. It shall be utilized for ranking the elements to a particular class. The results proved to have significant improvement in accuracy.
  arXiv  Detail & Related papers  (2020-02-09T01:03:25Z)
• Deep Attentive Ranking Networks for Learning to Order Sentences [31.70737249775046]
  We present an attention-based ranking framework for learning to order sentences given a paragraph. Our framework is built on a bidirectional sentence encoder and a self-attention based transformer network. It allows seamless training using a variety of ranking based loss functions, such as pointwise, pairwise, and listwise ranking.
  arXiv  Detail & Related papers  (2019-12-31T19:54:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.