Learning to Select from Multiple Options

• URL: http://arxiv.org/abs/2212.00301v3
• Date: Tue, 12 Sep 2023 00:29:12 GMT
• Title: Learning to Select from Multiple Options
• Abstract summary: Textual entailment (TE) has been shown as the state-of-the-art (SOTA) approach to dealing with selection problems. This work proposes a contextualized TE model (Context-TE) by appending other k options as the context of the current (P, H) modeling. Second, we speed up Context-TE by coming up with Parallel-TE, which learns the decisions of multiple options simultaneously.
• Score: 48.59962245388264
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many NLP tasks can be regarded as a selection problem from a set of options, such as classification tasks, multi-choice question answering, etc. Textual entailment (TE) has been shown as the state-of-the-art (SOTA) approach to dealing with those selection problems. TE treats input texts as premises (P), options as hypotheses (H), then handles the selection problem by modeling (P, H) pairwise. Two limitations: first, the pairwise modeling is unaware of other options, which is less intuitive since humans often determine the best options by comparing competing candidates; second, the inference process of pairwise TE is time-consuming, especially when the option space is large. To deal with the two issues, this work first proposes a contextualized TE model (Context-TE) by appending other k options as the context of the current (P, H) modeling. Context-TE is able to learn more reliable decision for the H since it considers various context. Second, we speed up Context-TE by coming up with Parallel-TE, which learns the decisions of multiple options simultaneously. Parallel-TE significantly improves the inference speed while keeping comparable performance with Context-TE. Our methods are evaluated on three tasks (ultra-fine entity typing, intent detection and multi-choice QA) that are typical selection problems with different sizes of options. Experiments show our models set new SOTA performance; particularly, Parallel-TE is faster than the pairwise TE by k times in inference. Our code is publicly available at https://github.com/jiangshdd/LearningToSelect.

Related papers

• REAR: Test-time Preference Realignment through Reward Decomposition [68.09214603569744]
  We introduce a novel framework that models the task as a realignment problem, since the base model often fails to align with the stated preference.<n>We derive a REAlignment Reward (REAR) that selectively rescales the proportions of these two reward terms.<n>We show that REAR can be formulated as a linear combination of token-level policy log-probabilities, making it computationally efficient and easy to integrate with various TTS algorithms.
  arXiv  Detail & Related papers  (2026-06-29T14:17:53Z)
• You May Speak Freely: Improving the Fine-Grained Visual Recognition Capabilities of Multimodal Large Language Models with Answer Extraction [24.029138898778626]
  nlg2choice is a simple two-stage method which first asks the MLLM an open-ended question for the task with minimal constraints.<n>We compute the probability of the constrained response taking that choice with an early stopping method to significantly improve throughput.<n>Our results show improvement over a suite of seven fine-grained visual datasets when evaluating in terms of classification and retrieval.
  arXiv  Detail & Related papers  (2025-10-16T17:04:25Z)
• CM-Align: Consistency-based Multilingual Alignment for Large Language Models [84.19366314925593]
  We propose a consistency-based data method to construct high-quality multilingual preference data.<n> Specifically, our method includes two parts: consistency-guided English reference selection and cross-lingual consistency-based multilingual preference data construction.
  arXiv  Detail & Related papers  (2025-09-10T12:40:49Z)
• Differentiating Choices via Commonality for Multiple-Choice Question Answering [54.04315943420376]
  Multiple-choice question answering can provide valuable clues for choosing the right answer. Existing models often rank each choice separately, overlooking the context provided by other choices. We propose a novel model by differentiating choices through identifying and eliminating their commonality, called DCQA.
  arXiv  Detail & Related papers  (2024-08-21T12:05:21Z)
• Transfer Learning Enhanced Single-choice Decision for Multi-choice Question Answering [27.601353412882258]
  Multi-choice Machine Reading (MMRC) aims to select the correct answer from a set of options based on a given passage and question. In this paper, we reconstruct multi-choice to single-choice by training a binary classification to distinguish whether a certain answer is correct. Our proposed method gets rid of the multi-choice framework and can leverage resources of other tasks.
  arXiv  Detail & Related papers  (2024-04-27T16:02:55Z)
• A post-selection algorithm for improving dynamic ensemble selection methods [6.59003008107689]
  Post-Selection Dynamic Ensemble Selection (PS-DES) is a post-selection scheme that evaluates ensembles selected by several DES techniques using different metrics. Using accuracy as a metric to select the ensembles, PS-DES performs better than individual DES techniques.
  arXiv  Detail & Related papers  (2023-09-25T17:25:39Z)
• IRRGN: An Implicit Relational Reasoning Graph Network for Multi-turn Response Selection [4.471148909362883]
  Implicit Reasoning to Graph Network aims to implicitly extract between utterances, as well as utterances and options. Model surpasses human performance for the first time on the MuTual dataset.
  arXiv  Detail & Related papers  (2022-12-01T13:17:25Z)
• Cost-Effective Online Contextual Model Selection [14.094350329970537]
  We formulate this task as an online contextual active model selection problem, where at each round the learner receives an unlabeled data point along with a context. The goal is to output the best model for any given context without obtaining an excessive amount of labels. We propose a contextual active model selection algorithm (CAMS), which relies on a novel uncertainty sampling query criterion defined on a given policy class for adaptive model selection.
  arXiv  Detail & Related papers  (2022-07-13T08:22:22Z)
• Relation-aware Video Reading Comprehension for Temporal Language Grounding [67.5613853693704]
  Temporal language grounding in videos aims to localize the temporal span relevant to the given query sentence. This paper will formulate temporal language grounding into video reading comprehension and propose a Relation-aware Network (RaNet) to address it.
  arXiv  Detail & Related papers  (2021-10-12T03:10:21Z)
• Scalable Feature Selection for (Multitask) Gradient Boosted Trees [10.48364482781362]
  We develop a scalable forward feature selection variant for Gradient Boosted Decision Trees (GBDTs) We show via experiments that the proposed method offers significant speedups in training time, while being as competitive as existing GBDT methods in terms of model performance metrics. We also extend the method to the multitask setting, allowing the practitioner to select common features across tasks, as well as selecting task-specific features.
  arXiv  Detail & Related papers  (2021-09-05T01:58:37Z)
• Adversarial Option-Aware Hierarchical Imitation Learning [89.92994158193237]
  We propose Option-GAIL, a novel method to learn skills at long horizon. The key idea of Option-GAIL is modeling the task hierarchy by options and train the policy via generative adversarial optimization. Experiments show that Option-GAIL outperforms other counterparts consistently across a variety of tasks.
  arXiv  Detail & Related papers  (2021-06-10T06:42:05Z)
• Response Selection for Multi-Party Conversations with Dynamic Topic Tracking [63.15158355071206]
  We frame response selection as a dynamic topic tracking task to match the topic between the response and relevant conversation context. We propose a novel multi-task learning framework that supports efficient encoding through large pretrained models. Experimental results on the DSTC-8 Ubuntu IRC dataset show state-of-the-art results in response selection and topic disentanglement tasks.
  arXiv  Detail & Related papers  (2020-10-15T14:21:38Z)
• Tradeoffs in Sentence Selection Techniques for Open-Domain Question Answering [54.541952928070344]
  We describe two groups of models for sentence selection: QA-based approaches, which run a full-fledged QA system to identify answer candidates, and retrieval-based models, which find parts of each passage specifically related to each question. We show that very lightweight QA models can do well at this task, but retrieval-based models are faster still.
  arXiv  Detail & Related papers  (2020-09-18T23:39:15Z)
• Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection [11.465266718370536]
  We study the task of selecting the optimal response given a user and system utterance history in retrieval-based dialog systems. We propose utterance manipulation strategies (UMS) to address this problem. UMS consist of several strategies (i.e., insertion, deletion, and search) which aid the response selection model towards maintaining dialog coherence.
  arXiv  Detail & Related papers  (2020-09-10T07:39:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.