RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsing

• URL: http://arxiv.org/abs/2109.10410v1
• Date: Tue, 21 Sep 2021 19:30:30 GMT
• Title: RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsing
• Authors: Vivek Gupta, Akshat Shrivastava, Adithya Sagar, Armen Aghajanyan and Denis Savenkov
• Abstract summary: We are applying retrieval-based modeling ideas to the problem of multi-domain task-oriented semantic parsing. Our approach, RetroNLU, extends a sequence-to-sequence model architecture with a retrieval component. We analyze the nearest neighbor retrieval component's quality, model sensitivity and break down the performance for semantic parses of different utterance complexity.
• Score: 11.157958012672202
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: While large pre-trained language models accumulate a lot of knowledge in their parameters, it has been demonstrated that augmenting it with non-parametric retrieval-based memory has a number of benefits from accuracy improvements to data efficiency for knowledge-focused tasks, such as question answering. In this paper, we are applying retrieval-based modeling ideas to the problem of multi-domain task-oriented semantic parsing for conversational assistants. Our approach, RetroNLU, extends a sequence-to-sequence model architecture with a retrieval component, used to fetch existing similar examples and provide them as an additional input to the model. In particular, we analyze two settings, where we augment an input with (a) retrieved nearest neighbor utterances (utterance-nn), and (b) ground-truth semantic parses of nearest neighbor utterances (semparse-nn). Our technique outperforms the baseline method by 1.5% absolute macro-F1, especially at the low resource setting, matching the baseline model accuracy with only 40% of the data. Furthermore, we analyze the nearest neighbor retrieval component's quality, model sensitivity and break down the performance for semantic parses of different utterance complexity.

Related papers

• Enhancing Question Answering Precision with Optimized Vector Retrieval and Instructions [1.2425910171551517]
  Question-answering (QA) is an important application of Information Retrieval (IR) and language models. We propose an innovative approach to improve QA task performances by integrating optimized vector retrievals and instruction methodologies.
  arXiv  Detail & Related papers  (2024-11-01T21:14:04Z)
• Exploring the State-of-the-Art Language Modeling Methods and Data Augmentation Techniques for Multilingual Clause-Level Morphology [3.8498574327875947]
  We present our work on all three parts of the shared task: inflection, reinflection, and analysis. We mainly explore two approaches: Transformer models in combination with data augmentation, and exploiting the state-of-the-art language modeling techniques for morphological analysis. Our methods achieved first place in each of the three tasks and outperforms mT5-baseline with 89% for inflection, 80% for reinflection and 12% for analysis.
  arXiv  Detail & Related papers  (2022-11-03T11:53:39Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis [90.24921443175514]
  We focus on aspect-based sentiment analysis, which involves extracting aspect term, category, and predicting their corresponding polarities. We propose to reformulate the extraction and prediction tasks into the sequence generation task, using a generative language model with unidirectional attention. Our approach outperforms the previous state-of-the-art (based on BERT) on average performance by a large margins in few-shot and full-shot settings.
  arXiv  Detail & Related papers  (2022-04-11T18:31:53Z)
• Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering [38.071375112873675]
  We propose a question-answer augmented encoder-decoder model and accompanying pretraining strategy. This yields an end-to-end system that outperforms prior QA retrieval methods on single-hop QA tasks.
  arXiv  Detail & Related papers  (2022-04-10T02:33:00Z)
• Efficient Nearest Neighbor Language Models [114.40866461741795]
  Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore. We show how to achieve up to a 6x speed-up in inference speed while retaining comparable performance.
  arXiv  Detail & Related papers  (2021-09-09T12:32:28Z)
• X2Parser: Cross-Lingual and Cross-Domain Framework for Task-Oriented Compositional Semantic Parsing [51.81533991497547]
  Task-oriented compositional semantic parsing (TCSP) handles complex nested user queries. We present X2 compared a transferable Cross-lingual and Cross-domain for TCSP. We propose to predict flattened intents and slots representations separately and cast both prediction tasks into sequence labeling problems.
  arXiv  Detail & Related papers  (2021-06-07T16:40:05Z)
• Improved Semantic Role Labeling using Parameterized Neighborhood Memory Adaptation [22.064890647610348]
  We propose a parameterized neighborhood memory adaptive (PNMA) method that uses a parameterized representation of the nearest neighbors of tokens in a memory of activations. We empirically show that PNMA consistently improves the SRL performance of the base model irrespective of types of word embeddings.
  arXiv  Detail & Related papers  (2020-11-29T22:51:25Z)
• Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing [85.35582118010608]
  Task-oriented semantic parsing is a critical component of virtual assistants. Recent advances in deep learning have enabled several approaches to successfully parse more complex queries. We propose a novel method that outperforms a supervised neural model at a 10-fold data reduction.
  arXiv  Detail & Related papers  (2020-10-07T17:47:53Z)
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks [133.93803565077337]
  retrieval-augmented generation models combine pre-trained parametric and non-parametric memory for language generation. We show that RAG models generate more specific, diverse and factual language than a state-of-the-art parametric-only seq2seq baseline.
  arXiv  Detail & Related papers  (2020-05-22T21:34:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.