Natural Language Decomposition and Interpretation of Complex Utterances

• URL: http://arxiv.org/abs/2305.08677v2
• Date: Mon, 8 Jan 2024 06:33:25 GMT
• Title: Natural Language Decomposition and Interpretation of Complex Utterances
• Authors: Harsh Jhamtani, Hao Fang, Patrick Xia, Eran Levy, Jacob Andreas, Ben Van Durme
• Abstract summary: We introduce an approach to handle complex-intent-bearing utterances from a user via a process of hierarchical natural language decomposition. Our approach uses a pre-trained language model to decompose a complex utterance into a sequence of simpler natural language steps. Experiments show that the proposed approach enables the interpretation of complex utterances with almost no complex training data.
• Score: 47.30126929007346
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Designing natural language interfaces has historically required collecting supervised data to translate user requests into carefully designed intent representations. This requires enumerating and labeling a long tail of user requests, which is challenging. At the same time, large language models (LLMs) encode knowledge about goals and plans that can help conversational assistants interpret user requests requiring numerous steps to complete. We introduce an approach to handle complex-intent-bearing utterances from a user via a process of hierarchical natural language decomposition and interpretation. Our approach uses a pre-trained language model to decompose a complex utterance into a sequence of simpler natural language steps and interprets each step using the language-to-program model designed for the interface. To test our approach, we collect and release DeCU -- a new NL-to-program benchmark to evaluate Decomposition of Complex Utterances. Experiments show that the proposed approach enables the interpretation of complex utterances with almost no complex training data, while outperforming standard few-shot prompting approaches.

Related papers

• Interpreting User Requests in the Context of Natural Language Standing Instructions [89.12540932734476]
  We develop NLSI, a language-to-program dataset consisting of over 2.4K dialogues spanning 17 domains. A key challenge in NLSI is to identify which subset of the standing instructions is applicable to a given dialogue.
  arXiv  Detail & Related papers  (2023-11-16T11:19:26Z)
• Can Large Language Models Understand Real-World Complex Instructions? [54.86632921036983]
  Large language models (LLMs) can understand human instructions, but struggle with complex instructions. Existing benchmarks are insufficient to assess LLMs' ability to understand complex instructions. We propose CELLO, a benchmark for evaluating LLMs' ability to follow complex instructions systematically.
  arXiv  Detail & Related papers  (2023-09-17T04:18:39Z)
• nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Models [3.1143846686797314]
  We present nl2spec, a framework for applying Large Language Models (LLMs) derive formal specifications from unstructured natural language. We introduce a new methodology to detect and resolve the inherent ambiguity of system requirements in natural language. Users iteratively add, delete, and edit these sub-translations to amend erroneous formalizations, which is easier than manually redrafting the entire formalization.
  arXiv  Detail & Related papers  (2023-03-08T20:08:53Z)
• Modeling Sequential Sentence Relation to Improve Cross-lingual Dense Retrieval [87.11836738011007]
  We propose a multilingual multilingual language model called masked sentence model (MSM) MSM consists of a sentence encoder to generate the sentence representations, and a document encoder applied to a sequence of sentence vectors from a document. To train the model, we propose a masked sentence prediction task, which masks and predicts the sentence vector via a hierarchical contrastive loss with sampled negatives.
  arXiv  Detail & Related papers  (2023-02-03T09:54:27Z)
• Prompting Is Programming: A Query Language for Large Language Models [5.8010446129208155]
  We present the novel idea of Language Model Programming (LMP) LMP generalizes language model prompting from pure text prompts to an intuitive combination of text prompting and scripting. We show that LMQL can capture a wide range of state-of-the-art prompting methods in an intuitive way.
  arXiv  Detail & Related papers  (2022-12-12T18:09:09Z)
• Benchmarking Language Models for Code Syntax Understanding [79.11525961219591]
  Pre-trained language models have demonstrated impressive performance in both natural language processing and program understanding. In this work, we perform the first thorough benchmarking of the state-of-the-art pre-trained models for identifying the syntactic structures of programs. Our findings point out key limitations of existing pre-training methods for programming languages, and suggest the importance of modeling code syntactic structures.
  arXiv  Detail & Related papers  (2022-10-26T04:47:18Z)
• Towards Transparent Interactive Semantic Parsing via Step-by-Step Correction [17.000283696243564]
  We investigate an interactive semantic parsing framework that explains the predicted logical form step by step in natural language. We focus on question answering over knowledge bases (KBQA) as an instantiation of our framework. Our experiments show that the interactive framework with human feedback has the potential to greatly improve overall parse accuracy.
  arXiv  Detail & Related papers  (2021-10-15T20:11:22Z)
• Learning Adaptive Language Interfaces through Decomposition [89.21937539950966]
  We introduce a neural semantic parsing system that learns new high-level abstractions through decomposition. Users interactively teach the system by breaking down high-level utterances describing novel behavior into low-level steps.
  arXiv  Detail & Related papers  (2020-10-11T08:27:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.