Semantics-Aware Inferential Network for Natural Language Understanding

• URL: http://arxiv.org/abs/2004.13338v1
• Date: Tue, 28 Apr 2020 07:24:43 GMT
• Title: Semantics-Aware Inferential Network for Natural Language Understanding
• Authors: Shuailiang Zhang, Hai Zhao, Junru Zhou
• Abstract summary: We propose a Semantics-Aware Inferential Network (SAIN) to meet such a motivation. Taking explicit contextualized semantics as a complementary input, the inferential module of SAIN enables a series of reasoning steps over semantic clues. Our model achieves significant improvement on 11 tasks including machine reading comprehension and natural language inference.
• Score: 79.70497178043368
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For natural language understanding tasks, either machine reading comprehension or natural language inference, both semantics-aware and inference are favorable features of the concerned modeling for better understanding performance. Thus we propose a Semantics-Aware Inferential Network (SAIN) to meet such a motivation. Taking explicit contextualized semantics as a complementary input, the inferential module of SAIN enables a series of reasoning steps over semantic clues through an attention mechanism. By stringing these steps, the inferential network effectively learns to perform iterative reasoning which incorporates both explicit semantics and contextualized representations. In terms of well pre-trained language models as front-end encoder, our model achieves significant improvement on 11 tasks including machine reading comprehension and natural language inference.

Related papers

• DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment [82.86363991170546]
  We propose a Descriptive Speech-Text Alignment approach that leverages speech captioning to bridge the gap between speech and text modalities. Our model demonstrates superior performance on the Dynamic-SUPERB benchmark, particularly in generalizing to unseen tasks. These findings highlight the potential to reshape instruction-following SLMs by incorporating descriptive rich, speech captions.
  arXiv  Detail & Related papers  (2024-06-27T03:52:35Z)
• Resolving Word Vagueness with Scenario-guided Adapter for Natural Language Inference [24.58277380514406]
  Natural Language Inference (NLI) is a crucial task in natural language processing. We propose an innovative ScenaFuse adapter that simultaneously integrates large-scale pre-trained linguistic knowledge and relevant visual information. Our approach bridges the gap between language and vision, leading to improved understanding and inference capabilities in NLI tasks.
  arXiv  Detail & Related papers  (2024-05-21T01:19:52Z)
• Punctuation Restoration Improves Structure Understanding without Supervision [6.4736137270915215]
  We show that punctuation restoration as a learning objective improves in- and out-of-distribution performance on structure-related tasks. Punctuation restoration is an effective learning objective that can improve structure understanding and yield a more robust structure-aware representations of natural language.
  arXiv  Detail & Related papers  (2024-02-13T11:22:52Z)
• Contextualized word senses: from attention to compositionality [0.10878040851637999]
  We propose a transparent, interpretable, and linguistically motivated strategy for encoding the contextual sense of words. Particular attention is given to dependency relations and semantic notions such as selection preferences and paradigmatic classes.
  arXiv  Detail & Related papers  (2023-12-01T16:04:00Z)
• Benchmarking Language Models for Code Syntax Understanding [79.11525961219591]
  Pre-trained language models have demonstrated impressive performance in both natural language processing and program understanding. In this work, we perform the first thorough benchmarking of the state-of-the-art pre-trained models for identifying the syntactic structures of programs. Our findings point out key limitations of existing pre-training methods for programming languages, and suggest the importance of modeling code syntactic structures.
  arXiv  Detail & Related papers  (2022-10-26T04:47:18Z)
• Transferring Semantic Knowledge Into Language Encoders [6.85316573653194]
  We introduce semantic form mid-tuning, an approach for transferring semantic knowledge from semantic meaning representations into language encoders. We show that this alignment can be learned implicitly via classification or directly via triplet loss. Our method yields language encoders that demonstrate improved predictive performance across inference, reading comprehension, textual similarity, and other semantic tasks.
  arXiv  Detail & Related papers  (2021-10-14T14:11:12Z)
• How could Neural Networks understand Programs? [67.4217527949013]
  It is difficult to build a model to better understand programs, by either directly applying off-the-shelf NLP pre-training techniques to the source code, or adding features to the model by theshelf. We propose a novel program semantics learning paradigm, that the model should learn from information composed of (1) the representations which align well with the fundamental operations in operational semantics, and (2) the information of environment transition.
  arXiv  Detail & Related papers  (2021-05-10T12:21:42Z)
• SG-Net: Syntax Guided Transformer for Language Representation [58.35672033887343]
  We propose using syntax to guide the text modeling by incorporating explicit syntactic constraints into attention mechanisms for better linguistically motivated word representations. In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention. Experiments on popular benchmark tasks, including machine reading comprehension, natural language inference, and neural machine translation show the effectiveness of the proposed SG-Net design.
  arXiv  Detail & Related papers  (2020-12-27T11:09:35Z)
• Infusing Finetuning with Semantic Dependencies [62.37697048781823]
  We show that, unlike syntax, semantics is not brought to the surface by today's pretrained models. We then use convolutional graph encoders to explicitly incorporate semantic parses into task-specific finetuning.
  arXiv  Detail & Related papers  (2020-12-10T01:27:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.