How May I Help You? Using Neural Text Simplification to Improve Downstream NLP Tasks

• URL: http://arxiv.org/abs/2109.04604v2
• Date: Tue, 14 Sep 2021 06:26:15 GMT
• Title: How May I Help You? Using Neural Text Simplification to Improve Downstream NLP Tasks
• Authors: Hoang Van, Zheng Tang, Mihai Surdeanu
• Abstract summary: We evaluate the use of neural TS in two ways: simplifying input texts at prediction time and augmenting data to provide machines with additional information during training. We demonstrate that the latter scenario provides positive effects on machine performance on two separate datasets. In particular, the latter use of TS improves the performances of LSTM (1.82-1.98%) and SpanBERT (0.7-1.3%) extractors on TACRED, a complex, large-scale, real-world relation extraction task.
• Score: 20.370296294233313
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The general goal of text simplification (TS) is to reduce text complexity for human consumption. This paper investigates another potential use of neural TS: assisting machines performing natural language processing (NLP) tasks. We evaluate the use of neural TS in two ways: simplifying input texts at prediction time and augmenting data to provide machines with additional information during training. We demonstrate that the latter scenario provides positive effects on machine performance on two separate datasets. In particular, the latter use of TS improves the performances of LSTM (1.82-1.98%) and SpanBERT (0.7-1.3%) extractors on TACRED, a complex, large-scale, real-world relation extraction task. Further, the same setting yields improvements of up to 0.65% matched and 0.62% mismatched accuracies for a BERT text classifier on MNLI, a practical natural language inference dataset.

Related papers

• Text Quality-Based Pruning for Efficient Training of Language Models [66.66259229732121]
  We propose a novel method for numerically evaluating text quality in large unlabelled NLP datasets. By proposing the text quality metric, the paper establishes a framework to identify and eliminate low-quality text instances. Experimental results over multiple models and datasets demonstrate the efficacy of this approach.
  arXiv  Detail & Related papers  (2024-04-26T18:01:25Z)
• Efficient data selection employing Semantic Similarity-based Graph Structures for model training [1.5845679507219355]
  This paper introduces Semantics for data SAliency in Model performance Estimation (SeSaME) It is an efficient data sampling mechanism solely based on textual information without passing the data through a compute-heavy model. The application of this approach is demonstrated in the use case of low-resource automated speech recognition (ASR) models.
  arXiv  Detail & Related papers  (2024-02-22T09:43:53Z)
• Improving Text Embeddings with Large Language Models [59.930513259982725]
  We introduce a novel and simple method for obtaining high-quality text embeddings using only synthetic data and less than 1k training steps. We leverage proprietary LLMs to generate diverse synthetic data for hundreds of thousands of text embedding tasks across 93 languages. Experiments demonstrate that our method achieves strong performance on highly competitive text embedding benchmarks without using any labeled data.
  arXiv  Detail & Related papers  (2023-12-31T02:13:18Z)
• SelfSeg: A Self-supervised Sub-word Segmentation Method for Neural Machine Translation [51.881877192924414]
  Sub-word segmentation is an essential pre-processing step for Neural Machine Translation (NMT) This paper introduces SelfSeg, a self-supervised neural sub-word segmentation method. SelfSeg is much faster to train/decode and requires only monolingual dictionaries instead of parallel corpora.
  arXiv  Detail & Related papers  (2023-07-31T04:38:47Z)
• Strategies for improving low resource speech to text translation relying on pre-trained ASR models [59.90106959717875]
  This paper presents techniques and findings for improving the performance of low-resource speech to text translation (ST) We conducted experiments on both simulated and real-low resource setups, on language pairs English - Portuguese, and Tamasheq - French respectively.
  arXiv  Detail & Related papers  (2023-05-31T21:58:07Z)
• Code-Switching Text Generation and Injection in Mandarin-English ASR [57.57570417273262]
  We investigate text generation and injection for improving the performance of an industry commonly-used streaming model, Transformer-Transducer (T-T) We first propose a strategy to generate code-switching text data and then investigate injecting generated text into T-T model explicitly by Text-To-Speech (TTS) conversion or implicitly by tying speech and text latent spaces. Experimental results on the T-T model trained with a dataset containing 1,800 hours of real Mandarin-English code-switched speech show that our approaches to inject generated code-switching text significantly boost the performance of T-T models.
  arXiv  Detail & Related papers  (2023-03-20T09:13:27Z)
• UZH_CLyp at SemEval-2023 Task 9: Head-First Fine-Tuning and ChatGPT Data Generation for Cross-Lingual Learning in Tweet Intimacy Prediction [3.1798318618973362]
  This paper describes the submission of UZH_CLyp for the SemEval 2023 Task 9 "Multilingual Tweet Intimacy Analysis" We achieved second-best results in all 10 languages according to the official Pearson's correlation regression evaluation measure.
  arXiv  Detail & Related papers  (2023-03-02T12:18:53Z)
• Speech-text based multi-modal training with bidirectional attention for improved speech recognition [26.47071418582507]
  We propose to employ a novel bidirectional attention mechanism (BiAM) to jointly learn both ASR encoder (bottom layers) and text encoder with a multi-modal learning method. BiAM is to facilitate feature sampling rate exchange, realizing the quality of the transformed features for the one kind to be measured in another space. Experimental results on Librispeech corpus show it can achieve up to 6.15% word error rate reduction (WERR) with only paired data learning, while 9.23% WERR when more unpaired text data is employed.
  arXiv  Detail & Related papers  (2022-11-01T08:25:11Z)
• Investigating Lexical Replacements for Arabic-English Code-Switched Data Augmentation [32.885722714728765]
  We investigate data augmentation techniques for code-switching (CS) NLP systems. We perform lexical replacements using word-aligned parallel corpora. We compare these approaches against dictionary-based replacements.
  arXiv  Detail & Related papers  (2022-05-25T10:44:36Z)
• USTED: Improving ASR with a Unified Speech and Text Encoder-Decoder [8.88137815551529]
  We propose training ASR model jointly with a set of text-to-text auxiliary tasks. We observe WER reductions of 16% and 20% on test-other and test-clean respectively over an ASR-only baseline. We achieve further improvements when we train masked language model on Librispeech data or when we use machine translation as the auxiliary task.
  arXiv  Detail & Related papers  (2022-02-12T11:35:59Z)
• SDA: Improving Text Generation with Self Data Augmentation [88.24594090105899]
  We propose to improve the standard maximum likelihood estimation (MLE) paradigm by incorporating a self-imitation-learning phase for automatic data augmentation. Unlike most existing sentence-level augmentation strategies, our method is more general and could be easily adapted to any MLE-based training procedure.
  arXiv  Detail & Related papers  (2021-01-02T01:15:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.