Text Simplification with Sentence Embeddings

• URL: http://arxiv.org/abs/2510.24365v1
• Date: Tue, 28 Oct 2025 12:41:10 GMT
• Title: Text Simplification with Sentence Embeddings
• Authors: Matthew Shardlow,
• Abstract summary: We learn to learn a transformation between sentence embeddings representing high-complexity and low-complexity texts.<n>We conclude that learning transformations in sentence embedding space is a promising direction for future research.
• Score: 4.484170173286332
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sentence embeddings can be decoded to give approximations of the original texts used to create them. We explore this effect in the context of text simplification, demonstrating that reconstructed text embeddings preserve complexity levels. We experiment with a small feed forward neural network to effectively learn a transformation between sentence embeddings representing high-complexity and low-complexity texts. We provide comparison to a Seq2Seq and LLM-based approach, showing encouraging results in our much smaller learning setting. Finally, we demonstrate the applicability of our transformation to an unseen simplification dataset (MedEASI), as well as datasets from languages outside the training data (ES,DE). We conclude that learning transformations in sentence embedding space is a promising direction for future research and has potential to unlock the ability to develop small, but powerful models for text simplification and other natural language generation tasks.

Related papers

• Beyond Repetition: Text Simplification and Curriculum Learning for Data-Constrained Pretraining [0.19258299315493077]
  We study curriculum learning in pretraining, focusing on text-complexity ordering and data augmentation via simplification.<n>We test four data schedules: repeated exposure, low-to-high complexity, high-to-low, and interleaved.<n>Our findings show that adding simplified data improves fine-tuning and zero-shot performance over a repeated-exposure baseline.
  arXiv  Detail & Related papers  (2025-09-29T06:54:59Z)
• Hallucination Detection and Mitigation in Scientific Text Simplification using Ensemble Approaches: DS@GT at CLEF 2025 SimpleText [0.0]
  We describe our methodology for the CLEF 2025 SimpleText Task 2, which focuses on detecting and evaluating creative generation and information distortion.<n>Our solution integrates multiple strategies: we construct an ensemble framework that leverages BERT-based classifier, semantic similarity measure, natural language inference model, and large language model.<n>For grounded generation, we employ an LLM-based post-editing system that revises simplifications based on the original input texts.
  arXiv  Detail & Related papers  (2025-08-15T21:57:27Z)
• Exploiting Summarization Data to Help Text Simplification [50.0624778757462]
  We analyzed the similarity between text summarization and text simplification and exploited summarization data to help simplify. We named these pairs Sum4Simp (S4S) and conducted human evaluations to show that S4S is high-quality.
  arXiv  Detail & Related papers  (2023-02-14T15:32:04Z)
• Composable Text Controls in Latent Space with ODEs [97.12426987887021]
  This paper proposes a new efficient approach for composable text operations in the compact latent space of text. By connecting pretrained LMs to the latent space through efficient adaption, we then decode the sampled vectors into desired text sequences. Experiments show that composing those operators within our approach manages to generate or edit high-quality text.
  arXiv  Detail & Related papers  (2022-08-01T06:51:45Z)
• SDA: Improving Text Generation with Self Data Augmentation [88.24594090105899]
  We propose to improve the standard maximum likelihood estimation (MLE) paradigm by incorporating a self-imitation-learning phase for automatic data augmentation. Unlike most existing sentence-level augmentation strategies, our method is more general and could be easily adapted to any MLE-based training procedure.
  arXiv  Detail & Related papers  (2021-01-02T01:15:57Z)
• Elaborative Simplification: Content Addition and Explanation Generation in Text Simplification [33.08519864889526]
  We present the first data-driven study of content addition in text simplification. We analyze how entities, ideas, and concepts are elaborated through the lens of contextual specificity. Our results illustrate the complexities of elaborative simplification, suggesting many interesting directions for future work.
  arXiv  Detail & Related papers  (2020-10-20T05:06:23Z)
• A Comparative Study on Structural and Semantic Properties of Sentence Embeddings [77.34726150561087]
  We propose a set of experiments using a widely-used large-scale data set for relation extraction. We show that different embedding spaces have different degrees of strength for the structural and semantic properties. These results provide useful information for developing embedding-based relation extraction methods.
  arXiv  Detail & Related papers  (2020-09-23T15:45:32Z)
• Neural CRF Model for Sentence Alignment in Text Simplification [31.62648025127563]
  We create two manually annotated sentence-aligned datasets from two commonly used text simplification corpora, Newsela and Wikipedia. Experiments demonstrate that our proposed approach outperforms all the previous work on monolingual sentence alignment task by more than 5 points in F1. A Transformer-based seq2seq model trained on our datasets establishes a new state-of-the-art for text simplification in both automatic and human evaluation.
  arXiv  Detail & Related papers  (2020-05-05T16:47:51Z)
• ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations [97.27005783856285]
  This paper introduces ASSET, a new dataset for assessing sentence simplification in English. We show that simplifications in ASSET are better at capturing characteristics of simplicity when compared to other standard evaluation datasets for the task.
  arXiv  Detail & Related papers  (2020-05-01T16:44:54Z)
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer [64.22926988297685]
  Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP) In this paper, we explore the landscape of introducing transfer learning techniques for NLP by a unified framework that converts all text-based language problems into a text-to-text format.
  arXiv  Detail & Related papers  (2019-10-23T17:37:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.