Simplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMs

• URL: http://arxiv.org/abs/2601.05794v1
• Date: Fri, 09 Jan 2026 13:46:52 GMT
• Title: Simplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMs
• Authors: Eilam Cohen, Itamar Bul, Danielle Inbar, Omri Loewenbach,
• Abstract summary: Large language models (LLMs) enable strong text generation, and in general there is a practical tradeoff between fine-tuning and prompt engineering.<n>We introduce Simplify-This, a comparative study evaluating both paradigms for text simplification with encoder-decoder LLMs.<n>Fine-tuned models consistently deliver stronger structural simplification, whereas prompting often attains higher semantic similarity scores yet tends to copy inputs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) enable strong text generation, and in general there is a practical tradeoff between fine-tuning and prompt engineering. We introduce Simplify-This, a comparative study evaluating both paradigms for text simplification with encoder-decoder LLMs across multiple benchmarks, using a range of evaluation metrics. Fine-tuned models consistently deliver stronger structural simplification, whereas prompting often attains higher semantic similarity scores yet tends to copy inputs. A human evaluation favors fine-tuned outputs overall. We release code, a cleaned derivative dataset used in our study, checkpoints of fine-tuned models, and prompt templates to facilitate reproducibility and future work.

Related papers

• Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings [25.565372681837697]
  We introduce a Multi-Granularity Hard-negative (MGH) synthesis framework to generate diverse negative samples with varying levels of similarity with the query.<n>We also propose an Anchor Token Aware (ATA) pooling method to improve text embedding accuracy without increasing model complexity.
  arXiv  Detail & Related papers  (2025-08-31T13:24:48Z)
• Re-Evaluating Code LLM Benchmarks Under Semantic Mutation [8.58692613099365]
  We present an empirical study to investigate prompt sensitivity in code benchmarks.<n>We propose a general framework that modifies prompt templates in a manner that preserves both their semantics and their structure.<n>Our findings suggest that even slight prompt variations can lead to significant shifts in performance.
  arXiv  Detail & Related papers  (2025-06-20T15:30:36Z)
• Evaluation Under Imperfect Benchmarks and Ratings: A Case Study in Text Simplification [13.381644813030725]
  We introduce a synthetic benchmark for text simplification featuring simplified sentences generated by models of varying sizes.<n>We show that human ratings on our benchmark exhibit high inter-annotator agreement and reflect the expected trend.<n>Second, we show that auto-evaluation with a panel of LLM judges (LLMs-as-a-jury) often suffices to obtain consistent ratings for the evaluation of text simplification.
  arXiv  Detail & Related papers  (2025-04-13T01:36:47Z)
• Scoring Verifiers: Evaluating Synthetic Verification for Code and Reasoning [59.25951947621526]
  We propose an approach which can transform existing coding benchmarks into scoring and ranking datasets to evaluate the effectiveness of synthetic verifiers.<n>We release four new benchmarks (HE-R, HE-R+, MBPP-R, and MBPP-R+), and analyzed synthetic verification methods with standard, reasoning-based, and reward-based LLMs.<n>Our experiments show that reasoning can significantly improve test case generation and that scaling the number of test cases enhances the verification accuracy.
  arXiv  Detail & Related papers  (2025-02-19T15:32:11Z)
• Detection and Measurement of Syntactic Templates in Generated Text [58.111650675717414]
  We offer an analysis of syntactic features to characterize general repetition in models. We find that models tend to produce templated text in downstream tasks at a higher rate than what is found in human-reference texts.
  arXiv  Detail & Related papers  (2024-06-28T19:34:23Z)
• A Better LLM Evaluator for Text Generation: The Impact of Prompt Output Sequencing and Optimization [17.38671584773247]
  This research investigates prompt designs of evaluating generated texts using large language models (LLMs) We found that the order of presenting reasons and scores significantly influences LLMs' scoring. An additional optimization may enhance scoring alignment if sufficient data is available.
  arXiv  Detail & Related papers  (2024-06-14T12:31:44Z)
• SemScore: Automated Evaluation of Instruction-Tuned LLMs based on Semantic Textual Similarity [3.3162484539136416]
  We propose a simple but remarkably effective evaluation metric called SemScore. We compare model outputs to gold target responses using semantic textual similarity (STS) We find that our proposed SemScore metric outperforms all other, in many cases more complex, evaluation metrics in terms of correlation to human evaluation.
  arXiv  Detail & Related papers  (2024-01-30T14:52:50Z)
• Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting [68.19544657508509]
  Large language models (LLMs) are adopted as a fundamental component of language technologies. We find that several widely used open-source LLMs are extremely sensitive to subtle changes in prompt format in few-shot settings. We propose an algorithm that rapidly evaluates a sampled set of plausible prompt formats for a given task, and reports the interval of expected performance without accessing model weights.
  arXiv  Detail & Related papers  (2023-10-17T15:03:30Z)
• MURMUR: Modular Multi-Step Reasoning for Semi-Structured Data-to-Text Generation [102.20036684996248]
  We propose MURMUR, a neuro-symbolic modular approach to text generation from semi-structured data with multi-step reasoning. We conduct experiments on two data-to-text generation tasks like WebNLG and LogicNLG.
  arXiv  Detail & Related papers  (2022-12-16T17:36:23Z)
• Classifiers are Better Experts for Controllable Text Generation [63.17266060165098]
  We show that the proposed method significantly outperforms recent PPLM, GeDi, and DExperts on PPL and sentiment accuracy based on the external classifier of generated texts. The same time, it is also easier to implement and tune, and has significantly fewer restrictions and requirements.
  arXiv  Detail & Related papers  (2022-05-15T12:58:35Z)
• SDA: Improving Text Generation with Self Data Augmentation [88.24594090105899]
  We propose to improve the standard maximum likelihood estimation (MLE) paradigm by incorporating a self-imitation-learning phase for automatic data augmentation. Unlike most existing sentence-level augmentation strategies, our method is more general and could be easily adapted to any MLE-based training procedure.
  arXiv  Detail & Related papers  (2021-01-02T01:15:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.