Taming CATS: Controllable Automatic Text Simplification through Instruction Fine-Tuning with Control Tokens

• URL: http://arxiv.org/abs/2604.01779v1
• Date: Thu, 02 Apr 2026 08:44:17 GMT
• Title: Taming CATS: Controllable Automatic Text Simplification through Instruction Fine-Tuning with Control Tokens
• Abstract summary: Controllability in Automatic Text Simplification is significantly constrained by data and evaluation.<n>We introduce a domain-agnostic CATS framework based on instruction fine-tuning with discrete control tokens.<n>We show that standard simplification and similarity metrics are insufficient for measuring control.
• Score: 2.4713807020542773
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Controllable Automatic Text Simplification (CATS) produces user-tailored outputs, yet controllability is often treated as a decoding problem and evaluated with metrics that are not reflective to the measure of control. We observe that controllability in ATS is significantly constrained by data and evaluation. To this end, we introduce a domain-agnostic CATS framework based on instruction fine-tuning with discrete control tokens, steering open-source models to target readability levels and compression rates. Across three model families with different model sizes (Llama, Mistral, Qwen; 1-14B) and four domains (medicine, public administration, news, encyclopedic text), we find that smaller models (1-3B) can be competitive, but reliable controllability strongly depends on whether the training data encodes sufficient variation in the target attribute. Readability control (FKGL, ARI, Dale-Chall) is learned consistently, whereas compression control underperforms due to limited signal variability in the existing corpora. We further show that standard simplification and similarity metrics are insufficient for measuring control, motivating error-based measures for target-output alignment. Finally, our sampling and stratification experiments demonstrate that naive splits can introduce distributional mismatch that undermines both training and evaluation.

Related papers

• CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs [100.38986535324284]
  We introduce textbfCIAware-Bench, a benchmark for measuring textbfcontrol textbfintervention (CI) awareness across frontier models.<n>CIAware-Bench tests whether models can distinguish their own trajectories from those modified by a control intervention.
  arXiv  Detail & Related papers  (2026-06-09T16:24:16Z)
• Auditable Decision Models with Learned Abstention and Real-Time Steering [6.287457666346811]
  Production AI systems often operate with incomplete, conflicting, or insufficient evidence.<n>We study operational decision control for AI systems, where uncertainty must be explicitly routable.<n>We present EvaluatorDPT, a bounded decision-control model that predicts YES, NO, or TBD.
  arXiv  Detail & Related papers  (2026-05-26T23:37:56Z)
• Large Language Models as Explainable Cyberattack Detectors for Energy Industrial Control Systems [1.784933900656067]
  We study whether an off-the-shelf large language model (LLM) can serve as a complementary, human-in-the-loop layer for ICS detectors.<n>We cast this as a binary network-side normal/critical decision task on two public ICS datasets.<n>LLM produces a normal/critical alert together with a concise, token-grounded incident record for analyst review.
  arXiv  Detail & Related papers  (2026-04-28T19:44:38Z)
• Sparse Visual Thought Circuits in Vision-Language Models [2.5754366051855837]
  We develop a causal pipeline to localize and test sparse visual thought circuits in Qwen3-VL-8B.<n>We train SAEs at this layer, construct task-selective sets via an explicit rule, and perform inference time scaling and ablation while quantifying accuracy and drift.
  arXiv  Detail & Related papers  (2026-03-26T06:24:36Z)
• FaithSteer-BENCH: A Deployment-Aligned Stress-Testing Benchmark for Inference-Time Steering [11.609466767597093]
  FaithSteer-BENCH is a stress-testing benchmark forference-time steering.<n>It evaluates steering methods at a fixed deployment-style operating point through three gate-wise criteria: controllability, utility preservation, and robustness.<n> Gate-wise benchmark results show that existing methods do not necessarily provide reliable controllability in deployment-oriented practical settings.
  arXiv  Detail & Related papers  (2026-03-18T22:28:36Z)
• AMPS: Adaptive Modality Preference Steering via Functional Entropy [66.69992693275061]
  We introduce an instance-aware diagnostic metric that quantifies each modality's information contribution and reveals sample-specific susceptibility to steering.<n> Experimental results show that our instance-aware steering outperforms conventional steering in modulating modality preference.
  arXiv  Detail & Related papers  (2026-02-13T02:29:06Z)
• Steering Language Models Before They Speak: Logit-Level Interventions [9.055997973281919]
  We propose a training-free inference-time logit intervention for controllable generation.<n>Our results show that statistically grounded logit steering can achieve large, consistent, and multi-task control gains.
  arXiv  Detail & Related papers  (2026-01-16T03:00:33Z)
• Unsupervised Conformal Inference: Bootstrapping and Alignment to Control LLM Uncertainty [49.19257648205146]
  We propose an unsupervised conformal inference framework for generation.<n>Our gates achieve close-to-nominal coverage and provide tighter, more stable thresholds than split UCP.<n>The result is a label-free, API-compatible gate for test-time filtering.
  arXiv  Detail & Related papers  (2025-09-26T23:40:47Z)
• Analysing Zero-Shot Readability-Controlled Sentence Simplification [54.09069745799918]
  We investigate how different types of contextual information affect a model's ability to generate sentences with the desired readability.<n>Results show that all tested models struggle to simplify sentences due to models' limitations and characteristics of the source sentences.<n>Our experiments also highlight the need for better automatic evaluation metrics tailored to RCTS.
  arXiv  Detail & Related papers  (2024-09-30T12:36:25Z)
• A Study of Unsupervised Evaluation Metrics for Practical and Automatic Domain Adaptation [15.728090002818963]
  Unsupervised domain adaptation (UDA) methods facilitate the transfer of models to target domains without labels. In this paper, we aim to find an evaluation metric capable of assessing the quality of a transferred model without access to target validation labels.
  arXiv  Detail & Related papers  (2023-08-01T05:01:05Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)
• Controller-Guided Partial Label Consistency Regularization with Unlabeled Data [49.24911720809604]
  We propose a controller-guided consistency regularization at both the label-level and representation-level. We dynamically adjust the confidence thresholds so that the number of samples of each class participating in consistency regularization remains roughly equal to alleviate the problem of class-imbalance.
  arXiv  Detail & Related papers  (2022-10-20T12:15:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.