POaaS: Minimal-Edit Prompt Optimization as a Service to Lift Accuracy and Cut Hallucinations on On-Device sLLMs

• URL: http://arxiv.org/abs/2603.16045v1
• Date: Tue, 17 Mar 2026 01:05:39 GMT
• Title: POaaS: Minimal-Edit Prompt Optimization as a Service to Lift Accuracy and Cut Hallucinations on On-Device sLLMs
• Abstract summary: Small language models (sLLMs) are increasingly deployed on-device, where imperfect user prompts can trigger factual errors and hallucinations.<n>We propose PO, a minimal-edit prompt optimization layer that routes each query to lightweight specialists.
• Score: 2.2406645586052925
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Small language models (sLLMs) are increasingly deployed on-device, where imperfect user prompts--typos, unclear intent, or missing context--can trigger factual errors and hallucinations. Existing automatic prompt optimization (APO) methods were designed for large cloud LLMs and rely on search that often produces long, structured instructions; when executed under an on-device constraint where the same small model must act as optimizer and solver, these pipelines can waste context and even hurt accuracy. We propose POaaS, a minimal-edit prompt optimization layer that routes each query to lightweight specialists (Cleaner, Paraphraser, Fact-Adder) and merges their outputs under strict drift and length constraints, with a conservative skip policy for well-formed prompts. Under a strict fixed-model setting with Llama-3.2-3B-Instruct and Llama-3.1-8B-Instruct, POaaS improves both task accuracy and factuality while representative APO baselines degrade them, and POaaS recovers up to +7.4% under token deletion and mixup. Overall, per-query conservative optimization is a practical alternative to search-heavy APO for on-device sLLMs.

Related papers

• Larger Context Window, Fewer Overcorrections: Optimizing Prompts and Batching for Minimal-Edit Grammatical Error Correction [0.0]
  Minimal-edit Grammatical Error Correction (GEC) is a challenging task for zero-shot and few-shot prompted Large Language Models (LLMs)<n>We introduce a prompt-based approach that closes the gap to fine-tuned models through three advances in GEC prompting methodology.<n> powered by Gemini 3.1Pro, our prompt $F_0.5=78.32$ achieves a new prompt-based SOTA while shrinking the gap to finetuned single-model SOTA to a mere $0.38$ points.
  arXiv  Detail & Related papers  (2026-09-09T20:24:48Z)
• Reflective Prompt Tuning through Language Model Function-Calling [16.552558967042312]
  Reflective Prompt Tuning (RPT) is a framework that simulates the iterative workflow of human prompt engineers.<n>RPT improves over initial prompts by up to 12.9 points, remains competitive with state of the art, and improves confidence calibration.<n>Our analyses show that RPT is especially effective on multi-hop and mathematical reasoning.
  arXiv  Detail & Related papers  (2026-05-20T22:21:29Z)
• Towards Self-Robust LLMs: Intrinsic Prompt Noise Resistance via CoIPO [38.36870540583669]
  Large language models (LLMs) have demonstrated remarkable and steadily improving performance across a wide range of tasks.<n>In real-world applications, user prompts provided to LLMs often contain imperfections, which may undermine the quality of the model's responses.<n>We propose a Contrastive Learning-based Inverse Direct Preference Optimization (CoIPO) method that minimizes the discrepancy between the label-aligned logits produced by the model under a clean prompt and its noisy counterpart.
  arXiv  Detail & Related papers  (2026-02-09T13:24:50Z)
• Modular Prompt Optimization: Optimizing Structured Prompts with Section-Local Textual Gradients [0.8604557306886812]
  We introduce a schema-based prompt optimization framework that treats prompts as structured objects composed of fixed semantic sections.<n>We evaluate MPO on two reasoning benchmarks, ARC-Challenge and MMLU, using LLaMA-3 8B-Instruct and Mistral-7B-Instruct as solver models.
  arXiv  Detail & Related papers  (2026-01-07T16:20:08Z)
• Adaptive Dependency-aware Prompt Optimization Framework for Multi-Step LLM Pipeline [9.013236765328301]
  We propose ADOPT, an Adaptive Dependency-aware Prompt Optimization framework for multi-step LLM pipelines.<n> ADOPT explicitly models the dependency between each LLM step and the final task outcome, enabling precise text-gradient estimation.<n>Experiments on real-world datasets and diverse pipeline structures show that ADOPT is effective and robust.
  arXiv  Detail & Related papers  (2025-12-31T15:46:37Z)
• PromptTailor: Multi-turn Intent-Aligned Prompt Synthesis for Lightweight LLMs [6.220520502038547]
  We introduce PromptTailor, a system for controllable prompt generation for open-ended text.<n>It improves model output quality by intent-aligned prompt synthesis.<n>The system is a quantized Llama3-8B model fine-tuned with a lightweight LoRA adapter.
  arXiv  Detail & Related papers  (2025-11-20T22:17:48Z)
• Grammar-Guided Evolutionary Search for Discrete Prompt Optimisation [63.97051732013936]
  We propose an evolutionary search approach to automated discrete prompt optimisation consisting of two phases.<n>In the first phase, grammar-guided genetic programming is invoked to synthesise prompt-creating programmes.<n>In the second phase, local search is applied to explore the neighbourhoods of best-performing programmes.
  arXiv  Detail & Related papers  (2025-07-14T14:34:15Z)
• Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models? [65.18157595903124]
  This work investigates iterative approximate evaluation for arbitrary prompts.<n>It introduces Model Predictive Prompt Selection (MoPPS), a Bayesian risk-predictive framework.<n>MoPPS reliably predicts prompt difficulty and accelerates training with significantly reduced rollouts.
  arXiv  Detail & Related papers  (2025-07-07T03:20:52Z)
• Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights [75.83625828306839]
  textbfDrag-and-Drop LLMs (textitDnD) eliminates per-task training by mapping a handful of unlabeled task prompts directly to LoRA weight updates.<n>A lightweight text encoder distills each prompt batch into condition embeddings, which are then transformed by a cascaded hyper-convolutional decoder into the full set of LoRA matrices.
  arXiv  Detail & Related papers  (2025-06-19T15:38:21Z)
• Rethinking Prompt Optimizers: From Prompt Merits to Optimization [14.01541576309104]
  MePO is a merit-guided, locally deployable prompt trained on a merit-guided prompt dataset.<n>MePO avoids online optimization, reduces privacy concerns, and, by learning clear, interpretable merits, generalizes effectively to both large-scale and lightweight inference models.
  arXiv  Detail & Related papers  (2025-05-15T03:31:37Z)
• Teaching Your Models to Understand Code via Focal Preference Alignment [70.71693365502212]
  In existing approaches, a set of n candidate solutions is evaluated based on test case success rates.<n>Because this approach aligns entire failing code blocks rather than pinpointing specific errors, it lacks the granularity necessary to capture meaningful error-correction relationships.<n>We propose Target-DPO, a new preference alignment framework that mimics human iterative debug to refine Code LLMs.
  arXiv  Detail & Related papers  (2025-03-04T16:56:34Z)
• GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers [52.17222304851524]
  We introduce GReaTer, a novel prompt optimization technique that directly incorporates gradient information over task-specific reasoning.<n>By utilizing task loss gradients, GReaTer enables self-optimization of prompts for open-source, lightweight language models.<n> GReaTer consistently outperforms previous state-of-the-art prompt optimization methods.
  arXiv  Detail & Related papers  (2024-12-12T20:59:43Z)
• In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement [71.60563181678323]
  Large language models (LLMs) have achieved great success across diverse tasks, and fine-tuning is sometimes needed to further enhance generation quality.<n>To handle these challenges, a direct solution is to generate high-confidence'' data from unsupervised downstream tasks.<n>We propose a novel approach, pseudo-supervised demonstrations aligned prompt optimization (PAPO) algorithm, which jointly refines both the prompt and the overall pseudo-supervision.
  arXiv  Detail & Related papers  (2024-10-04T03:39:28Z)
• AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs [51.217126257318924]
  Large Language Models (LLMs) are vulnerable to jailbreaking attacks that lead to generation of inappropriate or harmful content.<n>We present a novel method that uses another LLM, called AdvPrompter, to generate human-readable adversarial prompts in seconds.
  arXiv  Detail & Related papers  (2024-04-21T22:18:13Z)
• Automatic Prompt Optimization with "Gradient Descent" and Beam Search [64.08364384823645]
  Large Language Models (LLMs) have shown impressive performance as general purpose agents, but their abilities remain highly dependent on prompts. We propose a simple and nonparametric solution to this problem, Automatic Prompt Optimization (APO) APO uses minibatches of data to form natural language "gradients" that criticize the current prompt. The gradients are then "propagated" into the prompt by editing the prompt in the opposite semantic direction of the gradient.
  arXiv  Detail & Related papers  (2023-05-04T15:15:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.