Efficient Prompt Optimisation for Legal Text Classification with Proxy Prompt Evaluator

• URL: http://arxiv.org/abs/2510.08524v1
• Date: Thu, 09 Oct 2025 17:49:53 GMT
• Title: Efficient Prompt Optimisation for Legal Text Classification with Proxy Prompt Evaluator
• Authors: Hyunji Lee, Kevin Chenhao Li, Matthias Grabmair, Shanshan Xu,
• Abstract summary: We propose a framework that combines Monte Carlo Tree Search (MCTS) with a proxy prompt evaluator to more effectively explore the prompt space.<n> Experiments demonstrate that our approach achieves higher classification accuracy and efficiency than baseline methods under a constrained computation budget.
• Score: 15.858271325271438
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prompt optimization aims to systematically refine prompts to enhance a language model's performance on specific tasks. Fairness detection in Terms of Service (ToS) clauses is a challenging legal NLP task that demands carefully crafted prompts to ensure reliable results. However, existing prompt optimization methods are often computationally expensive due to inefficient search strategies and costly prompt candidate scoring. In this paper, we propose a framework that combines Monte Carlo Tree Search (MCTS) with a proxy prompt evaluator to more effectively explore the prompt space while reducing evaluation costs. Experiments demonstrate that our approach achieves higher classification accuracy and efficiency than baseline methods under a constrained computation budget.

Related papers

• Bayesian Optimization in Language Space: An Eval-Efficient AI Self-Improvement Framework [0.0]
  Large Language Models (LLMs) have recently enabled self-improving AI, i.e., AI that iteratively generates, evaluates, and refines its own outcomes.<n>In many societal applications, the primary limitation is not generating new solutions but evaluating them.<n>This paper overcomes this challenge by proving that the combination of the simple and widely used Best-of-N selection strategy and simple textual gradients statistically emulates the behavior of the gradients on the canonical UCB acquisition function.
  arXiv  Detail & Related papers  (2025-11-15T07:04:44Z)
• Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models? [65.18157595903124]
  This work investigates iterative approximate evaluation for arbitrary prompts.<n>It introduces Model Predictive Prompt Selection (MoPPS), a Bayesian risk-predictive framework.<n>MoPPS reliably predicts prompt difficulty and accelerates training with significantly reduced rollouts.
  arXiv  Detail & Related papers  (2025-07-07T03:20:52Z)
• Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective [65.12150411762273]
  We show that pruning random demonstrations into seemingly incoherent "gibberish" can remarkably improve performance across diverse tasks.<n>We propose a self-discover prompt optimization framework, PromptQuine, that automatically searches for the pruning strategy by itself using only low-data regimes.
  arXiv  Detail & Related papers  (2025-06-22T07:53:07Z)
• HPSS: Heuristic Prompting Strategy Search for LLM Evaluators [81.09765876000208]
  We propose a novel automatic prompting strategy optimization method called Heuristic Prompting Strategy Search (HPSS)<n>Inspired by the genetic algorithm, HPSS conducts an iterative search to find well-behaved prompting strategies for evaluators.<n>Extensive experiments across four evaluation tasks demonstrate the effectiveness of HPSS.
  arXiv  Detail & Related papers  (2025-02-18T16:46:47Z)
• A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Models [14.483240353801074]
  This paper proposes an optimal learning framework for automated prompt engineering.<n>It is designed to sequentially identify effective prompt features while efficiently allocating a limited evaluation budget.<n>Our framework provides a solution to deploying automated prompt engineering in a wider range applications.
  arXiv  Detail & Related papers  (2025-01-07T03:51:10Z)
• Hyperband-based Bayesian Optimization for Black-box Prompt Selection [15.756224286651237]
  Black-box prompt selection is challenging due to potentially large, search spaces, absence of gradient information, and high evaluation cost of prompts on a validation set.<n>We propose HbBoPs, a novel method that combines a structural-aware deep kernel Gaussian Process with Hyperband as a multi-fidelity scheduler.<n>HbBoPs outperforms state-of-the-art methods in both performance and efficiency.
  arXiv  Detail & Related papers  (2024-12-10T14:42:51Z)
• SCULPT: Systematic Tuning of Long Prompts [17.00433893207345]
  We propose a framework that treats prompt optimization as a hierarchical tree refinement problem.<n>SCULPT represents prompts as tree structures, enabling targeted modifications while preserving contextual integrity.<n>It produces more stable and interpretable prompt modifications, ensuring better generalization across tasks.
  arXiv  Detail & Related papers  (2024-10-28T07:10:10Z)
• In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement [71.60563181678323]
  Large language models (LLMs) have achieved great success across diverse tasks, and fine-tuning is sometimes needed to further enhance generation quality.<n>To handle these challenges, a direct solution is to generate high-confidence'' data from unsupervised downstream tasks.<n>We propose a novel approach, pseudo-supervised demonstrations aligned prompt optimization (PAPO) algorithm, which jointly refines both the prompt and the overall pseudo-supervision.
  arXiv  Detail & Related papers  (2024-10-04T03:39:28Z)
• Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL [62.824464372594576]
  We aim to enhance arithmetic reasoning ability of Large Language Models (LLMs) through zero-shot prompt optimization. We identify a previously overlooked objective of query dependency in such optimization. We introduce Prompt-OIRL, which harnesses offline inverse reinforcement learning to draw insights from offline prompting demonstration data.
  arXiv  Detail & Related papers  (2023-09-13T01:12:52Z)
• Cost-Efficient Online Hyperparameter Optimization [94.60924644778558]
  We propose an online HPO algorithm that reaches human expert-level performance within a single run of the experiment. Our proposed online HPO algorithm reaches human expert-level performance within a single run of the experiment, while incurring only modest computational overhead compared to regular training.
  arXiv  Detail & Related papers  (2021-01-17T04:55:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.