Optimising Hard Prompts with Few-Shot Meta-Prompting

• URL: http://arxiv.org/abs/2407.18920v1
• Date: Tue, 9 Jul 2024 07:02:57 GMT
• Title: Optimising Hard Prompts with Few-Shot Meta-Prompting
• Authors: Sayash Raaj Hiraou,
• Abstract summary: Contextual prompts include context in the form of a document or dialogue along with the natural language instructions to the Large Language Model (LLM) Masking the context, it acts as template for prompts. In this paper, we present an iterative method to generate better templates using an LLM from an existing set of prompt templates without revealing the context to the LLM.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prompting is a flexible and adaptable way of providing instructions to a Large Language Model (LLM). Contextual prompts include context in the form of a document or dialogue along with the natural language instructions to the LLM, often constraining the LLM to restrict facts to that of the given context while complying with the instructions. Masking the context, it acts as template for prompts. In this paper, we present an iterative method to generate better templates using an LLM from an existing set of prompt templates without revealing the context to the LLM. Multiple methods of optimising prompts using the LLM itself are explored to check the effect of few shot sampling methods on iterative propagation while maintaining linguistic styles and syntax on optimisation of prompt templates, yielding a 103.87% improvement using the best performing method. Comparison of the results of multiple contextual tasks demonstrate the ability of LLMs to maintain syntax while learning to replicate linguistic styles. Additionally, the effect on the output with different methods of prompt template generation is shown.

Related papers

• FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering [57.43420753842626]
  FinLFQA is a benchmark designed to evaluate the ability of Large Language Models to generate long-form answers to complex financial questions.<n>We provide an automatic evaluation framework covering both answer quality and attribution quality.
  arXiv  Detail & Related papers  (2025-10-07T20:06:15Z)
• Beyond Naïve Prompting: Strategies for Improved Zero-shot Context-aided Forecasting with LLMs [57.82819770709032]
  Large language models (LLMs) can be effective context-aided forecasters via na"ive direct prompting.<n>ReDP improves interpretability by eliciting explicit reasoning traces, allowing us to assess the model's reasoning over the context.<n>CorDP leverages LLMs solely to refine existing forecasts with context, enhancing their applicability in real-world forecasting pipelines.<n> IC-DP proposes embedding historical examples of context-aided forecasting tasks in the prompt, substantially improving accuracy even for the largest models.
  arXiv  Detail & Related papers  (2025-08-13T16:02:55Z)
• FinDPO: Financial Sentiment Analysis for Algorithmic Trading through Preference Optimization of LLMs [2.06242362470764]
  We introduce FinDPO, the first finance-specific sentiment analysis framework based on post-training human preference alignment.<n>The proposed FinDPO achieves state-of-the-art performance on standard sentiment classification benchmarks.<n>We show that FinDPO is the first sentiment-based approach to maintain substantial positive returns of 67% annually and strong risk-adjusted performance.
  arXiv  Detail & Related papers  (2025-07-24T13:57:05Z)
• Deriving Strategic Market Insights with Large Language Models: A Benchmark for Forward Counterfactual Generation [55.2788567621326]
  We introduce a novel benchmark, FIN-FORCE-FINancial FORward Counterfactual Evaluation.<n>By curating financial news headlines, FIN-FORCE supports LLM based forward counterfactual generation.<n>This paves the way for scalable and automated solutions for exploring and anticipating future market developments.
  arXiv  Detail & Related papers  (2025-05-26T02:41:50Z)
• Bridging Language Models and Financial Analysis [49.361943182322385]
  The rapid advancements in Large Language Models (LLMs) have unlocked transformative possibilities in natural language processing.<n>Financial data is often embedded in intricate relationships across textual content, numerical tables, and visual charts.<n>Despite the fast pace of innovation in LLM research, there remains a significant gap in their practical adoption within the finance industry.
  arXiv  Detail & Related papers  (2025-03-14T01:35:20Z)
• FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Trading [28.57263158928989]
  Large language models (LLMs) fine-tuned on multimodal financial data have demonstrated impressive reasoning capabilities.<n>We propose textscFLAG-Trader, a unified architecture integrating linguistic processing (via LLMs) with gradient-driven reinforcement learning (RL) policy optimization.
  arXiv  Detail & Related papers  (2025-02-17T04:45:53Z)
• Demystifying Domain-adaptive Post-training for Financial LLMs [87.28855088465197]
  FINDAP is a systematic and fine-grained investigation into domain-adaptive post-training of large language models.<n>Our approach consists of four key components: FinCap, FinRec, FinTrain and FinEval.<n>The resulting model, Llama-Fin, achieves, state-of-the-art performance across a wide range of financial tasks.
  arXiv  Detail & Related papers  (2025-01-09T04:26:15Z)
• IPO: Interpretable Prompt Optimization for Vision-Language Models [40.83071220530289]
  This paper introduces a simple but interpretable prompt (IPO) IPO utilizes large language models (LLMs) to generate textual prompts dynamically. We incorporate a large multimodal model (LMM) to condition on visual content by generating image descriptions.
  arXiv  Detail & Related papers  (2024-10-20T14:10:22Z)
• Privacy Policy Analysis through Prompt Engineering for LLMs [3.059256166047627]
  PAPEL (Privacy Policy Analysis through Prompt Engineering for LLMs) is a framework harnessing the power of Large Language Models (LLMs) to automate the analysis of privacy policies. It aims to streamline the extraction, annotation, and summarization of information from these policies, enhancing their accessibility and comprehensibility without requiring additional model training. We demonstrate the effectiveness of PAPEL with two applications: (i) annotation and (ii) contradiction analysis.
  arXiv  Detail & Related papers  (2024-09-23T10:23:31Z)
• Visual Prompting in Multimodal Large Language Models: A Survey [95.75225825537528]
  Multimodal large language models (MLLMs) equip pre-trained large-language models (LLMs) with visual capabilities. Visual prompting has emerged for more fine-grained and free-form visual instructions. This paper focuses on visual prompting, prompt generation, compositional reasoning, and prompt learning.
  arXiv  Detail & Related papers  (2024-09-05T08:47:34Z)
• Using Grammar Masking to Ensure Syntactic Validity in LLM-based Modeling Tasks [0.996023506058745]
  Grammar masking is used to guide large language models toward producing syntactically correct models for a given context-free grammar. We show that grammar masking can dramatically improve the modeling capabilities of several language models.
  arXiv  Detail & Related papers  (2024-07-08T17:19:59Z)
• MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization [73.7779735046424]
  We show that different prompts should be adapted to different Large Language Models (LLM) to enhance their capabilities across various downstream tasks in NLP. We then propose a model-adaptive prompt (MAPO) method that optimize the original prompts for each specific LLM in downstream tasks.
  arXiv  Detail & Related papers  (2024-07-04T18:39:59Z)
• Prompting Large Language Models with Audio for General-Purpose Speech Summarization [13.415189715216354]
  We introduce a framework for speech summarization that leverages the processing and reasoning capabilities of large language models (LLMs) We propose an end-to-end system that combines an instruction-tuned LLM with an audio encoder that converts speech into token representations that the LLM can interpret.
  arXiv  Detail & Related papers  (2024-06-10T02:04:28Z)
• A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law [65.87885628115946]
  Large language models (LLMs) are revolutionizing the landscapes of finance, healthcare, and law. We highlight the instrumental role of LLMs in enhancing diagnostic and treatment methodologies in healthcare, innovating financial analytics, and refining legal interpretation and compliance strategies. We critically examine the ethics for LLM applications in these fields, pointing out the existing ethical concerns and the need for transparent, fair, and robust AI systems.
  arXiv  Detail & Related papers  (2024-05-02T22:43:02Z)
• Efficient Prompting Methods for Large Language Models: A Survey [50.171011917404485]
  Prompting has become a mainstream paradigm for adapting large language models (LLMs) to specific natural language processing tasks. This approach brings the additional computational burden of model inference and human effort to guide and control the behavior of LLMs. We present the basic concepts of prompting, review the advances for efficient prompting, and highlight future research directions.
  arXiv  Detail & Related papers  (2024-04-01T12:19:08Z)
• FinLlama: Financial Sentiment Classification for Algorithmic Trading Applications [2.2661367844871854]
  Large Language Models (LLMs) can be used in this context, but they are not finance-specific and tend to require significant computational resources. We introduce a novel approach based on the Llama 2 7B foundational model, in order to benefit from its generative nature and comprehensive language manipulation. This is achieved by fine-tuning the Llama2 7B model on a small portion of supervised financial sentiment analysis data.
  arXiv  Detail & Related papers  (2024-03-18T22:11:00Z)
• Generative Context-aware Fine-tuning of Self-supervised Speech Models [54.389711404209415]
  We study the use of generative large language models (LLM) generated context information. We propose an approach to distill the generated information during fine-tuning of self-supervised speech models. We evaluate the proposed approach using the SLUE and Libri-light benchmarks for several downstream tasks: automatic speech recognition, named entity recognition, and sentiment analysis.
  arXiv  Detail & Related papers  (2023-12-15T15:46:02Z)
• CLAMP: Contrastive LAnguage Model Prompt-tuning [89.96914454453791]
  We show that large language models can achieve good image classification performance when adapted this way. Our approach beats state-of-the-art mLLMs by 13% and slightly outperforms contrastive learning with a custom text model.
  arXiv  Detail & Related papers  (2023-12-04T05:13:59Z)
• AlignedCoT: Prompting Large Language Models via Native-Speaking Demonstrations [52.43593893122206]
  Alignedcot is an in-context learning technique for invoking Large Language Models. It achieves consistent and correct step-wise prompts in zero-shot scenarios. We conduct experiments on mathematical reasoning and commonsense reasoning.
  arXiv  Detail & Related papers  (2023-11-22T17:24:21Z)
• Large Language Models in Finance: A Survey [12.243277149505364]
  Large language models (LLMs) have opened new possibilities for artificial intelligence applications in finance. Recent advances in large language models (LLMs) have opened new possibilities for artificial intelligence applications in finance.
  arXiv  Detail & Related papers  (2023-09-28T06:04:04Z)
• Context-Aware Prompt Tuning for Vision-Language Model with Dual-Alignment [15.180715595425864]
  We introduce a novel method to improve the prompt learning of vision-language models by incorporating pre-trained large language models (LLMs) With DuAl-PT, we propose to learn more context-aware prompts, benefiting from both explicit and implicit context modeling. Empirically, DuAl-PT achieves superior performance on 11 downstream datasets on few-shot recognition and base-to-new generalization.
  arXiv  Detail & Related papers  (2023-09-08T06:51:15Z)
• Guiding Large Language Models via Directional Stimulus Prompting [114.84930073977672]
  We introduce Directional Stimulus Prompting, a novel framework for guiding black-box large language models (LLMs) toward specific desired outputs. Instead of directly adjusting LLMs, our method employs a small tunable policy model to generate an auxiliary directional stimulus prompt for each input instance.
  arXiv  Detail & Related papers  (2023-02-22T17:44:15Z)
• A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT [1.2640882896302839]
  This paper provides contributions to research on prompt engineering that apply large language models (LLMs) to automate software development tasks. It provides a framework for documenting patterns for structuring prompts to solve a range of problems so that they can be adapted to different domains. Third, it explains how prompts can be built from multiple patterns and illustrates prompt patterns that benefit from combination with other prompt patterns.
  arXiv  Detail & Related papers  (2023-02-21T12:42:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.