Iterative Critique-Refine Framework for Enhancing LLM Personalization

• URL: http://arxiv.org/abs/2510.24469v1
• Date: Tue, 28 Oct 2025 14:36:22 GMT
• Title: Iterative Critique-Refine Framework for Enhancing LLM Personalization
• Authors: Durga Prasad Maram, Dhruvin Gandhi, Zonghai Yao, Gayathri Akkinapalli, Franck Dernoncourt, Yu Wang, Ryan A. Rossi, Nesreen K. Ahmed,
• Abstract summary: We present PerFine, a unified, training-free critique-refine framework for personalized text generation.<n>In each iteration, an LLM generator produces a draft conditioned on a retrieved profile, and a critic LLM - also conditioned on the same profile - provides structured feedback on tone, vocabulary, sentence structure, and topicality.<n>Across Yelp, Goodreads, and Amazon datasets, PerFine consistently improves personalization over PGraphRAG.
• Score: 67.77803308645511
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Personalized text generation requires models not only to produce coherent text but also to align with a target user's style, tone, and topical focus. Existing retrieval-augmented approaches such as LaMP and PGraphRAG enrich profiles with user and neighbor histories, but they stop at generation and often yield outputs that drift in tone, topic, or style. We present PerFine, a unified, training-free critique-refine framework that enhances personalization through iterative, profile-grounded feedback. In each iteration, an LLM generator produces a draft conditioned on the retrieved profile, and a critic LLM - also conditioned on the same profile - provides structured feedback on tone, vocabulary, sentence structure, and topicality. The generator then revises, while a novel knockout strategy retains the stronger draft across iterations. We further study additional inference-time strategies such as Best-of-N and Topic Extraction to balance quality and efficiency. Across Yelp, Goodreads, and Amazon datasets, PerFine consistently improves personalization over PGraphRAG, with GEval gains of +7-13%, steady improvements over 3-5 refinement iterations, and scalability with increasing critic size. These results highlight that post-hoc, profile-aware feedback offers a powerful paradigm for personalized LLM generation that is both training-free and model-agnostic.

Related papers

• Reflective Personalization Optimization: A Post-hoc Rewriting Framework for Black-Box Large Language Models [16.152962349146275]
  We propose Reflective Personalization Optimization (RPO), a framework that redefines the personalization paradigm by decoupling content generation from alignment.<n>RPO operates in two distinct stages: first, a base model generates a high-quality, generic response; then, an external reflection module explicitly rewrites this output to align with the user's preferences.<n> Comprehensive experiments on the LaMP benchmark demonstrate that RPO, by decoupling content generation from personalization, significantly outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-11-07T14:48:49Z)
• RecMind: LLM-Enhanced Graph Neural Networks for Personalized Consumer Recommendations [1.9556834179471867]
  Personalization is a core capability across consumer technologies, streaming, shopping, wearables, and voice.<n>We present RecMind, an LLM-enhanced graph recommender that treats the language model as a preference prior to a monolithic ranker.<n>On Yelp and Amazon-Electronics, RecMind attains the best results on all eight reported metrics, with relative improvements up to +4.53% (Recall@40) and +4.01% (NDCG@40) over strong baselines.
  arXiv  Detail & Related papers  (2025-09-08T02:15:55Z)
• Learning from Natural Language Feedback for Personalized Question Answering [21.115495457454365]
  Personalization is crucial for enhancing the effectiveness and user satisfaction of language technologies.<n>Current approaches for personalizing large language models (LLMs) often rely on retrieval-augmented generation (RAG)<n>We introduce Vac, a novel framework for personalized response generation that replaces scalar rewards with natural language feedback (NLF)
  arXiv  Detail & Related papers  (2025-08-14T14:36:53Z)
• The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvement [61.00950725408354]
  Large language models (LLMs) have recently transformed from text-based assistants to autonomous agents capable of planning, reasoning, and iteratively improving their actions.<n>In this work, we introduce Critique-Guided Improvement (CGI), a novel two-player framework, comprising an actor model that explores an environment and a critic model that generates detailed nature language feedback.
  arXiv  Detail & Related papers  (2025-03-20T10:42:33Z)
• Personalized Text Generation with Contrastive Activation Steering [63.60368120937822]
  We propose a training-free framework that disentangles and represents personalized writing style as a vector.<n>Our framework achieves a significant 8% relative improvement in personalized generation while reducing storage requirements by 1700 times over PEFT method.
  arXiv  Detail & Related papers  (2025-03-07T08:07:15Z)
• Personalized Graph-Based Retrieval for Large Language Models [51.7278897841697]
  We propose a framework that leverages user-centric knowledge graphs to enrich personalization.<n>By directly integrating structured user knowledge into the retrieval process and augmenting prompts with user-relevant context, PGraph enhances contextual understanding and output quality.<n>We also introduce the Personalized Graph-based Benchmark for Text Generation, designed to evaluate personalized text generation tasks in real-world settings where user history is sparse or unavailable.
  arXiv  Detail & Related papers  (2025-01-04T01:46:49Z)
• Guided Profile Generation Improves Personalization with LLMs [3.2685922749445617]
  In modern commercial systems, including Recommendation, Ranking, and E-Commerce platforms, there is a trend towards incorporating Personalization context as input into Large Language Models (LLMs) We propose Guided Profile Generation (GPG), a general method designed to generate personal profiles in natural language. Our experimental results show that GPG improves LLM's personalization ability across different tasks, for example, it increases 37% accuracy in predicting personal preference compared to directly feeding the LLMs with raw personal context.
  arXiv  Detail & Related papers  (2024-09-19T21:29:56Z)
• Pistis-RAG: Enhancing Retrieval-Augmented Generation with Human Feedback [41.88662700261036]
  RAG systems face limitations when semantic relevance alone does not guarantee improved generation quality. We propose Pistis-RAG, a new RAG framework designed with a content-centric approach to better align LLMs with human preferences.
  arXiv  Detail & Related papers  (2024-06-21T08:52:11Z)
• Exploring Precision and Recall to assess the quality and diversity of LLMs [82.21278402856079]
  We introduce a novel evaluation framework for Large Language Models (LLMs) such as textscLlama-2 and textscMistral. This approach allows for a nuanced assessment of the quality and diversity of generated text without the need for aligned corpora.
  arXiv  Detail & Related papers  (2024-02-16T13:53:26Z)
• CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation [87.44350003888646]
  Eval-Instruct can acquire pointwise grading critiques with pseudo references and revise these critiques via multi-path prompting. CritiqueLLM is empirically shown to outperform ChatGPT and all the open-source baselines.
  arXiv  Detail & Related papers  (2023-11-30T16:52:42Z)
• Self-Refine: Iterative Refinement with Self-Feedback [62.78755306241981]
  Self-Refine is an approach for improving initial outputs from large language models (LLMs) through iterative feedback and refinement. We evaluate Self-Refine across 7 diverse tasks, ranging from dialog response generation to mathematical reasoning, using state-of-the-art (GPT-3.5, ChatGPT, and GPT-4) LLMs. Our work demonstrates that even state-of-the-art LLMs like GPT-4 can be further improved at test time using our simple, standalone approach.
  arXiv  Detail & Related papers  (2023-03-30T18:30:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.