UltraGen: Extremely Fine-grained Controllable Generation via Attribute Reconstruction and Global Preference Optimization

• URL: http://arxiv.org/abs/2502.12375v1
• Date: Mon, 17 Feb 2025 23:28:58 GMT
• Title: UltraGen: Extremely Fine-grained Controllable Generation via Attribute Reconstruction and Global Preference Optimization
• Authors: Longfei Yun, Letian Peng, Jingbo Shang,
• Abstract summary: existing methods focus mainly on a small set of attributes like 3 to 5, and their degrades significantly when the number of attributes increases to magnitude. We propose a novel zero-shot approach for extremely finegrained controllable generation (EFCG) Our framework significantly improves the constraint satisfaction rate (CSR) and text quality for EFCG by mitigating bias and alleviating attention dilution.
• Score: 33.747872934103334
• License:
• Abstract: Fine granularity is an essential requirement for controllable text generation, which has seen rapid growth with the ability of LLMs. However, existing methods focus mainly on a small set of attributes like 3 to 5, and their performance degrades significantly when the number of attributes increases to the next order of magnitude. To address this challenge, we propose a novel zero-shot approach for extremely fine-grained controllable generation (EFCG), proposing auto-reconstruction (AR) and global preference optimization (GPO). In the AR phase, we leverage LLMs to extract soft attributes (e.g., Emphasis on simplicity and minimalism in design) from raw texts, and combine them with programmatically derived hard attributes (e.g., The text should be between 300 and 400 words) to construct massive (around 45) multi-attribute requirements, which guide the fine-grained text reconstruction process under weak supervision. In the GPO phase, we apply direct preference optimization (DPO) to refine text generation under diverse attribute combinations, enabling efficient exploration of the global combination space. Additionally, we introduce an efficient attribute sampling strategy to identify and correct potentially erroneous attributes, further improving global optimization. Our framework significantly improves the constraint satisfaction rate (CSR) and text quality for EFCG by mitigating position bias and alleviating attention dilution.

Related papers

• Towards Lightweight, Adaptive and Attribute-Aware Multi-Aspect Controllable Text Generation with Large Language Models [40.54453001537357]
  Multi-aspect controllable text generation aims to control text generation in attributes from multiple aspects. Supervised fine-tuning methods are often employed for this task due to their simplicity and effectiveness. We propose a lightweight, adaptive and attribute-aware framework for multi-aspect controllable text generation.
  arXiv  Detail & Related papers  (2025-02-19T06:56:02Z)
• Multi-Attribute Constraint Satisfaction via Language Model Rewriting [67.5778646504987]
  Multi-Attribute Constraint Satisfaction (MACS) is a method capable of finetuning language models to satisfy user-specified constraints on multiple external real-value attributes. Our work opens new avenues for generalized and real-value multi-attribute control, with implications for diverse applications spanning NLP and bioinformatics.
  arXiv  Detail & Related papers  (2024-12-26T12:36:39Z)
• Effective Text Adaptation for LLM-based ASR through Soft Prompt Fine-Tuning [12.676026149146772]
  Large Language Models (LLM) has reformed the Automatic Speech Recognition (ASR) Fine-tuning such ASR on text-only data without paired prompts may diminish the effectiveness of domain-specific knowledge. We propose a two-step soft prompt fine-tuning strategy that enhances domain-specific text adaptation.
  arXiv  Detail & Related papers  (2024-12-09T20:22:06Z)
• UniGen: A Unified Framework for Textual Dataset Generation Using Large Language Models [88.16197692794707]
  UniGen is a comprehensive framework designed to produce diverse, accurate, and highly controllable datasets. To augment data diversity, UniGen incorporates an attribute-guided generation module and a group checking feature. Extensive experiments demonstrate the superior quality of data generated by UniGen.
  arXiv  Detail & Related papers  (2024-06-27T07:56:44Z)
• TextGrad: Automatic "Differentiation" via Text [32.94896315864364]
  TextGrad backpropagates textual feedback to improve individual components of a compound AI system. It works out-of-the-box for a variety of tasks, where the users only provide the objective function without tuning components or prompts of the framework. We showcase TextGrad's effectiveness and generality across a diverse range of applications, from question answering and molecule optimization to radiotherapy treatment planning.
  arXiv  Detail & Related papers  (2024-06-11T17:32:21Z)
• One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models [67.49462724595445]
  Retrieval-augmented generation (RAG) is a promising way to improve large language models (LLMs) We propose a novel method that involves learning scalable and pluggable virtual tokens for RAG.
  arXiv  Detail & Related papers  (2024-05-30T03:44:54Z)
• Improving Cross-Domain Low-Resource Text Generation through LLM Post-Editing: A Programmer-Interpreter Approach [50.400999859808984]
  Post-editing has proven effective in improving the quality of text generated by large language models (LLMs) We propose a neural programmer-interpreter approach that preserves the domain generalization ability of LLMs when editing their output. Experiments demonstrate that the programmer-interpreter significantly enhances GPT-3.5's performance in logical form-to-text conversion and low-resource machine translation.
  arXiv  Detail & Related papers  (2024-02-07T06:13:14Z)
• Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuning [24.765911297156855]
  FISH-DIP is a sample-aware dynamic sparse finetuning strategy that selectively focuses on a fraction of parameters. We demonstrate that FISH-DIP can smoothly optimize the model in low resource settings offering upto 40% performance improvements.
  arXiv  Detail & Related papers  (2023-11-07T06:19:37Z)
• Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers [70.18534453485849]
  EvoPrompt is a framework for discrete prompt optimization. It borrows the idea of evolutionary algorithms (EAs) as they exhibit good performance and fast convergence. It significantly outperforms human-engineered prompts and existing methods for automatic prompt generation.
  arXiv  Detail & Related papers  (2023-09-15T16:50:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.