PatternGPT :A Pattern-Driven Framework for Large Language Model Text Generation

• URL: http://arxiv.org/abs/2307.00470v4
• Date: Thu, 20 Jul 2023 03:03:25 GMT
• Title: PatternGPT :A Pattern-Driven Framework for Large Language Model Text Generation
• Authors: Le Xiao and Xin Shan
• Abstract summary: This paper proposes PatternGPT, a pattern-driven text generation framework for Large Language Models. The framework utilizes the extraction capability of Large Language Models to generate rich and diversified structured and formalized patterns. external knowledge such as judgment criteria and optimization algorithms are used to search for high-quality patterns.
• Score: 1.7259824817932292
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models(LLMS)have shown excellent text generation capabilities, capable of generating fluent human-like responses for many downstream tasks. However, applying large language models to real-world critical tasks remains challenging due to their susceptibility to hallucinations and inability to directly use external knowledge. To cope with the above challenges, this paper proposes PatternGPT, a pattern-driven text generation framework for Large Language Models. Firstly, the framework utilizes the extraction capability of Large Language Models to generate rich and diversified structured and formalized patterns, which facilitates the introduction of external knowledge to do the computation, and then draws on the idea of federated learning to use multiple agents to achieve the sharing in order to obtain more diversified patterns, and finally uses judgment criteria and optimization algorithm to search for high-quality patterns to guide the generation of models. Finally, external knowledge such as judgment criteria and optimization algorithms are used to search for high-quality patterns, and the searched patterns are used to guide model generation. This framework has the advantages of generating diversified patterns, protecting data privacy, combining external knowledge, and improving the quality of generation, which provides an effective method to optimize the text generation capability of large language models, and make it better applied to the field of intelligent dialogue and content generation.

Related papers

• Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generation [0.20971479389679337]
  We introduce adaptive contrastive search, a novel decoding strategy extending contrastive search. Our findings indicate performance enhancement in both aspects, across different model architectures and datasets.
  arXiv  Detail & Related papers  (2024-07-26T12:23:54Z)
• Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation [87.50120181861362]
  VisionPrefer is a high-quality and fine-grained preference dataset that captures multiple preference aspects. We train a reward model VP-Score over VisionPrefer to guide the training of text-to-image generative models and the preference prediction accuracy of VP-Score is comparable to human annotators.
  arXiv  Detail & Related papers  (2024-04-23T14:53:15Z)
• Retrieval is Accurate Generation [99.24267226311157]
  We introduce a novel method that selects context-aware phrases from a collection of supporting documents. Our model achieves the best performance and the lowest latency among several retrieval-augmented baselines.
  arXiv  Detail & Related papers  (2024-02-27T14:16:19Z)
• A Comparative Analysis of Conversational Large Language Models in Knowledge-Based Text Generation [5.661396828160973]
  We conduct an empirical analysis of conversational large language models in generating natural language text from semantic triples. We compare four large language models of varying sizes with different prompting techniques. Our findings show that the capabilities of large language models in triple verbalization can be significantly improved through few-shot prompting, post-processing, and efficient fine-tuning techniques.
  arXiv  Detail & Related papers  (2024-02-02T15:26:39Z)
• RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model [93.8067369210696]
  Text-to-image generation (TTI) refers to the usage of models that could process text input and generate high fidelity images based on text descriptions. Diffusion models are one prominent type of generative model used for the generation of images through the systematic introduction of noises with repeating steps. In the era of large models, scaling up model size and the integration with large language models have further improved the performance of TTI models.
  arXiv  Detail & Related papers  (2023-09-02T03:27:20Z)
• Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy [164.83371924650294]
  We show that strong performance can be achieved by a method we call Iter-RetGen, which synergizes retrieval and generation in an iterative manner. A model output shows what might be needed to finish a task, and thus provides an informative context for retrieving more relevant knowledge. Iter-RetGen processes all retrieved knowledge as a whole and largely preserves the flexibility in generation without structural constraints.
  arXiv  Detail & Related papers  (2023-05-24T16:17:36Z)
• Controllable Text Generation for Open-Domain Creativity and Fairness [36.744208990024575]
  I introduce our recent works on controllable text generation to enhance the creativity and fairness of language generation models. We explore hierarchical generation and constrained decoding, with applications to creative language generation including story, poetry, and figurative languages.
  arXiv  Detail & Related papers  (2022-09-24T22:40:01Z)
• Twist Decoding: Diverse Generators Guide Each Other [116.20780037268801]
  We introduce Twist decoding, a simple and general inference algorithm that generates text while benefiting from diverse models. Our method does not assume the vocabulary, tokenization or even generation order is shared.
  arXiv  Detail & Related papers  (2022-05-19T01:27:53Z)
• Incorporating Stylistic Lexical Preferences in Generative Language Models [10.62343151429147]
  We present an approach to induce certain target-author attributes by incorporating continuous multi-dimensional lexical preferences of an author into generative language models. Our experiments demonstrate that the proposed approach can generate text that distinctively aligns with a given target author's lexical style.
  arXiv  Detail & Related papers  (2020-10-22T09:24:05Z)
• Improving Adversarial Text Generation by Modeling the Distant Future [155.83051741029732]
  We consider a text planning scheme and present a model-based imitation-learning approach to alleviate the aforementioned issues. We propose a novel guider network to focus on the generative process over a longer horizon, which can assist next-word prediction and provide intermediate rewards for generator optimization.
  arXiv  Detail & Related papers  (2020-05-04T05:45:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.