Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning

• URL: http://arxiv.org/abs/2308.05317v1
• Date: Thu, 10 Aug 2023 03:09:12 GMT
• Title: Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning
• Authors: Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, Jie Ma, Patrick Ng, Zhiguo Wang, Bonan Min, William Wang, Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang
• Abstract summary: We present a novel approach for structured data-to-text generation that addresses the limitations of existing methods. Our proposed method aims to improve performance in multi-task training, zero-shot and few-shot scenarios.
• Score: 114.54944761345594
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a novel approach for structured data-to-text generation that addresses the limitations of existing methods that primarily focus on specific types of structured data. Our proposed method aims to improve performance in multi-task training, zero-shot and few-shot scenarios by providing a unified representation that can handle various forms of structured data such as tables, knowledge graph triples, and meaning representations. We demonstrate that our proposed approach can effectively adapt to new structured forms, and can improve performance in comparison to current methods. For example, our method resulted in a 66% improvement in zero-shot BLEU scores when transferring models trained on table inputs to a knowledge graph dataset. Our proposed method is an important step towards a more general data-to-text generation framework.

Related papers

• An Active Learning Framework for Inclusive Generation by Large Language Models [32.16984263644299]
  Large Language Models (LLMs) generate text representative of diverse sub-populations. We propose a novel clustering-based active learning framework, enhanced with knowledge distillation. We construct two new datasets in tandem with model training, showing a performance improvement of 2%-10% over baseline models.
  arXiv  Detail & Related papers  (2024-10-17T15:09:35Z)
• Pointer-Guided Pre-Training: Infusing Large Language Models with Paragraph-Level Contextual Awareness [3.2925222641796554]
  "pointer-guided segment ordering" (SO) is a novel pre-training technique aimed at enhancing the contextual understanding of paragraph-level text representations. Our experiments show that pointer-guided pre-training significantly enhances the model's ability to understand complex document structures.
  arXiv  Detail & Related papers  (2024-06-06T15:17:51Z)
• One-Shot Learning as Instruction Data Prospector for Large Language Models [108.81681547472138]
  textscNuggets uses one-shot learning to select high-quality instruction data from extensive datasets. We show that instruction tuning with the top 1% of examples curated by textscNuggets substantially outperforms conventional methods employing the entire dataset.
  arXiv  Detail & Related papers  (2023-12-16T03:33:12Z)
• Faithful Low-Resource Data-to-Text Generation through Cycle Training [14.375070014155817]
  Methods to generate text from structured data have advanced significantly in recent years. Cycle training uses two models which are inverses of each other. We show that cycle training achieves nearly the same performance as fully supervised approaches.
  arXiv  Detail & Related papers  (2023-05-24T06:44:42Z)
• Towards Table-to-Text Generation with Pretrained Language Model: A Table Structure Understanding and Text Deliberating Approach [60.03002572791552]
  We propose a table structure understanding and text deliberating approach, namely TASD. Specifically, we devise a three-layered multi-head attention network to realize the table-structure-aware text generation model. Our approach can generate faithful and fluent descriptive texts for different types of tables.
  arXiv  Detail & Related papers  (2023-01-05T14:03:26Z)
• Schema-aware Reference as Prompt Improves Data-Efficient Knowledge Graph Construction [57.854498238624366]
  We propose a retrieval-augmented approach, which retrieves schema-aware Reference As Prompt (RAP) for data-efficient knowledge graph construction. RAP can dynamically leverage schema and knowledge inherited from human-annotated and weak-supervised data as a prompt for each sample.
  arXiv  Detail & Related papers  (2022-10-19T16:40:28Z)
• Curriculum-Based Self-Training Makes Better Few-Shot Learners for Data-to-Text Generation [56.98033565736974]
  We propose Curriculum-Based Self-Training (CBST) to leverage unlabeled data in a rearranged order determined by the difficulty of text generation. Our method can outperform fine-tuning and task-adaptive pre-training methods, and achieve state-of-the-art performance in the few-shot setting of data-to-text generation.
  arXiv  Detail & Related papers  (2022-06-06T16:11:58Z)
• Data-to-text Generation with Variational Sequential Planning [74.3955521225497]
  We consider the task of data-to-text generation, which aims to create textual output from non-linguistic input. We propose a neural model enhanced with a planning component responsible for organizing high-level information in a coherent and meaningful way. We infer latent plans sequentially with a structured variational model, while interleaving the steps of planning and generation.
  arXiv  Detail & Related papers  (2022-02-28T13:17:59Z)
• A Span Extraction Approach for Information Extraction on Visually-Rich Documents [2.3131309703965135]
  We present a new approach to improve the capability of language model pre-training on visually-rich documents (VRDs) Firstly, we introduce a new IE model that is query-based and employs the span extraction formulation instead of the commonly used sequence labelling approach. We also propose a new training task which focuses on modelling the relationships between semantic entities within a document.
  arXiv  Detail & Related papers  (2021-06-02T06:50:04Z)
• Iterative Data Programming for Expanding Text Classification Corpora [9.152045698511506]
  Real-world text classification tasks often require many labeled training examples that are expensive to obtain. Recent advancements in machine teaching, specifically the data programming paradigm, facilitate the creation of training data sets quickly. We present a fast, simple data programming method for augmenting text data sets by generating neighborhood-based weak models.
  arXiv  Detail & Related papers  (2020-02-04T17:12:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.