TextBox 2.0: A Text Generation Library with Pre-trained Language Models

• URL: http://arxiv.org/abs/2212.13005v1
• Date: Mon, 26 Dec 2022 03:50:36 GMT
• Title: TextBox 2.0: A Text Generation Library with Pre-trained Language Models
• Abstract summary: This paper presents a comprehensive and unified library, TextBox 2.0, focusing on the use of pre-trained language models (PLMs) To be comprehensive, our library covers $13$ common text generation tasks and their corresponding $83$ datasets. We also implement $4$ efficient training strategies and provide $4$ generation objectives for pre-training new PLMs from scratch.
• Score: 72.49946755856935
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To facilitate research on text generation, this paper presents a comprehensive and unified library, TextBox 2.0, focusing on the use of pre-trained language models (PLMs). To be comprehensive, our library covers $13$ common text generation tasks and their corresponding $83$ datasets and further incorporates $45$ PLMs covering general, translation, Chinese, dialogue, controllable, distilled, prompting, and lightweight PLMs. We also implement $4$ efficient training strategies and provide $4$ generation objectives for pre-training new PLMs from scratch. To be unified, we design the interfaces to support the entire research pipeline (from data loading to training and evaluation), ensuring that each step can be fulfilled in a unified way. Despite the rich functionality, it is easy to use our library, either through the friendly Python API or command line. To validate the effectiveness of our library, we conduct extensive experiments and exemplify four types of research scenarios. The project is released at the link: https://github.com/RUCAIBox/TextBox.

Related papers

• OpenLanguageModel: Readable and Composable Small-Language-Model Pretraining for Education and Research [1.1654568410166084]
  OpenLanguageModel (OLM) is an open-source PyTorch library for building and pretraining small language models.<n>The package includes 27 presets across nine familiar model families and documentation that progresses from LM fundamentals to architecture research.
  arXiv  Detail & Related papers  (2026-07-18T06:55:18Z)
• ORBIT: Scalable and Verifiable Data Generation for Search Agents on a Tight Budget [56.6320292944541]
  ORBIT is a training dataset with 20K reasoning-intensive queries with short verifiable answers.<n>Our framework, code and datasets are open-sourced and available publicly.
  arXiv  Detail & Related papers  (2026-04-01T17:42:41Z)
• Agnostics: Learning to Code in Any Programming Language via Reinforcement with a Universal Learning Environment [44.47221244109926]
  Agnostics is a language-agnostic post-training pipeline that eliminates per-language engineering.<n>Key idea is to judge code solely by its externally observable behavior, so a single verifier can test solutions written in any language.<n>We will release the language-agnostic training datasets (Ag-MBPP-X, Ag-Codeforces-X, Ag-LiveCodeBench-X), training code, and ready-to-use configurations, making RL post-training in any programming language as simple as editing a short YAML file.
  arXiv  Detail & Related papers  (2025-08-06T20:30:55Z)
• Zero-Shot Statistical Tests for LLM-Generated Text Detection using Finite Sample Concentration Inequalities [13.657259851747126]
  Verifying provenance of content is crucial to the function of many organizations, e.g., educational institutions, social media platforms, firms, etc.<n>This problem is becoming increasingly challenging as text generated by Large Language Models (LLMs) becomes almost indistinguishable from human-generated content.<n>In this paper, we answer the following question: Given a piece of text, can we identify whether it was produced by a particular LLM or not?<n>We model LLM-generated text as a sequential process with complete dependence on history. We then design zero-shot statistical tests to distinguish between text generated by two different known sets of LLM
  arXiv  Detail & Related papers  (2025-01-04T23:51:43Z)
• Large Language Models for Mobile GUI Text Input Generation: An Empirical Study [24.256184336154544]
  Large Language Models (LLMs) have demonstrated excellent text-generation capabilities. This paper extensively investigates the effectiveness of nine state-of-the-art LLMs in Android text-input generation for UI pages.
  arXiv  Detail & Related papers  (2024-04-13T09:56:50Z)
• How do Large Language Models Handle Multilingualism? [81.15060972112563]
  This study explores how large language models (LLMs) handle multilingualism. LLMs initially understand the query, converting multilingual inputs into English for task-solving. In the intermediate layers, they employ English for thinking and incorporate multilingual knowledge with self-attention and feed-forward structures.
  arXiv  Detail & Related papers  (2024-02-29T02:55:26Z)
• LAMPAT: Low-Rank Adaption for Multilingual Paraphrasing Using Adversarial Training [19.173992333194683]
  Paraphrases are texts that convey the same meaning while using different words or sentence structures. Previous studies have leveraged the knowledge from the machine translation field, forming a paraphrase through zero-shot machine translation in the same language. We propose the first unsupervised multilingual paraphrasing model, LAMPAT, by which monolingual dataset is sufficient enough to generate a human-like and diverse sentence.
  arXiv  Detail & Related papers  (2024-01-09T04:19:16Z)
• Learning to Prompt with Text Only Supervision for Vision-Language Models [107.282881515667]
  One branch of methods adapts CLIP by learning prompts using visual information. An alternative approach resorts to training-free methods by generating class descriptions from large language models. We propose to combine the strengths of both streams by learning prompts using only text data.
  arXiv  Detail & Related papers  (2024-01-04T18:59:49Z)
• A Simple yet Efficient Ensemble Approach for AI-generated Text Detection [0.5840089113969194]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in generating text that closely resembles human writing. It is essential to build automated approaches capable of distinguishing between artificially generated text and human-authored text. We propose a simple yet efficient solution by ensembling predictions from multiple constituent LLMs.
  arXiv  Detail & Related papers  (2023-11-06T13:11:02Z)
• Private-Library-Oriented Code Generation with Large Language Models [52.73999698194344]
  This paper focuses on utilizing large language models (LLMs) for code generation in private libraries. We propose a novel framework that emulates the process of programmers writing private code. We create four private library benchmarks, including TorchDataEval, TorchDataComplexEval, MonkeyEval, and BeatNumEval.
  arXiv  Detail & Related papers  (2023-07-28T07:43:13Z)
• Learning to Transfer Prompts for Text Generation [97.64625999380425]
  We propose a novel prompt-based method (PTG) for text generation in a transferable setting. First, PTG learns a set of source prompts for various source generation tasks and then transfers these prompts as target prompts to perform target generation tasks. In extensive experiments, PTG yields competitive or better results than fine-tuning methods.
  arXiv  Detail & Related papers  (2022-05-03T14:53:48Z)
• A Survey of Pretrained Language Models Based Text Generation [97.64625999380425]
  Text Generation aims to produce plausible and readable text in human language from input data. Deep learning has greatly advanced this field by neural generation models, especially the paradigm of pretrained language models (PLMs) Grounding text generation on PLMs is seen as a promising direction in both academia and industry.
  arXiv  Detail & Related papers  (2022-01-14T01:44:58Z)
• Zero-Shot Recommendation as Language Modeling [1.0312968200748118]
  We propose a framework for recommendation with off-the-shelf pretrained language models (LM) We construct a textual prompt to estimate the affinity between $u$ and $m$ with LM likelihood. We motivate our idea with a corpus analysis, evaluate several prompt structures, and we compare LM-based recommendation with standard matrix factorization trained on different data regimes.
  arXiv  Detail & Related papers  (2021-12-08T09:16:03Z)
• GrASP: A Library for Extracting and Exploring Human-Interpretable Textual Patterns [25.350957495556226]
  We provide a Python library for GrASP, an algorithm for drawing patterns from textual data. The library is equipped with a web-based interface empowering human users to conveniently explore the data and the extracted patterns.
  arXiv  Detail & Related papers  (2021-04-08T17:58:03Z)
• A Multi-Perspective Architecture for Semantic Code Search [58.73778219645548]
  We propose a novel multi-perspective cross-lingual neural framework for code--text matching. Our experiments on the CoNaLa dataset show that our proposed model yields better performance than previous approaches.
  arXiv  Detail & Related papers  (2020-05-06T04:46:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.