NL2TL: Transforming Natural Languages to Temporal Logics using Large Language Models

• URL: http://arxiv.org/abs/2305.07766v2
• Date: Fri, 22 Mar 2024 00:28:51 GMT
• Title: NL2TL: Transforming Natural Languages to Temporal Logics using Large Language Models
• Authors: Yongchao Chen, Rujul Gandhi, Yang Zhang, Chuchu Fan,
• Abstract summary: Temporal Logic (TL) can be used to rigorously specify complex high-level specification for systems in many engineering applications. In this paper, we propose an accurate and generalizable transformation framework of English instructions from NL to TL.
• Score: 11.913662099158033
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Temporal Logic (TL) can be used to rigorously specify complex high-level specification for systems in many engineering applications. The translation between natural language (NL) and TL has been under-explored due to the lack of dataset and generalizable model across different application domains. In this paper, we propose an accurate and generalizable transformation framework of English instructions from NL to TL, exploring the use of Large Language Models (LLMs) at multiple stages. Our contributions are twofold. First, we develop a framework to create a dataset of NL-TL pairs combining LLMs and human annotation. We publish a dataset with 28K NL-TL pairs. Then, we finetune T5 models on the lifted versions (i.e., the specific Atomic Propositions (AP) are hidden) of the NL and TL. The enhanced generalizability originates from two aspects: 1) Usage of lifted NL-TL characterizes common logical structures, without constraints of specific domains. 2) Application of LLMs in dataset creation largely enhances corpus richness. We test the generalization of trained models on five varied domains. To achieve full NL-TL transformation, we either combine the lifted model with AP recognition task or do the further finetuning on each specific domain. During the further finetuning, our model achieves higher accuracy (>95%) using only <10% training data, compared with the baseline sequence to sequence (Seq2Seq) model.

Related papers

• Enhancing Transformation from Natural Language to Signal Temporal Logic Using LLMs with Diverse External Knowledge [23.50725254650578]
  We propose an NL-STL dataset named STL-Diversity-Enhanced (STL-DivEn), which comprises 16,000 samples enriched with diverse patterns.<n>To develop the dataset, we first manually create a small-scale seed set of NL-STL pairs.<n> representative examples are identified through clustering and used to guide large language models.<n>Finally, diversity and accuracy are ensured through rigorous rule-based filters and human validation.
  arXiv  Detail & Related papers  (2025-05-27T03:07:25Z)
• The Unreasonable Effectiveness of Model Merging for Cross-Lingual Transfer in LLMs [54.59207567677249]
  Large language models (LLMs) still struggle across tasks outside of high-resource languages.<n>In this work, we investigate cross-lingual transfer to lower-resource languages where task-specific post-training data is scarce.
  arXiv  Detail & Related papers  (2025-05-23T20:28:31Z)
• TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking [6.070192392563392]
  We present TituLLMs, the first large pretrained Bangla LLMs, available in 1b and 3b parameter sizes. To train TituLLMs, we collected a pretraining dataset of approximately 37 billion tokens. We extended the Llama-3.2 tokenizer to incorporate language- and culture-specific knowledge.
  arXiv  Detail & Related papers  (2025-02-16T16:22:23Z)
• On Domain-Specific Post-Training for Multimodal Large Language Models [72.67107077850939]
  This paper systematically investigates domain adaptation of MLLMs through post-training. We focus on data synthesis, training pipelines, and task evaluation. We conduct experiments in high-impact domains such as biomedicine, food, and remote sensing.
  arXiv  Detail & Related papers  (2024-11-29T18:42:28Z)
• SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning [70.21358720599821]
  Large language models (LLMs) hold the promise of solving diverse tasks when provided with appropriate natural language prompts. We propose SELF-GUIDE, a multi-stage mechanism in which we synthesize task-specific input-output pairs from the student LLM. We report an absolute improvement of approximately 15% for classification tasks and 18% for generation tasks in the benchmark's metrics.
  arXiv  Detail & Related papers  (2024-07-16T04:41:58Z)
• VANER: Leveraging Large Language Model for Versatile and Adaptive Biomedical Named Entity Recognition [3.4923338594757674]
  Large language models (LLMs) can be used to train a model capable of extracting various types of entities. In this paper, we utilize the open-sourced LLM LLaMA2 as the backbone model, and design specific instructions to distinguish between different types of entities and datasets. Our model VANER, trained with a small partition of parameters, significantly outperforms previous LLMs-based models and, for the first time, as a model based on LLM, surpasses the majority of conventional state-of-the-art BioNER systems.
  arXiv  Detail & Related papers  (2024-04-27T09:00:39Z)
• GeMQuAD : Generating Multilingual Question Answering Datasets from Large Language Models using Few Shot Learning [4.8838210812204235]
  In this paper, we propose GeMQuAD - a semi-supervised learning approach, applied to a dataset generated through ICL with just one example in the target language. We iteratively identify high-quality data to enhance model performance, especially for low-resource multilingual setting. Our framework outperforms the machine translation-augmented model by 0.22/1.68 F1/EM points for Hindi and 0.82/1.37 F1/EM points for Spanish on the MLQA dataset.
  arXiv  Detail & Related papers  (2024-04-14T06:55:42Z)
• TAT-LLM: A Specialized Language Model for Discrete Reasoning over Tabular and Textual Data [73.29220562541204]
  We consider harnessing the amazing power of language models (LLMs) to solve our task. We develop a TAT-LLM language model by fine-tuning LLaMA 2 with the training data generated automatically from existing expert-annotated datasets.
  arXiv  Detail & Related papers  (2024-01-24T04:28:50Z)
• LLM Augmented LLMs: Expanding Capabilities through Composition [56.40953749310957]
  CALM -- Composition to Augment Language Models -- introduces cross-attention between models to compose their representations and enable new capabilities. We illustrate that augmenting PaLM2-S with a smaller model trained on low-resource languages results in an absolute improvement of up to 13% on tasks like translation into English. When PaLM2-S is augmented with a code-specific model, we see a relative improvement of 40% over the base model for code generation and explanation tasks.
  arXiv  Detail & Related papers  (2024-01-04T18:53:01Z)
• Data Transformation to Construct a Dataset for Generating Entity-Relationship Model from Natural Language [39.53954130028595]
  In order to reduce the manual cost of ER models, recent approaches have been proposed to address the task of NL2ERM. These approaches are typically rule-based ones, which rely on rigid rules. Despite having better generalization than rule-based approaches, deep-based models are lacking for NL2ERM due to lacking a large-scale dataset.
  arXiv  Detail & Related papers  (2023-12-21T09:45:13Z)
• Prompt2Model: Generating Deployable Models from Natural Language Instructions [74.19816829003729]
  Large language models (LLMs) enable system builders to create competent NLP systems through prompting. In other ways, LLMs are a step backward from traditional special-purpose NLP models. We propose Prompt2Model, a general-purpose method that takes a natural language task description like the prompts provided to LLMs.
  arXiv  Detail & Related papers  (2023-08-23T17:28:21Z)
• Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models [106.65127123304842]
  Branch-Train-Merge (BTM) is an efficient algorithm for parallel training of large language models (LLMs) BTM learns a set of independent expert LMs (ELMs), each specialized to a different textual domain. Experiments show that BTM improves in- and out-of-domain perplexities as compared to GPT-style Transformer LMs.
  arXiv  Detail & Related papers  (2022-08-05T17:46:38Z)
• A Variational Hierarchical Model for Neural Cross-Lingual Summarization [85.44969140204026]
  Cross-lingual summarization () is to convert a document in one language to a summary in another one. Existing studies on CLS mainly focus on utilizing pipeline methods or jointly training an end-to-end model. We propose a hierarchical model for the CLS task, based on the conditional variational auto-encoder.
  arXiv  Detail & Related papers  (2022-03-08T02:46:11Z)
• CoreLM: Coreference-aware Language Model Fine-Tuning [0.0]
  We propose a Fine-Tuning framework, named CoreLM, that extends the architecture of current Pretrained Language Models. We make available information outside the contextual space of the model, which results in a better Language Model for a fraction of the computational cost. Our proposed model achieves a lower Perplexity in GUMBY and LAMBDADA datasets when compared to GPT2 and a fine-tuned version of GPT2 without any changes.
  arXiv  Detail & Related papers  (2021-11-04T08:44:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.