Learning from flowsheets: A generative transformer model for autocompletion of flowsheets

• URL: http://arxiv.org/abs/2208.00859v1
• Date: Mon, 1 Aug 2022 13:43:58 GMT
• Title: Learning from flowsheets: A generative transformer model for autocompletion of flowsheets
• Authors: Gabriel Vogel and Lukas Schulze Balhorn and Artur M. Schweidtmann
• Abstract summary: We represent flowsheets as strings using the text-based SFILES 2.0 notation. We learn the grammatical structure of the SFILES 2.0 language and common patterns in flowsheets using a transformer-based language model.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel method enabling autocompletion of chemical flowsheets. This idea is inspired by the autocompletion of text. We represent flowsheets as strings using the text-based SFILES 2.0 notation and learn the grammatical structure of the SFILES 2.0 language and common patterns in flowsheets using a transformer-based language model. We pre-train our model on synthetically generated flowsheets to learn the flowsheet language grammar. Then, we fine-tune our model in a transfer learning step on real flowsheet topologies. Finally, we use the trained model for causal language modeling to autocomplete flowsheets. Eventually, the proposed method can provide chemical engineers with recommendations during interactive flowsheet synthesis. The results demonstrate a high potential of this approach for future AI-assisted process synthesis.

Related papers

• In-Context Language Learning: Architectures and Algorithms [73.93205821154605]
  We study ICL through the lens of a new family of model problems we term in context language learning (ICLL) We evaluate a diverse set of neural sequence models on regular ICLL tasks.
  arXiv  Detail & Related papers  (2024-01-23T18:59:21Z)
• Toward autocorrection of chemical process flowsheets using large language models [0.0]
  We propose a novel generative AI methodology for identifying errors in flowsheets and suggesting corrections to the user. The input to the model is a potentially erroneous flowsheet and the output of the model are suggestions for a corrected flowsheet. The model achieves a top-1 accuracy of 80% and a top-5 accuracy of 84% on an independent test dataset of synthetically generated flowsheets.
  arXiv  Detail & Related papers  (2023-12-05T16:39:41Z)
• On Conditional and Compositional Language Model Differentiable Prompting [75.76546041094436]
  Prompts have been shown to be an effective method to adapt a frozen Pretrained Language Model (PLM) to perform well on downstream tasks. We propose a new model, Prompt Production System (PRopS), which learns to transform task instructions or input metadata, into continuous prompts.
  arXiv  Detail & Related papers  (2023-07-04T02:47:42Z)
• Data augmentation for machine learning of chemical process flowsheets [0.0]
  We show that proposed data augmentation improves the performance of artificial intelligence-based process design models. In our case study flowsheet data augmentation improved the prediction uncertainty of the flowsheet autocompletion model by 14.7%.
  arXiv  Detail & Related papers  (2023-02-07T10:35:24Z)
• FusionRetro: Molecule Representation Fusion via In-Context Learning for Retrosynthetic Planning [58.47265392465442]
  Retrosynthetic planning aims to devise a complete multi-step synthetic route from starting materials to a target molecule. Current strategies use a decoupled approach of single-step retrosynthesis models and search algorithms. We propose a novel framework that utilizes context information for improved retrosynthetic planning.
  arXiv  Detail & Related papers  (2022-09-30T08:44:58Z)
• AutoFlow: Learning a Better Training Set for Optical Flow [62.40293188964933]
  AutoFlow is a method to render training data for optical flow. AutoFlow achieves state-of-the-art accuracy in pre-training both PWC-Net and RAFT.
  arXiv  Detail & Related papers  (2021-04-29T17:55:23Z)
• Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Events [8.655294504286635]
  We propose to formulate this task as a question answering problem. This enables us to use pre-trained language models on other QA benchmarks by adapting those to the procedural text understanding. Our model evaluated on the Propara dataset shows improvements on the published state-of-the-art results with a $3.1%$ increase in F1 score.
  arXiv  Detail & Related papers  (2021-04-15T17:50:41Z)
• Automated Synthesis of Steady-State Continuous Processes using Reinforcement Learning [0.0]
  Reinforcement learning can be used for automated flowsheet synthesis without prior knowledge of conceptual design. Flowsheet synthesis is modelled as a game of two competing players. The method is applied successfully to a reaction-distillation process in a quaternary system.
  arXiv  Detail & Related papers  (2021-01-12T11:49:34Z)
• Syntax-Enhanced Pre-trained Model [49.1659635460369]
  We study the problem of leveraging the syntactic structure of text to enhance pre-trained models such as BERT and RoBERTa. Existing methods utilize syntax of text either in the pre-training stage or in the fine-tuning stage, so that they suffer from discrepancy between the two stages. We present a model that utilizes the syntax of text in both pre-training and fine-tuning stages.
  arXiv  Detail & Related papers  (2020-12-28T06:48:04Z)
• Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis [23.115879727598262]
  Flowtron is an autoregressive flow-based generative network for text-to-speech synthesis. We provide results on control of speech variation, between samples and style transfer between speakers seen and unseen during training.
  arXiv  Detail & Related papers  (2020-05-12T17:57:17Z)
• DiscreTalk: Text-to-Speech as a Machine Translation Problem [52.33785857500754]
  This paper proposes a new end-to-end text-to-speech (E2E-TTS) model based on neural machine translation (NMT) The proposed model consists of two components; a non-autoregressive vector quantized variational autoencoder (VQ-VAE) model and an autoregressive Transformer-NMT model.
  arXiv  Detail & Related papers  (2020-05-12T02:45:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.