A Knowledge-Informed Large Language Model Framework for U.S. Nuclear Power Plant Shutdown Initiating Event Classification for Probabilistic Risk Assessment

• URL: http://arxiv.org/abs/2410.00929v1
• Date: Mon, 30 Sep 2024 20:35:03 GMT
• Title: A Knowledge-Informed Large Language Model Framework for U.S. Nuclear Power Plant Shutdown Initiating Event Classification for Probabilistic Risk Assessment
• Authors: Min Xian, Tao Wang, Sai Zhang, Fei Xu, Zhegang Ma,
• Abstract summary: We propose a hybrid pipeline that integrates a knowledge-informed machine learning mode to prescreen non-SDIEs and a large language model (LLM) to classify SDIEs into four types. The proposed approaches are evaluated on a dataset with 10,928 events using precision, recall ratio, F1 score, and average accuracy.
• Score: 6.200373536860575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Identifying and classifying shutdown initiating events (SDIEs) is critical for developing low power shutdown probabilistic risk assessment for nuclear power plants. Existing computational approaches cannot achieve satisfactory performance due to the challenges of unavailable large, labeled datasets, imbalanced event types, and label noise. To address these challenges, we propose a hybrid pipeline that integrates a knowledge-informed machine learning mode to prescreen non-SDIEs and a large language model (LLM) to classify SDIEs into four types. In the prescreening stage, we proposed a set of 44 SDIE text patterns that consist of the most salient keywords and phrases from six SDIE types. Text vectorization based on the SDIE patterns generates feature vectors that are highly separable by using a simple binary classifier. The second stage builds Bidirectional Encoder Representations from Transformers (BERT)-based LLM, which learns generic English language representations from self-supervised pretraining on a large dataset and adapts to SDIE classification by fine-tuning it on an SDIE dataset. The proposed approaches are evaluated on a dataset with 10,928 events using precision, recall ratio, F1 score, and average accuracy. The results demonstrate that the prescreening stage can exclude more than 97% non-SDIEs, and the LLM achieves an average accuracy of 93.4% for SDIE classification.

Related papers

• An LLM-driven Scenario Generation Pipeline Using an Extended Scenic DSL for Autonomous Driving Safety Validation [4.602386383455713]
  Real-world crash reports are valuable for scenario-based testing of autonomous driving systems.<n>Current methods cannot effectively translate this multimodal data into precise, executable simulation scenarios.<n>We propose a scalable and verifiable pipeline that uses a large language model and a probabilistic intermediate representation.
  arXiv  Detail & Related papers  (2026-02-24T07:44:26Z)
• ESLM: Risk-Averse Selective Language Modeling for Efficient Pretraining [53.893792844055106]
  Large language model pretraining is compute-intensive, yet many tokens contribute marginally to learning, resulting in inefficiency.<n>We introduce Selective Efficient Language Modeling, a risk-aware algorithm that improves training efficiency and distributional robustness by performing online token-level batch selection.<n> Experiments on GPT-2 pretraining show that ESLM significantly reduces training FLOPs while maintaining or improving both perplexity and downstream performance compared to baselines.
  arXiv  Detail & Related papers  (2025-05-26T12:23:26Z)
• Random-Set Large Language Models [4.308457163593758]
  Large Language Models (LLMs) are known to produce very high-quality tests and responses to our queries. But how much can we trust this generated text? We propose a novel Random-Set Large Language Model (RSLLM) approach which predicts finite random sets (belief functions) over the token space.
  arXiv  Detail & Related papers  (2025-04-25T05:25:27Z)
• Detecting and Filtering Unsafe Training Data via Data Attribution with Denoised Representation [8.963777475007669]
  Large language models (LLMs) are highly sensitive to even small amounts of unsafe training data.<n>We propose Denoised Representation (DRA), a novel representation-based data attribution approach.
  arXiv  Detail & Related papers  (2025-02-17T03:50:58Z)
• Beyond Exact Match: Semantically Reassessing Event Extraction by Large Language Models [69.38024658668887]
  Current evaluation method for event extraction relies on token-level exact match. We propose RAEE, an automatic evaluation framework that accurately assesses event extraction results at semantic-level instead of token-level.
  arXiv  Detail & Related papers  (2024-10-12T07:54:01Z)
• Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Method [108.56493934296687]
  We introduce a divergence-based calibration method, inspired by the divergence-from-randomness concept, to calibrate token probabilities for pretraining data detection. We have developed a Chinese-language benchmark, PatentMIA, to assess the performance of detection approaches for LLMs on Chinese text.
  arXiv  Detail & Related papers  (2024-09-23T07:55:35Z)
• OV-DINO: Unified Open-Vocabulary Detection with Language-Aware Selective Fusion [88.59397418187226]
  We propose a novel unified open-vocabulary detection method called OV-DINO. It is pre-trained on diverse large-scale datasets with language-aware selective fusion in a unified framework. We evaluate the performance of the proposed OV-DINO on popular open-vocabulary detection benchmarks.
  arXiv  Detail & Related papers  (2024-07-10T17:05:49Z)
• Improved Out-of-Scope Intent Classification with Dual Encoding and Threshold-based Re-Classification [6.975902383951604]
  Current methodologies face difficulties with the unpredictable distribution of outliers. We present the Dual for Threshold-Based Re-Classification (DETER) to address these challenges. Our model outperforms previous benchmarks, increasing up to 13% and 5% in F1 score for known and unknown intents.
  arXiv  Detail & Related papers  (2024-05-30T11:46:42Z)
• Feature Fusion for Improved Classification: Combining Dempster-Shafer Theory and Multiple CNN Architectures [1.0678175996321808]
  This paper introduces a novel algorithm leveraging Dempster-Shafer Theory (DST) to integrate multiple pre-trained models to form an ensemble capable of providing more reliable and enhanced classifications. Several experiments have been conducted on CIFAR-10 and CIFAR-100 datasets, demonstrating superior classification accuracy of the proposed DST-based method.
  arXiv  Detail & Related papers  (2024-05-23T20:44:10Z)
• TeLeS: Temporal Lexeme Similarity Score to Estimate Confidence in End-to-End ASR [1.8477401359673709]
  Class-probability-based confidence scores do not accurately represent quality of overconfident ASR predictions. We propose a novel Temporal-Lexeme Similarity (TeLeS) confidence score to train Confidence Estimation Model (CEM) We conduct experiments with ASR models trained in three languages, namely Hindi, Tamil, and Kannada, with varying training data sizes.
  arXiv  Detail & Related papers  (2024-01-06T16:29:13Z)
• A Meta-Learning Approach to Predicting Performance and Data Requirements [163.4412093478316]
  We propose an approach to estimate the number of samples required for a model to reach a target performance. We find that the power law, the de facto principle to estimate model performance, leads to large error when using a small dataset. We introduce a novel piecewise power law (PPL) that handles the two data differently.
  arXiv  Detail & Related papers  (2023-03-02T21:48:22Z)
• ASR in German: A Detailed Error Analysis [0.0]
  This work presents a selection of ASR model architectures that are pretrained on the German language and evaluates them on a benchmark of diverse test datasets. It identifies cross-architectural prediction errors, classifies those into categories and traces the sources of errors per category back into training data.
  arXiv  Detail & Related papers  (2022-04-12T08:25:01Z)
• From Good to Best: Two-Stage Training for Cross-lingual Machine Reading Comprehension [51.953428342923885]
  We develop a two-stage approach to enhance the model performance. The first stage targets at recall: we design a hard-learning (HL) algorithm to maximize the likelihood that the top-k predictions contain the accurate answer. The second stage focuses on precision: an answer-aware contrastive learning mechanism is developed to learn the fine difference between the accurate answer and other candidates.
  arXiv  Detail & Related papers  (2021-12-09T07:31:15Z)
• Robust Event Classification Using Imperfect Real-world PMU Data [58.26737360525643]
  We study robust event classification using imperfect real-world phasor measurement unit (PMU) data. We develop a novel machine learning framework for training robust event classifiers.
  arXiv  Detail & Related papers  (2021-10-19T17:41:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.