Class Model Generation from Requirements using Large Language Models

• URL: http://arxiv.org/abs/2603.09100v1
• Date: Tue, 10 Mar 2026 02:20:35 GMT
• Title: Class Model Generation from Requirements using Large Language Models
• Abstract summary: Large Language Models (LLMs) can generate class diagrams from natural language requirements automatically.<n>This paper investigates the capabilities of state-of-the-art LLMs, including GPT-5, Claude Sonnet 4.0, Gemini 2.5 Flash Thinking, and Llama-3.1-8-B-Instruct.
• Score: 5.685497917524985
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The emergence of Large Language Models (LLMs) has opened new opportunities to automate software engineering activities that traditionally require substantial manual effort. Among these, class diagram generation represents a critical yet resource-intensive phase in software design. This paper investigates the capabilities of state-of-the-art LLMs, including GPT-5, Claude Sonnet 4.0, Gemini 2.5 Flash Thinking, and Llama-3.1-8B-Instruct, to generate UML class diagrams from natural language requirements automatically. To evaluate the effectiveness and reliability of LLM-based model generation, we propose a comprehensive dual-validation framework that integrates an LLM-as-a-Judge methodology with human-in-the-loop assessment. Using eight heterogeneous datasets, we apply chain-of-thought prompting to extract domain entities, attributes, and associations, generating corresponding PlantUML representations. The resulting models are evaluated across five quality dimensions: completeness, correctness, conformance to standards, comprehensibility, and terminological alignment. Two independent LLM judges (Grok and Mistral) perform structured pairwise comparisons, and their judgments are further validated against expert evaluations. Our results demonstrate that LLMs can generate structurally coherent and semantically meaningful UML diagrams, achieving substantial alignment with human evaluators. The consistency observed between LLM-based and human-based assessments highlights the potential of LLMs not only as modeling assistants but also as reliable evaluators in automated requirements engineering workflows, offering practical insights into the capabilities and limitations of LLM-driven UML class diagram automation.

Related papers

• Automatic Translation of Unstructured Requirements into Linear Temporal Logic through Large Language Models [46.28810845443601]
  This paper evaluates whether contemporary off-the-shelf Large Language Models (LLMs) can help bridge this gap by generating Linear Temporal Logic (LTL) formulas directly from unstructured requirements.<n>The study examines six modern LLMs using a few-shot prompting strategy on a heterogeneous benchmark of 15 structurally varied requirements.<n>The results indicate that current general-purpose LLMs can achieve practically significant performance on the unstructured NLto-LTL task without task-specific fine-tuning.
  arXiv  Detail & Related papers  (2026-08-06T17:13:07Z)
• Beyond Grading Accuracy: Exploring Alignment of TAs and LLMs [1.529342790344802]
  We investigate the potential of open-source Large Language Models (LLMs) for grading Unified Language (UML) class diagrams.<n>Our results show per-criterion accuracy of up to 88.56% and a Pearson correlation coefficient of up to 0.78, representing a substantial improvement over previous work.
  arXiv  Detail & Related papers  (2026-03-17T10:40:35Z)
• Learning to Judge: LLMs Designing and Applying Evaluation Rubrics [18.936553687978087]
  Large language models (LLMs) are increasingly used as evaluators for natural language generation.<n>We introduce GER-Eval to investigate whether LLMs can design and apply their own evaluation rubrics.
  arXiv  Detail & Related papers  (2026-02-09T13:56:06Z)
• Assessing the Business Process Modeling Competences of Large Language Models [40.495149980011924]
  Large language models (LLMs) have significantly expanded the possibilities for generating Business Process Model and Notation (BPMN) models directly from natural language.<n>We introduce BEF4LLM, a novel evaluation framework comprising four perspectives: syntactic quality, pragmatic quality, semantic quality, and validity.<n>Using BEF4LLM, we conduct a comprehensive analysis of open-source LLMs and benchmark their performance against human modeling experts.
  arXiv  Detail & Related papers  (2026-01-29T14:34:20Z)
• NOMAD: A Multi-Agent LLM System for UML Class Diagram Generation from Natural Language Requirements [20.080985332719383]
  Large Language Models (LLMs) are increasingly utilised in software engineering, yet their ability to generate structured artefacts such as diagrams remains underexplored.<n>In this work we present NOMAD, a cognitively inspired, modular multi-agent framework that decomposes generation into a series of role-specialised subtasks.<n>Each agent handles a distinct modelling activity, such as entity extraction, relationship classification, synthesis diagram, mirroring the goal-directed reasoning processes of an engineer.
  arXiv  Detail & Related papers  (2025-11-27T12:36:25Z)
• LM4Opt-RA: A Multi-Candidate LLM Framework with Structured Ranking for Automating Network Resource Allocation [0.7933039558471408]
  We tackle complex analytical and mathematical reasoning tasks requiring nuanced contextual understanding.<n>Existing benchmarks datasets cannot address the complexities of such problems with dynamic environments, variables, and heterogeneous constraints.<n>We introduce NL4RA, a curated dataset comprising 50 resource allocation optimization problems formulated as LP, ILP, and MILP.<n>We then evaluate the performance of well-known open-source LLMs with varying parameter counts.
  arXiv  Detail & Related papers  (2025-11-13T23:19:43Z)
• Evaluating Language Models for Generating and Judging Programming Feedback [4.743413681603463]
  Large language models (LLMs) have transformed research and practice across a wide range of domains. We evaluate the efficiency of open-source LLMs in generating high-quality feedback for programming assignments.
  arXiv  Detail & Related papers  (2024-07-05T21:44:11Z)
• Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimension [63.330262740414646]
  We study how to characterize and predict the truthfulness of texts generated from large language models (LLMs) We suggest investigating internal activations and quantifying LLM's truthfulness using the local intrinsic dimension (LID) of model activations.
  arXiv  Detail & Related papers  (2024-02-28T04:56:21Z)
• Knowledge Fusion of Large Language Models [73.28202188100646]
  This paper introduces the notion of knowledge fusion for large language models (LLMs) We externalize their collective knowledge and unique strengths, thereby elevating the capabilities of the target model beyond those of any individual source LLM. Our findings confirm that the fusion of LLMs can improve the performance of the target model across a range of capabilities such as reasoning, commonsense, and code generation.
  arXiv  Detail & Related papers  (2024-01-19T05:02:46Z)
• CLOMO: Counterfactual Logical Modification with Large Language Models [109.60793869938534]
  We introduce a novel task, Counterfactual Logical Modification (CLOMO), and a high-quality human-annotated benchmark. In this task, LLMs must adeptly alter a given argumentative text to uphold a predetermined logical relationship. We propose an innovative evaluation metric, the Self-Evaluation Score (SES), to directly evaluate the natural language output of LLMs.
  arXiv  Detail & Related papers  (2023-11-29T08:29:54Z)
• Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization [132.25202059478065]
  We benchmark large language models (LLMs) on instruction controllable text summarization. Our study reveals that instruction controllable text summarization remains a challenging task for LLMs.
  arXiv  Detail & Related papers  (2023-11-15T18:25:26Z)
• Improving Open Information Extraction with Large Language Models: A Study on Demonstration Uncertainty [52.72790059506241]
  Open Information Extraction (OIE) task aims at extracting structured facts from unstructured text. Despite the potential of large language models (LLMs) like ChatGPT as a general task solver, they lag behind state-of-the-art (supervised) methods in OIE tasks.
  arXiv  Detail & Related papers  (2023-09-07T01:35:24Z)
• On Learning to Summarize with Large Language Models as References [101.79795027550959]
  Large language models (LLMs) are favored by human annotators over the original reference summaries in commonly used summarization datasets. We study an LLM-as-reference learning setting for smaller text summarization models to investigate whether their performance can be substantially improved.
  arXiv  Detail & Related papers  (2023-05-23T16:56:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.