LLM-based Automated Architecture View Generation: Where Are We Now?

• URL: http://arxiv.org/abs/2603.21178v1
• Date: Sun, 22 Mar 2026 11:41:37 GMT
• Title: LLM-based Automated Architecture View Generation: Where Are We Now?
• Abstract summary: We evaluate the ability of LLMs and agentic approaches to generate architecture views from source code.<n>Few-shot prompting reduces clarity failures by 9.2% compared to zero-shot baselines.<n>Custom agentic approach consistently outperforms the general-purpose agent.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Architecture views are essential for software architecture documentation, yet their manual creation is labor intensive and often leads to outdated artifacts. As systems grow in complexity, the automated generation of views from source code becomes increasingly valuable. Goal: We empirically evaluate the ability of LLMs and agentic approaches to generate architecture views from source code. Method: We analyze 340 open-source repositories across 13 experimental configurations using 3 LLMs with 3 prompting techniques and 2 agentic approaches, yielding 4,137 generated views. We evaluate the generated views by comparing them with the ground-truth using a combination of automated metrics complemented by human evaluations. Results: Prompting strategies offer marginal improvements. Few-shot prompting reduces clarity failures by 9.2% compared to zero-shot baselines. The custom agentic approach consistently outperforms the general-purpose agent, achieving the best clarity (22.6% failure rate) and level-of-detail success (50%). Conclusions: LLM and agentic approaches demonstrate capabilities in generating syntactically valid architecture views. However, they consistently exhibit granularity mismatches, operating at the code level rather than architectural abstractions. This suggests that there is still a need for human expertise, positioning LLMs and agents as assistive tools rather than autonomous architects.

Related papers

• Recovering Software Architecture Intent from Historical Work Items using Generative AI: A Mixed-Methods Industry Case Study [2.5501791028999583]
  This work investigates the recovery of C4 architecture diagrams from historical agile work items using an LLM-based pipeline.<n>The semi-automatic five-step workflow employs a prompt chain, bidirectional traceability, and Chain-of-Thought reasoning.<n>The proposed workflow demonstrates the practical viability of LLM-assisted architectural recovery based on development process artifacts.
  arXiv  Detail & Related papers  (2026-08-28T14:56:00Z)
• Semantic-Enhanced Automatic Refinement of Architecture Recovery Results Using LLMs [15.87039675625043]
  Semref is a framework that combines LLMs with dependency analysis to automatically refine architectures recovered by existing architecture recovery tools.<n>We show that Semref improves accuracy across various metrics, with normalized gains ranges from 17.72% to 43.35%.
  arXiv  Detail & Related papers  (2026-07-26T17:51:23Z)
• SAKE: Software Architectural Knowledge Evaluation Benchmark for Large Language Models [0.0]
  Large Language Models (LLMs) are increasingly used as assistants across the software development lifecycle.<n> Architectural decision-making depends on quality attribute trade-offs, design patterns, and system-level constraints.<n>We introduce SAKE, a standardized and reproducible benchmark for assessing software architectural knowledge.
  arXiv  Detail & Related papers  (2026-06-28T17:31:14Z)
• Code2UML: Agentic LLMs with context engineering for scalable software visualization [7.069738194755401]
  This paper introduces an agentic architecture with context engineering for automated diagram generation from source code repositories.<n>It employs a hierarchy of five specialized agents: PlannerAgent, Analyzer DiagramAgent, CorrectorAgent and DependencyAnalyzerAgent, built on the Claude Agent SDK.<n>A deterministic, importance-weighted IR compaction layer transforms full project IRs into diagram-specific views guaranteed to fit within token constraints.
  arXiv  Detail & Related papers  (2026-05-23T08:01:26Z)
• LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering [90.84806758077536]
  We introduce textbfLoCoBench-Agent, a comprehensive evaluation framework specifically designed to assess large language models (LLMs) agents in realistic, long-context software engineering.<n>Our framework extends LoCoBench's 8,000 scenarios into interactive agent environments, enabling systematic evaluation of multi-turn conversations.<n>Our framework provides agents with 8 specialized tools (file operations, search, code analysis) and evaluates them across context lengths ranging from 10K to 1M tokens.
  arXiv  Detail & Related papers  (2025-11-17T23:57:24Z)
• Beyond Synthetic Benchmarks: Evaluating LLM Performance on Real-World Class-Level Code Generation [3.9189409002585567]
  Large language models (LLMs) have demonstrated strong performance on function-level code generation benchmarks.<n>We introduce a benchmark derived from real-world open-source repositories to evaluate generalization under practical conditions.<n>We examine how input specification completeness and retrieval-augmented generation affect class-level correctness across multiple state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2025-10-30T04:30:23Z)
• ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation [51.297873393639456]
  ArtifactsBench is a framework for automated visual code generation evaluation.<n>Our framework renders each generated artifact and captures its dynamic behavior through temporal screenshots.<n>We construct a new benchmark of 1,825 diverse tasks and evaluate over 30 leading Large Language Models.
  arXiv  Detail & Related papers  (2025-07-07T12:53:00Z)
• Evaluating the Use of LLMs for Documentation to Code Traceability [3.076436880934678]
  Large Language Models can establish trace links between various software documentation and source code.<n>We create two novel datasets from two open-source projects (Unity Catalog and Crawl4AI)<n>Results show that the best-performing LLM achieves F1-scores of 79.4% and 80.4% across the two datasets.
  arXiv  Detail & Related papers  (2025-06-19T16:18:53Z)
• AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Search [58.98450205734779]
  Large language model (LLM) agents have demonstrated strong capabilities across diverse domains.<n>Existing agent search methods suffer from three major limitations.<n>We introduce a comprehensive framework to address these challenges.
  arXiv  Detail & Related papers  (2025-06-06T12:07:23Z)
• AIRepr: An Analyst-Inspector Framework for Evaluating Reproducibility of LLMs in Data Science [8.281093505963158]
  Large language models (LLMs) are increasingly used to automate data analysis through executable code generation.<n>We present AIRepr, an Analyst-Inspector framework for automatically evaluating and improving the of LLM-generated data analysis.
  arXiv  Detail & Related papers  (2025-02-23T01:15:50Z)
• AvaTaR: Optimizing LLM Agents for Tool Usage via Contrastive Reasoning [93.96463520716759]
  Large language model (LLM) agents have demonstrated impressive capabilities in utilizing external tools and knowledge to boost accuracy and hallucinations. Here, we introduce AvaTaR, a novel and automated framework that optimize an LLM agent to effectively leverage provided tools, improving performance on a given task.
  arXiv  Detail & Related papers  (2024-06-17T04:20:02Z)
• Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub [79.31134731122462]
  We introduce OpenAct benchmark to evaluate the open-domain task-solving capability, built on human expert consultation and repositories in GitHub.<n>We present OpenAgent, a novel LLM-based agent system that can tackle evolving queries in open domains through autonomously integrating specialized tools from GitHub.
  arXiv  Detail & Related papers  (2023-12-28T15:47:30Z)
• Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes [54.13559879916708]
  EVAPORATE is a prototype system powered by large language models (LLMs)<n>Code synthesis is cheap, but far less accurate than directly processing each document with the LLM.<n>We propose an extended code implementation, EVAPORATE-CODE+, which achieves better quality than direct extraction.
  arXiv  Detail & Related papers  (2023-04-19T06:00:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.