CIAO - Code In Architecture Out - Automated Software Architecture Documentation with Large Language Models

• URL: http://arxiv.org/abs/2604.08293v1
• Date: Thu, 09 Apr 2026 14:29:17 GMT
• Title: CIAO - Code In Architecture Out - Automated Software Architecture Documentation with Large Language Models
• Abstract summary: This paper presents a structured process for automatically generating system-level architectural documentation directly from GitHub repositories using Large Language Models.<n>The process, called CIAO (Code In Architecture Out), defines an LLM-based workflow that takes a repository as input and produces system-level architectural documentation.<n>We evaluated the process through a study with 22 developers, each reviewing the documentation generated for a repository they had contributed to.
• Score: 7.539790545610935
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Software architecture documentation is essential for system comprehension, yet it is often unavailable or incomplete. While recent LLM-based techniques can generate documentation from code, they typically address local artifacts rather than producing coherent, system-level architectural descriptions. This paper presents a structured process for automatically generating system-level architectural documentation directly from GitHub repositories using Large Language Models. The process, called CIAO (Code In Architecture Out), defines an LLM-based workflow that takes a repository as input and produces system-level architectural documentation following a template derived from ISO/IEC/IEEE 42010, SEI Views \& Beyond, and the C4 model. The resulting documentation can be directly added to the target repository. We evaluated the process through a study with 22 developers, each reviewing the documentation generated for a repository they had contributed to. The evaluation shows that developers generally perceive the produced documentation as valuable, comprehensible, and broadly accurate with respect to the source code, while also highlighting limitations in diagram quality, high-level context modeling, and deployment views. We also assessed the operational cost of the process, finding that generating a complete architectural document requires only a few minutes and is inexpensive to run. Overall, the results indicate that a structured, standards-oriented approach can effectively guide LLMs in producing system-level architectural documentation that is both usable and cost-effective.

Related papers

• Recovering Software Architecture Intent from Historical Work Items using Generative AI: A Mixed-Methods Industry Case Study [2.5501791028999583]
  This work investigates the recovery of C4 architecture diagrams from historical agile work items using an LLM-based pipeline.<n>The semi-automatic five-step workflow employs a prompt chain, bidirectional traceability, and Chain-of-Thought reasoning.<n>The proposed workflow demonstrates the practical viability of LLM-assisted architectural recovery based on development process artifacts.
  arXiv  Detail & Related papers  (2026-08-28T14:56:00Z)
• RealBench: A Repo-Level Code Generation Benchmark Aligned with Real-World Software Development Practices [54.956760584923295]
  Researchers have made substantial progress using Large Language Models (LLMs) for code generation.<n>However, developers typically write code based on structured designs or specifications rather than raw natural language descriptions.<n>This gap between existing benchmarks and real industry development practices means that current benchmark scores may not accurately reflect how much code generation can help automate development tasks.
  arXiv  Detail & Related papers  (2026-04-24T15:35:54Z)
• Text2Arch: A Dataset for Generating Scientific Architecture Diagrams from Natural Language Descriptions [8.259454114005989]
  A system that automatically generates scientific architecture diagrams from text with high semantic fidelity can be useful in multiple applications.<n>No clean large-scale open-access dataset exists, implying lack of any effective open models for this task.<n>We make the code, data and models publicly available.
  arXiv  Detail & Related papers  (2026-04-16T12:36:22Z)
• Large Language Models for Analyzing Enterprise Architecture Debt in Unstructured Documentation [0.0]
  Enterprise Architecture Debt (EA Debt) arises from suboptimal design decisions and misaligned components that can degrade an organization's IT landscape over time.<n>Early indicators, Enterprise Architecture Smells (EA Smells), are currently mainly detected manually or only from structured artifacts.<n>This study proposes an approach using a large language model (LLM) to identify and quantify EA Debt in unstructured architectural documentation.
  arXiv  Detail & Related papers  (2026-03-29T21:45:15Z)
• LLM-based Automated Architecture View Generation: Where Are We Now? [0.0]
  We evaluate the ability of LLMs and agentic approaches to generate architecture views from source code.<n>Few-shot prompting reduces clarity failures by 9.2% compared to zero-shot baselines.<n>Custom agentic approach consistently outperforms the general-purpose agent.
  arXiv  Detail & Related papers  (2026-03-22T11:41:37Z)
• CodeWiki: Evaluating AI's Ability to Generate Holistic Documentation for Large-Scale Codebases [7.75137961900221]
  We present bftextCodeWiki, a unified framework for automated repository-level documentation across seven programming languages.<n>CodeWiki introduces three key innovations: (i) hierarchical decomposition that preserves architectural context across multiple levels of granularity, (ii) recursive multi-agent processing with dynamic task delegation for scalable generation, and (iii) multi-modal synthesis that integrates textual descriptions with visual artifacts such as architecture diagrams and data-flow representations.<n>CodeWiki achieves a 68.79% quality score with proprietary models, outperforming the closed-source DeepWiki baseline (64.06%) by 4.73%
  arXiv  Detail & Related papers  (2025-10-28T13:52:46Z)
• Collaborative LLM Agents for C4 Software Architecture Design Automation [0.0]
  This study contributes to automated software architecture design and its evaluation methods.<n>We introduce an LLM-based multi-agent system that automates the production of a C4 software architecture model.<n>Tested on five canonical system briefs, the workflow demonstrates fast C4 model creation, sustains high compilation success, and delivers semantic fidelity.
  arXiv  Detail & Related papers  (2025-10-26T18:43:59Z)
• Beyond Isolated Dots: Benchmarking Structured Table Construction as Deep Knowledge Extraction [80.88654868264645]
  Arranged and Organized Extraction Benchmark designed to evaluate ability of large language models to comprehend fragmented documents.<n>AOE includes 11 carefully crafted tasks across three diverse domains, requiring models to generate context-specific schema tailored to varied input queries.<n>Results show that even the most advanced models struggled significantly.
  arXiv  Detail & Related papers  (2025-07-22T06:37:51Z)
• Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning [70.04746094652653]
  We introduce PaperCoder, a framework that transforms machine learning papers into functional code repositories.<n>PaperCoder operates in three stages: planning, designs the system architecture with diagrams, identifies file dependencies, and generates configuration files.<n>We then evaluate PaperCoder on generating code implementations from machine learning papers based on both model-based and human evaluations.
  arXiv  Detail & Related papers  (2025-04-24T01:57:01Z)
• DOCBENCH: A Benchmark for Evaluating LLM-based Document Reading Systems [99.17123445211115]
  We introduce DocBench, a benchmark to evaluate large language model (LLM)-based document reading systems. Our benchmark involves the recruitment of human annotators and the generation of synthetic questions. It includes 229 real documents and 1,102 questions, spanning across five different domains and four major types of questions.
  arXiv  Detail & Related papers  (2024-07-15T13:17:42Z)
• RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation [79.83270415843857]
  We introduce RepoAgent, a large language model powered open-source framework aimed at proactively generating, maintaining, and updating code documentation. We have validated the effectiveness of our approach, showing that RepoAgent excels in generating high-quality repository-level documentation.
  arXiv  Detail & Related papers  (2024-02-26T15:39:52Z)
• A Study of Documentation for Software Architecture [7.011803832284996]
  We asked 65 participants to answer software architecture understanding questions. Answers to questions that require applying and creating activities were statistically significantly associated with the use of the system's source code. We conclude that, in the limited experimental context studied, our results contradict the hypothesis that the format of architectural documentation matters.
  arXiv  Detail & Related papers  (2023-05-26T22:14:53Z)
• Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes [54.13559879916708]
  EVAPORATE is a prototype system powered by large language models (LLMs)<n>Code synthesis is cheap, but far less accurate than directly processing each document with the LLM.<n>We propose an extended code implementation, EVAPORATE-CODE+, which achieves better quality than direct extraction.
  arXiv  Detail & Related papers  (2023-04-19T06:00:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.