What is the Return on Investment of Digital Engineering for Complex Systems Development? Findings from a Mixed-Methods Study on the Post-production Design Change Process of Navy Assets

• URL: http://arxiv.org/abs/2511.00077v3
• Date: Mon, 10 Nov 2025 03:16:15 GMT
• Title: What is the Return on Investment of Digital Engineering for Complex Systems Development? Findings from a Mixed-Methods Study on the Post-production Design Change Process of Navy Assets
• Abstract summary: Complex engineered systems routinely face schedule and cost overruns, along with poor post-deployment performance.<n>Championed by both INCOSE and the U.S. Department of Defense (DoD), the systems engineering (SE) community has increasingly looked to Digital Engineering (DE) as a potential remedy.<n>This research presents findings from a case study on a Navy SE team responsible for the preliminary design phase of post-production design change projects for Navy assets.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Complex engineered systems routinely face schedule and cost overruns, along with poor post-deployment performance. Championed by both INCOSE and the U.S. Department of Defense (DoD), the systems engineering (SE) community has increasingly looked to Digital Engineering (DE) as a potential remedy. Despite this growing advocacy, most of DE's purported benefits remain anecdotal, and its return on investment (ROI) remains poorly understood. This research presents findings from a case study on a Navy SE team responsible for the preliminary design phase of post-production design change projects for Navy assets. Using a mixed-methods approach, we document why complex system sustainment projects are routinely late, where and to what extent schedule slips arise, and how a DE transformation could improve schedule adherence. This study makes three contributions. First, it identifies four archetypical inefficiency modes that drive schedule overruns and explains how these mechanisms unfold in their organizational context. Second, it quantifies the magnitude and variation of schedule slips. Third, it creates a hypothetical digitally transformed version of the current process, aligned with DoD DE policy, and compares it to the current state to estimate potential schedule gains. Our findings suggest that a DE transformation could reduce the median project duration by 50.1% and reduce the standard deviation by 41.5%, leading to faster and more predictable timelines. However, the observed gains are not uniform across task categories. Overall, this study provides initial quantitative evidence of DE's potential ROI and its value in improving the efficiency and predictability of complex system sustainment projects.

Related papers

• Long-Horizon Autonomous Architecture Research with a Language-Model Agent: A Behavioural Case Study [0.3683202928838613]
  We study what happens when a single general-purpose large language model acts as the sole researcher on a long-horizon neural architecture design problem.<n>The study comprises three phases, separated by human-declared transitions, that progressively expand the agent's tool surface or problem scale.<n>We conclude that workflow design was at least as influential as agent capability in this study and propose diversified search, budgeted moonshot hypotheses, explicit forks, and regime-aware re-validation.
  arXiv  Detail & Related papers  (2026-08-03T09:56:34Z)
• Observationally Informed Adaptive Causal Experimental Design [55.998153710215654]
  We propose Active Residual Learning, a new paradigm that leverages the observational model as a foundational prior.<n>This approach shifts the experimental focus from learning target causal quantities from scratch to efficiently estimating the residuals required to correct observational bias.<n> Experiments on synthetic and semi-synthetic benchmarks demonstrate that R-Design significantly outperforms baselines.
  arXiv  Detail & Related papers  (2026-03-04T06:52:37Z)
• Evaluating Stochasticity in Deep Research Agents [34.59664118367103]
  Deep Research Agents (DRAs) are promising agentic systems to support research across domains such as financial decision-making, medical analysis, and scientific discovery.<n>Despite recent improvements in research quality, DRA system design often overlooks a critical barrier to real-world deployment:ity.<n>Under identical queries, repeated executions of DRAs can exhibit substantial variability in terms of research outcome, findings, and citations.
  arXiv  Detail & Related papers  (2026-02-26T17:46:42Z)
• Towards a Science of Collective AI: LLM-based Multi-Agent Systems Need a Transition from Blind Trial-and-Error to Rigorous Science [70.3658845234978]
  Large Language Models (LLMs) have greatly extended the capabilities of Multi-Agent Systems (MAS)<n>Despite this rapid progress, the field still relies heavily on empirical trial-and-error.<n>This bottleneck stems from the ambiguity of attribution.<n>We propose a factor attribution paradigm to systematically identify collaboration-driving factors.
  arXiv  Detail & Related papers  (2026-02-05T04:19:52Z)
• SSA3D: Text-Conditioned Assisted Self-Supervised Framework for Automatic Dental Abutment Design [52.57094737117145]
  We propose a Self-supervised assisted automatic abutment design framework (SS$A3$D), which employs a dual-branch architecture with a reconstruction branch and a regression branch.<n>The regression branch then predicts the abutment parameters under supervised learning, which eliminates the separate pre-training and fine-tuning process.<n>It also achieves state-of-the-art performance compared to other methods, significantly improving the accuracy and efficiency of automated abutment design.
  arXiv  Detail & Related papers  (2025-12-12T12:08:05Z)
• The Stagnant Persistence Paradox: Survival Analysis and Temporal Efficiency in Exact Sciences and Engineering Education [0.0]
  This study applies a dual-outcome survival analysis framework to two key outcomes: definitive dropout and first major switch.<n>Results uncover a critical systemic inefficiency: a global median survival time of 4.33 years prior to definitive dropout, with a pronounced long tail of extended enrolment.<n>We argue that academic failure in rigid engineering curricula is not a sudden outcome but a long-tail process that generates high opportunity costs.
  arXiv  Detail & Related papers  (2025-12-04T14:11:28Z)
• The Promotion Wall: Efficiency-Equity Trade-offs of Direct Promotion Regimes in Engineering Education [0.0]
  Article uses a calibrated agent-based model to examine how alternative progression regimes reconfigure dropout, time-to-degree, equity and students' psychological experience.
  arXiv  Detail & Related papers  (2025-11-21T12:04:31Z)
• ResT: Reshaping Token-Level Policy Gradients for Tool-Use Large Language Models [62.82372407840088]
  Large language models (LLMs) transcend passive generation and act as goal-directed agents by invoking external tools.<n>textbfReshaped textbfToken-level policy gradients (textbfResT) for tool-use tasks.<n>textbfResT achieves state-of-the-art results, outperforming prior methods by up to $8.76%$.
  arXiv  Detail & Related papers  (2025-09-26T03:38:27Z)
• BranchGRPO: Stable and Efficient GRPO with Structured Branching in Diffusion Models [57.304411396229035]
  We present BranchGRPO, a method that restructures the rollout process into a branching tree.<n>On HPDv2.1 image alignment, BranchGRPO improves alignment scores by up to textbf16% over DanceGRPO.<n>A hybrid variant, BranchGRPO-Mix, further accelerates training to 4.7x faster than DanceGRPO without degrading alignment.
  arXiv  Detail & Related papers  (2025-09-07T12:53:06Z)
• Anytime Incremental $ρ$POMDP Planning in Continuous Spaces [5.767643556541711]
  We present an anytime solver that dynamically refines belief representations, with formal guarantees of improvement over time.<n>We demonstrate its effectiveness for common entropy estimators, reducing computational cost by orders of magnitude.<n> Experimental results show that $rho$POMCPOW outperforms state-of-the-art solvers in both efficiency and solution quality.
  arXiv  Detail & Related papers  (2025-02-04T18:19:40Z)
• Management Decisions in Manufacturing using Causal Machine Learning -- To Rework, or not to Rework? [0.5772546394254112]
  We present a data-driven model for estimating optimal rework policies in manufacturing systems. We consider a single production stage within a multistage, lot-based system that allows for optional rework steps. We apply methods from causal machine learning, in particular double/debiased machine learning (DML) techniques, to estimate conditional treatment effects from data.
  arXiv  Detail & Related papers  (2024-06-17T08:14:40Z)
• Can LLMs Generate Architectural Design Decisions? -An Exploratory Empirical study [10.051928501109]
  An essential artifact of Architectural Knowledge Management (AKM) is the Architecture Decision Records ( ADRs) Despite their benefits, ADR adoption in software development has been slow due to challenges like time constraints and inconsistent uptake. Recent advancements in Large Language Models (LLMs) may help bridge this adoption gap by facilitating ADR generation.
  arXiv  Detail & Related papers  (2024-03-04T03:56:14Z)
• An Investigation of Time Reversal Symmetry in Reinforcement Learning [18.375784421726287]
  We formalize a concept of time reversal symmetry in a Markov decision process (MDP) We observe that utilizing the structure of time reversal in an MDP allows every environment transition experienced by an agent to be transformed into a feasible reverse-time transition. To test the usefulness of this newly synthesized data, we develop a novel approach called time symmetric data augmentation (TSDA)
  arXiv  Detail & Related papers  (2023-11-28T18:02:06Z)
• Ladder-of-Thought: Using Knowledge as Steps to Elevate Stance Detection [73.31406286956535]
  We introduce the Ladder-of-Thought (LoT) for the stance detection task. LoT directs the small LMs to assimilate high-quality external knowledge, refining the intermediate rationales produced. Our empirical evaluations underscore LoT's efficacy, marking a 16% improvement over GPT-3.5 and a 10% enhancement compared to GPT-3.5 with CoT on stance detection task.
  arXiv  Detail & Related papers  (2023-08-31T14:31:48Z)
• Integration of Domain Expert-Centric Ontology Design into the CRISP-DM for Cyber-Physical Production Systems [45.05372822216111]
  Methods from Machine Learning (ML) and Data Mining (DM) have proven to be promising in extracting complex and hidden patterns from the data collected. However, such data-driven projects, usually performed with the Cross-Industry Standard Process for Data Mining (CRISPDM), often fail due to the disproportionate amount of time needed for understanding and preparing the data. This contribution intends present an integrated approach so that data scientists are able to more quickly and reliably gain insights into the CPPS challenges.
  arXiv  Detail & Related papers  (2023-07-21T15:04:00Z)
• A Machine Learning Approach to Improving Timing Consistency between Global Route and Detailed Route [3.202646674984817]
  Inaccurate timing prediction wastes design effort, hurts circuit performance, and may lead to design failure. This work focuses on timing prediction after clock tree synthesis and placement legalization, which is the earliest opportunity to time and optimize a "complete" netlist. To bridge the gap between GR-based parasitic and timing estimation and post-DR results during post-GR optimization, machine learning (ML)-based models are proposed.
  arXiv  Detail & Related papers  (2023-05-11T16:01:23Z)
• Transformer Hawkes Process [79.16290557505211]
  We propose a Transformer Hawkes Process (THP) model, which leverages the self-attention mechanism to capture long-term dependencies. THP outperforms existing models in terms of both likelihood and event prediction accuracy by a notable margin. We provide a concrete example, where THP achieves improved prediction performance for learning multiple point processes when incorporating their relational information.
  arXiv  Detail & Related papers  (2020-02-21T13:48:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.