Assured Autonomy: How Operations Research Powers and Orchestrates Generative AI Systems

• URL: http://arxiv.org/abs/2512.23978v1
• Date: Tue, 30 Dec 2025 04:24:06 GMT
• Title: Assured Autonomy: How Operations Research Powers and Orchestrates Generative AI Systems
• Authors: Tinglong Dai, David Simchi-Levi, Michelle Xiao Wu, Yao Xie,
• Abstract summary: We argue generative models can be fragile in operational domains unless paired with mechanisms that provide feasibility, robustness to distribution shift, and stress testing.<n>We develop a conceptual framework for assured autonomy grounded in operations research.<n>These elements define a research agenda for assured autonomy in safety-critical, reliability-sensitive operational domains.
• Score: 18.881800772626427
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative artificial intelligence (GenAI) is shifting from conversational assistants toward agentic systems -- autonomous decision-making systems that sense, decide, and act within operational workflows. This shift creates an autonomy paradox: as GenAI systems are granted greater operational autonomy, they should, by design, embody more formal structure, more explicit constraints, and stronger tail-risk discipline. We argue stochastic generative models can be fragile in operational domains unless paired with mechanisms that provide verifiable feasibility, robustness to distribution shift, and stress testing under high-consequence scenarios. To address this challenge, we develop a conceptual framework for assured autonomy grounded in operations research (OR), built on two complementary approaches. First, flow-based generative models frame generation as deterministic transport characterized by an ordinary differential equation, enabling auditability, constraint-aware generation, and connections to optimal transport, robust optimization, and sequential decision control. Second, operational safety is formulated through an adversarial robustness lens: decision rules are evaluated against worst-case perturbations within uncertainty or ambiguity sets, making unmodeled risks part of the design. This framework clarifies how increasing autonomy shifts OR's role from solver to guardrail to system architect, with responsibility for control logic, incentive protocols, monitoring regimes, and safety boundaries. These elements define a research agenda for assured autonomy in safety-critical, reliability-sensitive operational domains.

Related papers

• SAGE-LLM: Towards Safe and Generalizable LLM Controller with Fuzzy-CBF Verification and Graph-Structured Knowledge Retrieval for UAV Decision [46.089736018739295]
  Large Language Models (LLM) lack domain-specific UAV control knowledge and formal safety assurances.<n>This paper proposes a train-free two-layer decision architecture based on LLMs, integrating high-level safety planning with low-level precise control.
  arXiv  Detail & Related papers  (2026-02-27T06:41:04Z)
• Agentic Problem Frames: A Systematic Approach to Engineering Reliable Domain Agents [0.0]
  Large Language Models (LLMs) are evolving into autonomous agents, yet current "frameless" development--relying on ambiguous natural language--leads to critical risks such as scope creep and open-loop failures.<n>This study proposes Agentic Problem Frames (APF), a systematic engineering framework that shifts focus from internal model intelligence to the structured interaction between the agent and its environment.
  arXiv  Detail & Related papers  (2026-02-22T06:32:32Z)
• Agentic AI for Cybersecurity: A Meta-Cognitive Architecture for Governable Autonomy [0.0]
  This paper argues that cybersecurity orchestration should be reconceptualized as an agentic, multi-agent cognitive system.<n>We introduce a conceptual framework in which heterogeneous AI agents responsible for detection, hypothesis formation, contextual interpretation, explanation, and governance are coordinated through an explicit meta-cognitive judgement function.<n>Our contribution is to make this cognitive structure architecturally explicit and governable by embedding meta-cognitive judgement as a first-class system function.
  arXiv  Detail & Related papers  (2026-02-12T12:52:49Z)
• Self-Guard: Defending Large Reasoning Models via enhanced self-reflection [54.775612141528164]
  Self-Guard is a lightweight safety defense framework for Large Reasoning Models.<n>It bridges the awareness-compliance gap, achieving robust safety performance without compromising model utility.<n>Self-Guard exhibits strong generalization across diverse unseen risks and varying model scales.
  arXiv  Detail & Related papers  (2026-01-31T13:06:11Z)
• Interpreting Agentic Systems: Beyond Model Explanations to System-Level Accountability [0.6745502291821954]
  Agentic systems have transformed how Large Language Models can be leveraged to create autonomous systems with goal-directed behaviors.<n>Current interpretability techniques, developed primarily for static models, show limitations when applied to agentic systems.<n>This paper assesses the suitability and limitations of existing interpretability methods in the context of agentic systems.
  arXiv  Detail & Related papers  (2026-01-23T21:05:32Z)
• Agentic Uncertainty Quantification [76.94013626702183]
  We propose a unified Dual-Process Agentic UQ (AUQ) framework that transforms verbalized uncertainty into active, bi-directional control signals.<n>Our architecture comprises two complementary mechanisms: System 1 (Uncertainty-Aware Memory, UAM), which implicitly propagates verbalized confidence and semantic explanations to prevent blind decision-making; and System 2 (Uncertainty-Aware Reflection, UAR), which utilizes these explanations as rational cues to trigger targeted inference-time resolution only when necessary.
  arXiv  Detail & Related papers  (2026-01-22T07:16:26Z)
• Institutional AI: A Governance Framework for Distributional AGI Safety [1.3763052684269788]
  We identify three structural problems that emerge from core properties of AI models.<n>The solution is Institutional AI, a system-level approach that treats alignment as a question of effective governance of AI agent collectives.
  arXiv  Detail & Related papers  (2026-01-15T17:08:26Z)
• Towards Responsible and Explainable AI Agents with Consensus-Driven Reasoning [4.226647687395254]
  This paper presents a Responsible(RAI) and Explainable(XAI) AI Agent Architecture for production-grade agentic based on multi-model consensus and reasoning-layer governance.<n>In the proposed design, a consortium of heterogeneous LLM and VLM agents independently generates candidate outputs from a shared input context.<n>A dedicated reasoning agent then performs structured consolidation across these outputs, enforcing safety and policy constraints, mitigating hallucinations and bias, and producing auditable, evidence-backed decisions.
  arXiv  Detail & Related papers  (2025-12-25T14:49:25Z)
• Perspectives on a Reliability Monitoring Framework for Agentic AI Systems [5.539407031861404]
  We derive the main reliability challenges of agentic AI systems during operation based on their characteristics.<n>We propose a two-layered reliability monitoring framework for agentic AI systems.
  arXiv  Detail & Related papers  (2025-11-12T10:19:17Z)
• A cybersecurity AI agent selection and decision support framework [0.0]
  This paper presents a novel, structured decision support framework that aligns AI agent architectures, reactive, cognitive, hybrid, and learning.<n>By integrating agent theory with industry guidelines, this framework provides a transparent and stepwise methodology for selecting and deploying AI solutions.
  arXiv  Detail & Related papers  (2025-10-02T07:38:21Z)
• A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents [45.53643260046778]
  Recent advances in large language models (LLMs) have catalyzed the rise of autonomous AI agents.<n>These large-model agents mark a paradigm shift from static inference systems to interactive, memory-augmented entities.
  arXiv  Detail & Related papers  (2025-06-30T13:34:34Z)
• Human-AI Governance (HAIG): A Trust-Utility Approach [0.0]
  This paper introduces the HAIG framework for analysing trust dynamics across evolving human-AI relationships.<n>Our analysis reveals how technical advances in self-supervision, reasoning authority, and distributed decision-making drive non-uniform trust evolution.
  arXiv  Detail & Related papers  (2025-05-03T01:57:08Z)
• Know Where You're Uncertain When Planning with Multimodal Foundation Models: A Formal Framework [54.40508478482667]
  We present a comprehensive framework to disentangle, quantify, and mitigate uncertainty in perception and plan generation.<n>We propose methods tailored to the unique properties of perception and decision-making.<n>We show that our uncertainty disentanglement framework reduces variability by up to 40% and enhances task success rates by 5% compared to baselines.
  arXiv  Detail & Related papers  (2024-11-03T17:32:00Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.