DECO: Life-Cycle Management of Enterprise-Grade Copilots

• URL: http://arxiv.org/abs/2412.06099v2
• Date: Mon, 10 Mar 2025 05:24:19 GMT
• Title: DECO: Life-Cycle Management of Enterprise-Grade Copilots
• Abstract summary: DECO is a comprehensive framework for developing, deploying, and managing enterprise-grade copilots.<n>It supports efficient and customized retrieval-augmented-generation (RAG) algorithms.<n>DECO has demonstrated its effectiveness through widespread adoption, enabling tens of thousands of interactions.
• Score: 9.908567982584815
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Software engineers frequently grapple with the challenge of accessing disparate documentation and telemetry data, including TroubleShooting Guides (TSGs), incident reports, code repositories, and various internal tools developed by multiple stakeholders. While on-call duties are inevitable, incident resolution becomes even more daunting due to the obscurity of legacy sources and the pressures of strict time constraints. To enhance the efficiency of on-call engineers (OCEs) and streamline their daily workflows, we introduced DECO-a comprehensive framework for developing, deploying, and managing enterprise-grade copilots tailored to improve productivity in engineering routines. This paper details the design and implementation of the DECO framework, emphasizing its innovative NL2SearchQuery functionality and a lightweight agentic framework. These features support efficient and customized retrieval-augmented-generation (RAG) algorithms that not only extract relevant information from diverse sources but also select the most pertinent skills in response to user queries. This enables the addressing of complex technical questions and provides seamless, automated access to internal resources. Additionally, DECO incorporates a robust mechanism for converting unstructured incident logs into user-friendly, structured guides, effectively bridging the documentation gap. Since its launch in September 2023, DECO has demonstrated its effectiveness through widespread adoption, enabling tens of thousands of interactions and engaging hundreds of monthly active users (MAU) across dozens of organizations within the company.

Related papers

• PRAXIS: Graph-Grounded Tacit Knowledge for Domain Code Generation [61.752289169878566]
  We identify the root cause of the agent's lack of tacit knowledge, including domain-specific business rules, interface contracts, and operational conventions that developers internalize through practice but never document.<n>This knowledge is deeply buried beneath the domain code, across code entities and their dependency relations, and invisible to the agent that lacks it.<n>We propose PRAXIS, a framework that enables agents to extract, represent, and reuse tacit knowledge for domain code generation.
  arXiv  Detail & Related papers  (2026-08-20T08:28:28Z)
• Meta-Engineering Harnesses for AI-Native Software Production: A Contract-Driven Adversarial Verification Architecture with Early Deployment Report [0.0]
  We present a software-production architecture that transforms operational and product feature requirements into explicit contracts.<n>We describe the layered architecture, including two-pass contract compilation.<n>We report results from an early production deployment spanning 17 features over several weeks.
  arXiv  Detail & Related papers  (2026-05-25T10:15:24Z)
• Building AI Coding Agents for the Terminal: Scaffolding, Harness, Context Engineering, and Lessons Learned [9.127884945730019]
  We present OPENDEV, an open-source, command-line coding agent engineered specifically for this new paradigm.<n>It overcomes these challenges through a compound AI system architecture with workload-specialized model routing.<n>It employs an automated memory system to accumulate project-specific knowledge across sessions and counteract instruction fade-out.
  arXiv  Detail & Related papers  (2026-03-05T16:21:08Z)
• Steering LLMs via Scalable Interactive Oversight [74.12746881843044]
  Large Language Models increasingly automate complex, long-horizon tasks such as emphvibe coding, a supervision gap has emerged.<n>It presents a critical challenge in scalable oversight: enabling humans to responsibly steer AI systems on tasks that surpass their own ability to specify or verify.
  arXiv  Detail & Related papers  (2026-02-04T04:52:00Z)
• ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development [72.4729759618632]
  We introduce ABC-Bench, a benchmark to evaluate agentic backend coding within a realistic, executable workflow.<n>We curated 224 practical tasks spanning 8 languages and 19 frameworks from open-source repositories.<n>Our evaluation reveals that even state-of-the-art models struggle to deliver reliable performance on these holistic tasks.
  arXiv  Detail & Related papers  (2026-01-16T08:23:52Z)
• Towards Reliable ML Feature Engineering via Planning in Constrained-Topology of LLM Agents [1.991571265620589]
  Recent advances in code generation models have unlocked unprecedented opportunities for automating feature engineering.<n>Their adoption in real-world ML teams remains constrained by critical challenges.<n>We address these challenges with a planner-guided, constrained-topology multi-agent framework.
  arXiv  Detail & Related papers  (2026-01-15T19:33:42Z)
• LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering [90.84806758077536]
  We introduce textbfLoCoBench-Agent, a comprehensive evaluation framework specifically designed to assess large language models (LLMs) agents in realistic, long-context software engineering.<n>Our framework extends LoCoBench's 8,000 scenarios into interactive agent environments, enabling systematic evaluation of multi-turn conversations.<n>Our framework provides agents with 8 specialized tools (file operations, search, code analysis) and evaluates them across context lengths ranging from 10K to 1M tokens.
  arXiv  Detail & Related papers  (2025-11-17T23:57:24Z)
• DRBench: A Realistic Benchmark for Enterprise Deep Research [81.49694432639406]
  DRBench is a benchmark for evaluating AI agents on complex, open-ended deep research tasks in enterprise settings.<n>We release 15 deep research tasks across 10 domains, such as Sales, Cybersecurity, and Compliance.
  arXiv  Detail & Related papers  (2025-09-30T18:47:20Z)
• Large Language Model Powered Automated Modeling and Optimization of Active Distribution Network Dispatch Problems [22.491530071431107]
  This paper proposes a large language model (LLM) powered automated modeling and optimization approach.<n>The proposed approach features a user-centric interface that enables ADN operators to derive dispatch strategies via simple natural language queries.
  arXiv  Detail & Related papers  (2025-07-25T07:46:25Z)
• ORMind: A Cognitive-Inspired End-to-End Reasoning Framework for Operations Research [53.736407871322314]
  We introduce ORMind, a cognitive-inspired framework that enhances optimization through counterfactual reasoning.<n>Our approach emulates human cognition, implementing an end-to-end workflow that transforms requirements into mathematical models and executable code.<n>It is currently being tested internally in Lenovo's AI Assistant, with plans to enhance optimization capabilities for both business and consumer customers.
  arXiv  Detail & Related papers  (2025-06-02T05:11:21Z)
• Co-Saving: Resource Aware Multi-Agent Collaboration for Software Development [65.94639060883475]
  We propose a resource-aware multi-agent system -- Co-Saving.<n>Our key innovation is the introduction of "shortcuts"<n>Compared to the state-of-the-art MAS ChatDev, our method achieves an average reduction of 50.85% in token usage.
  arXiv  Detail & Related papers  (2025-05-28T02:23:53Z)
• Get Experience from Practice: LLM Agents with Record & Replay [16.179801770737892]
  This paper proposes a new paradigm called AgentRR (Agent Record & Replay), which introduces the classical record-and-replay mechanism into AI agent frameworks.<n>We detail a multi-level experience abstraction method and a check function mechanism in AgentRR.<n>In addition, we explore multiple application modes of AgentRR, including user-recorded task demonstration, large-small model collaboration and privacy-aware agent execution.
  arXiv  Detail & Related papers  (2025-05-23T10:33:14Z)
• Orchestrating Agents and Data for Enterprise: A Blueprint Architecture for Compound AI [11.859180018313147]
  We propose a 'blueprint architecture' for compound AI systems for orchestrating agents and data for enterprise applications. Existing proprietary models and APIs in the enterprise are mapped to 'agents', defined in an 'agent registry' Agents can utilize proprietary data through a 'data registry' that similarly registers enterprise data of various modalities.
  arXiv  Detail & Related papers  (2025-04-10T22:19:41Z)
• RoboFactory: Exploring Embodied Agent Collaboration with Compositional Constraints [27.467048581838405]
  We propose the concept of compositional constraints for embodied multi-agent systems. We design interfaces tailored to different types of constraints, enabling seamless interaction with the physical world. We introduce the first benchmark for embodied multi-agent manipulation, RoboFactory.
  arXiv  Detail & Related papers  (2025-03-20T17:58:38Z)
• Intelligent Mobile AI-Generated Content Services via Interactive Prompt Engineering and Dynamic Service Provisioning [55.641299901038316]
  AI-generated content can organize collaborative Mobile AIGC Service Providers (MASPs) at network edges to provide ubiquitous and customized content for resource-constrained users. Such a paradigm faces two significant challenges: 1) raw prompts often lead to poor generation quality due to users' lack of experience with specific AIGC models, and 2) static service provisioning fails to efficiently utilize computational and communication resources. We develop an interactive prompt engineering mechanism that leverages a Large Language Model (LLM) to generate customized prompt corpora and employs Inverse Reinforcement Learning (IRL) for policy imitation.
  arXiv  Detail & Related papers  (2025-02-17T03:05:20Z)
• KIMAs: A Configurable Knowledge Integrated Multi-Agent System [46.91903900679881]
  This technical report presents a knowledge integrated multi-agent system, KIMAs, to address these challenges. Our work provides a scalable framework for advancing the deployment of large language models in real-world settings.
  arXiv  Detail & Related papers  (2025-02-13T18:51:12Z)
• Agentic AI-Driven Technical Troubleshooting for Enterprise Systems: A Novel Weighted Retrieval-Augmented Generation Paradigm [0.0]
  This paper presents a novel agentic AI solution built on a Weighted Retrieval-Augmented Generation (RAG) Framework tailored for enterprise technical troubleshooting. By dynamically weighting retrieval sources such as product manuals, internal knowledge bases, FAQ, and troubleshooting guides, the framework prioritizes the most relevant data. Preliminary evaluations on large enterprise datasets demonstrate the framework's efficacy in improving troubleshooting accuracy, reducing resolution times, and adapting to varied technical challenges.
  arXiv  Detail & Related papers  (2024-12-16T17:32:38Z)
• HierTOD: A Task-Oriented Dialogue System Driven by Hierarchical Goals [4.630232280155836]
  Task-Oriented Dialogue (TOD) systems assist users in completing tasks through natural language interactions. In this work, we introduce HierTOD, an enterprise TOD system driven by hierarchical goals and can support composite. Our system implementation unifies two TOD paradigms: slot-filling for information collection and step-by-step guidance for task execution.
  arXiv  Detail & Related papers  (2024-11-11T17:28:19Z)
• CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments [90.29937153770835]
  We introduce CRMArena, a benchmark designed to evaluate AI agents on realistic tasks grounded in professional work environments. We show that state-of-the-art LLM agents succeed in less than 40% of the tasks with ReAct prompting, and less than 55% even with function-calling abilities. Our findings highlight the need for enhanced agent capabilities in function-calling and rule-following to be deployed in real-world work environments.
  arXiv  Detail & Related papers  (2024-11-04T17:30:51Z)
• Towards Human-Level Understanding of Complex Process Engineering Schematics: A Pedagogical, Introspective Multi-Agent Framework for Open-Domain Question Answering [0.0]
  In the chemical and process industries, Process Flow Diagrams (PFDs) and Piping and Instrumentation Diagrams (P&IDs) are critical for design, construction, and maintenance. Recent advancements in Generative AI have shown promise in understanding and interpreting process diagrams for Visual Question Answering (VQA) We propose a secure, on-premises enterprise solution using a hierarchical, multi-agent Retrieval Augmented Generation (RAG) framework.
  arXiv  Detail & Related papers  (2024-08-24T19:34:04Z)
• Agent-Driven Automatic Software Improvement [55.2480439325792]
  This research proposal aims to explore innovative solutions by focusing on the deployment of agents powered by Large Language Models (LLMs) The iterative nature of agents, which allows for continuous learning and adaptation, can help surpass common challenges in code generation. We aim to use the iterative feedback in these systems to further fine-tune the LLMs underlying the agents, becoming better aligned to the task of automated software improvement.
  arXiv  Detail & Related papers  (2024-06-24T15:45:22Z)
• Towards a RAG-based Summarization Agent for the Electron-Ion Collider [0.5504260452953508]
  A Retrieval Augmented Generation (RAG)--based Summarization AI for EIC (RAGS4EIC) is under development. This AI-Agent not only condenses information but also effectively references relevant responses, offering substantial advantages for collaborators. Our project involves a two-step approach: first, querying a comprehensive vector database containing all pertinent experiment information; second, utilizing a Large Language Model (LLM) to generate concise summaries enriched with citations based on user queries and retrieved data.
  arXiv  Detail & Related papers  (2024-03-23T05:32:46Z)
• Experiential Co-Learning of Software-Developing Agents [83.34027623428096]
  Large language models (LLMs) have brought significant changes to various domains, especially in software development. We introduce Experiential Co-Learning, a novel LLM-agent learning framework. Experiments demonstrate that the framework enables agents to tackle unseen software-developing tasks more effectively.
  arXiv  Detail & Related papers  (2023-12-28T13:50:42Z)
• ChatDev: Communicative Agents for Software Development [84.90400377131962]
  ChatDev is a chat-powered software development framework in which specialized agents are guided in what to communicate. These agents actively contribute to the design, coding, and testing phases through unified language-based communication.
  arXiv  Detail & Related papers  (2023-07-16T02:11:34Z)
• Flexible Job Shop Scheduling via Dual Attention Network Based Reinforcement Learning [73.19312285906891]
  In flexible job shop scheduling problem (FJSP), operations can be processed on multiple machines, leading to intricate relationships between operations and machines. Recent works have employed deep reinforcement learning (DRL) to learn priority dispatching rules (PDRs) for solving FJSP. This paper presents a novel end-to-end learning framework that weds the merits of self-attention models for deep feature extraction and DRL for scalable decision-making.
  arXiv  Detail & Related papers  (2023-05-09T01:35:48Z)
• Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanism [120.1998866178014]
  We present a flexible framework for continual object detection via pRotOtypical taSk corrElaTion guided gaTingAnism (ROSETTA) Concretely, a unified framework is shared by all tasks while task-aware gates are introduced to automatically select sub-models for specific tasks. Experiments on COCO-VOC, KITTI-Kitchen, class-incremental detection on VOC and sequential learning of four tasks show that ROSETTA yields state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-05-06T07:31:28Z)
• Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming [77.38174112525168]
  We present Nemo, an end-to-end interactive Supervision system that improves overall productivity of WS learning pipeline by an average 20% (and up to 47% in one task) compared to the prevailing WS supervision approach.
  arXiv  Detail & Related papers  (2022-03-02T19:57:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.