DevNous: An LLM-Based Multi-Agent System for Grounding IT Project Management in Unstructured Conversation

• URL: http://arxiv.org/abs/2508.08761v1
• Date: Tue, 12 Aug 2025 09:08:29 GMT
• Title: DevNous: An LLM-Based Multi-Agent System for Grounding IT Project Management in Unstructured Conversation
• Abstract summary: We introduce DevNous, a Large Language Model-based (LLM) multi-agent expert system to automate this unstructured-to-structured translation process.<n>DevNous integrates directly into team chat environments, identifying actionable intents from informal dialogue.<n>To quantitatively evaluate the system, we introduce a new benchmark of 160 realistic, interactive conversational turns.
• Score: 2.1301560294088318
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The manual translation of unstructured team dialogue into the structured artifacts required for Information Technology (IT) project governance is a critical bottleneck in modern information systems management. We introduce DevNous, a Large Language Model-based (LLM) multi-agent expert system, to automate this unstructured-to-structured translation process. DevNous integrates directly into team chat environments, identifying actionable intents from informal dialogue and managing stateful, multi-turn workflows for core administrative tasks like automated task formalization and progress summary synthesis. To quantitatively evaluate the system, we introduce a new benchmark of 160 realistic, interactive conversational turns. The dataset was manually annotated with a multi-label ground truth and is publicly available. On this benchmark, DevNous achieves an exact match turn accuracy of 81.3\% and a multiset F1-Score of 0.845, providing strong evidence for its viability. The primary contributions of this work are twofold: (1) a validated architectural pattern for developing ambient administrative agents, and (2) the introduction of the first robust empirical baseline and public benchmark dataset for this challenging problem domain.

Related papers

• OpenCompass: A Universal Evaluation Platform for Large Language Models [62.59671563145442]
  General-purpose large language models (LLMs) have become a critical link in advancing technological development.<n>Mainstream benchmark datasets face challenges such as the diversity of task types, inconsistent evaluation criteria, and fragmentation of data and processing.<n>This paper proposes and open-source Open LLMs, a one-stop, scalable, and high-concurrency general-purpose evaluation platform.
  arXiv  Detail & Related papers  (2026-05-19T02:50:11Z)
• Autonoma: A Hierarchical Multi-Agent Framework for End-to-End Workflow Automation [0.0]
  This paper introduces Autonoma, a structured, hierarchical multi-agent framework designed for end-to-end workflow automation from natural language prompts.<n>Autonoma employs a principled multi-tiered architecture where a high-level Coordinator validates user intent, a Planner generates structured, and a Supervisor manages the execution by orchestrating a suite of specialized agents.<n>Autonoma achieved a 97% task completion rate and a 98% successful agent handoff rate, confirming its operational reliability and efficient collaboration.
  arXiv  Detail & Related papers  (2026-02-27T11:00:37Z)
• TAMUSA-Chat: A Domain-Adapted Large Language Model Conversational System for Research and Responsible Deployment [0.0]
  TAMUSA-Chat is a framework for building domain-adapted large language model conversational systems.<n>We show how academic institutions can develop contextually grounded conversational agents while maintaining transparency, governance compliance, and responsible AI practices.
  arXiv  Detail & Related papers  (2026-02-14T04:44:52Z)
• Agyn: A Multi-Agent System for Team-Based Autonomous Software Engineering [0.09046463333989574]
  Real-world software development is organized as a collaborative activity carried out by teams following shared methodologies.<n>We present a fully automated multi-agent system that explicitly models software engineering as an organizational process.<n>Our results suggest that replicating team structure, methodology, and communication is a powerful paradigm for autonomous software engineering.
  arXiv  Detail & Related papers  (2026-02-01T22:17:19Z)
• LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering [90.84806758077536]
  We introduce textbfLoCoBench-Agent, a comprehensive evaluation framework specifically designed to assess large language models (LLMs) agents in realistic, long-context software engineering.<n>Our framework extends LoCoBench's 8,000 scenarios into interactive agent environments, enabling systematic evaluation of multi-turn conversations.<n>Our framework provides agents with 8 specialized tools (file operations, search, code analysis) and evaluates them across context lengths ranging from 10K to 1M tokens.
  arXiv  Detail & Related papers  (2025-11-17T23:57:24Z)
• MMWOZ: Building Multimodal Agent for Task-oriented Dialogue [61.816787158531874]
  We develop a new multimodal dialogue dataset that is extended from MultiWOZ 2.3 dataset.<n>We propose a novel multimodal model called MATE as the baseline model for the MMWOZ dataset.
  arXiv  Detail & Related papers  (2025-11-16T13:08:03Z)
• A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System [56.40989626804489]
  This survey provides the first holistic analysis of Large Language Models-powered software engineering.<n>We review over 150 recent papers and propose a taxonomy along two key dimensions: (1) Solutions, categorized into prompt-based, fine-tuning-based, and agent-based paradigms, and (2) Benchmarks, including tasks such as code generation, translation, and repair.
  arXiv  Detail & Related papers  (2025-10-10T06:56:50Z)
• AgentMaster: A Multi-Agent Conversational Framework Using A2A and MCP Protocols for Multimodal Information Retrieval and Analysis [0.0]
  We present a pilot study of AgentMaster, a novel modular multi-protocol MAS framework with self-implemented A2A and MCP.<n>The system supports natural language interaction without prior technical expertise and responds to multimodal queries.<n>Overall, our proposed framework contributes to the potential capabilities of domain-specific, cooperative, and scalable conversational AI powered by MAS.
  arXiv  Detail & Related papers  (2025-07-08T03:34:26Z)
• A Survey on (M)LLM-Based GUI Agents [62.57899977018417]
  Graphical User Interface (GUI) Agents have emerged as a transformative paradigm in human-computer interaction.<n>Recent advances in large language models and multimodal learning have revolutionized GUI automation across desktop, mobile, and web platforms.<n>This survey identifies key technical challenges, including accurate element localization, effective knowledge retrieval, long-horizon planning, and safety-aware execution control.
  arXiv  Detail & Related papers  (2025-03-27T17:58:31Z)
• OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Models [58.45517851437422]
  Visually-situated text parsing (VsTP) has recently seen notable advancements, driven by the growing demand for automated document understanding.<n>Existing solutions often rely on task-specific architectures and objectives for individual tasks.<n>In this paper, we introduce Omni V2, a universal model that unifies VsTP typical tasks, including text spotting, key information extraction, table recognition, and layout analysis.
  arXiv  Detail & Related papers  (2025-02-22T09:32:01Z)
• Autonomous Deep Agent [0.7489814067742621]
  Deep Agent is an advanced autonomous AI system designed to manage complex multi-phase tasks.<n>The system's foundation is built on our Hierarchical Task DAG framework.<n>Deep Agent establishes a novel paradigm in self-governing AI systems.
  arXiv  Detail & Related papers  (2025-02-10T21:46:54Z)
• InstructERC: Reforming Emotion Recognition in Conversation with Multi-task Retrieval-Augmented Large Language Models [9.611864685207056]
  We propose a novel approach, InstructERC, to reformulate the emotion recognition task from a discriminative framework to a generative framework based on Large Language Models (LLMs) InstructERC makes three significant contributions: (1) it introduces a simple yet effective retrieval template module, which helps the model explicitly integrate multi-granularity dialogue supervision information; (2) we introduce two additional emotion alignment tasks, namely speaker identification and emotion prediction tasks, to implicitly model the dialogue role relationships and future emotional tendencies in conversations; and (3) Pioneeringly, we unify emotion labels across benchmarks through the feeling wheel to fit real application scenarios.
  arXiv  Detail & Related papers  (2023-09-21T09:22:07Z)
• LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark [81.42376626294812]
  We present Language-Assisted Multi-Modal instruction tuning dataset, framework, and benchmark. Our aim is to establish LAMM as a growing ecosystem for training and evaluating MLLMs. We present a comprehensive dataset and benchmark, which cover a wide range of vision tasks for 2D and 3D vision.
  arXiv  Detail & Related papers  (2023-06-11T14:01:17Z)
• Zero-Shot Generalizable End-to-End Task-Oriented Dialog System using Context Summarization and Domain Schema [2.7178968279054936]
  State-of-the-art approaches in task-oriented dialog systems formulate the problem as a conditional sequence generation task. This requires labeled training data for each new domain or task. We introduce a novel Zero-Shot generalizable end-to-end Task-oriented Dialog system, ZS-ToD.
  arXiv  Detail & Related papers  (2023-03-28T18:56:31Z)
• A Machine Learning Approach to Classifying Construction Cost Documents into the International Construction Measurement Standard [0.0]
  We introduce the first automated models for classifying natural language descriptions provided in cost documents called "Bills of Quantities" We learn from a dataset of more than 50 thousand descriptions of items retrieved from 24 large infrastructure construction projects across the United Kingdom.
  arXiv  Detail & Related papers  (2022-10-24T11:35:53Z)
• A Data-Centric Framework for Composable NLP Workflows [109.51144493023533]
  Empirical natural language processing systems in application domains (e.g., healthcare, finance, education) involve interoperation among multiple components. We establish a unified open-source framework to support fast development of such sophisticated NLP in a composable manner.
  arXiv  Detail & Related papers  (2021-03-02T16:19:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.