An Agentic AI Workflow for Detecting Cognitive Concerns in Real-world Data

• URL: http://arxiv.org/abs/2502.01789v1
• Date: Mon, 03 Feb 2025 20:08:33 GMT
• Title: An Agentic AI Workflow for Detecting Cognitive Concerns in Real-world Data
• Abstract summary: Early identification of cognitive concerns is critical but often hindered by subtle symptom presentation.<n>This study developed and validated a fully automated, multi-agent AI workflow using LLaMA 3 8B to identify cognitive concerns in 3,338 clinical notes from Mass General Brigham.
• Score: 4.161501422808857
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Early identification of cognitive concerns is critical but often hindered by subtle symptom presentation. This study developed and validated a fully automated, multi-agent AI workflow using LLaMA 3 8B to identify cognitive concerns in 3,338 clinical notes from Mass General Brigham. The agentic workflow, leveraging task-specific agents that dynamically collaborate to extract meaningful insights from clinical notes, was compared to an expert-driven benchmark. Both workflows achieved high classification performance, with F1-scores of 0.90 and 0.91, respectively. The agentic workflow demonstrated improved specificity (1.00) and achieved prompt refinement in fewer iterations. Although both workflows showed reduced performance on validation data, the agentic workflow maintained perfect specificity. These findings highlight the potential of fully automated multi-agent AI workflows to achieve expert-level accuracy with greater efficiency, offering a scalable and cost-effective solution for detecting cognitive concerns in clinical settings.

Related papers

• Diagnosis-Driven Automatic Repair for Agentic Workflow via Symbolic Inference [23.048678930743467]
  FlowFixer is a diagnosis-driven automated repair framework for agenticcution.<n>We evaluate FlowFixer on workflow failures collected from three popular development platforms: Dify, Coze and n8n.<n>Results show that FlowFixer achieves a repair success rate of 71.3%, outperforming state-of-the-art baselines by 11.9% to 27.6%.
  arXiv  Detail & Related papers  (2026-07-03T02:28:32Z)
• AutoMedBench: Towards Medical AutoResearch with Agentic AI Models [31.335309699875264]
  AutoMedBench is a workflow-aware benchmark for autonomous medical-AI research.<n>It comprises long-horizon tasks with each run averaging 33 agent turns.
  arXiv  Detail & Related papers  (2026-06-01T09:22:55Z)
• Towards a Virtual Neuroscientist: Autonomous Neuroimaging Analysis via Multi-Agent Collaboration [53.772014300855375]
  We introduce NIAgent, a multi-agent system for autonomous end-to-end neuroimaging analysis.<n>Unlike conventional flat tool-calling agents, NIAgent adopts a code-centric execution paradigm.<n>We propose a hierarchical verification framework for autonomous quality control, integrating cohort-level metric screening with agentic visual inspection.
  arXiv  Detail & Related papers  (2026-05-10T06:30:19Z)
• Reliability of AI Bots Footprints in GitHub Actions CI/CD Workflows [2.283561089098417]
  Continuous Integration and Deployment (CI/CD) are central to modern software delivery, yet the reliability of agentic AI bots operating within these remain underexplored.<n>Using pull requests (PRs), commits, and repositories from the AIDev dataset, we retrieved associated CI/CD workflow runs via the GitHub API.<n>We analyzed 61,837 runs from 2,355 repositories, all triggered by PRs generated by five AI bots: Claude, Devin, Cursor, Copilot, and Codex.
  arXiv  Detail & Related papers  (2026-04-20T14:34:14Z)
• Human-Guided Agentic AI for Multimodal Clinical Prediction: Lessons from the AgentDS Healthcare Benchmark [0.5066646435185324]
  We investigate how human guidance of agentic AI can improve multimodal clinical prediction.<n>We present our approach to three benchmark challenges: 30-day hospital prediction, emergency department cost forecasting, and discharge readiness assessment.<n>Our approach ranked 5th overall in the healthcare domain, with a 3rd-place finish on the discharge readiness task.
  arXiv  Detail & Related papers  (2026-02-23T04:37:45Z)
• AgentIF-OneDay: A Task-level Instruction-Following Benchmark for General AI Agents in Daily Scenarios [49.90735676070039]
  The capacity of AI agents to effectively handle tasks of increasing duration and complexity continues to grow.<n>We argue that current evaluations prioritize increasing task difficulty without sufficiently addressing the diversity of agentic tasks.<n>We propose AgentIF-OneDay, aimed at determining whether general users can utilize natural language instructions and AI agents to complete a diverse array of daily tasks.
  arXiv  Detail & Related papers  (2026-01-28T13:49:18Z)
• AI-assisted Protocol Information Extraction For Improved Accuracy and Efficiency in Clinical Trial Workflows [0.0]
  Structuring protocol content into standard formats has the potential to improve efficiency, support documentation quality, and strengthen compliance.<n>We evaluate an Artificial Intelligence (AI) system using generative LLMs with RetrievalAugmented Generation (RAG) for automated clinical trial protocol information extraction.
  arXiv  Detail & Related papers  (2026-01-19T18:38:36Z)
• CangLing-KnowFlow: A Unified Knowledge-and-Flow-fused Agent for Comprehensive Remote Sensing Applications [25.74279545322058]
  This paper introduces CangLing-KnowFlow, a unified intelligent agent framework that integrates a Procedural Adjustment, and an Evolutionary Memory Module.<n>During runtime failures, Dynamic Adjustment guides planning and diagnoses, while the Evolutionary Memory Module continuously learns from these events.<n>This synergy enables CangLing-KnowFlow to adapt, learn, and operate reliably across diverse runtime tasks.
  arXiv  Detail & Related papers  (2025-12-17T09:31:57Z)
• Counterfactual-based Agent Influence Ranker for Agentic AI Workflows [4.971684462894703]
  An Agentic AI (AAW) assembles several LLM-based agents to work collaboratively towards a shared goal.<n>There are no existing methods to assess the influence of each agent on the AAW's final output.<n>We present Counterfactual-based Agent Influence Ranker (CAIR) - the first method for assessing the influence level of each agent on the AAW's output.
  arXiv  Detail & Related papers  (2025-10-29T15:17:31Z)
• An Interpretable Transformer-Based Foundation Model for Cross-Procedural Skill Assessment Using Raw fNIRS Signals [0.0]
  We introduce an interpretable transformer-based foundation model trained on minimally processed fNIRS signals for cross-procedural skill assessment.<n>The model achieves greater than 88% classification accuracy on all tasks, with Matthews Correlation Coefficient exceeding 0.91 on ETI.<n>It generalizes to a novel emergency airway procedure--cricothyrotomy--using fewer than 30 labeled samples and a lightweight (less than 2k parameter) adapter module.
  arXiv  Detail & Related papers  (2025-06-21T18:30:58Z)
• Agentic Predictor: Performance Prediction for Agentic Workflows via Multi-View Encoding [56.565200973244146]
  Agentic Predictor is a lightweight predictor for efficient agentic workflow evaluation.<n>By learning to approximate task success rates, Agentic Predictor enables fast and accurate selection of optimal agentic workflow configurations.
  arXiv  Detail & Related papers  (2025-05-26T09:46:50Z)
• SEW: Self-Evolving Agentic Workflows for Automated Code Generation [24.16770109875788]
  We propose textbfSelf-textbfEvolving textbfWork (textbfSEW), a novel framework that automatically generates and optimises multi-agentflow.<n>Our SEW can automatically design agentic and optimise them through self-evolution, bringing up to 33% improvement on LiveCodeBench.
  arXiv  Detail & Related papers  (2025-05-24T11:12:14Z)
• TUMLS: Trustful Fully Unsupervised Multi-Level Segmentation for Whole Slide Images of Histology [41.94295877935867]
  We present a trustful fully unsupervised multi-level segmentation methodology (TUMLS) for whole slide images (WSIs) TUMLS adopts an autoencoder (AE) as a feature extractor to identify the different tissue types within low-resolution training data. This solution integrates seamlessly into clinicians, transforming the examination of a whole WSI into a review of concise, interpretable cross-level insights.
  arXiv  Detail & Related papers  (2025-04-17T07:48:05Z)
• On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows [71.92083784393418]
  Agentic AI (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low.<n>Inference-time alignment relies on three components: sampling, evaluation, and feedback.<n>We introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques.
  arXiv  Detail & Related papers  (2025-04-02T17:40:47Z)
• DeepPerception: Advancing R1-like Cognitive Visual Perception in MLLMs for Knowledge-Intensive Visual Grounding [61.26026947423187]
  Human experts excel at fine-grained visual discrimination by leveraging domain knowledge to refine perceptual features. Current Multimodal Large Language Models (MLLMs) struggle to integrate reasoning into visual perception. We propose DeepPerception, an MLLM enhanced with cognitive visual perception capabilities.
  arXiv  Detail & Related papers  (2025-03-17T04:06:34Z)
• Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases [48.87360916431396]
  We introduce MedR-Bench, a benchmarking dataset of 1,453 structured patient cases, annotated with reasoning references. We propose a framework encompassing three critical examination recommendation, diagnostic decision-making, and treatment planning, simulating the entire patient care journey. Using this benchmark, we evaluate five state-of-the-art reasoning LLMs, including DeepSeek-R1, OpenAI-o3-mini, and Gemini-2.0-Flash Thinking, etc.
  arXiv  Detail & Related papers  (2025-03-06T18:35:39Z)
• Turning Conversations into Workflows: A Framework to Extract and Evaluate Dialog Workflows for Service AI Agents [65.36060818857109]
  We present a novel framework for extracting and evaluating dialog from historical interactions. Our extraction process consists of two key stages: (1) a retrieval step to select relevant conversations based on key procedural elements, and (2) a structured workflow generation process using a question-answer-based chain-of-thought (QA-CoT) prompting.
  arXiv  Detail & Related papers  (2025-02-24T16:55:15Z)
• Interactive Agents to Overcome Ambiguity in Software Engineering [61.40183840499932]
  AI agents are increasingly being deployed to automate tasks, often based on ambiguous and underspecified user instructions. Making unwarranted assumptions and failing to ask clarifying questions can lead to suboptimal outcomes. We study the ability of LLM agents to handle ambiguous instructions in interactive code generation settings by evaluating proprietary and open-weight models on their performance.
  arXiv  Detail & Related papers  (2025-02-18T17:12:26Z)
• The Impact of AI Assistance on Radiology Reporting: A Pilot Study Using Simulated AI Draft Reports [1.717011912812067]
  Radiologists face increasing workload pressures amid growing imaging volumes, creating risks of burnout and delayed reporting times.<n>While artificial intelligence (AI) based automated radiology report generation shows promise for reporting workflow optimization, evidence of its real-world impact on clinical accuracy and efficiency remains limited.
  arXiv  Detail & Related papers  (2024-12-16T18:10:49Z)
• Benchmarking Agentic Workflow Generation [80.74757493266057]
  We introduce WorFBench, a unified workflow generation benchmark with multi-faceted scenarios and intricate graph workflow structures. We also present WorFEval, a systemic evaluation protocol utilizing subsequence and subgraph matching algorithms. We observe that the generated can enhance downstream tasks, enabling them to achieve superior performance with less time during inference.
  arXiv  Detail & Related papers  (2024-10-10T12:41:19Z)
• Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinement [50.481380478458945]
  Iterative step-level Process Refinement (IPR) framework provides detailed step-by-step guidance to enhance agent training. Our experiments on three complex agent tasks demonstrate that our framework outperforms a variety of strong baselines.
  arXiv  Detail & Related papers  (2024-06-17T03:29:13Z)
• Concept Matching with Agent for Out-of-Distribution Detection [19.407364109506904]
  We propose a new method that integrates the agent paradigm into out-of-distribution (OOD) detection task.<n>Our proposed method, Concept Matching with Agent (CMA), employs neutral prompts as agents to augment the CLIP-based OOD detection process.<n>Our extensive experimental results showcase the superior performance of CMA over both zero-shot and training-required methods.
  arXiv  Detail & Related papers  (2024-05-27T02:27:28Z)
• Exploring LLM Multi-Agents for ICD Coding [15.730751450511333]
  The proposed multi-agent method for ICD coding effectively mimics the real-world coding process and improves performance on both common and rare codes. Our method achieves comparable results to state-of-the-art ICD coding methods that require extensive pre-training or fine-tuning, and outperforms them in rare code accuracy, and explainability.
  arXiv  Detail & Related papers  (2024-04-01T15:17:39Z)
• ADT: Agent-based Dynamic Thresholding for Anomaly Detection [4.356615197661274]
  We propose an agent-based dynamic thresholding (ADT) framework based on a deep Q-network. An auto-encoder is utilized in this study to obtain feature representations and produce anomaly scores for complex input data. ADT can adjust thresholds adaptively by utilizing the anomaly scores from the auto-encoder.
  arXiv  Detail & Related papers  (2023-12-03T19:07:30Z)
• Ladder-of-Thought: Using Knowledge as Steps to Elevate Stance Detection [73.31406286956535]
  We introduce the Ladder-of-Thought (LoT) for the stance detection task. LoT directs the small LMs to assimilate high-quality external knowledge, refining the intermediate rationales produced. Our empirical evaluations underscore LoT's efficacy, marking a 16% improvement over GPT-3.5 and a 10% enhancement compared to GPT-3.5 with CoT on stance detection task.
  arXiv  Detail & Related papers  (2023-08-31T14:31:48Z)
• Anomaly Detection Based on Selection and Weighting in Latent Space [73.01328671569759]
  We propose a novel selection-and-weighting-based anomaly detection framework called SWAD. Experiments on both benchmark and real-world datasets have shown the effectiveness and superiority of SWAD.
  arXiv  Detail & Related papers  (2021-03-08T10:56:38Z)
• Detecting Affective Flow States of Knowledge Workers Using Physiological Sensors [0.9912861954256301]
  Flow-like experiences at work are important for productivity and worker well-being. It is difficult to objectively detect when workers are experiencing flow in their work. In this paper, we investigate how to predict a worker's focus state based on physiological signals.
  arXiv  Detail & Related papers  (2020-06-18T15:59:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.