ESL-Bench: An Event-Driven Synthetic Longitudinal Benchmark for Health Agents

• URL: http://arxiv.org/abs/2604.02834v1
• Date: Fri, 03 Apr 2026 07:55:56 GMT
• Title: ESL-Bench: An Event-Driven Synthetic Longitudinal Benchmark for Health Agents
• Abstract summary: ESLBench is an event-driven synthesis framework providing 100 synthetic users.<n>We evaluate 13 methods spanning LLMs with tools, DB-native agents, and memory-augmented RAG.
• Score: 10.567096101127886
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Longitudinal health agents must reason across multi-source trajectories that combine continuous device streams, sparse clinical exams, and episodic life events - yet evaluating them is hard: real-world data cannot be released at scale, and temporally grounded attribution questions seldom admit definitive answers without structured ground truth. We present ESL-Bench, an event-driven synthesis framework and benchmark providing 100 synthetic users, each with a 1-5 year trajectory comprising a health profile, a multi-phase narrative plan, daily device measurements, periodic exam records, and an event log with explicit per-indicator impact parameters. Each indicator follows a baseline stochastic process driven by discrete events with sigmoid-onset, exponential-decay kernels under saturation and projection constraints; a hybrid pipeline delegates sparse semantic artifacts to LLM-based planning and dense indicator dynamics to algorithmic simulation with hard physiological bounds. Users are each paired with 100 evaluation queries across five dimensions - Lookup, Trend, Comparison, Anomaly, Explanation - stratified into Easy, Medium, and Hard tiers, with all ground-truth answers programmatically computable from the recorded event-indicator relationships. Evaluating 13 methods spanning LLMs with tools, DB-native agents, and memory-augmented RAG, we find that DB agents (48-58%) substantially outperform memory RAG baselines (30-38%), with the gap concentrated on Comparison and Explanation queries where multi-hop reasoning and evidence attribution are required.

Related papers

• ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation [65.39532118490511]
  ProMediConv is a novel benchmarking framework that models mediation as a proactive, multi-stage, and party-aware dialogue process.<n>Using 972 complete real-world cases, we construct a high-fidelity mediation dataset with utterance-level annotations of strategies and BP states.<n>We also propose MAD (Mean Attribute Difference), a fine-grained metric that captures BP shifts throughout the dialogue.
  arXiv  Detail & Related papers  (2026-09-10T05:26:40Z)
• WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data [37.47499553332249]
  WearableQA is a benchmark for evaluating AI systems' reasoning over a real user's longitudinal wearable record.<n>We introduce 16 question types organized along two complementary axes: data versus health reasoning, and single- versus cross-signal reasoning.<n> Evaluation of 14 proprietary and open-source LLMs demonstrates that WearableQA effectively differentiates model capabilities, with performance ranging from 19.6% to 72.9% against a 10% chance baseline.
  arXiv  Detail & Related papers  (2026-09-04T17:52:37Z)
• DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflows [69.12339622053588]
  We introduce DuMateBench, a real-session benchmark reconstructed from anonymized and privacy-screened user sessions.<n>The resulting benchmark comprises 200 tasks spanning 8 broad scenarios and 17 fine-grained capability categories.<n>We execute these tasks in isolated Docker containers injected with three forms of real-world environmental complexity: Insufficient, Unstable, and Noisy.
  arXiv  Detail & Related papers  (2026-08-27T02:36:23Z)
• Synergistic Perception-Reasoning Governance: Grounding Medical MLLMs with Verifiable Anatomical Evidence [45.78409219344429]
  Multimodal large language models (MLLMs) show strong promise for clinical VQA and radiology report generation, yet inference-time hallucinations undermine trustworthy use.<n>We propose a holistic, training-free evidence-injection framework that systematically mitigates hallucinations through dual-side evidence injection.<n>We conduct systematic evaluations on 2 tasks and 5 datasets using textttLLaVA-1.5-7B, texttLLaVA-Med-1.5-7B, texttQwen3-VL-8B/32B, and textttInternVL
  arXiv  Detail & Related papers  (2026-06-30T08:07:25Z)
• Influence-Guided Symbolic Regression: Scientific Discovery via LLM-Driven Equation Search with Granular Feedback [56.69850045068714]
  We introduce textitInfluence-Guided Regression (IGSR), a method that frames equation discovery as an iterative two-step process.<n>We demonstrate IGSRs effectiveness on a diverse suite of benchmarks, including LLM-SRBench, pharmacological PKPD models, an epidemiological simulation, and real-world genomic data.
  arXiv  Detail & Related papers  (2026-05-27T23:48:01Z)
• Query-Conditioned Graph Retrieval for Contextualized LLM Reasoning in Personalized Wearable Data [4.084164541001164]
  Wearable As Graph (WAG) organizes wearable metrics and user-specific signals into a personalized knowledge graph.<n>WAG retrieves a query-conditioned subgraph to support downstream generation.<n>We evaluate WAG on over 10,000 data-grounded queries from real-world wearable datasets.
  arXiv  Detail & Related papers  (2026-04-10T13:13:17Z)
• Can LLM Agents Generate Real-World Evidence? Evaluating Observational Studies in Medical Databases [17.35673829214932]
  We introduce RWE-bench, a benchmark grounded in MIMIC-IV and derived from peer-reviewed observational studies.<n>Each task provides the corresponding study protocol as the reference standard, requiring agents to execute experiments in a real database.<n>Across 162 tasks, task success is low: the best agent reaches 39.9%, and the best open-source model reaches 30.4%.
  arXiv  Detail & Related papers  (2026-03-24T03:50:34Z)
• Do Generative Metrics Predict YOLO Performance? An Evaluation Across Models, Augmentation Ratios, and Dataset Complexity [43.338311770275745]
  We present a controlled evaluation of synthetic augmentation for YOLOv11 across three single-class detection regimes.<n>We benchmark six GAN-, diffusion-, and hybrid-based generators over augmentation ratios from 10% to 150% of the real training split.<n>For each dataset-generator-augmentation configuration, we compute pre-training dataset metrics under a matched-size bootstrap protocol.
  arXiv  Detail & Related papers  (2026-02-20T03:02:36Z)
• AgentLongBench: A Controllable Long Benchmark For Long-Contexts Agents via Environment Rollouts [78.33143446024485]
  We introduce textbfAgentLongBench, which evaluates agents through simulated environment rollouts based on Lateral Thinking Puzzles.<n>This framework generates rigorous interaction trajectories across knowledge-intensive and knowledge-free scenarios.
  arXiv  Detail & Related papers  (2026-01-28T16:05:44Z)
• Real-Time Health Analytics Using Ontology-Driven Complex Event Processing and LLM Reasoning: A Tuberculosis Case Study [4.0954316720608634]
  This study presents an ontology-enabled real-time analytics framework that integrates Complex Event Processing (CEP) and Large Language Models (LLMs)<n>Patient data is ingested and processed using Apache Kafka and Spark Streaming, where CEP engines detect clinically significant event patterns.<n>The framework is evaluated using a dataset of 1,000 Tuberculosis (TB) patients as a use case, demonstrating low-latency event detection, scalable reasoning, and high model performance.
  arXiv  Detail & Related papers  (2025-10-05T14:21:46Z)
• A Graph-Based Test-Harness for LLM Evaluation [0.8164433158925593]
  We present a first known prototype of a dynamic, systematic benchmark of medical guidelines for 400+ questions.<n>We transform the WHO IMCI handbook into a directed graph with 200+ nodes and generate questions that incorporate age-specific scenarios.<n>We find models excel at symptom recognition but struggle with triaging severity, treatment protocols and follow-up care.
  arXiv  Detail & Related papers  (2025-08-28T14:10:59Z)
• Benchmarking Deep Search over Heterogeneous Enterprise Data [73.55304268238474]
  We present a new benchmark for evaluating a form of retrieval-augmented generation (RAG)<n>RAG requires source-aware, multi-hop reasoning over diverse, sparsed, but related sources.<n>We build it using a synthetic data pipeline that simulates business across product planning, development, and support stages.
  arXiv  Detail & Related papers  (2025-06-29T08:34:59Z)
• A Sample Efficient Conditional Independence Test in the Presence of Discretization [54.047334792855345]
  Conditional Independence (CI) tests directly to discretized data can lead to incorrect conclusions.<n>Recent advancements have sought to infer the correct CI relationship between the latent variables through binarizing observed data.<n>Motivated by this, this paper introduces a sample-efficient CI test that does not rely on the binarization process.
  arXiv  Detail & Related papers  (2025-06-10T12:41:26Z)
• Human-inspired Episodic Memory for Infinite Context LLMs [20.265521531429943]
  Large language models (LLMs) have shown remarkable capabilities, but still struggle with processing extensive contexts.<n>We introduce EM-LLM, a novel approach that integrates key aspects of human episodic memory and event cognition into LLMs.<n> EM-LLM organises sequences of tokens into coherent episodic events using a combination of Bayesian surprise and graph-theoretic boundary refinement.
  arXiv  Detail & Related papers  (2024-07-12T17:34:03Z)
• Soft Random Sampling: A Theoretical and Empirical Analysis [59.719035355483875]
  Soft random sampling (SRS) is a simple yet effective approach for efficient deep neural networks when dealing with massive data. It selects a uniformly speed at random with replacement from each data set in each epoch. It is shown to be a powerful and competitive strategy with significant and competitive performance on real-world industrial scale.
  arXiv  Detail & Related papers  (2023-11-21T17:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.