ArogyaSutra: A Multi-Agent Framework for Multimodal Medical Reasoning in Indic Languages

• URL: http://arxiv.org/abs/2606.13572v1
• Date: Thu, 11 Jun 2026 16:59:42 GMT
• Title: ArogyaSutra: A Multi-Agent Framework for Multimodal Medical Reasoning in Indic Languages
• Abstract summary: We introduce ArogyaBodha, a large-scale multilingual medical question-answer dataset constructed from eight sources.<n>We propose ArogyaSutra, a framework that integrates tool grounding with dual-memory mechanisms for step-wise, reasoning-aware decision making.<n>Experiments show that our dataset and framework improve multilingual medical reasoning accuracy across all Indic languages.
• Score: 19.24164302861434
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal Large Language Models (MLLMs) have shown promising reasoning capabilities in general domains, yet their performance remains limited in specialized settings such as healthcare, especially in multilingual and low-resource scenarios. This gap is critical in regions like rural India, where patients often express complex medical queries in native Indic languages and rely on multimodal inputs such as medical images. Existing English-centric MLLMs struggle to support such use cases, limiting equitable access to AI-driven healthcare assistance. To address this challenge, we introduce ArogyaBodha, a large-scale multilingual multimodal medical question-answer dataset constructed from eight heterogeneous sources, covering 31 body systems, six imaging modalities, and 21 clinical domains across English and seven major Indian languages. We further propose ArogyaSutra, an actor-critic-based multi-agent framework that integrates tool grounding with dual-memory mechanisms for step-wise, reasoning-aware decision making, and uses stored actor-critic simulation trajectories for distillation. Experiments show that our dataset and framework improve multilingual medical reasoning accuracy across all Indic languages, with ablations validating the contribution of each component. The source code and dataset are available at: https://iitp-cse.github.io/ ArogyaSutra/

Related papers

• HealMed: Multilingual Evaluation of Large Language Models in Medicine [82.79641241632788]
  We present HealMed, an expert-reviewed benchmark for multilingual evaluation of large language models in medicine.<n>The benchmark was developed over two years by 23 physicians and medical experts based across nine countries and regions.<n>The strongest proprietary models were the most stable across languages, whereas many open-source and medically specialized models showed larger and less consistent gaps.
  arXiv  Detail & Related papers  (2026-08-20T12:52:09Z)
• IndicMedDialog: A Parallel Multi-Turn Medical Dialogue Dataset for Accessible Healthcare in Indic Languages [1.3141424119891407]
  IndicMedDialog is a parallel multi-turn medical dialogue dataset spanning English and nine Indic languages.<n>The dataset extends MDDial with LLM-generated synthetic consultations, translated using TranslateGemma, verified by native speakers.<n>We fine-tune IndicMedLM via parameter-efficient adaptation of a quantized small language model, incorporating optional patient pre-context.
  arXiv  Detail & Related papers  (2026-05-13T10:06:38Z)
• MultiMed: Multilingual Medical Speech Recognition via Attention Encoder Decoder [5.3903790635541515]
  We introduce MultiMed, the first multilingual medical ASR dataset, along with the first collection of small-to-large end-to-end medical ASR models.<n>To our best knowledge, MultiMed stands as the world's largest medical ASR dataset across all major benchmarks.<n>We present the first multilinguality study for medical ASR, which includes reproducible empirical baselines, a monolinguality-multilinguality analysis, Attention Decoder (AED) vs Hybrid comparative study and a linguistic analysis.
  arXiv  Detail & Related papers  (2024-09-21T09:05:48Z)
• Towards Building Multilingual Language Model for Medicine [54.1382395897071]
  We construct a multilingual medical corpus, containing approximately 25.5B tokens encompassing 6 main languages. We propose a multilingual medical multi-choice question-answering benchmark with rationale, termed as MMedBench. Our final model, MMed-Llama 3, with only 8B parameters, achieves superior performance compared to all other open-source models on both MMedBench and English benchmarks.
  arXiv  Detail & Related papers  (2024-02-21T17:47:20Z)
• MedEval: A Multi-Level, Multi-Task, and Multi-Domain Medical Benchmark for Language Model Evaluation [22.986061896641083]
  MedEval is a multi-level, multi-task, and multi-domain medical benchmark to facilitate the development of language models for healthcare. With 22,779 collected sentences and 21,228 reports, we provide expert annotations at multiple levels, offering a granular potential usage of the data.
  arXiv  Detail & Related papers  (2023-10-21T18:59:41Z)
• Mitigating Data Imbalance and Representation Degeneration in Multilingual Machine Translation [103.90963418039473]
  Bi-ACL is a framework that uses only target-side monolingual data and a bilingual dictionary to improve the performance of the MNMT model. We show that Bi-ACL is more effective both in long-tail languages and in high-resource languages.
  arXiv  Detail & Related papers  (2023-05-22T07:31:08Z)
• Intent Identification and Entity Extraction for Healthcare Queries in Indic Languages [16.39183257795811]
  Scarcity of data and technological limitations for resource-poor languages in developing countries like India poses a threat to the development of sophisticated NLU systems for healthcare. This paper studies the problem by proposing two different Healthcare datasets. Our aim is to detect query intents and extract corresponding entities.
  arXiv  Detail & Related papers  (2023-02-19T22:53:03Z)
• MULTI3NLU++: A Multilingual, Multi-Intent, Multi-Domain Dataset for Natural Language Understanding in Task-Oriented Dialogue [115.32009638844059]
  We extend the English only NLU++ dataset to include manual translations into a range of high, medium, and low resource languages. Because of its multi-intent property, MULTI3NLU++ represents complex and natural user goals. We use MULTI3NLU++ to benchmark state-of-the-art multilingual models for the Natural Language Understanding tasks of intent detection and slot labelling.
  arXiv  Detail & Related papers  (2022-12-20T17:34:25Z)
• A Review of Multilingualism in and for Ontologies [1.5736899098702972]
  This review seeks to elucidate and compare the modelling options for OWL, (i) examine extant for their multilingualism, and (ii) evaluate editors for their ability to manage multilinguality. For the DataBench and LOVBench repositories, we extracted production-level and accessible from BioPortal and LOVBench respectively, where most of them have only partial translations and they all use a labels approach only. Based on a set of nine tool requirements for managing multilingual-based editors, the assessment of seven stock-taking editors showed that there are significant gaps in tooling support.
  arXiv  Detail & Related papers  (2022-10-06T10:35:07Z)
• AM2iCo: Evaluating Word Meaning in Context across Low-ResourceLanguages with Adversarial Examples [51.048234591165155]
  We present AM2iCo, Adversarial and Multilingual Meaning in Context. It aims to faithfully assess the ability of state-of-the-art (SotA) representation models to understand the identity of word meaning in cross-lingual contexts. Results reveal that current SotA pretrained encoders substantially lag behind human performance.
  arXiv  Detail & Related papers  (2021-04-17T20:23:45Z)
• Crossing the Conversational Chasm: A Primer on Multilingual Task-Oriented Dialogue Systems [51.328224222640614]
  Current state-of-the-art ToD models based on large pretrained neural language models are data hungry. Data acquisition for ToD use cases is expensive and tedious.
  arXiv  Detail & Related papers  (2021-04-17T15:19:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.