Trust, Safety, and Accuracy: Assessing LLMs for Routine Maternity Advice

• URL: http://arxiv.org/abs/2603.16872v1
• Date: Mon, 12 Jan 2026 13:48:22 GMT
• Title: Trust, Safety, and Accuracy: Assessing LLMs for Routine Maternity Advice
• Abstract summary: This study evaluates large language models (LLMs) like ChatGPT-4o, Perplexity AI, and GeminiAI to provide reliable and understandable pregnancy-related information.<n> Evaluations used semantic similarity, noun overlap, and readability metrics to measure content quality.<n>Results show Perplexity closely matched expert semantics, while ChatGPT-4o produced clearer, more understandable text with better medical terminology.
• Score: 0.8749675983608171
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Access to reliable maternal healthcare information is a major challenge in rural India due to limited medical resources and infrastructure. With over 830 million internet users and nearly half of rural women online, digital tools offer new opportunities for health education. This study evaluates large language models (LLMs) like ChatGPT-4o, Perplexity AI, and GeminiAI to provide reliable and understandable pregnancy-related information. Seventeen pregnancy-focused questions were posed to each model and compared with responses from maternal health professionals. Evaluations used semantic similarity, noun overlap, and readability metrics to measure content quality. Results show Perplexity closely matched expert semantics, while ChatGPT-4o produced clearer, more understandable text with better medical terminology. As internet access grows in rural areas, LLMs could serve as scalable aids for maternal health education. The study highlights the need for AI tools that balance accuracy and clarity to improve healthcare communication in underserved regions.

Related papers

• Comparative Analysis of Large Language Models in Generating Telugu Responses for Maternal Health Queries [0.8749675983608171]
  Large Language Models (LLMs) have been progressively exhibiting there capabilities in various areas of research.<n>This study presents how ChatGPT-4o, GeminiAI, and Perplexity AI respond to pregnancy related questions asked in different languages.
  arXiv  Detail & Related papers  (2026-03-19T13:39:38Z)
• Developing and evaluating a chatbot to support maternal health care [20.297280325309924]
  We present a bot for maternal health in India developed through a partnership between academic researchers, a health tech company, a public health nonprofit, and a hospital.<n>The system combines (1) stage-aware triage, routing high-risk queries to expert templates, (2) hybrid retrieval over curated maternal/newborn guidelines, and (3) evidence-conditioned generation from an LLM.<n>Our core contribution is an evaluation workflow for high-stakes deployment under limited expert supervision.
  arXiv  Detail & Related papers  (2026-03-13T17:02:05Z)
• Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning [57.873833577058]
  We build a multimodal dataset enriched with extensive medical knowledge.<n>We then introduce our medical-specialized MLLM: Lingshu.<n>Lingshu undergoes multi-stage training to embed medical expertise and enhance its task-solving capabilities.
  arXiv  Detail & Related papers  (2025-06-08T08:47:30Z)
• Scope of Online Maternal Health Information in Kinyarwanda and Opportunities for Digital Health Developers [0.0]
  There is an increasing proliferation of websites dedicated to maternal health education, but the scope and quality of their content varies widely.<n>This study analyzed the main topics covered on maternal health websites that offer content in the low-resource Kinyarwanda language.<n>We found five main topics: pregnancy danger signs, (2) child care, (3) intimacy (sex), (4) nutrition, and (5) the importance of doctor consultations.
  arXiv  Detail & Related papers  (2025-04-04T09:55:23Z)
• Edge AI for Real-time Fetal Assessment in Rural Guatemala [5.360772430004525]
  Perinatal complications represent a significant burden on maternal and neonatal health worldwide.<n>We have developed an AI-enabled smartphone application designed to provide decision support at the point-of-care.<n>This tool aims to enhance health monitoring during pregnancy by leveraging machine learning (ML) techniques.
  arXiv  Detail & Related papers  (2025-03-12T12:31:07Z)
• GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI [67.09501109871351]
  Large Vision-Language Models (LVLMs) are capable of handling diverse data types such as imaging, text, and physiological signals. GMAI-MMBench is the most comprehensive general medical AI benchmark with well-categorized data structure and multi-perceptual granularity to date. It is constructed from 284 datasets across 38 medical image modalities, 18 clinical-related tasks, 18 departments, and 4 perceptual granularities in a Visual Question Answering (VQA) format.
  arXiv  Detail & Related papers  (2024-08-06T17:59:21Z)
• AI as a Medical Ally: Evaluating ChatGPT's Usage and Impact in Indian Healthcare [2.259877069661293]
  This study investigates the integration and impact of Large Language Models (LLMs), like ChatGPT, in India's healthcare sector. Our findings reveal that healthcare professionals value ChatGPT in medical education and preliminary clinical settings, but exercise caution due to concerns about reliability, privacy, and the need for cross-verification with medical references. General users show a preference for AI interactions in healthcare, but concerns regarding accuracy and trust persist.
  arXiv  Detail & Related papers  (2024-01-28T08:20:36Z)
• ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences [51.66185471742271]
  We propose ChiMed-GPT, a benchmark LLM designed explicitly for Chinese medical domain. ChiMed-GPT undergoes a comprehensive training regime with pre-training, SFT, and RLHF. We analyze possible biases through prompting ChiMed-GPT to perform attitude scales regarding discrimination of patients.
  arXiv  Detail & Related papers  (2023-11-10T12:25:32Z)
• Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answering [48.17095875619711]
  We present a system called LLMs Augmented with Medical Textbooks (LLM-AMT)<n>LLM-AMT integrates authoritative medical textbooks into the LLMs' framework using plug-and-play modules.<n>We found that medical textbooks as a retrieval corpus is proven to be a more effective knowledge database than Wikipedia in the medical domain.
  arXiv  Detail & Related papers  (2023-09-05T13:39:38Z)
• MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records [60.35217378132709]
  Large language models (LLMs) can follow natural language instructions with human-level fluency. evaluating LLMs on realistic text generation tasks for healthcare remains challenging. We introduce MedAlign, a benchmark dataset of 983 natural language instructions for EHR data.
  arXiv  Detail & Related papers  (2023-08-27T12:24:39Z)
• ChatCAD+: Towards a Universal and Reliable Interactive CAD using LLMs [48.11532667875847]
  ChatCAD+ is a tool to generate high-quality medical reports and provide reliable medical advice. The Reliable Report Generation module is capable of interpreting medical images and generate high-quality medical reports. The Reliable Interaction module leverages up-to-date information from reputable medical websites to provide reliable medical advice.
  arXiv  Detail & Related papers  (2023-05-25T12:03:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.