Designing Safety-Constrained LLM Systems for Public Health Information Access

• URL: http://arxiv.org/abs/2607.13038v1
• Date: Thu, 11 Jun 2026 21:06:55 GMT
• Title: Designing Safety-Constrained LLM Systems for Public Health Information Access
• Abstract summary: We present the design and implementation of a safety constrained large language model (LLM) system for public health information access.<n>We introduce a multi-layered architecture that integrates domain-restricted retrieval augmented generation (RAG), strict boundary enforcement, and comprehensive audit logging for monitoring and compliance.<n>Results show consistent enforcement of safety constraints, reliable resource grounding, and stable system performance, with an average response time of 5.3 seconds.
• Score: 7.505217332626521
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present the design and implementation of a safety constrained large language model (LLM) system for public health information access, focusing on maternal and child health (MCH) resource navigation. While LLM based systems offer flexible and natural interfaces for information retrieval, their deployment in healthcare contexts introduces risks related to safety, trust, and uncontrolled generation. This work explores practical design patterns for constraining LLM behavior in safety critical environments. We introduce a multi-layered architecture that integrates domain-restricted retrieval augmented generation (RAG), strict boundary enforcement to prevent medical advice, anonymous multiuser session management, and comprehensive audit logging for monitoring and compliance. A key aspect of the design is a controlled data pipeline that grounds all responses in curated public health resources, avoiding reliance on the model pretrained medical knowledge. We implement the system in a real world public health setting and conduct scenario-based validation across in scope, out of scope, and emergency queries. Results show consistent enforcement of safety constraints, reliable resource grounding, and stable system performance, with an average response time of 5.3 seconds. Beyond the specific application, we discuss design trade offs and lessons learned in balancing safety, usability, and system flexibility. Our findings provide practical guidance for deploying LLM based systems in healthcare and other domains where strict information boundaries and accountability are required.

Related papers

• Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability [65.56750741053266]
  Large language models (LLMs) have transformed misinformation from a content-centric problem into a broader ecosystem-level security challenge.<n>In this paper, we introduce a role-layer framework to unify these risks and defenses.<n>The role characterizes LLMs as attackers, defenders, and vulnerable components of verification systems, while the layer dimension covers content, social contexts, evidence environments, and verification.
  arXiv  Detail & Related papers  (2026-07-11T17:04:17Z)
• SoK: Privacy-aware LLM in Healthcare: Threat Model, Privacy Techniques, Challenges and Recommendations [0.6533091401094101]
  Large Language Models (LLMs) are increasingly adopted in healthcare to support clinical decision-making and enhance patient care.<n>This work examines the evolving threat landscape across the three core LLM phases: Data preprocessing, Fine-tuning, and Inference within realistic healthcare settings.<n>We present a detailed threat model that characterizes adversaries, capabilities, and attack surfaces at each phase, and we systematize how existing privacy-preserving techniques (PPTs) attempt to mitigate these vulnerabilities.
  arXiv  Detail & Related papers  (2026-01-15T02:28:57Z)
• Safely Learning Controlled Stochastic Dynamics [61.82896036131116]
  We introduce a method that ensures safe exploration and efficient estimation of system dynamics.<n>After training, the learned model enables predictions of the system's dynamics and permits safety verification of any given control.<n>We provide theoretical guarantees for safety and derive adaptive learning rates that improve with increasing Sobolev regularity of the true dynamics.
  arXiv  Detail & Related papers  (2025-06-03T11:17:07Z)
• LLM Agents Should Employ Security Principles [60.03651084139836]
  This paper argues that the well-established design principles in information security should be employed when deploying Large Language Model (LLM) agents at scale.<n>We introduce AgentSandbox, a conceptual framework embedding these security principles to provide safeguards throughout an agent's life-cycle.
  arXiv  Detail & Related papers  (2025-05-29T21:39:08Z)
• LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey [34.275920463375684]
  Large language models (LLMs) have sparked growing interest in building fully autonomous agents.<n>LLM-HAS incorporate human-provided information, feedback, or control into the agent system to enhance system performance, reliability and safety.<n>This paper provides the first comprehensive and structured survey of LLM-HAS.
  arXiv  Detail & Related papers  (2025-05-01T08:29:26Z)
• Open Problems in Machine Unlearning for AI Safety [61.43515658834902]
  Machine unlearning -- the ability to selectively forget or suppress specific types of knowledge -- has shown promise for privacy and data removal tasks.<n>In this paper, we identify key limitations that prevent unlearning from serving as a comprehensive solution for AI safety.
  arXiv  Detail & Related papers  (2025-01-09T03:59:10Z)
• Guiding IoT-Based Healthcare Alert Systems with Large Language Models [22.54714587190204]
  Healthcare alert systems (HAS) are undergoing rapid evolution, propelled by advancements in artificial intelligence (AI), Internet of Things (IoT) technologies, and increasing health consciousness. Despite significant progress, a fundamental challenge remains: balancing the accuracy of personalized health alerts with stringent privacy protection in HAS environments constrained by resources. We introduce a uniform framework, LLM-HAS, which incorporates Large Language Models (LLM) into HAS to significantly boost the accuracy, ensure user privacy, and enhance personalized health service.
  arXiv  Detail & Related papers  (2024-08-23T13:55:36Z)
• Current state of LLM Risks and AI Guardrails [0.0]
  Large language models (LLMs) have become increasingly sophisticated, leading to widespread deployment in sensitive applications where safety and reliability are paramount. These risks necessitate the development of "guardrails" to align LLMs with desired behaviors and mitigate potential harm. This work explores the risks associated with deploying LLMs and evaluates current approaches to implementing guardrails and model alignment techniques.
  arXiv  Detail & Related papers  (2024-06-16T22:04:10Z)
• Trustworthy and Practical AI for Healthcare: A Guided Deferral System with Large Language Models [1.2281181385434294]
  Large language models (LLMs) offer a valuable technology for various applications in healthcare.<n>Their tendency to hallucinate and the existing reliance on proprietary systems pose challenges in environments concerning critical decision-making.<n>This paper presents a novel HAIC guided deferral system that can simultaneously parse medical reports for disorder classification, and defer uncertain predictions with intelligent guidance to humans.
  arXiv  Detail & Related papers  (2024-06-11T12:41:54Z)
• Empowering Autonomous Driving with Large Language Models: A Safety Perspective [82.90376711290808]
  This paper explores the integration of Large Language Models (LLMs) into Autonomous Driving systems. LLMs are intelligent decision-makers in behavioral planning, augmented with a safety verifier shield for contextual safety learning. We present two key studies in a simulated environment: an adaptive LLM-conditioned Model Predictive Control (MPC) and an LLM-enabled interactive behavior planning scheme with a state machine.
  arXiv  Detail & Related papers  (2023-11-28T03:13:09Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)
• Towards Compliant Data Management Systems for Healthcare ML [6.057289837472806]
  We review how data flows within machine learning projects in healthcare from source to storage to use in training algorithms and beyond. Our objective is to design tools to detect and track sensitive data across machines and users across the life cycle of a project. We build a prototype of the solution that demonstrates the difficulties in this domain.
  arXiv  Detail & Related papers  (2020-11-15T15:27:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.