Secure On-Premise Deployment of Open-Weights Large Language Models in Radiology: An Isolation-First Architecture with Prospective Pilot Evaluation

• URL: http://arxiv.org/abs/2604.22768v1
• Date: Wed, 25 Mar 2026 17:38:20 GMT
• Title: Secure On-Premise Deployment of Open-Weights Large Language Models in Radiology: An Isolation-First Architecture with Prospective Pilot Evaluation
• Abstract summary: The proposed isolation-first on-premise architecture enabled overcoming regulatory borders.<n>The system served the open-weights DeepSeek-R1 model via vLLM.
• Score: 29.427162122010216
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Purpose: To design, implement, evaluate, and report on the regulatory requirements of a self-hosted LLM infrastructure for radiology adhering to the principle of least privilege, emphasizing technical feasibility, network isolation, and clinical utility. Materials and Methods: The isolation-first, containerized LLM inference stack relies on strict network segmentation, host-enforced egress filtering, and active isolation monitoring preventing unauthorized external connectivity. An accompanying deployment package provides automated isolation and hardening tests. The system served the open-weights DeepSeek-R1 model via vLLM. In a one-week pilot phase, 22 residents and radiologists were free to use 10 predefined prompt-templates whenever they considered them useful in daily work. Afterward, they rated clinical utility and system stability on an 0-10 Likert scale and reported observed critical errors in model output. Results: The applied institutional governance pathway achieved approval from clinic management, compliance, data protection and information security officers for processing unanonymized PHI. The system was rated stable and user friendly during the pilot. Source text-anchored tasks, such as report corrections or simplifications, and radiology guideline recommendations received the highest utility ratings, whereas open-ended conclusion generation based on findings resulted in the highest frequency of critical errors, such as clinically relevant hallucinations or omissions. Conclusion: The proposed isolation-first on-premise architecture enabled overcoming regulatory borders, showed promising clinical utility in text-anchored tasks and is the current base to serve open-weights LLMs as an official service of a German University Hospital with over 10,000 employees. The deployment package were made publicly available (https://github.com/ukbonn/ukb-gpt).

Related papers

• FMRP-LEAN: A HIPAA-Compliant AI-Augmented LIMS Architecture for End-to-End Clinical Assay Workflow Optimization [0.0]
  This paper presents FMRP-LEAN, a HIPAA-compliant, AI-augmented Laboratory Information Management System (LIMS) architecture that formalizes bio-specimen lifecycle management through a finite-state workflow model with explicit transition guards and dwell-time observability.
  arXiv  Detail & Related papers  (2026-07-22T17:11:52Z)
• Language Models as Interfaces, Not Oracles: A Hybrid LLM-ML System for Pediatric Appendicitis [1.0312968200748116]
  ClaMPAPP (Clinical Language-assisted Machine-learning Pipeline for Appendicitis) is a hybrid system that uses an LLM as an interface rather than as the final decision-maker.<n>We evaluated ClaMPAPP on two independent pediatric appendicitis cohorts from German hospitals.
  arXiv  Detail & Related papers  (2026-06-17T15:22:51Z)
• MedCTA: A Benchmark for Clinical Tool Agents [47.47354499871572]
  We introduce MedCTA, a benchmark for evaluating medical tool agents on clinician-validated, step-implicit tasks.<n>MedCTA comprises 107 real-world clinical tasks with clinician-verified executable trajectories over 5 deployed tools.<n>We benchmark 18 open- and closed-source multimodal models and find that even frontier systems remain brittle in multi-step clinical tool use.
  arXiv  Detail & Related papers  (2026-06-10T06:26:52Z)
• Automated Clinical Report Generation for Remote Cognitive Remediation: Comparing Knowledge-Engineered Templates and LLMs in Low-Resource Settings [4.406830461180521]
  This paper investigates automated clinical report generation for avatar-guided, home-based cognitive remediation sessions in a low-resource setting with no reference reports.<n>We present and compare two approaches: (1) a rule-based template system encoding speech therapy domain knowledge as explicit decision rules and validated templates; and (2) a zero-shot LLM-based approach (GPT-4) aimed at more fluent and concise output.<n>Results reveal a clear trade-off between clinical reliability and linguistic quality. The template-based system scored higher on fluidity, coherence, and results presentation, while GPT-4 produced more concise output.
  arXiv  Detail & Related papers  (2026-05-07T17:20:22Z)
• Statistics, Not Scale: Modular Medical Dialogue with Bayesian Belief Engine [51.722324399751294]
  Large language models are increasingly deployed as autonomous diagnostic agents, yet they conflate two fundamentally different capabilities.<n>We introduce BMBE, a modular diagnostic dialogue framework that enforces a strict separation between language and reasoning.
  arXiv  Detail & Related papers  (2026-04-21T21:59:57Z)
• Toward Zero-Egress Psychiatric AI: On-Device LLM Deployment for Privacy-Preserving Mental Health Decision Support [3.560737321459875]
  Privacy represents one of the most critical yet under barriers to AI adoption in mental healthcare.<n>Existing AI-enabled psychiatric decision support systems rely on cloud-based inference pipelines.<n>We propose a zero-egress, on-device AI platform for privacy-preserving psychiatric decision support.
  arXiv  Detail & Related papers  (2026-04-20T14:09:01Z)
• Adoption and Effectiveness of AI-Based Anomaly Detection for Cross Provider Health Data Exchange [0.0]
  The study proposes a staged deployment strategy combining rules for coverage and machine learning for prioritisation, supported by explainability and continuous monitoring.<n>Results show that rule-based methods achieve high recall but generate higher alert volumes, while Isolation Forest reduces alert burden at the cost of lower sensitivity.
  arXiv  Detail & Related papers  (2026-03-19T15:22:02Z)
• PREBA: Surgical Duration Prediction via PCA-Weighted Retrieval-Augmented LLMs and Bayesian Averaging Aggregation [51.96735866702332]
  PREBA is a retrieval-augmented framework that integrates PCA-weighted retrieval and Bayesian averaging aggregation.<n> PREBA significantly improves performance-for instance, reducing MAE by up to 40% and raising R2 from -0.13 to 0.62 over zero-shot inference.
  arXiv  Detail & Related papers  (2026-02-27T07:19:23Z)
• ClinAlign: Scaling Healthcare Alignment from Clinician Preference [46.61989657665566]
  We introduce HealthRubrics, a dataset of 7,034 physician-verified preference examples.<n>We distill these rubrics into HealthPrinciples: 119 broadly reusable, clinically grounded principles.<n>A 30B-A3B model trained with our framework achieves 33.4% on HealthBench-Hard.
  arXiv  Detail & Related papers  (2026-02-10T11:02:57Z)
• AgentScore: Autoformulation of Deployable Clinical Scoring Systems [45.88028371034407]
  We introduce AgentScore, which performs semantically guided optimization in unit-weighted clinical checklists.<n>AgentScore outperforms existing score-generation methods and achieves AUC comparable to more flexible interpretable models.<n>On two additional externally validated tasks, AgentScore achieves higher discrimination than established guideline-based scores.
  arXiv  Detail & Related papers  (2026-01-29T21:11:06Z)
• A Federated and Parameter-Efficient Framework for Large Language Model Training in Medicine [59.78991974851707]
  Large language models (LLMs) have demonstrated strong performance on medical benchmarks, including question answering and diagnosis.<n>Most medical LLMs are trained on data from a single institution, which faces limitations in generalizability and safety in heterogeneous systems.<n>We introduce the model-agnostic and parameter-efficient federated learning framework for adapting LLMs to medical applications.
  arXiv  Detail & Related papers  (2026-01-29T18:48:21Z)
• Benchmarking Egocentric Clinical Intent Understanding Capability for Medical Multimodal Large Language Models [48.95516224614331]
  We introduce MedGaze-Bench, the first benchmark leveraging clinician gaze as a Cognitive Cursor to assess intent understanding across surgery, emergency simulation, and diagnostic interpretation.<n>Our benchmark addresses three fundamental challenges: visual homogeneity of anatomical structures, strict temporal-causal dependencies in clinical, and implicit adherence to safety protocols.
  arXiv  Detail & Related papers  (2026-01-11T02:20:40Z)
• Generalist Large Language Models Outperform Clinical Tools on Medical Benchmarks [1.2773749417703923]
  Generalist models consistently outperformed clinical tools.<n>OpenEvidence and UpToDate Expert AI demonstrated deficits in completeness, communication quality, context awareness, and systems-based safety reasoning.
  arXiv  Detail & Related papers  (2025-12-01T02:14:43Z)
• Toward Reliable Clinical Coding with Language Models: Verification and Lightweight Adaptation [3.952186976672079]
  We show that lightweight interventions, including prompt engineering and small-scale fine-tuning, can improve accuracy without the computational overhead of search-based methods.<n>To address hierarchically near-miss errors, we introduce clinical code verification as both a standalone task and a pipeline component.
  arXiv  Detail & Related papers  (2025-10-08T23:50:58Z)
• MedPAO: A Protocol-Driven Agent for Structuring Medical Reports [0.13029689752120577]
  We introduce MedPAO, a novel agentic framework that ensures accuracy and verifiable reasoning.<n> MedPAO decomposes the report structuring task into a transparent process managed by a Plan-Act-Observe (PAO) loop and specialized tools.<n>The efficacy of our approach is demonstrated through rigorous evaluation: MedPAO achieves an F1-score of 0.96 on the critical sub-task of concept categorization.
  arXiv  Detail & Related papers  (2025-10-06T09:32:23Z)
• Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases [48.87360916431396]
  We introduce MedR-Bench, a benchmarking dataset of 1,453 structured patient cases, annotated with reasoning references.<n>We propose a framework encompassing three critical examination recommendation, diagnostic decision-making, and treatment planning, simulating the entire patient care journey.<n>Using this benchmark, we evaluate five state-of-the-art reasoning LLMs, including DeepSeek-R1, OpenAI-o3-mini, and Gemini-2.0-Flash Thinking, etc.
  arXiv  Detail & Related papers  (2025-03-06T18:35:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.