Beyond Human Performance: A Vision-Language Multi-Agent Approach for Quality Control in Pharmaceutical Manufacturing

• URL: http://arxiv.org/abs/2602.20543v1
• Date: Tue, 24 Feb 2026 04:48:05 GMT
• Title: Beyond Human Performance: A Vision-Language Multi-Agent Approach for Quality Control in Pharmaceutical Manufacturing
• Abstract summary: Colony-forming unit (CFU) detection is critical in pharmaceutical manufacturing.<n>To address this, we developed a multi-agent framework combining deep learning (DL) with vision-language models (VLMs)<n>Initial DL-based automation reduced human verification by 50 percent across vaccine manufacturing sites.
• Score: 1.5362004217750165
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Colony-forming unit (CFU) detection is critical in pharmaceutical manufacturing, serving as a key component of Environmental Monitoring programs and ensuring compliance with stringent quality standards. Manual counting is labor-intensive and error-prone, while deep learning (DL) approaches, though accurate, remain vulnerable to sample quality variations and artifacts. Building on our earlier CNN-based framework (Beznik et al., 2020), we evaluated YOLOv5, YOLOv7, and YOLOv8 for CFU detection; however, these achieved only 97.08 percent accuracy, insufficient for pharmaceutical-grade requirements. A custom Detectron2 model trained on GSK's dataset of over 50,000 Petri dish images achieved 99 percent detection rate with 2 percent false positives and 0.6 percent false negatives. Despite high validation accuracy, Detectron2 performance degrades on outlier cases including contaminated plates, plastic artifacts, or poor optical clarity. To address this, we developed a multi-agent framework combining DL with vision-language models (VLMs). The VLM agent first classifies plates as valid or invalid. For valid samples, both DL and VLM agents independently estimate colony counts. When predictions align within 5 percent, results are automatically recorded in Postgres and SAP; otherwise, samples are routed for expert review. Expert feedback enables continuous retraining and self-improvement. Initial DL-based automation reduced human verification by 50 percent across vaccine manufacturing sites. With VLM integration, this increased to 85 percent, delivering significant operational savings. The proposed system provides a scalable, auditable, and regulation-ready solution for microbiological quality control, advancing automation in biopharmaceutical production.

Related papers

• Perception, Verdict, and Evolution: Hindsight-Driven Self-Refining Forensics Agent for AI-Generated Image Detection [80.15317858033534]
  ForeAgent is an agentic forensics framework for AI-generated image detection with iterative self-evolution.<n>ForeAgent achieves state-of-the-art performance on the Chameleon benchmark, reaching 82.18% accuracy.<n>ForeAgent produces more consistent and causally grounded reasoning compared to GPT-5 and GPT-5-mini.
  arXiv  Detail & Related papers  (2026-06-25T02:59:33Z)
• Fine-Tuning Vision-Language Models for Understanding Current Damage and Scoring Priority with Quality Guard Agent [0.0]
  Bridge inspection in Japan requires mandatory visual assessments every five years.<n> qualitative damage ratings (levels a-e) assigned by different engineers exhibit significant inter-rater variability.<n>This paper presents a methodology for automating bridge damage understanding and repair priority scoring.
  arXiv  Detail & Related papers  (2026-05-24T21:11:50Z)
• Trustworthy Federated Label Distribution Learning under Annotation Quality Disparity [70.60312607268871]
  FedQual is a quality-aware Fed-LDL framework with two mechanisms.<n>We construct four new Fed-LDL benchmarks with controlled annotation quality disparity.
  arXiv  Detail & Related papers  (2026-05-06T12:20:20Z)
• Auditing Frontier Vision-Language Models for Trustworthy Medical VQA: Grounding Failures, Format Collapse, and Domain Adaptation [4.177028541321909]
  We audit five vision-language models (VLMs) on Medical VQA along two trust-relevant axes.<n>Models localize anatomical and pathological targets poorly.<n>A self-grounding pipeline, where the same model localizes then answers, degrades VQA accuracy.
  arXiv  Detail & Related papers  (2026-04-30T11:11:47Z)
• OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language Environment Simulation [57.505743202759646]
  OccuBench is a benchmark covering 100 real-world professional task scenarios across 10 industry categories and 65 specialized domains.<n>Our multi-agent synthesis pipeline automatically produces evaluation instances with guaranteed solvability, calibrated difficulty, and document-grounded diversity.
  arXiv  Detail & Related papers  (2026-04-13T00:27:32Z)
• Measuring Mid-2025 LLM-Assistance on Novice Performance in Biology [0.0]
  Large language models (LLMs) perform strongly on biological benchmarks, raising concerns that they may help novice actors acquire dual-use laboratory skills.<n>We evaluated whether LLMs improve novice performance in tasks that collectively model a viral reverse genetics workflow.<n>LLMs did not substantially increase novice completion of complex laboratory procedures but were associated with a modest performance benefit.
  arXiv  Detail & Related papers  (2026-02-18T18:51:28Z)
• Process-Supervised Multi-Agent Reinforcement Learning for Reliable Clinical Reasoning [15.47321745394914]
  We introduce an agent-as-tool reinforcement learning framework for gene-disease validity curation.<n>One critical real-world case is gene-disease validity curation, where experts must determine whether a gene is causally implicated in a disease.<n>Our evaluation shows that with outcome-only rewards, MAS with a GRPO-trained supervisor agent substantially improves final outcome accuracy from 0.195 with a base model supervisor to 0.732.<n>With process + outcome rewards, MAS with GRPO-trained supervisor achieves higher outcome accuracy (0.750) while significantly improving process fidelity to 0.520 F1.
  arXiv  Detail & Related papers  (2026-02-15T14:21:21Z)
• Are vision-language models ready to zero-shot replace supervised classification models in agriculture? [0.8594140167290097]
  Vision models (VLMs) are proposed as general-purpose solutions for visual recognition tasks.<n>We benchmark a diverse set open and closed-source VLMs on 27 agricultural classification datasets from the AgML collection.
  arXiv  Detail & Related papers  (2025-12-17T21:22:44Z)
• AutoMalDesc: Large-Scale Script Analysis for Cyber Threat Research [81.04845910798387]
  Generating natural language explanations for threat detections remains an open problem in cybersecurity research.<n>We present AutoMalDesc, an automated static analysis summarization framework that operates independently at scale.<n>We publish our complete dataset of more than 100K script samples, including annotated seed (0.9K) datasets, along with our methodology and evaluation framework.
  arXiv  Detail & Related papers  (2025-11-17T13:05:25Z)
• From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations [45.414878840652115]
  Large language models (LLMs) have demonstrated promising performance on medical benchmarks.<n>However, their ability to perform medical calculations remains underexplored and poorly evaluated.<n>In this work, we revisit medical calculation evaluation with a stronger focus on clinical trustworthiness.
  arXiv  Detail & Related papers  (2025-09-20T09:10:26Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)
• Beyond Benchmarks: Dynamic, Automatic And Systematic Red-Teaming Agents For Trustworthy Medical Language Models [87.66870367661342]
  Large language models (LLMs) are used in AI applications in healthcare.<n>Red-teaming framework that continuously stress-test LLMs can reveal significant weaknesses in four safety-critical domains.<n>A suite of adversarial agents is applied to autonomously mutate test cases, identify/evolve unsafe-triggering strategies, and evaluate responses.<n>Our framework delivers an evolvable, scalable, and reliable safeguard for the next generation of medical AI.
  arXiv  Detail & Related papers  (2025-07-30T08:44:22Z)
• Rethinking Verification for LLM Code Generation: From Generation to Testing [44.46778801679273]
  Large language models (LLMs) have recently achieved notable success in code-generation benchmarks such as HumanEval and LiveCodeBench.<n>We propose a new multi-dimensional metrics designed to rigorously quantify test-suite.<n> Experiments show that SAGA achieves a detection rate of 90.62% and a verifier accuracy of 32.58% on TCGBench.
  arXiv  Detail & Related papers  (2025-07-09T14:58:47Z)
• Semi-Supervised Defect Detection via Conditional Diffusion and CLIP-Guided Noise Filtering [8.132909775584395]
  This paper introduces a semi-supervised defect detection framework based on conditional diffusion (DSYM)<n>A conditional diffusion model synthesizes multi-scale pseudo-defect samples, while a CLIP cross-modal feature-based noise filtering mechanism mitigates label contamination.<n>This research provides a high-precision, low-labeling-dependent solution for defect detection in industrial quality inspection scenarios.
  arXiv  Detail & Related papers  (2025-07-08T01:53:34Z)
• Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models [42.958880063727996]
  CDD stands for Contamination Detection via output Distribution for LLMs. To mitigate the impact of data contamination in evaluation, we also present TED: Trustworthy Evaluation via output Distribution.
  arXiv  Detail & Related papers  (2024-02-24T23:54:41Z)
• Multi-step Problem Solving Through a Verifier: An Empirical Analysis on Model-induced Process Supervision [40.984680166762345]
  We introduce Model-induced Process Supervision (MiPS), a novel method for automating data curation. MiPS annotates an intermediate step by sampling completions of this solution through the reasoning model, and obtaining an accuracy defined as the proportion of correct completions. Our approach significantly improves the performance of PaLM 2 on math and coding tasks.
  arXiv  Detail & Related papers  (2024-02-05T00:57:51Z)
• Conservative Prediction via Data-Driven Confidence Minimization [70.93946578046003]
  In safety-critical applications of machine learning, it is often desirable for a model to be conservative. We propose the Data-Driven Confidence Minimization framework, which minimizes confidence on an uncertainty dataset.
  arXiv  Detail & Related papers  (2023-06-08T07:05:36Z)
• Synthetic Data Augmentation Using GAN For Improved Automated Visual Inspection [0.440401067183266]
  State-of-the-art unsupervised defect detection does not match the performance of supervised models. Best classification performance was achieved considering GAN-based data generation with AUC ROC scores equal to or higher than 0,9898.
  arXiv  Detail & Related papers  (2022-12-19T09:31:15Z)
• Recognition of Defective Mineral Wool Using Pruned ResNet Models [88.24021148516319]
  We developed a visual quality control system for mineral wool. X-ray images of wool specimens were collected to create a training set of defective and non-defective samples. We obtained a model with more than 98% accuracy, which in comparison to the current procedure used at the company, it can recognize 20% more defective products.
  arXiv  Detail & Related papers  (2022-11-01T13:58:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.