Zero-Shot Respiratory Sound Classification through LLM-Augmented Audio-Text Alignment

• URL: http://arxiv.org/abs/2609.00055v1
• Date: Sun, 30 Aug 2026 11:49:54 GMT
• Title: Zero-Shot Respiratory Sound Classification through LLM-Augmented Audio-Text Alignment
• Abstract summary: Self-supervised respiratory encoders lack semantic grounding in clinical domain needed for zero-shot inference.<n>We propose a framework that aligns these encoders with medical terminology in a shared latent space turning them into a zero-shot-capable foundation model.
• Score: 29.08356862626618
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervised respiratory encoders lack semantic grounding in clinical domain needed for zero-shot inference, limiting their utility without task-specific labeled data. We propose a framework that aligns these encoders with medical terminology in a shared latent space turning them into a zero-shot-capable foundation model. To address paired data scarcity, we use a medical LLM to synthesize structured reports from metadata, creating dense semantic anchors for contrastive learning. Our training combines a sigmoid-based contrastive loss with encoder's native SSL objective and similarity-aware negative sampling to sharpen pathological boundaries. Across 9 tasks on 6 datasets, our method achieves a 61.3% mean zero-shot AUC, surpassing CLAP (51.4%) and Qwen2-Audio (54.9%) while reaching the highest linear probing AUC (71.6%) with only 43% of data used by full-scale baselines, showing that structured semantic alignment outperforms large-scale, general-purpose models in clinical diagnostics.

Related papers

• Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI [0.0]
  Deep networks trained on structural MRI for Alzheimer's disease often reach reasonable accuracy while attending to anatomically irrelevant regions.<n>We use FastSurfer segmentations as an anatomical reference.<n>We adapt a CLIP-style image and organize ADNIMERGE variables along a label-leakage spectrum.
  arXiv  Detail & Related papers  (2026-09-14T17:13:12Z)
• Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography [75.7789001619107]
  Vision-language pre-training holds great promise for general-purpose medical AI.<n>We propose a tailored framework featuring three key components.<n>A diagnosis-aware prompt strategy employs real clinical phrases and aggregated disease prototypes to bridge the pre-training-inference gap.
  arXiv  Detail & Related papers  (2026-06-24T08:24:45Z)
• Exploring Prompt Alignment with Clinical Factors in Zero-Shot Segmentation VLMs for NSCLC Tumor Segmentation [0.05597620745943382]
  Zero-shot vision-language models (VLMs) offer a promptable alternative to task-specific training for gross tumor volume delineation.<n>We study alignment directions in VoxTell on a held-out internal NSCLC tumor dataset.<n> Alignment analyses revealed that anatomical location is the dominant driver of VoxTell's spatial attention.
  arXiv  Detail & Related papers  (2026-05-02T05:49:32Z)
• A Semi-Automated Annotation Workflow for Paediatric Histopathology Reports Using Small Language Models [0.0]
  We develop a resource-efficient semi-automated annotation workflow using small language models (SLMs) to extract structured information from unstructured EPR data.<n>As a proof-of-concept, we apply the workflow to paediatric renal biopsy reports, a domain chosen for its constrained diagnostic scope and well-defined underlying biology.<n>We develop the workflow iteratively with clinical oversight across three meetings, manually annotating 400 reports from a dataset of 2,111 at Great Ormond Street Hospital as a gold standard, while developing an automated information extraction approach using SLMs.
  arXiv  Detail & Related papers  (2026-04-05T16:15:39Z)
• Something from Nothing: Data Augmentation for Robust Severity Level Estimation of Dysarthric Speech [69.86604856129883]
  Dysarthric speech quality assessment (DSQA) is critical for clinical diagnostics and inclusive speech technologies.<n>We propose a three-stage framework that leverages unlabeled dysarthric speech and large-scale typical speech datasets to scale training.
  arXiv  Detail & Related papers  (2026-03-16T23:00:07Z)
• Patient-Level Multimodal Question Answering from Multi-Site Auscultation Recordings [7.307923185691307]
  Auscultation is a vital diagnostic tool, yet its utility is often limited by subjective interpretation.<n>We propose a framework that aligns multi-site auscultation recordings directly with a frozen Large Language Model (LLM) embedding space via gated cross-attention.
  arXiv  Detail & Related papers  (2026-03-09T10:29:47Z)
• A Detection-Gated Pipeline for Robust Glottal Area Waveform Extraction and Clinical Pathology Assessment [0.0]
  We propose a detection-gated pipeline that integrates a localizer with a segmenter.<n>The pipeline achieved state-of-the-art consistency on the GIRAFE and BAGLS benchmarks.
  arXiv  Detail & Related papers  (2026-03-02T17:05:41Z)
• Language Models as Semantic Teachers: Post-Training Alignment for Medical Audio Understanding [15.79973026677169]
  Pre-trained audio models excel at detecting acoustic patterns in auscultation sounds but often fail to grasp their clinical significance.<n>We introduce AcuLa, a framework that instills semantic understanding into any audio encoder by aligning it with a medical language model.<n>Our work demonstrates that this audio-language alignment transforms purely acoustic models into clinically-aware diagnostic tools.
  arXiv  Detail & Related papers  (2025-12-04T14:30:58Z)
• Zero-Shot ATC Coding with Large Language Models for Clinical Assessments [40.72273945475456]
  Manual assignment of Anatomical Therapeutic Chemical codes to prescription records is a significant bottleneck.<n>We develop a practical approach using locally deployable large language models (LLMs)<n>We evaluate our approach using GPT-4o as an accuracy ceiling and focus development on open-source Llama models suitable for privacy-sensitive deployment.
  arXiv  Detail & Related papers  (2024-12-10T18:43:02Z)
• Self-supervised contrastive learning of echocardiogram videos enables label-efficient cardiac disease diagnosis [48.64462717254158]
  We developed a self-supervised contrastive learning approach, EchoCLR, to catered to echocardiogram videos. When fine-tuned on small portions of labeled data, EchoCLR pretraining significantly improved classification performance for left ventricular hypertrophy (LVH) and aortic stenosis (AS) EchoCLR is unique in its ability to learn representations of medical videos and demonstrates that SSL can enable label-efficient disease classification from small, labeled datasets.
  arXiv  Detail & Related papers  (2022-07-23T19:17:26Z)
• Semi-Supervised Speech Recognition via Local Prior Matching [42.311823406287864]
  Local prior matching is a semi-supervised objective that distills knowledge from a strong prior. We demonstrate that LPM is theoretically well-native, simple to implement, and superior to existing knowledge distillation techniques.
  arXiv  Detail & Related papers  (2020-02-24T16:07:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.