BUSTR: Breast Ultrasound Text Reporting with a Descriptor-Aware Vision-Language Model

• URL: http://arxiv.org/abs/2511.20956v1
• Date: Wed, 26 Nov 2025 01:22:29 GMT
• Title: BUSTR: Breast Ultrasound Text Reporting with a Descriptor-Aware Vision-Language Model
• Abstract summary: Automated radiology report generation (RRG) for breast ultrasound (BUS) is limited by the lack of paired image-report datasets.<n>We propose BUSTR, a multitask vision-language framework that generates BUS reports without requiring paired image-report supervision.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated radiology report generation (RRG) for breast ultrasound (BUS) is limited by the lack of paired image-report datasets and the risk of hallucinations from large language models. We propose BUSTR, a multitask vision-language framework that generates BUS reports without requiring paired image-report supervision. BUSTR constructs reports from structured descriptors (e.g., BI-RADS, pathology, histology) and radiomics features, learns descriptor-aware visual representations with a multi-head Swin encoder trained using a multitask loss over dataset-specific descriptor sets, and aligns visual and textual tokens via a dual-level objective that combines token-level cross-entropy with a cosine-similarity alignment loss between input and output representations. We evaluate BUSTR on two public BUS datasets, BrEaST and BUS-BRA, which differ in size and available descriptors. Across both datasets, BUSTR consistently improves standard natural language generation metrics and clinical efficacy metrics, particularly for key targets such as BI-RADS category and pathology. Our results show that this descriptor-aware vision model, trained with a combined token-level and alignment loss, improves both automatic report metrics and clinical efficacy without requiring paired image-report data. The source code can be found at https://github.com/AAR-UNLV/BUSTR

Related papers

• HarMoE: Multi-Source Chest Radiograph Pretraining with Dataset-Disentangled Experts [6.917612472314974]
  HarMoE is a dataset-aware mixture-of-experts framework that learns shared cross-dataset medical semantics.<n> Experiments on large-scale chest X-ray benchmarks show that HarMoE consistently improves zero-shot classification transfer.
  arXiv  Detail & Related papers  (2026-08-03T14:00:16Z)
• Gated Multi-Graph Fusion via Graph Attention Networks for Alzheimer's Disease Detection [52.36198268369384]
  Spontaneous speech is a vital non-invasive biomarker for Alzheimer's Disease (AD)<n>We propose a Multi-View Gated Graph Attention Network that transcribes audio via Automatic Speech Recognition (ASR) to construct semantic, dependency, and co-occurrence graphs.
  arXiv  Detail & Related papers  (2026-06-30T06:15:59Z)
• RIHA: Report-Image Hierarchical Alignment for Radiology Report Generation [14.781219175876883]
  Radiology report generation (RRG) has emerged as a promising approach to alleviate radiologists' workload and reduce human errors.<n>A key challenge in RRG is achieving fine-grained alignment between complex visual features and the structure of long-form radiology reports.<n>We propose RIHA, a novel end-to-end framework that performs multi-level alignment between radiological images and their corresponding reports.
  arXiv  Detail & Related papers  (2026-04-30T08:08:46Z)
• Structure Observation Driven Image-Text Contrastive Learning for Computed Tomography Report Generation [51.509572354327986]
  This work introduces a novel two-stage (structure- and report-learning) framework tailored for Computed Tomography Report Generation (CTRG)<n>In the first stage, a set of learnable structure-specific visual queries observe corresponding structures in a CT image. The resulting observation tokens are contrasted with structure-specific textual features extracted from the accompanying radiology report with a structure-wise image-text contrastive loss.<n>In the second stage, the visual structure queries are frozen and used to select the critical image patch embeddings depicting each anatomical structure, minimizing distractions from irrelevant areas while reducing memory consumption.
  arXiv  Detail & Related papers  (2026-03-05T07:07:07Z)
• Automated Histopathology Report Generation via Pyramidal Feature Extraction and the UNI Foundation Model [0.0]
  We propose a hierarchical vision language framework that combines a frozen pathology foundation model with a Transformer decoder for report generation.<n>To make WSI processing tractable, we perform multi resolution pyramidal patch selection and remove background and artifacts.<n>We tokenize the output using BioGPT to better represent biomedical terminology.
  arXiv  Detail & Related papers  (2026-02-18T12:55:20Z)
• BTReport: A Framework for Brain Tumor Radiology Report Generation with Clinically Relevant Features [2.5111131141274328]
  BTReport is an open-source framework that constructs natural language radiology reports using deterministically extracted imaging features.<n>We show that the features used for report generation are predictive of key clinical outcomes, including survival and IDH mutation status.<n>Finally, we introduce BTReport-BraTS, a companion dataset that augments BraTS imaging with synthetically generated radiology reports produced with BTReport.
  arXiv  Detail & Related papers  (2026-02-17T20:55:00Z)
• $μ^2$Tokenizer: Differentiable Multi-Scale Multi-Modal Tokenizer for Radiology Report Generation [9.947108972979155]
  We propose $mu2$LLM, a $underlinetextbfmu$ltiscale $underlinetextbfmu$ltimodal large language models for radiology report generation tasks.<n>The novel $mu2$Tokenizer, as an intermediate layer, integrates multi-modal features from the multiscale visual tokenizer and the text tokenizer.<n>For prompt engineering, we introduce a five-stage, LLM-driven pipeline that converts routine CT reports into paired visual-question-answer triples and citation-linked reasoning narratives.
  arXiv  Detail & Related papers  (2025-06-30T23:14:49Z)
• Ultrasound Report Generation with Multimodal Large Language Models for Standardized Texts [15.349894506969074]
  We propose a unified framework for multi-organ and multilingual US report generation.<n>Method achieves consistent and clinically accurate text generation across organ sites and languages.
  arXiv  Detail & Related papers  (2025-05-13T08:27:01Z)
• RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignment [10.67889367763112]
  RadAlign is a novel framework that combines the predictive accuracy of vision-language models with the reasoning capabilities of large language models.<n>Our framework maintains strong clinical interpretability while reducing hallucinations, advancing automated medical imaging and report analysis through integrated predictive and generative AI.
  arXiv  Detail & Related papers  (2025-01-13T17:55:32Z)
• Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-ray Report Generation [54.631356899598956]
  We propose a novel associative memory-enhanced X-ray report generation model that effectively mimics the process of professional doctors writing medical reports.<n>We employ a visual Hopfield network to establish memory associations for disease-related tokens, and a report Hopfield network to retrieve report memory information.
  arXiv  Detail & Related papers  (2025-01-07T01:19:48Z)
• Factual Serialization Enhancement: A Key Innovation for Chest X-ray Report Generation [15.82363717056198]
  A radiology report comprises presentation-style vocabulary, which ensures clarity and organization. A critical step in this process is to align radiographs with their corresponding reports. To address this issue, we propose FSE, a two-stage Factual Serialization Enhancement method.
  arXiv  Detail & Related papers  (2024-05-15T07:56:38Z)
• Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime [70.04389979779195]
  This paper explores training medical vision-language models (VLMs) where the visual and language inputs are embedded into a common space. We explore several candidate methods to improve low-data performance, including adapting generic pre-trained models to novel image and text domains. Using text-to-image retrieval as a benchmark, we evaluate the performance of these methods with variable sized training datasets of paired chest X-rays and radiological reports.
  arXiv  Detail & Related papers  (2023-03-30T18:20:00Z)
• Cross-Modal Causal Intervention for Medical Report Generation [107.76649943399168]
  Radiology Report Generation (RRG) is essential for computer-aided diagnosis and medication guidance.<n> generating accurate lesion descriptions remains challenging due to spurious correlations from visual-linguistic biases.<n>We propose a two-stage framework named CrossModal Causal Representation Learning (CMCRL)<n> Experiments on IU-Xray and MIMIC-CXR show that our CMCRL pipeline significantly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-03-16T07:23:55Z)
• Unified Visual Relationship Detection with Vision and Language Models [89.77838890788638]
  This work focuses on training a single visual relationship detector predicting over the union of label spaces from multiple datasets. We propose UniVRD, a novel bottom-up method for Unified Visual Relationship Detection by leveraging vision and language models. Empirical results on both human-object interaction detection and scene-graph generation demonstrate the competitive performance of our model.
  arXiv  Detail & Related papers  (2023-03-16T00:06:28Z)
• Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing [53.89917396428747]
  Self-supervised learning in vision-language processing exploits semantic alignment between imaging and text modalities. We explicitly account for prior images and reports when available during both training and fine-tuning. Our approach, named BioViL-T, uses a CNN-Transformer hybrid multi-image encoder trained jointly with a text model.
  arXiv  Detail & Related papers  (2023-01-11T16:35:33Z)
• Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation [107.3538598876467]
  We propose an Auxiliary Signal-Guided Knowledge-Decoder (ASGK) to mimic radiologists' working patterns. ASGK integrates internal visual feature fusion and external medical linguistic information to guide medical knowledge transfer and learning.
  arXiv  Detail & Related papers  (2020-06-06T01:00:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.