Robust Summarization of Doctor-Patient Conversations: TalTech Systems for the Beyond Transcription Challenge

• URL: http://arxiv.org/abs/2607.17230v1
• Date: Sun, 19 Jul 2026 12:49:35 GMT
• Title: Robust Summarization of Doctor-Patient Conversations: TalTech Systems for the Beyond Transcription Challenge
• Abstract summary: This paper describes TalTech's submissions to the Beyond Transcription Challenge (BeTraC)<n>The challenge requires generating SOAP notes directly from long doctor-patient conversation recordings, without intermediate transcription.<n>We adapted Voxtral Mini and Voxtral Small for long-audio robustness, with LoRA supervised fine-tuning followed by DAPO reinforcement learning.<n>We find that fine-tuning on text transcripts transfers well to speech input and appears to improve robustness on out-of-domain real recordings.
• Score: 6.193919591996934
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper describes TalTech's submissions to the Beyond Transcription Challenge (BeTraC), which requires generating SOAP notes directly from long doctor-patient conversation recordings, without intermediate transcription. After screening open-weight speech LLMs for long-audio robustness, we adapted Voxtral Mini (lightweight track) and Voxtral Small (heavyweight track) with LoRA supervised fine-tuning followed by DAPO reinforcement learning that uses the challenge metric, Open Medical Concept F1, as its reward. Our systems ranked first in both tracks, and an independent LLM-as-a-judge evaluation showed the lowest hallucination rate among all submissions, indicating that reinforcement learning against a concept-matching metric need not compromise factual reliability. We also find that fine-tuning on text transcripts transfers well to speech input and appears to improve robustness on out-of-domain real recordings.

Related papers

• Who Spoke What When? Evaluating Spoken Language Models for Conversational ASR with Semantic and Overlap-Aware Metrics [99.89493037369071]
  Conversational automatic speech recognition remains challenging due to overlapping speech, far-field noise, and varying speaker counts.<n>Recent LLM-based systems perform well on single-speaker benchmarks, but their robustness in multi-speaker settings is unclear.<n>We systematically compare LLM-based and modular approaches along four axes: overlap, semantic fidelity, speaker count, and single- versus multi-channel input.
  arXiv  Detail & Related papers  (2026-03-24T02:01:21Z)
• Beyond Transcripts: A Renewed Perspective on Audio Chaptering [66.61445564139052]
  We show that a novel audio-only architecture (AudioSeg) outperforms text-based approaches for segmenting long-form audio into coherent sections.<n>Our experiments on YTSeg reveal that AudioSeg substantially outperforms text-based approaches, pauses provide the largest acoustic gains, and MLLMs remain limited by context length and weak instruction following.
  arXiv  Detail & Related papers  (2026-02-09T18:28:10Z)
• Hearing Between the Lines: Unlocking the Reasoning Power of LLMs for Speech Evaluation [19.92868268408954]
  Large Language Model (LLM) judges exhibit strong reasoning capabilities but are limited to textual content.<n>We propose TRACE, a novel framework that enables LLM judges to reason over audio cues to achieve cost-efficient and human-aligned S2S evaluation.<n>We will release the HCoT annotations and the TRACE framework to enable scalable and human-aligned S2S evaluation.
  arXiv  Detail & Related papers  (2026-01-20T08:57:02Z)
• MOSS Transcribe Diarize: Accurate Transcription with Speaker Diarization [68.87577482940664]
  Speaker-Attributed, Time-Stamped Transcription aims to transcribe what is said and to precisely determine the timing of each speaker.<n>Existing SATS systems rarely adopt an end-to-end formulation and are further constrained by limited context windows, weak long-range speaker memory, and the inability to output timestamps.<n>We present MOSS Transcribe Diarize, a unified multimodal large language model that jointly performs Speaker-Attributed, Time-Stamped Transcription in an end-to-end paradigm.
  arXiv  Detail & Related papers  (2026-01-04T15:01:10Z)
• Eliminating stability hallucinations in llm-based tts models via attention guidance [54.69225348397135]
  This paper focuses on resolving stability hallucinations in Text-to-Speech (TTS) models by improving and leveraging the attention mechanism.<n>We proposed a metric termed the Optimal Alignment Score (OAS), which employs the Viterbi algorithm to evaluate text-speech alignment quality.<n>The proposed methods can effectively reduce the stability of CosyVoice2 without introducing additional negative effects.
  arXiv  Detail & Related papers  (2025-09-24T07:47:52Z)
• Blending LLMs into Cascaded Speech Translation: KIT's Offline Speech Translation System for IWSLT 2024 [61.189875635090225]
  Large Language Models (LLMs) are currently under exploration for various tasks, including Automatic Speech Recognition (ASR), Machine Translation (MT), and even End-to-End Speech Translation (ST)
  arXiv  Detail & Related papers  (2024-06-24T16:38:17Z)
• The Sound of Healthcare: Improving Medical Transcription ASR Accuracy with Large Language Models [0.0]
  Large Language Models (LLMs) can enhance the accuracy of Automatic Speech Recognition (ASR) systems in medical transcription. Our research focuses on improvements in Word Error Rate (WER), Medical Concept WER (MC-WER) for the accurate transcription of essential medical terms, and speaker diarization accuracy.
  arXiv  Detail & Related papers  (2024-02-12T14:01:12Z)
• DiariST: Streaming Speech Translation with Speaker Diarization [53.595990270899414]
  We propose DiariST, the first streaming ST and SD solution. It is built upon a neural transducer-based streaming ST system and integrates token-level serialized output training and t-vector. Our system achieves a strong ST and SD capability compared to offline systems based on Whisper, while performing streaming inference for overlapping speech.
  arXiv  Detail & Related papers  (2023-09-14T19:33:27Z)
• Improving Deliberation by Text-Only and Semi-Supervised Training [42.942428288428836]
  We propose incorporating text-only and semi-supervised training into an attention-based deliberation model. We achieve 4%-12% WER reduction for various tasks compared to the baseline deliberation. We show that the deliberation model also achieves a positive human side-by-side evaluation.
  arXiv  Detail & Related papers  (2022-06-29T15:30:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.