MOSS Transcribe Diarize: Accurate Transcription with Speaker Diarization

• URL: http://arxiv.org/abs/2601.01554v3
• Date: Thu, 08 Jan 2026 04:58:04 GMT
• Title: MOSS Transcribe Diarize: Accurate Transcription with Speaker Diarization
• Abstract summary: Speaker-Attributed, Time-Stamped Transcription aims to transcribe what is said and to precisely determine the timing of each speaker.<n>Existing SATS systems rarely adopt an end-to-end formulation and are further constrained by limited context windows, weak long-range speaker memory, and the inability to output timestamps.<n>We present MOSS Transcribe Diarize, a unified multimodal large language model that jointly performs Speaker-Attributed, Time-Stamped Transcription in an end-to-end paradigm.
• Score: 68.87577482940664
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Speaker-Attributed, Time-Stamped Transcription (SATS) aims to transcribe what is said and to precisely determine the timing of each speaker, which is particularly valuable for meeting transcription. Existing SATS systems rarely adopt an end-to-end formulation and are further constrained by limited context windows, weak long-range speaker memory, and the inability to output timestamps. To address these limitations, we present MOSS Transcribe Diarize, a unified multimodal large language model that jointly performs Speaker-Attributed, Time-Stamped Transcription in an end-to-end paradigm. Trained on extensive real wild data and equipped with a 128k context window for up to 90-minute inputs, MOSS Transcribe Diarize scales well and generalizes robustly. Across comprehensive evaluations, it outperforms state-of-the-art commercial systems on multiple public and in-house benchmarks.

Related papers

• Speaker-Reasoner: Scaling Interaction Turns and Reasoning Patterns for Timestamped Speaker-Attributed ASR [18.90091826363701]
  Speaker-Reasoner is an end-to-end Speech LLM with agentic multi-turn temporal reasoning.<n>Speaker-Reasoner achieves consistent improvements over strong baselines on AliMeeting and AISHELL-4 datasets.
  arXiv  Detail & Related papers  (2026-04-03T14:52:20Z)
• G-STAR: End-to-End Global Speaker-Tracking Attributed Recognition [32.00022309010023]
  Speech-LLM systems tend to prioritize either local diarization or global labeling, but often lack the ability to capture fine-grained temporal boundaries.<n>We propose G-STAR, an end-to-end system that couples a time-aware speaker-tracking module with a Speech-LLM transcription backbone.<n>G-STAR supports both component-wise optimization and joint end-to-end training, enabling flexible learning under heterogeneous supervision and domain shift.
  arXiv  Detail & Related papers  (2026-03-11T06:40:01Z)
• Smooth Operators: LLMs Translating Imperfect Hints into Disfluency-Rich Transcripts [5.439020425819001]
  Large language models (LLMs) are versatile learners capable of processing both lexical and non-lexical inputs.<n>We propose a novel approach to disfluencies as explicit tokens with timestamps, enabling the generation of fully annotated disfluency-rich transcripts.
  arXiv  Detail & Related papers  (2025-06-23T11:04:20Z)
• MSA-ASR: Efficient Multilingual Speaker Attribution with frozen ASR Models [59.80042864360884]
  Speaker-attributed automatic speech recognition (SA-ASR) aims to transcribe speech while assigning transcripts to the corresponding speakers accurately.<n>This paper introduces a novel approach, leveraging a frozen multilingual ASR model to incorporate speaker attribution into the transcriptions.
  arXiv  Detail & Related papers  (2024-11-27T09:01:08Z)
• Robust Speech Recognition via Large-Scale Weak Supervision [69.63329359286419]
  We study the capabilities of speech processing systems trained simply to predict large amounts of transcripts of audio on the internet. When scaled to 680,000 hours of multilingual and multitask supervision, the resulting models generalize well to standard benchmarks. We are releasing models and inference code to serve as a foundation for further work on robust speech processing.
  arXiv  Detail & Related papers  (2022-12-06T18:46:04Z)
• A unified one-shot prosody and speaker conversion system with self-supervised discrete speech units [94.64927912924087]
  Existing systems ignore the correlation between prosody and language content, leading to degradation of naturalness in converted speech. We devise a cascaded modular system leveraging self-supervised discrete speech units as language representation. Experiments show that our system outperforms previous approaches in naturalness, intelligibility, speaker transferability, and prosody transferability.
  arXiv  Detail & Related papers  (2022-11-12T00:54:09Z)
• Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings [53.11450530896623]
  This paper presents a streaming speaker-attributed automatic speech recognition (SA-ASR) model that can recognize "who spoke what" Our model is based on token-level serialized output training (t-SOT) which was recently proposed to transcribe multi-talker speech in a streaming fashion. The proposed model achieves substantially better accuracy than a prior streaming model and shows comparable or sometimes even superior results to the state-of-the-art offline SA-ASR model.
  arXiv  Detail & Related papers  (2022-03-30T21:42:00Z)
• Directed Speech Separation for Automatic Speech Recognition of Long Form Conversational Speech [10.291482850329892]
  We propose a speaker conditioned separator trained on speaker embeddings extracted directly from the mixed signal. We achieve significant improvements on Word error rate (WER) for real conversational data without the need for an additional re-stitching step.
  arXiv  Detail & Related papers  (2021-12-10T23:07:48Z)
• Consistent Transcription and Translation of Speech [13.652411093089947]
  We explore the task of jointly transcribing and translating speech. While high accuracy of transcript and translation are crucial, even highly accurate systems can suffer from inconsistencies between both outputs. We find that direct models are poorly suited to the joint transcription/translation task, but that end-to-end models that feature a coupled inference procedure are able to achieve strong consistency.
  arXiv  Detail & Related papers  (2020-07-24T19:17:26Z)
• Adapting End-to-End Speech Recognition for Readable Subtitles [15.525314212209562]
  In some use cases such as subtitling, verbatim transcription would reduce output readability given limited screen size and reading time. We first investigate a cascaded system, where an unsupervised compression model is used to post-edit the transcribed speech. Experiments show that with limited data far less than needed for training a model from scratch, we can adapt a Transformer-based ASR model to incorporate both transcription and compression capabilities.
  arXiv  Detail & Related papers  (2020-05-25T14:42:26Z)
• Semi-supervised Learning for Multi-speaker Text-to-speech Synthesis Using Discrete Speech Representation [125.59372403631006]
  We propose a semi-supervised learning approach for multi-speaker text-to-speech (TTS) A multi-speaker TTS model can learn from the untranscribed audio via the proposed encoder-decoder framework with discrete speech representation. We found the model can benefit from the proposed semi-supervised learning approach even when part of the unpaired speech data is noisy.
  arXiv  Detail & Related papers  (2020-05-16T15:47:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.