An Omnilingual-ASR-Based Speech-LLM System for the 2nd MLC-SLM Challenge

• URL: http://arxiv.org/abs/2607.12468v1
• Date: Tue, 14 Jul 2026 07:52:39 GMT
• Title: An Omnilingual-ASR-Based Speech-LLM System for the 2nd MLC-SLM Challenge
• Abstract summary: We describe a cascaded diarization-then-recognition system that combines DiariZen-Large-s80 (WavLM-Large) segmentation, embedding-based two-adapted clustering, and a LoRA- omniASR LLM 7B v2 recognizer.<n>On the official Development set (150 conversations, 21 language/accent categories), the system attains a macroMER of 29.27%, versus 79.15% for the official baseline; on the Evaluation set it scores 50.23%.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We describe our submission to Task 1 of the 2nd MLCSLM Challenge: a cascaded diarization-then-recognition system that combines DiariZen-Large-s80 (WavLM-Large) segmentation, CAM++ embedding-based two-speaker clustering, and a LoRA-adapted omniASR LLM 7B v2 recognizer, with no oracle segmentation or speaker labels at test time. On the official Development set (150 conversations, 21 language/accent categories) the system attains a macro tcpMER of 29.27%, versus 79.15% for the official baseline; on the Evaluation set it scores 50.23%. We also analyze two engineering choices that substantially affect tcpMER. First, embedding-based speaker clustering outperforms an end-to-end-style alternative that assigns speakers from ASR <sc> turn markers alone. Second, overlap-aware segmentation, although intended to raise diarization recall, increases tcpMER because overlapped speech is transcribed twice.

Related papers

• Diarization-Guided Qwen-ASR Adaptation for Multilingual Two-Speaker Conversational Speech [14.048968602801606]
  This paper describes our self-designed hallucination system for Task 1 of the MLC-SLM 2026 Challenge for multilingual two-speaker conversational speech.<n>The system combines a modular speaker diarization front end with a challenge-adapted Qwen3-ASR-1.7B recognizer.
  arXiv  Detail & Related papers  (2026-07-09T08:07:34Z)
• Who Spoke What When? Evaluating Spoken Language Models for Conversational ASR with Semantic and Overlap-Aware Metrics [99.89493037369071]
  Conversational automatic speech recognition remains challenging due to overlapping speech, far-field noise, and varying speaker counts.<n>Recent LLM-based systems perform well on single-speaker benchmarks, but their robustness in multi-speaker settings is unclear.<n>We systematically compare LLM-based and modular approaches along four axes: overlap, semantic fidelity, speaker count, and single- versus multi-channel input.
  arXiv  Detail & Related papers  (2026-03-24T02:01:21Z)
• Bridging the gap: A comparative exploration of Speech-LLM and end-to-end architecture for multilingual conversational ASR [16.090902570653803]
  We present an enhanced LLM-based ASR framework that combines fine-tuned Whisper and mHuBERT encoders with an LLM to enrich speech representations.<n>Our system achieves a CER/WER of 10.69%, ranking on par with the top-ranked Track 1 systems.
  arXiv  Detail & Related papers  (2026-01-04T10:08:53Z)
• MEGConformer: Conformer-Based MEG Decoder for Robust Speech and Phoneme Classification [1.0896567381206717]
  We present Conformer-based decoders for the LibriBrain 2025 PNPL competition.<n>Our approach adapts a compact Conformer to raw 306-channel MEG signals.<n>For Speech Detection, a MEG-oriented SpecAugment provided a first exploration of MEG-specific augmentation.<n>For Phoneme Classification, we used inverse-square-root class weighting and a dynamic grouping loader.
  arXiv  Detail & Related papers  (2025-12-01T09:25:22Z)
• Unifying Diarization, Separation, and ASR with Multi-Speaker Encoder [53.00939565103065]
  We present a novel architecture that jointly learns representations for speaker diarization (SD), speech separation (SS), and multi-speaker automatic speech recognition (ASR) tasks.<n>We leverage the hidden representations from multiple layers of UME as a residual weighted-sum encoding (RWSE) to effectively use information from different semantic levels.<n>This joint training approach captures the inherent interdependencies among the tasks, enhancing overall performance on overlapping speech data.
  arXiv  Detail & Related papers  (2025-08-28T06:50:57Z)
• SLICER: Learning universal audio representations using low-resource self-supervised pre-training [53.06337011259031]
  We present a new Self-Supervised Learning approach to pre-train encoders on unlabeled audio data. Our primary aim is to learn audio representations that can generalize across a large variety of speech and non-speech tasks.
  arXiv  Detail & Related papers  (2022-11-02T23:45:33Z)
• Bi-LSTM Scoring Based Similarity Measurement with Agglomerative Hierarchical Clustering (AHC) for Speaker Diarization [0.0]
  A typical conversation between two speakers consists of segments where their voices overlap, interrupt each other or halt their speech in between multiple sentences. Recent advancements in diarization technology leverage neural network-based approaches to improvise speaker diarization system. We propose a Bi-directional Long Short-term Memory network for estimating the elements present in the similarity matrix.
  arXiv  Detail & Related papers  (2022-05-19T17:20:51Z)
• Speaker Embedding-aware Neural Diarization: a Novel Framework for Overlapped Speech Diarization in the Meeting Scenario [51.5031673695118]
  We reformulate overlapped speech diarization as a single-label prediction problem. We propose the speaker embedding-aware neural diarization (SEND) system.
  arXiv  Detail & Related papers  (2022-03-18T06:40:39Z)
• The USTC-Ximalaya system for the ICASSP 2022 multi-channel multi-party meeting transcription (M2MeT) challenge [43.262531688434215]
  We propose two improvements to target-speaker voice activity detection (TS-VAD) These techniques are designed to handle multi-speaker conversations in real-world meeting scenarios with high speaker-overlap ratios and under heavy reverberant and noisy condition.
  arXiv  Detail & Related papers  (2022-02-10T06:06:48Z)
• Audio-visual Multi-channel Recognition of Overlapped Speech [79.21950701506732]
  This paper presents an audio-visual multi-channel overlapped speech recognition system featuring tightly integrated separation front-end and recognition back-end. Experiments suggest that the proposed multi-channel AVSR system outperforms the baseline audio-only ASR system by up to 6.81% (26.83% relative) and 22.22% (56.87% relative) absolute word error rate (WER) reduction on overlapped speech constructed using either simulation or replaying of the lipreading sentence 2 dataset respectively.
  arXiv  Detail & Related papers  (2020-05-18T10:31:19Z)
• Unsupervised Speaker Adaptation using Attention-based Speaker Memory for End-to-End ASR [61.55606131634891]
  We propose an unsupervised speaker adaptation method inspired by the neural Turing machine for end-to-end (E2E) automatic speech recognition (ASR) The proposed model contains a memory block that holds speaker i-vectors extracted from the training data and reads relevant i-vectors from the memory through an attention mechanism. We show that M-vectors, which do not require an auxiliary speaker embedding extraction system at test time, achieve similar word error rates (WERs) compared to i-vectors for single speaker utterances and significantly lower WERs for utterances in which there are speaker changes
  arXiv  Detail & Related papers  (2020-02-14T18:31:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.