G-STAR: End-to-End Global Speaker-Tracking Attributed Recognition

• URL: http://arxiv.org/abs/2603.10468v1
• Date: Wed, 11 Mar 2026 06:40:01 GMT
• Title: G-STAR: End-to-End Global Speaker-Tracking Attributed Recognition
• Abstract summary: Speech-LLM systems tend to prioritize either local diarization or global labeling, but often lack the ability to capture fine-grained temporal boundaries.<n>We propose G-STAR, an end-to-end system that couples a time-aware speaker-tracking module with a Speech-LLM transcription backbone.<n>G-STAR supports both component-wise optimization and joint end-to-end training, enabling flexible learning under heterogeneous supervision and domain shift.
• Score: 32.00022309010023
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study timestamped speaker-attributed ASR for long-form, multi-party speech with overlap, where chunk-wise inference must preserve meeting-level speaker identity consistency while producing time-stamped, speaker-labeled transcripts. Previous Speech-LLM systems tend to prioritize either local diarization or global labeling, but often lack the ability to capture fine-grained temporal boundaries or robust cross-chunk identity linking. We propose G-STAR, an end-to-end system that couples a time-aware speaker-tracking module with a Speech-LLM transcription backbone. The tracker provides structured speaker cues with temporal grounding, and the LLM generates attributed text conditioned on these cues. G-STAR supports both component-wise optimization and joint end-to-end training, enabling flexible learning under heterogeneous supervision and domain shift. Experiments analyze cue fusion, local versus long-context trade-offs and hierarchical objectives.

Related papers

• UAF: A Unified Audio Front-end LLM for Full-Duplex Speech Interaction [7.775050285048427]
  Speech assistant is driving artificial propagation toward human-like conversational systems.<n>Front-end components, such as voice activity detection (VAD) and turn-taking detection (TD) are crucial for speech assistant.<n>We propose the first unified audio frontend LLM (UAF) tailored for fullgressive speech systems.
  arXiv  Detail & Related papers  (2026-04-21T08:24:55Z)
• A Novel Automatic Framework for Speaker Drift Detection in Synthesized Speech [34.33842514204989]
  Recent diffusion-based text-to-speech (TTS) models achieve high naturalness and expressiveness, yet often suffer from speaker drift.<n>We introduce the first automatic framework for detecting speaker drift by formulating it as a binary classification task over utterance-level speaker consistency.<n>Our method computes cosine similarity across overlapping segments of synthesized speech and prompts large language models with structured representations to assess drift.
  arXiv  Detail & Related papers  (2026-04-07T18:05:28Z)
• Speaker-Reasoner: Scaling Interaction Turns and Reasoning Patterns for Timestamped Speaker-Attributed ASR [18.90091826363701]
  Speaker-Reasoner is an end-to-end Speech LLM with agentic multi-turn temporal reasoning.<n>Speaker-Reasoner achieves consistent improvements over strong baselines on AliMeeting and AISHELL-4 datasets.
  arXiv  Detail & Related papers  (2026-04-03T14:52:20Z)
• Who Spoke What When? Evaluating Spoken Language Models for Conversational ASR with Semantic and Overlap-Aware Metrics [99.89493037369071]
  Conversational automatic speech recognition remains challenging due to overlapping speech, far-field noise, and varying speaker counts.<n>Recent LLM-based systems perform well on single-speaker benchmarks, but their robustness in multi-speaker settings is unclear.<n>We systematically compare LLM-based and modular approaches along four axes: overlap, semantic fidelity, speaker count, and single- versus multi-channel input.
  arXiv  Detail & Related papers  (2026-03-24T02:01:21Z)
• TagSpeech: End-to-End Multi-Speaker ASR and Diarization with Fine-Grained Temporal Grounding [15.908533215017059]
  We present TagSpeech, a unified framework that utilizes Temporal Anchor Grounding for joint multi-speaker ASR and diarization.<n>The framework is built on two key designs: (1) decoupled semantic and speaker streams fine-tuned via Serialized Output Training (SOT) to learn turn-taking dynamics; and (2) an interleaved time anchor mechanism that acts as a synchronization signal between semantic understanding and speaker tracking.
  arXiv  Detail & Related papers  (2026-01-11T12:40:07Z)
• MOSS Transcribe Diarize: Accurate Transcription with Speaker Diarization [68.87577482940664]
  Speaker-Attributed, Time-Stamped Transcription aims to transcribe what is said and to precisely determine the timing of each speaker.<n>Existing SATS systems rarely adopt an end-to-end formulation and are further constrained by limited context windows, weak long-range speaker memory, and the inability to output timestamps.<n>We present MOSS Transcribe Diarize, a unified multimodal large language model that jointly performs Speaker-Attributed, Time-Stamped Transcription in an end-to-end paradigm.
  arXiv  Detail & Related papers  (2026-01-04T15:01:10Z)
• Beamforming-LLM: What, Where and When Did I Miss? [0.6655749439594806]
  We present Beamforming-LLM, a system that enables users to semantically recall conversations they may have missed in multi-speaker environments.<n>The system combines spatial audio capture using a microphone array with retrieval-augmented generation (RAG) to support natural language queries.
  arXiv  Detail & Related papers  (2025-09-07T21:52:26Z)
• Detect Any Sound: Open-Vocabulary Sound Event Detection with Multi-Modal Queries [18.147981850263708]
  We propose a query-based framework for open-vocabulary SED guided by multi-modal queries.<n>DASM formulates SED as a frame-level retrieval task, where audio features are matched against query vectors from text or audio prompts.<n>DASM effectively balances localization accuracy with generalization to novel classes, outperforming CLAP-based methods in open-vocabulary setting.
  arXiv  Detail & Related papers  (2025-07-22T08:24:01Z)
• Speaker Embeddings to Improve Tracking of Intermittent and Moving Speakers [53.12031345322412]
  We propose to perform identity reassignment post-tracking, using speaker embeddings.<n>Beamforming is used to enhance the signal towards the speakers' positions in order to compute speaker embeddings.<n>We evaluate the performance of the proposed speaker embedding-based identity reassignment method on a dataset where speakers change position during inactivity periods.
  arXiv  Detail & Related papers  (2025-06-23T13:02:20Z)
• Text-Queried Audio Source Separation via Hierarchical Modeling [53.94434504259829]
  We propose a hierarchical decomposition framework, HSM-TSS, that decouples the task into global-local semantic-guided feature separation and structure-preserving acoustic reconstruction.<n>A Q-Audio architecture is employed to align audio and text modalities, serving as pretrained global-semantic encoders.<n>Our method achieves state-of-the-art separation performance with data-efficient training while maintaining superior semantic consistency with queries in complex auditory scenes.
  arXiv  Detail & Related papers  (2025-05-27T11:00:38Z)
• Towards Word-Level End-to-End Neural Speaker Diarization with Auxiliary Network [28.661704280484457]
  We propose Word-level End-to-End Neural Diarization (WEEND) with auxiliary network. We find WEEND has the potential to deliver high quality diarized text.
  arXiv  Detail & Related papers  (2023-09-15T15:48:45Z)
• Controllable speech synthesis by learning discrete phoneme-level prosodic representations [53.926969174260705]
  We present a novel method for phoneme-level prosody control of F0 and duration using intuitive discrete labels. We propose an unsupervised prosodic clustering process which is used to discretize phoneme-level F0 and duration features from a multispeaker speech dataset.
  arXiv  Detail & Related papers  (2022-11-29T15:43:36Z)
• Extended Graph Temporal Classification for Multi-Speaker End-to-End ASR [77.82653227783447]
  We propose an extension of GTC to model the posteriors of both labels and label transitions by a neural network. As an example application, we use the extended GTC (GTC-e) for the multi-speaker speech recognition task.
  arXiv  Detail & Related papers  (2022-03-01T05:02:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.