OmniTrace: A Unified Framework for Generation-Time Attribution in Omni-Modal LLMs

• URL: http://arxiv.org/abs/2604.13073v1
• Date: Fri, 20 Mar 2026 17:25:00 GMT
• Title: OmniTrace: A Unified Framework for Generation-Time Attribution in Omni-Modal LLMs
• Abstract summary: We introduce OmniTrace, a lightweight and model-agnostic framework that formalizes attribution as a generation-time tracing problem.<n>We show that generation-aware span-level attribution produces more stable and interpretable explanations than naive self-attribution.<n>Our results suggest that treating attribution as a structured generation-time tracing problem provides a scalable foundation for transparency in omni-modal language models.
• Score: 31.589945976149973
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern multimodal large language models (MLLMs) generate fluent responses from interleaved text, image, audio, and video inputs. However, identifying which input sources support each generated statement remains an open challenge. Existing attribution methods are primarily designed for classification settings, fixed prediction targets, or single-modality architectures, and do not naturally extend to autoregressive, decoder-only models performing open-ended multimodal generation. We introduce OmniTrace, a lightweight and model-agnostic framework that formalizes attribution as a generation-time tracing problem over the causal decoding process. OmniTrace provides a unified protocol that converts arbitrary token-level signals such as attention weights or gradient-based scores into coherent span-level, cross-modal explanations during decoding. It traces each generated token to multimodal inputs, aggregates signals into semantically meaningful spans, and selects concise supporting sources through confidence-weighted and temporally coherent aggregation, without retraining or supervision. Evaluations on Qwen2.5-Omni and MiniCPM-o-4.5 across visual, audio, and video tasks demonstrate that generation-aware span-level attribution produces more stable and interpretable explanations than naive self-attribution and embedding-based baselines, while remaining robust across multiple underlying attribution signals. Our results suggest that treating attribution as a structured generation-time tracing problem provides a scalable foundation for transparency in omni-modal language models.

Related papers

• Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMs [7.894740505622926]
  Diffusion multimodal large language models (dMLLMs) produce long-form outputs marred by semantic drift and repetition.<n>We propose ours (textbfContext-textbfAware textbfCluster textbfDecoding), a training-free decoding method that scores each masked position by a multiplicative composite of softmax confidence and neighbor proximity.
  arXiv  Detail & Related papers  (2026-08-23T11:16:04Z)
• AURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modeling [60.04959047453115]
  AURORA-LM is a continuous-latent language model that separates the construction of a decodable text representation from the modeling of its distribution.<n>AURORA-LM achieves the strongest performance among evaluated continuous and diffusion-based language models on OpenWebText free generation and XSum summarization.
  arXiv  Detail & Related papers  (2026-08-03T17:59:50Z)
• Sandboxed Coding Agents are Competitive Omni-modal Task Solvers [22.269016330825128]
  We show that coding agents with only text+image access and a sandboxed tool-use interface can match SOTA native omnimodal models.<n>Our trajectory analysis suggests that their strength comes from writing code and orchestrating tools to extract relevant evidence.<n>We introduce Code-X, a training recipe with the OmniCoding trajectory dataset and verifiable reward, and provide baselines on Qwen-3.5-9B and Qwen-3.6-27B.
  arXiv  Detail & Related papers  (2026-05-30T07:04:27Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• LongCat-Next: Lexicalizing Modalities as Discrete Tokens [111.50794040093656]
  LongCat-Next is a native multimodal model that processes text, vision, and audio under a single autoregressive objective.<n>In particular, LongCat-Next addresses the long-standing performance ceiling of discrete vision modeling on understanding tasks.<n>As an attempt toward native multimodality, we open-source the LongCat-Next and its tokenizers, hoping to foster further research and development in the community.
  arXiv  Detail & Related papers  (2026-03-29T06:35:03Z)
• Kelix Technical Report [86.64551727600104]
  We present Kelix, a fully discrete autoregressive unified model that closes the understanding gap between discrete and continuous visual representations.<n>Recent work has explored discrete visual tokenization to enable fully autoregressive multimodal modeling.
  arXiv  Detail & Related papers  (2026-02-10T14:48:26Z)
• CAAD: Context-Aware Adaptive Decoding for Truthful Text Generation [31.469511576774252]
  We propose a context-aware adaptive decoding method for large language models.<n>Our approach achieves a 2.8 percent average improvement on TruthfulQA.<n>Our model-agnostic, scalable, and efficient method requires only a single generation pass.
  arXiv  Detail & Related papers  (2025-08-04T08:28:25Z)
• Video-Level Language-Driven Video-Based Visible-Infrared Person Re-Identification [47.40091830500585]
  Video-based Visible-basedInfrared Person Re-Identification (VVIReID) aims to match pedestrian sequences across modalities by extracting modality-in sequence-level features.<n>A framework, video-level language-driven VVI-ReID (VLD), consists of two core modules: inmodality language (IMLP) and spatialtemporal aggregation.
  arXiv  Detail & Related papers  (2025-06-03T04:49:08Z)
• OmniBench: Towards The Future of Universal Omni-Language Models [63.16606414452612]
  We introduce OmniBench, a novel benchmark designed to evaluate models' ability to recognize, interpret, and reason across visual, acoustic, and textual inputs simultaneously.<n>Our evaluation reveals that open-source OLMs show significant limitations in instruction-following and reasoning in tri-modal contexts.<n>We advocate for developing more robust tri-modal integration techniques and training strategies to enhance OLM performance.
  arXiv  Detail & Related papers  (2024-09-23T17:59:05Z)
• FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent Space [7.324708513042455]
  This paper proposes a novel contrastive learning framework, called FOCAL, for extracting comprehensive features from multimodal time-series sensing signals. It consistently outperforms the state-of-the-art baselines in downstream tasks with a clear margin.
  arXiv  Detail & Related papers  (2023-10-30T22:55:29Z)
• FLIP: Fine-grained Alignment between ID-based Models and Pretrained Language Models for CTR Prediction [49.510163437116645]
  Click-through rate (CTR) prediction plays as a core function module in personalized online services. Traditional ID-based models for CTR prediction take as inputs the one-hot encoded ID features of tabular modality. Pretrained Language Models(PLMs) has given rise to another paradigm, which takes as inputs the sentences of textual modality. We propose to conduct Fine-grained feature-level ALignment between ID-based Models and Pretrained Language Models(FLIP) for CTR prediction.
  arXiv  Detail & Related papers  (2023-10-30T11:25:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.