TTE-Flash: Accelerating Reasoning-based Multimodal Representations via Think-Then-Embed Tokens

• URL: http://arxiv.org/abs/2605.16638v1
• Date: Fri, 15 May 2026 21:10:56 GMT
• Title: TTE-Flash: Accelerating Reasoning-based Multimodal Representations via Think-Then-Embed Tokens
• Abstract summary: We introduce TTE-Flash-2B, a reasoning-aware multimodal representation model that outperforms its explicit-CoT counterpart on the MMEB-v2 benchmark.<n>Our study investigates two key architectural designs: 1) how think and embeddings tokens should be extracted from the same LLM backbone, and 2) how the tokens should be trained as two dependent tasks.
• Score: 32.700228795574255
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent research has demonstrated that Universal Multimodal Embedding (UME) benefits significantly from Chain-of-Thought (CoT) reasoning. In this paradigm, a generative model produces explicit reasoning traces for a multimodal query, with the final representation extracted from an <eos> embedding token attending to both the query and the reasoning. Despite its effectiveness, the computational overhead of generating explicit CoT traces is often prohibitive. In this work, we propose replacing explicit CoT with latent think tokens, which are interpreted as latent variables that can produce explicit CoT traces as observed variables. By optimizing think tokens using CoT generation loss and subsequent embedding tokens using contrastive loss, we produce high-performance, reasoning-aware representations at a constant inference cost. Our study investigates two key architectural designs: 1) how think and embeddings tokens should be extracted from the same LLM backbone. 2) how the tokens should be trained as two dependent tasks. We introduce TTE-Flash-2B, a reasoning-aware multimodal representation model that outperforms its explicit-CoT counterpart on the MMEB-v2 benchmark, while producing latent think tokens that are interpretable both textually and visually. Furthermore, zero-shot evaluation across 15 video datasets reveals scaling behavior as the number of think tokens increases, and motivating a pilot study of adaptive think budget allocation based on task requirements.

Related papers

• Integrated and Cross-Architecture Interpretation of LLM Reasoning [48.58940522466915]
  Integrated, cross-Architecture Reasoning (IAR) framework designed to provide a unified approach to LLM reasoning interpretability.<n>We first propose to use bandwidth-calibrated MIP coupled with Tukey IQR peak-detection to isolate reasoning-crucial tokens at the output layer.<n>We then perform an overlap analysis between MIP-picked tokens and computation-deep tokens to trace the cross-layer trajectories of those tokens.
  arXiv  Detail & Related papers  (2026-05-27T05:56:35Z)
• Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs [51.60575965819268]
  We propose a plug-and-play Token-Reweighting (ToR) strategy that explicitly models this interdependence.<n>ToR delivers consistent performance gains across multiple multi-modal reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-03-26T06:25:27Z)
• Latent Reasoning with Supervised Thinking States [60.09942890192309]
  Reasoning with a chain-of-thought (CoT) enables Large Language Models (LLMs) to solve complex tasks but incurs significant inference costs.<n>We propose Thinking States, a method that performs reasoning em while the input is processing.<n>We show Thinking States leads to stronger reasoning behavior than CoT, successfully extrapolating to longer sequences than seen during training.
  arXiv  Detail & Related papers  (2026-02-09T07:12:41Z)
• ImgCoT: Compressing Long Chain of Thought into Compact Visual Tokens for Efficient Reasoning of Large Language Model [34.90582960625524]
  Long chains of thought (CoT) into compact latent tokens is crucial for efficient reasoning with large language models (LLMs)<n>We propose ImgCoT that replaces the reconstruction target from textual CoT to the visual CoT obtained by rendering CoT into images.<n>This substitutes linguistic bias with spatial inductive bias, enabling latent tokens to better capture global reasoning structure.
  arXiv  Detail & Related papers  (2026-01-30T09:06:45Z)
• FrugalPrompt: Reducing Contextual Overhead in Large Language Models via Token Attribution [3.4666771782038652]
  Large language models (LLMs) owe much of their stellar performance to expansive input contexts, yet such verbosity inflates monetary costs, carbon footprint, and inference-time latency.<n>We introduce FrugalPrompt, a novel prompt compression framework for LLMs, which retains only the most semantically significant tokens.<n>We evaluate the approach across four NLP tasks: Sentiment Analysis, Commonsense QA, Summarization, and Mathematical Reasoning.
  arXiv  Detail & Related papers  (2025-10-18T10:22:13Z)
• MARCOS: Deep Thinking by Markov Chain of Continuous Thoughts [82.46857666702924]
  We present a new paradigm for reasoning in large language models (LLMs)<n>Instead of autoregressively generating tokens, we model reasoning as a hidden Markov chain of continuous, high-dimensional "thoughts"<n>For the first time, MARCOS achieves performance comparable to token-based CoT, even surpassing it by 4.7% on GSM8K with up to 15.7x speedup in inference.
  arXiv  Detail & Related papers  (2025-09-29T16:44:22Z)
• ToDRE: Visual Token Pruning via Diversity and Task Awareness for Efficient Large Vision-Language Models [59.47738955960352]
  ToDRE is a two-stage and training-free token compression framework.<n>It achieves superior performance by pruning tokens based on token Diversity and token-task RElevance.
  arXiv  Detail & Related papers  (2025-05-24T15:47:49Z)
• Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching [64.74765550805024]
  Chain-of-Thought prompting elicits step-by-step problem solving, but often at the cost of excessive verbosity in intermediate outputs.<n>We propose Sketch-of-Thought (SoT), a prompting framework that integrates cognitively inspired reasoning paradigms with linguistic constraints.<n>SoT achieves token reductions of up to 84% with minimal accuracy loss across 18 reasoning datasets.
  arXiv  Detail & Related papers  (2025-03-07T06:57:17Z)
• Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning [53.57895922042783]
  Large Language Models (LLMs) excel at reasoning and planning when trained on chainof-thought (CoT) data.<n>We propose a hybrid representation of the reasoning process, where we partially abstract away the initial reasoning steps using latent discrete tokens.
  arXiv  Detail & Related papers  (2025-02-05T15:33:00Z)
• Disentangling Reasoning Tokens and Boilerplate Tokens For Language Model Fine-tuning [46.43130011147807]
  We argue that tokens serving different roles - specifically, reasoning tokens versus boilerplate tokens - differ significantly in importance and learning complexity.<n>We propose a novel Shuffle-Aware Discriminator (SHAD) for adaptive token discrimination.<n>Using SHAD, we propose the Reasoning-highlighted Fine-Tuning (RFT) method, which adaptively emphasizes reasoning tokens during fine-tuning.
  arXiv  Detail & Related papers  (2024-12-19T12:06:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.