Revisiting Chain-of-Thought Reasoning under Limited Supervision: Semi-supervised Chain-of-Thought Learning

• URL: http://arxiv.org/abs/2607.01511v1
• Date: Wed, 01 Jul 2026 22:17:39 GMT
• Title: Revisiting Chain-of-Thought Reasoning under Limited Supervision: Semi-supervised Chain-of-Thought Learning
• Abstract summary: We propose a framework that uses unlabeled questions to construct pseudo reasoning supervision.<n>Semi-CoT samples multiple pseudo-CoTs for each unlabeled question, estimates answer-level semantic entropy, and selects low-entropy reasoning chains as reliable pseudo-CoT demonstrations.<n>Pilot experiments on AQuA, SVAMP, GSM8K, and MultiArith show that the entropy gate selects high-precision pseudo-CoTs, with pseudo-answer precision ranging from $91.36%$ to $100%$.
• Score: 17.893249958828218
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Chain-of-thought (CoT) reasoning has emerged as an effective approach for activating latent reasoning capabilities in large language models. However, most existing CoT methods use reasoning chains mainly as inference-time prompts, while the generated reasoning traces are rarely reused as semi-supervised learning signals. In this report, we define \textbf{Semi-supervised Chain-of-Thought Learning} and propose \textbf{Semi-CoT}, a simple framework that uses unlabeled questions to construct pseudo reasoning supervision. Semi-CoT samples multiple pseudo-CoTs for each unlabeled question, estimates answer-level semantic entropy, and selects low-entropy reasoning chains as reliable pseudo-CoT demonstrations. This extends the self-training view of CoT from inference-time refinement to semi-supervised pseudo-supervision. Pilot experiments on AQuA, SVAMP, GSM8K, and MultiArith show that the entropy gate selects high-precision pseudo-CoTs, with pseudo-answer precision ranging from $91.36\%$ to $100\%$. Semi-CoT also gives small gains on SVAMP and GSM8K, while AQuA shows negative transfer and MultiArith reaches a ceiling. These results suggest that unlabeled questions can provide reliable pseudo reasoning signals, but their effective use still requires stronger demonstration selection or student training.

Related papers

• Demystifying Entropy-based Selection for Chain-of-Thought Compression in Large Reasoning Models [16.372524974789492]
  Entropy-based pruning has been proposed as an effective method for compressing Chain-of-Thought (CoT) reasoning with negligible accuracy loss.<n>We show that entropy offers no advantage over random pruning in any evaluated setting.
  arXiv  Detail & Related papers  (2026-07-30T15:59:51Z)
• WhisperRec: Latent Reasoning for Efficient Foundation Recommendation Models [51.88075626243057]
  We propose WhisperRec, an efficient latent reasoning framework for foundation recommendation models (FRMs)<n>WhisperRec compresses teacher-generated Chain-of-Thought (CoT) into learnable latent reasoning tokens.<n>We show that WhisperRec consistently outperforms explicit-CoT methods and conventional baselines.
  arXiv  Detail & Related papers  (2026-07-29T08:48:35Z)
• Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probing [14.176238828329952]
  Large language models (LLMs) can produce chains of thought (CoT) that do not accurately reflect the actual factors driving their answers.<n>We show that motivated reasoning can be identified by probing internal activations.
  arXiv  Detail & Related papers  (2026-03-17T23:03:21Z)
• CoLT: Reasoning with Chain of Latent Tool Calls [31.228763375347608]
  Chain-of-Thought (CoT) is a critical technique in enhancing the reasoning ability of Large Language Models (LLMs)<n>We propose CoLT, a novel framework that implements latent reasoning as tool calls''
  arXiv  Detail & Related papers  (2026-02-04T06:12:53Z)
• ChainV: Atomic Visual Hints Make Multimodal Reasoning Shorter and Better [59.29940512530982]
  We propose ChainV, a framework that dynamically integrates visual hints into the reasoning process.<n>Our method significantly improves reasoning accuracy and efficiency, especially on math-intensive benchmarks.
  arXiv  Detail & Related papers  (2025-11-21T10:11:17Z)
• Reinforced Latent Reasoning for LLM-based Recommendation [92.56166822197919]
  Large Language Models (LLMs) have demonstrated impressive reasoning capabilities in complex problem-solving tasks.<n>Existing methods typically rely on fine-tuning with explicit chain-of-thought (CoT) data.<n>In this work, we explore an alternative approach that shifts from explicit CoT reasoning to compact, information-dense latent reasoning.
  arXiv  Detail & Related papers  (2025-05-25T11:03:45Z)
• CoT Information: Improved Sample Complexity under Chain-of-Thought Supervision [10.29575414214269]
  Chain-of-thought (CoT) supervision provides intermediate reasoning steps together with the final output.<n>This paper develops a statistical theory of learning under CoT supervision.
  arXiv  Detail & Related papers  (2025-05-21T18:28:54Z)
• Fractured Chain-of-Thought Reasoning [61.647243580650446]
  We introduce Fractured Sampling, a unified inference-time strategy that interpolates between full CoT and solution-only sampling.<n>We show that Fractured Sampling consistently achieves superior accuracy-cost trade-offs, yielding steep log-linear scaling gains in Pass@k versus token budget.
  arXiv  Detail & Related papers  (2025-05-19T11:30:41Z)
• Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought [64.43689151961054]
  We prove that a two-layer transformer with $D$ steps of continuous CoTs can solve the directed graph reachability problem.<n>In our construction, each continuous thought vector is a superposition state that encodes multiple search frontiers simultaneously.
  arXiv  Detail & Related papers  (2025-05-18T18:36:53Z)
• Reasoning Models Don't Always Say What They Think [48.05987314492555]
  Chain-of-thought (CoT) allows monitoring a model's intentions and reasoning processes.<n>We evaluate CoT faithfulness of state-of-the-art reasoning models across 6 reasoning hints presented in prompts.
  arXiv  Detail & Related papers  (2025-05-08T16:51:43Z)
• Mitigating Misleading Chain-of-Thought Reasoning with Selective Filtering [59.495717939664246]
  Large language models have manifested remarkable capabilities by leveraging chain-of-thought (CoT) reasoning techniques to solve intricate questions. We propose a novel approach called the selective filtering reasoner (SelF-Reasoner) that assesses the entailment relationship between the question and the candidate reasoning chain. SelF-Reasoner improves the fine-tuned T5 baseline consistently over the ScienceQA, ECQA, and LastLetter tasks.
  arXiv  Detail & Related papers  (2024-03-28T06:28:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.