OracleTSC: Oracle-Informed Reward Hurdle and Uncertainty Regularization for Traffic Signal Control

• URL: http://arxiv.org/abs/2605.08516v1
• Date: Fri, 08 May 2026 21:55:41 GMT
• Title: OracleTSC: Oracle-Informed Reward Hurdle and Uncertainty Regularization for Traffic Signal Control
• Abstract summary: Reinforcement learning-based traffic signal control (TSC) systems function as black boxes with limited interpretability.<n>We introduce OracleTSC, which stabilizes LLM-based TSC through two mechanisms.<n> Experiments on the LibSignal benchmark show that OracleTSC enables a compact LLaMA3-8B model to substantially improve traffic efficiency.
• Score: 15.140797677788717
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transparent decision-making is essential for traffic signal control (TSC) systems to earn public trust. However, traditional reinforcement learning-based TSC methods function as black boxes with limited interpretability. Although large language models (LLMs) can provide natural language reasoning, reinforcement finetuning for TSC remains unstable because feedback is sparse and delayed, while most actions produce only marginal changes in congestion metrics. We introduce OracleTSC, which stabilizes LLM-based TSC through two mechanisms: (1) a reward hurdle mechanism that filters weak learning signals by subtracting a calibrated threshold from environmental rewards, and (2) uncertainty regularization that maximizes the probability of the selected response to encourage consistent decisions across sampled outputs. Experiments on the LibSignal benchmark show that OracleTSC enables a compact LLaMA3-8B model to substantially improve traffic efficiency, achieving a 75% reduction in travel time and a 67% decrease in queue length compared with the pretrained baseline while preserving interpretability through natural language explanations. OracleTSC also demonstrates strong cross-intersection generalization: a policy trained on one intersection transfers to a structurally different intersection with 17% lower travel time and 39% lower queue length without additional finetuning. These results suggest that uncertainty-aware reward shaping can improve the stability and effectiveness of reinforcement fine-tuning for TSC.

Related papers

• Mitigating Bus Bunching with Reinforcement Learning Enhanced by Semantic Stop Embedding [6.0671705325267835]
  Bus bunching degrades service regularity and increases passenger waiting in high-frequency transit.<n>This study introduces an LLM-assisted semantic stop representation for event-driven bus holding control.
  arXiv  Detail & Related papers  (2026-08-10T20:21:52Z)
• SignalClaw: LLM-Guided Evolutionary Synthesis of Interpretable Traffic Signal Control Skills [8.299375016847524]
  Traffic signal control TSC requires strategies that are both effective and interpretable for deployment.<n>We present SIGNALCLAW, a framework that uses large language models as evolutionary skill generators.<n>Each skill includes rationale, selection guidance, and executable code, making policies human-inspectable and self-documenting.
  arXiv  Detail & Related papers  (2026-04-07T07:35:45Z)
• What If Consensus Lies? Selective-Complementary Reinforcement Learning at Test Time [57.533031432715084]
  Test-Time Reinforcement Learning (TTRL) enables Large Language Models (LLMs) to enhance reasoning capabilities on unlabeled test streams.<n>Existing TTRL methods rely exclusively on positive pseudo-labeling strategies.<n>We propose SCRL, a robust test-time reinforcement learning framework that effectively mitigates label noise amplification.
  arXiv  Detail & Related papers  (2026-03-20T11:47:12Z)
• Baguan-TS: A Sequence-Native In-Context Learning Model for Time Series Forecasting with Covariates [31.296823831987748]
  Baguan-TS integrates the raw-sequence representation learning with ICL, instantiated by a 3D Transformer.<n>We tackle two key hurdles: (i) calibration and training stability, improved with a feature-agnostic, target-space retrieval-based local calibration, and (ii) output oversmoothing, mitigated via context-overfitting strategy.
  arXiv  Detail & Related papers  (2026-03-18T07:24:19Z)
• A Robust and Efficient Multi-Agent Reinforcement Learning Framework for Traffic Signal Control [14.135215838687207]
  This paper proposes a robust Multi-Agent Reinforcement Learning framework validated in the Vissim traffic simulator.<n> Experimental results demonstrate that our framework outperforms standard RL baselines, reducing average waiting time by over 10%.
  arXiv  Detail & Related papers  (2026-03-12T16:02:28Z)
• Constraint-Rectified Training for Efficient Chain-of-Thought [60.52883907721588]
  Chain-of-Thought (CoT) has significantly enhanced the reasoning capabilities of Large Language Models (LLMs)<n>While longer reasoning traces can improve answer quality and unlock abilities such as self-correction, they also incur high inference costs and often introduce redundant steps, known as overthinking.<n>Recent research seeks to develop efficient reasoning strategies that balance reasoning length and accuracy.
  arXiv  Detail & Related papers  (2026-02-13T02:13:45Z)
• SED-SFT: Selectively Encouraging Diversity in Supervised Fine-Tuning [54.393763477932474]
  Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) has emerged as the standard post-training paradigm for large language models (LLMs)<n>We propose SED-SFT, which adaptively encourages diversity based on the token exploration space.<n>This framework introduces a selective entropy regularization term with a selective masking mechanism into the optimization objective.
  arXiv  Detail & Related papers  (2026-02-07T09:39:21Z)
• Enhancing the Outcome Reward-based RL Training of MLLMs with Self-Consistency Sampling [90.87033586963828]
  Outcome-reward reinforcement learning (RL) is a common and increasingly significant way to refine the step-by-step reasoning of multimodal large language models (MLLMs)<n>We propose Self-Consistency Sampling (SCS) to correct this issue.<n>Based on Qwen2.5-VL-7B-Instruct, SCS improves accuracy by up to 7.7 percentage points on six multimodal benchmarks with negligible extra computation.
  arXiv  Detail & Related papers  (2025-11-13T18:59:57Z)
• CoT-Saliency: Unified Chain-of-Thought Reasoning for Heterogeneous Saliency Tasks [96.64597365827046]
  We present the first unified framework that jointly handles three operationally heterogeneous saliency tasks.<n>We introduce a Chain-of-Thought (CoT) reasoning process in a Vision-Language Model (VLM) to bridge task heterogeneity.<n>We show our model matches or outperforms specialized SOTA methods and strong closed-source VLMs across all tasks.
  arXiv  Detail & Related papers  (2025-11-01T04:37:01Z)
• Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning [18.934789236342244]
  Large language models (LLMs) primarily rely on supervised fine-tuning (SFT) to adapt pre-trained models to domain-specific tasks such as mathematical reasoning.<n>Standard SFT uniformly penalizes all tokens, neglecting that only a small subset of critical tokens determines reasoning correctness.<n>We propose Critical Token Fine-tuning (CFT), a simple yet effective approach that updates only tokens identified as functionally indispensable via counterfactual perturbations.
  arXiv  Detail & Related papers  (2025-10-13T03:25:36Z)
• Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistency [3.6199690908942546]
  Self-Consistency (SC) generates multiple reasoning chains in parallel and selects the final answer via majority voting.<n>We propose Slim-SC, a step-wise pruning strategy that identifies and removes redundant chains using inter-chain similarity at the thought level.<n> Experiments show that Slim-SC reduces latency and KVC usage by up to 45% and 26%, respectively, with R1-Distill.
  arXiv  Detail & Related papers  (2025-09-17T14:00:51Z)
• Guiding not Forcing: Enhancing the Transferability of Jailbreaking Attacks on LLMs via Removing Superfluous Constraints [81.14852921721793]
  This study aims to understand and enhance the transferability of gradient-based jailbreaking methods.<n>We introduce a novel conceptual framework to elucidate transferability and identify superfluous constraints.<n>Our method increases the overall Transfer Attack Success Rate (T-ASR) across a set of target models with varying safety levels from 18.4% to 50.3%.
  arXiv  Detail & Related papers  (2025-02-25T07:47:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.