CIG: Exploration via Conditional Information Gain

• URL: http://arxiv.org/abs/2605.20878v1
• Date: Wed, 20 May 2026 08:15:57 GMT
• Title: CIG: Exploration via Conditional Information Gain
• Abstract summary: Intrinsic rewards for exploration in reinforcement learning condition on different contexts.<n> Conditional Information Gain (CIG) reward as a tractable surrogate.<n>We instantiate CIG in a model-based setting, where rollouts are short and within-rollout corrections remain largely unexplored.
• Score: 10.289348661352841
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Intrinsic rewards for exploration in reinforcement learning condition on different contexts: lifelong rewards score each transition against accumulated experience but ignore within-rollout redundancy; episodic rewards penalize intra-trajectory repetition but discard lifetime progress. Hybrid methods combine both signals through heuristic weights or require Gaussian-process dynamics that do not scale beyond low-dimensional state spaces. Trajectory-level information gain decomposes into per-step terms that condition on the replay buffer and rollout prefix simultaneously, but remains intractable for deep models. We derive the Conditional Information Gain (CIG) reward as a tractable surrogate: a log-determinant objective over an ensemble disagreement kernel whose Cholesky factorization yields causal per-step rewards that retain both conditioning sets while scaling to high-dimensional state spaces. We instantiate CIG in a model-based setting, where rollouts are short and within-rollout corrections remain largely unexplored. Across twelve tasks spanning discrete (MiniGrid) and continuous control (OGBench), in both clean and stochastic-distractor settings, CIG outperforms or matches prior exploration methods while remaining robust to stochastic distractors.

Related papers

• ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning [74.19730275669227]
  ARISE-RL is a novel full-cycle self-evolution framework for training open-ended agents.<n>We present Reward-Gated Self-Evolution Distillation (RG-SED), which distills a memory-augmented variant of the same policy back into itself.<n>We also present ECR-Bench, an expert-calibrated rubric benchmark suite covering single-tool deep research and multi-tool travel planning.
  arXiv  Detail & Related papers  (2026-09-01T10:54:13Z)
• iSTAR: an algebraic-collapse framework for variational reduction in quantum-inspired continuous Ising solvers [0.9704960686964833]
  Continuous Ising solvers embed a discrete optimization problem into a continuous dynamical system and recover the spin configuration by sign readout.<n>We show that this cost is not intrinsic; during late-stage simulated bifurcation the trajectory collapses onto a lower-dimensional active subspace.<n>An online certified implementation on the G-set benchmark preserves the same-seed baseline in all runs and removes on average 64.4% of the dense interaction work.
  arXiv  Detail & Related papers  (2026-07-05T02:02:19Z)
• REAL-OW: Rehearsal-free Open World Object Detection with Low-Rank Adaptation and Dual-Stage Objectness Modeling [56.888597798431704]
  Open-World Object Detection (OWOD) requires detectors to identify previously unseen objects as unknown and incrementally incorporate them into the set of known categories.<n>Existing frameworks rely heavily on exemplar replay to mitigate catastrophic forgetting, but in some real applications, storing raw data conflicts with data access restrictions and leads to data exposure risks.<n>We propose REAL-OW, a novel rehearsal-free framework that decouples incremental knowledge through a collaborative adapter architecture.
  arXiv  Detail & Related papers  (2026-07-03T06:38:54Z)
• Reward-free Pretraining for Reinforcement Learning via Occupancy Coverage Maximization [31.035684025590417]
  Sparse rewards pose a central challenge in reinforcement learning, since agents receive no informative signal until they reach their goal.<n>We present a method to pre-train transferable exploration policies that rapidly adapt to sparse rewards at downstream task time.
  arXiv  Detail & Related papers  (2026-06-19T09:47:38Z)
• Controller-Augmented Hidden Markov Models: A Computational Framework for Constrained Sequential Inference [0.0]
  We present Controller-Augmented Hidden Markov Models (CHMMs)<n>CHMMs is a framework that compiles each constraint into a finite-state controller tracking the minimal sufficient history.<n>We evaluate CHMMs against 6 alternative decoders on 3 real-world sequence-labeling tasks.
  arXiv  Detail & Related papers  (2026-06-11T19:29:09Z)
• Cycle Consistency in Video Object-Centric Learning [33.833061023960546]
  Self-supervised video Object-Centric Learning aims to discover distinct objects and associate them across time.<n>Cycle Consistency (CC) cannot naively or explicitly apply to the latent slot space of OCL.<n>We propose Implicit Cycle Consistency (ICC) which shifts the cycle-consistency constraint from the restrictive slot space to the continuous reconstruction manifold.
  arXiv  Detail & Related papers  (2026-05-28T16:45:24Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• QHyer: Q-conditioned Hybrid Attention-mamba Transformer for Offline Goal-conditioned RL [34.9638718909932]
  offline goal-conditioned RL (GCRL) learns goal-reaching policies from static datasets.<n>Real-world datasets are often partially observable and history-dependent, exhibiting a mix of Markovian and non-Markovian.<n>textbfQHyer introduces a gated Hybrid Attention-Mamba backbone that performs content-adaptive history compression.
  arXiv  Detail & Related papers  (2026-05-03T13:11:28Z)
• Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibration [76.08899010904652]
  CapCal is a training-free framework that mechanically decouples positional bias from ranking decisions.<n>It achieves superior performance among training-free methods while preserving single-pass efficiency.
  arXiv  Detail & Related papers  (2026-04-11T10:47:22Z)
• Dual-Stage Invariant Continual Learning under Extreme Visual Sparsity [8.16821029459195]
  We show that background-driven gradients destabilize the feature backbone during sequential domain shifts.<n>We propose a dual-stage invariant continual learning framework via joint distillation.<n> Experiments on a high-resolution space-based RSO detection dataset show consistent improvement over established continual object detection methods.
  arXiv  Detail & Related papers  (2026-03-27T09:03:49Z)
• Certified Quantum Schrödinger Control via Hierarchical Tucker Models [2.033424698590539]
  This paper develops a framework for sampled-data feedback control implemented with fixed-rank HT projections.<n>By viewing each truncation as a bounded, rank-dependent of the nominal closed loop, we show that the HT-projected dynamics are practically exponentially stable.
  arXiv  Detail & Related papers  (2026-03-20T17:46:50Z)
• Forecasting as Rendering: A 2D Gaussian Splatting Framework for Time Series Forecasting [79.37674445572462]
  Time series forecasting (TSF) remains a challenging problem due to the intricate entanglement of intraperiod-fluctuations and interperiod-trends.<n>Treating reshaped tensors as static images results in a topological mismatch.<n> relying on uniform fixed-size representations allocates modeling capacity inefficiently.<n>TimeGS is a novel framework that fundamentally shifts the forecasting paradigm from regression to 2D generative rendering.
  arXiv  Detail & Related papers  (2026-02-10T14:13:36Z)
• TGSFormer: Scalable Temporal Gaussian Splatting for Embodied Semantic Scene Completion [38.31128137855632]
  Embodied 3D Semantic Scene Completion infers dense geometry and semantics from continuous egocentric observations.<n>Recent depth-guided approach alleviates this issue but remains local, suffering from latency and memory overhead as scale increases.<n>We propose TGSFormer, a scalable Temporal Gaussian Splatting framework for embodied SSC.
  arXiv  Detail & Related papers  (2025-11-29T03:47:14Z)
• Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Model [63.336123527432136]
  We introduce Bench2Drive-R, a generative framework that enables reactive closed-loop evaluation.<n>Unlike existing video generative models for autonomous driving, the proposed designs are tailored for interactive simulation.<n>We compare the generation quality of Bench2Drive-R with existing generative models and achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-12-11T06:35:18Z)
• OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learning [57.43911113915546]
  Few-Shot Class-Incremental Learning (FSCIL) introduces a paradigm in which the problem space expands with limited data. FSCIL methods inherently face the challenge of catastrophic forgetting as data arrives incrementally. We propose the OrCo framework built on two core principles: features' orthogonality in the representation space, and contrastive learning.
  arXiv  Detail & Related papers  (2024-03-27T13:30:48Z)
• Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding [55.107555305760954]
  We propose a conceptually simple yet effective method that attributes forgetting to layer-wise parameter overwriting and the resulting decision boundary distortion. Our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02x the base model.
  arXiv  Detail & Related papers  (2024-01-17T09:01:29Z)
• Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic [59.94347858883343]
  This paper investigates the motion planning of autonomous dynamical systems modeled by Markov decision processes (MDP) The novelty is to design an embedded product MDP (EP-MDP) between the LDGBA and the MDP. The proposed LDGBA-based reward shaping and discounting schemes for the model-free reinforcement learning (RL) only depend on the EP-MDP states.
  arXiv  Detail & Related papers  (2021-02-24T01:11:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.