Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloning

• URL: http://arxiv.org/abs/2506.10137v2
• Date: Wed, 15 Oct 2025 15:29:39 GMT
• Title: Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloning
• Abstract summary: We show how long-range temporal consistency via successor representations can facilitate generalization.<n>We then propose a simple yet effective representation learning objective, $textBYOL-gamma$ for GCBC.
• Score: 20.1753113722028
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While goal-conditioned behavior cloning (GCBC) methods can perform well on in-distribution training tasks, they do not necessarily generalize zero-shot to tasks that require conditioning on novel state-goal pairs, i.e. combinatorial generalization. In part, this limitation can be attributed to a lack of temporal consistency in the state representation learned by BC; if temporally correlated states are properly encoded to similar latent representations, then the out-of-distribution gap for novel state-goal pairs would be reduced. We formalize this notion by demonstrating how encouraging long-range temporal consistency via successor representations (SR) can facilitate generalization. We then propose a simple yet effective representation learning objective, $\text{BYOL-}\gamma$ for GCBC, which theoretically approximates the successor representation in the finite MDP case through self-predictive representations, and achieves competitive empirical performance across a suite of challenging tasks requiring combinatorial generalization.

Related papers

• Demonstrating Generalization Failures via Mixtures of Conditional Policies [5.640047328020258]
  We propose a simple and flexible way to construct language models which fail to generalize in controllable ways when subsequently trained with Reinforcement Learning (RL)<n>We observe that RL training then selects for policies that obtain the highest reward on the training distribution.<n>We also use our construction to illustrate two novel ways in which generalization may fail in future language models.
  arXiv  Detail & Related papers  (2026-07-03T16:43:59Z)
• Cross-Modal-Domain Generalization Through Semantically Aligned Discrete Representations [4.67724003380452]
  Multimodal learning seeks to integrate information across diverse sensory sources.<n>Current approaches struggle to balance cross-modal generalizability with modality-specific structure.<n>We introduce CoDAAR, a novel framework that establishes semantic consensus across modality-specific codebooks.
  arXiv  Detail & Related papers  (2026-05-12T14:03:30Z)
• Dual Goal Representations [57.43956630070019]
  We introduce dual goal representations for goal-conditioned reinforcement learning (GCRL)<n>A dual goal representation characterizes a state by "the set of temporal distances from all other states"<n>We empirically show that dual goal representations consistently improve offline goal-reaching performance across 20 state- and pixel-based tasks.
  arXiv  Detail & Related papers  (2025-10-08T07:07:39Z)
• Understanding and Improving Length Generalization in Recurrent Models [21.417068818034817]
  recurrent models can process arbitrarily long sequences, but their performance sometimes drops considerably beyond their training context lengths.<n>We show that models fail to length generalize when during training they are only exposed to a limited subset of the distribution of all attainable states.<n>We investigate simple training interventions that aim to increase the coverage of the states that the model is trained on.
  arXiv  Detail & Related papers  (2025-07-03T16:45:50Z)
• Non-autoregressive Sequence-to-Sequence Vision-Language Models [59.445765313094434]
  We propose a parallel decoding sequence-to-sequence vision-language model that marginalizes over multiple inference paths in the decoder.<n>The model achieves performance on-par with its state-of-the-art autoregressive counterpart, but is faster at inference time.
  arXiv  Detail & Related papers  (2024-03-04T17:34:59Z)
• Foundation Policies with Hilbert Representations [54.44869979017766]
  We propose an unsupervised framework to pre-train generalist policies from unlabeled offline data. Our key insight is to learn a structured representation that preserves the temporal structure of the underlying environment. Our experiments show that our unsupervised policies can solve goal-conditioned and general RL tasks in a zero-shot fashion.
  arXiv  Detail & Related papers  (2024-02-23T19:09:10Z)
• Time-series Generation by Contrastive Imitation [87.51882102248395]
  We study a generative framework that seeks to combine the strengths of both: Motivated by a moment-matching objective to mitigate compounding error, we optimize a local (but forward-looking) transition policy. At inference, the learned policy serves as the generator for iterative sampling, and the learned energy serves as a trajectory-level measure for evaluating sample quality.
  arXiv  Detail & Related papers  (2023-11-02T16:45:25Z)
• A State Representation for Diminishing Rewards [20.945260614372327]
  A common setting in multitask reinforcement learning (RL) demands that an agent rapidly adapt to various stationary reward functions randomly sampled from a fixed distribution. In the natural world, sequential tasks are rarely independent, and instead reflect shifting priorities based on the availability and subjective perception of rewarding stimuli. We introduce the $lambda$ representation ($lambda$R) which, surprisingly, is required for policy evaluation in this setting.
  arXiv  Detail & Related papers  (2023-09-07T13:38:36Z)
• Self-regulating Prompts: Foundational Model Adaptation without Forgetting [112.66832145320434]
  We introduce a self-regularization framework for prompting called PromptSRC. PromptSRC guides the prompts to optimize for both task-specific and task-agnostic general representations.
  arXiv  Detail & Related papers  (2023-07-13T17:59:35Z)
• Coherent Soft Imitation Learning [17.345411907902932]
  Imitation learning methods seek to learn from an expert either through behavioral cloning (BC) of the policy or inverse reinforcement learning (IRL) of the reward. This work derives an imitation method that captures the strengths of both BC and IRL.
  arXiv  Detail & Related papers  (2023-05-25T21:54:22Z)
• Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning [79.83792914684985]
  We prove a new identifiability result that provides conditions under which maximally sparse base-predictors yield disentangled representations. Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem.
  arXiv  Detail & Related papers  (2022-11-26T21:02:09Z)
• Behavior Prior Representation learning for Offline Reinforcement Learning [23.200489608592694]
  We introduce a simple, yet effective approach for learning state representations. Our method, Behavior Prior Representation (BPR), learns state representations with an easy-to-integrate objective based on behavior cloning of the dataset. We show that BPR combined with existing state-of-the-art Offline RL algorithms leads to significant improvements across several offline control benchmarks.
  arXiv  Detail & Related papers  (2022-11-02T04:15:20Z)
• Unsupervised Learning of Structured Representations via Closed-Loop Transcription [21.78655495464155]
  This paper proposes an unsupervised method for learning a unified representation that serves both discriminative and generative purposes. We show that a unified representation can enjoy the mutual benefits of having both. These structured representations enable classification close to state-of-the-art unsupervised discriminative representations.
  arXiv  Detail & Related papers  (2022-10-30T09:09:05Z)
• Self-Supervised Learning via Maximum Entropy Coding [57.56570417545023]
  We propose Maximum Entropy Coding (MEC) as a principled objective that explicitly optimize on the structure of the representation. MEC learns a more generalizable representation than previous methods based on specific pretext tasks. It achieves state-of-the-art performance consistently on various downstream tasks, including not only ImageNet linear probe, but also semi-supervised classification, object detection, instance segmentation, and object tracking.
  arXiv  Detail & Related papers  (2022-10-20T17:58:30Z)
• Contrastive Learning for Fair Representations [50.95604482330149]
  Trained classification models can unintentionally lead to biased representations and predictions. Existing debiasing methods for classification models, such as adversarial training, are often expensive to train and difficult to optimise. We propose a method for mitigating bias by incorporating contrastive learning, in which instances sharing the same class label are encouraged to have similar representations.
  arXiv  Detail & Related papers  (2021-09-22T10:47:51Z)
• Parallelized Reverse Curriculum Generation [62.25453821794469]
  For reinforcement learning, it is challenging for an agent to master a task that requires a specific series of actions due to sparse rewards. reverse curriculum generation (RCG) provides a reverse expansion approach that automatically generates a curriculum for the agent to learn. We propose a parallelized approach that simultaneously trains multiple AC pairs and periodically exchanges their critics.
  arXiv  Detail & Related papers  (2021-08-04T15:58:35Z)
• Conditional Meta-Learning of Linear Representations [57.90025697492041]
  Standard meta-learning for representation learning aims to find a common representation to be shared across multiple tasks. In this work we overcome this issue by inferring a conditioning function, mapping the tasks' side information into a representation tailored to the task at hand. We propose a meta-algorithm capable of leveraging this advantage in practice.
  arXiv  Detail & Related papers  (2021-03-30T12:02:14Z)
• Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependencies [91.92060221982064]
  We propose Non-Autoregressive Predictive Coding (NPC), a self-supervised method to learn a speech representation in a non-autoregressive manner. NPC has a conceptually simple objective and can be implemented easily with the introduced Masked Convolution Blocks. We show that the NPC representation is comparable to other methods in speech experiments on phonetic and speaker classification while being more efficient.
  arXiv  Detail & Related papers  (2020-11-01T02:48:37Z)
• Augmenting GAIL with BC for sample efficient imitation learning [5.199454801210509]
  We present a simple and elegant method to combine behavior cloning and GAIL to enable stable and sample efficient learning. Our algorithm is very simple to implement and integrates with different policy gradient algorithms. We demonstrate the effectiveness of the algorithm in low dimensional control tasks, gridworlds and in high dimensional image-based tasks.
  arXiv  Detail & Related papers  (2020-01-21T22:28:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.