Diffusion Imitation from Observation

• URL: http://arxiv.org/abs/2410.05429v1
• Date: Mon, 7 Oct 2024 18:49:55 GMT
• Title: Diffusion Imitation from Observation
• Authors: Bo-Ruei Huang, Chun-Kai Yang, Chun-Mao Lai, Dai-Jie Wu, Shao-Hua Sun,
• Abstract summary: adversarial imitation learning approaches learn a generator agent policy to produce state transitions that are indistinguishable to a discriminator. Motivated by the recent success of diffusion models in generative modeling, we propose to integrate a diffusion model into the adversarial imitation learning from observation framework.
• Score: 4.205946699819021
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning from observation (LfO) aims to imitate experts by learning from state-only demonstrations without requiring action labels. Existing adversarial imitation learning approaches learn a generator agent policy to produce state transitions that are indistinguishable to a discriminator that learns to classify agent and expert state transitions. Despite its simplicity in formulation, these methods are often sensitive to hyperparameters and brittle to train. Motivated by the recent success of diffusion models in generative modeling, we propose to integrate a diffusion model into the adversarial imitation learning from observation framework. Specifically, we employ a diffusion model to capture expert and agent transitions by generating the next state, given the current state. Then, we reformulate the learning objective to train the diffusion model as a binary classifier and use it to provide "realness" rewards for policy learning. Our proposed framework, Diffusion Imitation from Observation (DIFO), demonstrates superior performance in various continuous control domains, including navigation, locomotion, manipulation, and games. Project page: https://nturobotlearninglab.github.io/DIFO

Related papers

• Diffusing States and Matching Scores: A New Framework for Imitation Learning [16.941612670582522]
  Adversarial Imitation Learning is traditionally framed as a two-player zero-sum game between a learner and an adversarially chosen cost function. In recent years, diffusion models have emerged as a non-adversarial alternative to GANs. We show our approach outperforms GAN-style imitation learning baselines across various continuous control problems.
  arXiv  Detail & Related papers  (2024-10-17T17:59:25Z)
• Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Models [63.43422118066493]
  Machine unlearning (MU) is a crucial foundation for developing safe, secure, and trustworthy GenAI models. Traditional MU methods often rely on stringent assumptions and require access to real data. This paper introduces Score Forgetting Distillation (SFD), an innovative MU approach that promotes the forgetting of undesirable information in diffusion models.
  arXiv  Detail & Related papers  (2024-09-17T14:12:50Z)
• Diffusion Augmented Agents: A Framework for Efficient Exploration and Transfer Learning [6.06616040517684]
  DAAG hindsight relabels the agent's past experience by using diffusion models to transform videos. Large language model orchestrates this autonomous process without requiring human supervision. Results show that DAAG improves learning of reward detectors, transferring past experience, and acquiring new tasks.
  arXiv  Detail & Related papers  (2024-07-30T13:01:31Z)
• Guided Diffusion from Self-Supervised Diffusion Features [49.78673164423208]
  Guidance serves as a key concept in diffusion models, yet its effectiveness is often limited by the need for extra data annotation or pretraining. We propose a framework to extract guidance from, and specifically for, diffusion models.
  arXiv  Detail & Related papers  (2023-12-14T11:19:11Z)
• DiffAIL: Diffusion Adversarial Imitation Learning [32.90853955228524]
  Imitation learning aims to solve the problem of defining reward functions in real-world decision-making tasks. We propose a method named diffusion adversarial imitation learning (DiffAIL) Our method achieves state-of-the-art performance and significantly surpasses expert demonstration on two benchmark tasks.
  arXiv  Detail & Related papers  (2023-12-11T12:53:30Z)
• Unsupervised Discovery of Interpretable Directions in h-space of Pre-trained Diffusion Models [63.1637853118899]
  We propose the first unsupervised and learning-based method to identify interpretable directions in h-space of pre-trained diffusion models. We employ a shift control module that works on h-space of pre-trained diffusion models to manipulate a sample into a shifted version of itself. By jointly optimizing them, the model will spontaneously discover disentangled and interpretable directions.
  arXiv  Detail & Related papers  (2023-10-15T18:44:30Z)
• DiffAug: Enhance Unsupervised Contrastive Learning with Domain-Knowledge-Free Diffusion-based Data Augmentation [48.25619775814776]
  This paper proposes DiffAug, a novel unsupervised contrastive learning technique with diffusion mode-based positive data generation. DiffAug consists of a semantic encoder and a conditional diffusion model; the conditional diffusion model generates new positive samples conditioned on the semantic encoding. Experimental evaluations show that DiffAug outperforms hand-designed and SOTA model-based augmentation methods on DNA sequence, visual, and bio-feature datasets.
  arXiv  Detail & Related papers  (2023-09-10T13:28:46Z)
• Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learning [42.009856923352864]
  diffusion models have been adopted for behavioral cloning in a sequence modeling fashion. We propose Crossway Diffusion, a simple yet effective method to enhance diffusion-based visuomotor policy learning. Our experiments demonstrate the effectiveness of Crossway Diffusion in various simulated and real-world robot tasks.
  arXiv  Detail & Related papers  (2023-07-04T17:59:29Z)
• Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models [77.83923746319498]
  We propose a framework called Diff-Instruct to instruct the training of arbitrary generative models. We show that Diff-Instruct results in state-of-the-art single-step diffusion-based models. Experiments on refining GAN models show that the Diff-Instruct can consistently improve the pre-trained generators of GAN models.
  arXiv  Detail & Related papers  (2023-05-29T04:22:57Z)
• DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models [81.84866217721361]
  DiffusionBERT is a new generative masked language model based on discrete diffusion models. We propose a new noise schedule for the forward diffusion process that controls the degree of noise added at each step. Experiments on unconditional text generation demonstrate that DiffusionBERT achieves significant improvement over existing diffusion models for text.
  arXiv  Detail & Related papers  (2022-11-28T03:25:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.