Memory-Consistent Neural Networks for Imitation Learning

• URL: http://arxiv.org/abs/2310.06171v2
• Date: Sat, 16 Mar 2024 15:06:07 GMT
• Title: Memory-Consistent Neural Networks for Imitation Learning
• Authors: Kaustubh Sridhar, Souradeep Dutta, Dinesh Jayaraman, James Weimer, Insup Lee,
• Abstract summary: Imitation learning considerably simplifies policy synthesis compared to alternative approaches. Errors away from the training samples are particularly critical for such imitation policies. We design a model class to counter the compounding error phenomenon.
• Score: 19.849671682644104
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Imitation learning considerably simplifies policy synthesis compared to alternative approaches by exploiting access to expert demonstrations. For such imitation policies, errors away from the training samples are particularly critical. Even rare slip-ups in the policy action outputs can compound quickly over time, since they lead to unfamiliar future states where the policy is still more likely to err, eventually causing task failures. We revisit simple supervised ``behavior cloning'' for conveniently training the policy from nothing more than pre-recorded demonstrations, but carefully design the model class to counter the compounding error phenomenon. Our ``memory-consistent neural network'' (MCNN) outputs are hard-constrained to stay within clearly specified permissible regions anchored to prototypical ``memory'' training samples. We provide a guaranteed upper bound for the sub-optimality gap induced by MCNN policies. Using MCNNs on 10 imitation learning tasks, with MLP, Transformer, and Diffusion backbones, spanning dexterous robotic manipulation and driving, proprioceptive inputs and visual inputs, and varying sizes and types of demonstration data, we find large and consistent gains in performance, validating that MCNNs are better-suited than vanilla deep neural networks for imitation learning applications. Website: https://sites.google.com/view/mcnn-imitation

Related papers

• Robust Instant Policy: Leveraging Student's t-Regression Model for Robust In-context Imitation Learning of Robot Manipulation [4.545367391076448]
  We propose a new robust in-context imitation learning algorithm called the robust instant policy (RIP)<n>RIP generates several candidate robot trajectories to complete a given task from an LLM and aggregates them using the Student's t-distribution.<n>Our experiments, conducted in both simulated and real-world environments, show that RIP significantly outperforms state-of-the-art IL methods.
  arXiv  Detail & Related papers  (2025-06-18T06:02:06Z)
• MOREL: Enhancing Adversarial Robustness through Multi-Objective Representation Learning [1.534667887016089]
  deep neural networks (DNNs) are vulnerable to slight adversarial perturbations. We show that strong feature representation learning during training can significantly enhance the original model's robustness. We propose MOREL, a multi-objective feature representation learning approach, encouraging classification models to produce similar features for inputs within the same class, despite perturbations.
  arXiv  Detail & Related papers  (2024-10-02T16:05:03Z)
• Augmented Neural Fine-Tuning for Efficient Backdoor Purification [16.74156528484354]
  Recent studies have revealed the vulnerability of deep neural networks (DNNs) to various backdoor attacks. We propose Neural mask Fine-Tuning (NFT) with an aim to optimally re-organize the neuron activities. NFT relaxes the trigger synthesis process and eliminates the requirement of the adversarial search module.
  arXiv  Detail & Related papers  (2024-07-14T02:36:54Z)
• Decouple Graph Neural Networks: Train Multiple Simple GNNs Simultaneously Instead of One [60.5818387068983]
  Graph neural networks (GNN) suffer from severe inefficiency. We propose to decouple a multi-layer GNN as multiple simple modules for more efficient training. We show that the proposed framework is highly efficient with reasonable performance.
  arXiv  Detail & Related papers  (2023-04-20T07:21:32Z)
• Intelligence Processing Units Accelerate Neuromorphic Learning [52.952192990802345]
  Spiking neural networks (SNNs) have achieved orders of magnitude improvement in terms of energy consumption and latency. We present an IPU-optimized release of our custom SNN Python package, snnTorch.
  arXiv  Detail & Related papers  (2022-11-19T15:44:08Z)
• Desire Backpropagation: A Lightweight Training Algorithm for Multi-Layer Spiking Neural Networks based on Spike-Timing-Dependent Plasticity [13.384228628766236]
  Spiking neural networks (SNNs) are a viable alternative to conventional artificial neural networks. We present desire backpropagation, a method to derive the desired spike activity of all neurons, including the hidden ones. We trained three-layer networks to classify MNIST and Fashion-MNIST images and reached an accuracy of 98.41% and 87.56%, respectively.
  arXiv  Detail & Related papers  (2022-11-10T08:32:13Z)
• RelaxLoss: Defending Membership Inference Attacks without Losing Utility [68.48117818874155]
  We propose a novel training framework based on a relaxed loss with a more achievable learning target. RelaxLoss is applicable to any classification model with added benefits of easy implementation and negligible overhead. Our approach consistently outperforms state-of-the-art defense mechanisms in terms of resilience against MIAs.
  arXiv  Detail & Related papers  (2022-07-12T19:34:47Z)
• Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State [66.2457134675891]
  Spiking neural networks (SNNs) are brain-inspired models that enable energy-efficient implementation on neuromorphic hardware. Most existing methods imitate the backpropagation framework and feedforward architectures for artificial neural networks. We propose a novel training method that does not rely on the exact reverse of the forward computation.
  arXiv  Detail & Related papers  (2021-09-29T07:46:54Z)
• Stochastic-Shield: A Probabilistic Approach Towards Training-Free Adversarial Defense in Quantized CNNs [0.0]
  Quantized neural networks (NNs) are the common standard to efficiently deploy deep learning models on tiny hardware platforms. We show that it is possible to jointly achieve efficiency and robustness by accurately enabling each module without the burden of re-retraining or ad hoc fine-tuning.
  arXiv  Detail & Related papers  (2021-05-13T18:59:15Z)
• S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration [74.5509794733707]
  We present a novel guided learning paradigm from real-valued to distill binary networks on the final prediction distribution. Our proposed method can boost the simple contrastive learning baseline by an absolute gain of 5.515% on BNNs. Our method achieves substantial improvement over the simple contrastive learning baseline, and is even comparable to many mainstream supervised BNN methods.
  arXiv  Detail & Related papers  (2021-02-17T18:59:28Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.