Demonstration Informed Specification Search
- URL: http://arxiv.org/abs/2112.10807v4
- Date: Mon, 24 Apr 2023 04:55:57 GMT
- Title: Demonstration Informed Specification Search
- Authors: Marcell Vazquez-Chanlatte, Ameesh Shah, Gil Lederman, Sanjit A. Seshia
- Abstract summary: This paper considers the problem of learning temporal task specifications, e.g. automata and temporal logic, from expert demonstrations.
Three features make learning temporal task specifications difficult: (1) the (countably) infinite number of tasks under consideration; (2) an a-priori ignorance of what memory is needed to encode the task; and (3) the discrete solution space.
We propose Demonstration Informed Specification Search (DISS), a family of algorithms requiring only black box access to a maximum entropy planner and a task sampler from labeled examples.
- Score: 9.03950827864517
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper considers the problem of learning temporal task specifications,
e.g. automata and temporal logic, from expert demonstrations. Task
specifications are a class of sparse memory augmented rewards with explicit
support for temporal and Boolean composition. Three features make learning
temporal task specifications difficult: (1) the (countably) infinite number of
tasks under consideration; (2) an a-priori ignorance of what memory is needed
to encode the task; and (3) the discrete solution space - typically addressed
by (brute force) enumeration. To overcome these hurdles, we propose
Demonstration Informed Specification Search (DISS): a family of algorithms
requiring only black box access to a maximum entropy planner and a task sampler
from labeled examples. DISS then works by alternating between conjecturing
labeled examples to make the provided demonstrations less surprising and
sampling tasks consistent with the conjectured labeled examples. We provide a
concrete implementation of DISS in the context of tasks described by
Deterministic Finite Automata, and show that DISS is able to efficiently
identify tasks from only one or two expert demonstrations.
Related papers
- Temporal2Seq: A Unified Framework for Temporal Video Understanding Tasks [26.007846170517055]
We propose a single unified framework, coined as Temporal2Seq, to formulate the output of temporal video understanding tasks as a sequence of discrete tokens.
With this unified token representation, Temporal2Seq can train a generalist model within a single architecture on different video understanding tasks.
We evaluate our Temporal2Seq generalist model on the corresponding test sets of three tasks, demonstrating that Temporal2Seq can produce reasonable results on various tasks.
arXiv Detail & Related papers (2024-09-27T06:37:47Z) - A Minimalist Prompt for Zero-Shot Policy Learning [61.65128628634916]
Transformer-based methods have exhibited significant generalization ability when prompted with target-domain demonstrations or example solutions during inference.
We show that conditioning a decision transformer on task parameters alone can enable zero-shot generalization on par with or better than its demonstration-conditioned counterpart.
We introduce an additional learnable prompt which is demonstrated to further boost zero-shot generalization across a range of robotic control, manipulation, and navigation benchmark tasks.
arXiv Detail & Related papers (2024-05-09T19:15:33Z) - Data-CUBE: Data Curriculum for Instruction-based Sentence Representation
Learning [85.66907881270785]
We propose a data curriculum method, namely Data-CUBE, that arranges the orders of all the multi-task data for training.
In the task level, we aim to find the optimal task order to minimize the total cross-task interference risk.
In the instance level, we measure the difficulty of all instances per task, then divide them into the easy-to-difficult mini-batches for training.
arXiv Detail & Related papers (2024-01-07T18:12:20Z) - DemoNSF: A Multi-task Demonstration-based Generative Framework for Noisy
Slot Filling Task [22.105151515616363]
We propose a demonstration based generative framework for noisy slot filling, named DemoNSF.
Specifically, we introduce three noisy auxiliary tasks, namely noisy recovery (NR), random mask (RM), and hybrid discrimination (HD)
In the downstream main task, we design a noisy demonstration construction strategy for the generative framework, which explicitly incorporates task-specific information and perturbed distribution during training and inference.
Experiments on two benchmarks demonstrate that DemoNSF outperforms all baseline methods and achieves strong generalization.
arXiv Detail & Related papers (2023-10-16T08:16:53Z) - A Dynamic Feature Interaction Framework for Multi-task Visual Perception [100.98434079696268]
We devise an efficient unified framework to solve multiple common perception tasks.
These tasks include instance segmentation, semantic segmentation, monocular 3D detection, and depth estimation.
Our proposed framework, termed D2BNet, demonstrates a unique approach to parameter-efficient predictions for multi-task perception.
arXiv Detail & Related papers (2023-06-08T09:24:46Z) - Unified Demonstration Retriever for In-Context Learning [56.06473069923567]
Unified Demonstration Retriever (textbfUDR) is a single model to retrieve demonstrations for a wide range of tasks.
We propose a multi-task list-wise ranking training framework, with an iterative mining strategy to find high-quality candidates.
Experiments on 30+ tasks across 13 task families and multiple data domains show that UDR significantly outperforms baselines.
arXiv Detail & Related papers (2023-05-07T16:07:11Z) - On Steering Multi-Annotations per Sample for Multi-Task Learning [79.98259057711044]
The study of multi-task learning has drawn great attention from the community.
Despite the remarkable progress, the challenge of optimally learning different tasks simultaneously remains to be explored.
Previous works attempt to modify the gradients from different tasks. Yet these methods give a subjective assumption of the relationship between tasks, and the modified gradient may be less accurate.
In this paper, we introduce Task Allocation(STA), a mechanism that addresses this issue by a task allocation approach, in which each sample is randomly allocated a subset of tasks.
For further progress, we propose Interleaved Task Allocation(ISTA) to iteratively allocate all
arXiv Detail & Related papers (2022-03-06T11:57:18Z) - Supervised Bayesian Specification Inference from Demonstrations [11.855400596862275]
We present a probabilistic model for inferring task specification as a temporal logic formula.
We demonstrate the efficacy of our model for inferring specifications, with over 90% similarity observed between the inferred specification and the ground truth.
arXiv Detail & Related papers (2021-07-06T21:16:37Z) - MIST: Multiple Instance Self-Training Framework for Video Anomaly
Detection [76.80153360498797]
We develop a multiple instance self-training framework (MIST) to efficiently refine task-specific discriminative representations.
MIST is composed of 1) a multiple instance pseudo label generator, which adapts a sparse continuous sampling strategy to produce more reliable clip-level pseudo labels, and 2) a self-guided attention boosted feature encoder.
Our method performs comparably to or even better than existing supervised and weakly supervised methods, specifically obtaining a frame-level AUC 94.83% on ShanghaiTech.
arXiv Detail & Related papers (2021-04-04T15:47:14Z) - Conditional Channel Gated Networks for Task-Aware Continual Learning [44.894710899300435]
Convolutional Neural Networks experience catastrophic forgetting when optimized on a sequence of learning problems.
We introduce a novel framework to tackle this problem with conditional computation.
We validate our proposal on four continual learning datasets.
arXiv Detail & Related papers (2020-03-31T19:35:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.