Modular Networks for Compositional Instruction Following

• URL: http://arxiv.org/abs/2010.12764v2
• Date: Tue, 13 Apr 2021 05:34:01 GMT
• Title: Modular Networks for Compositional Instruction Following
• Authors: Rodolfo Corona, Daniel Fried, Coline Devin, Dan Klein, Trevor Darrell
• Abstract summary: We propose a modular architecture for following natural language instructions that describe sequences of diverse subgoals. A sequence of modules to execute is chosen by learning to segment the instructions and predicting a subgoal type for each segment. When compared to standard, non-modular sequence-to-sequence approaches on ALFRED, a challenging instruction following benchmark, we find that modularization improves generalization to novel subgoal compositions.
• Score: 102.152217117883
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Standard architectures used in instruction following often struggle on novel compositions of subgoals (e.g. navigating to landmarks or picking up objects) observed during training. We propose a modular architecture for following natural language instructions that describe sequences of diverse subgoals. In our approach, subgoal modules each carry out natural language instructions for a specific subgoal type. A sequence of modules to execute is chosen by learning to segment the instructions and predicting a subgoal type for each segment. When compared to standard, non-modular sequence-to-sequence approaches on ALFRED, a challenging instruction following benchmark, we find that modularization improves generalization to novel subgoal compositions, as well as to environments unseen in training.

Related papers

• Controllable Navigation Instruction Generation with Chain of Thought Prompting [74.34604350917273]
  We propose C-Instructor, which utilizes the chain-of-thought-style prompt for style-controllable and content-controllable instruction generation. C-Instructor renders generated instructions more accessible to follow and offers greater controllability over the manipulation of landmark objects.
  arXiv  Detail & Related papers  (2024-07-10T07:37:20Z)
• Chain-of-Instructions: Compositional Instruction Tuning on Large Language Models [15.444719480373001]
  We propose a novel concept of compositional instructions called chain-of-instructions (CoI) Unlike the conventional practice of solving single instruction tasks, our proposed method encourages a model to solve each subtask step by step until the final answer is reached. CoI-tuning improves the model's ability to handle instructions composed of multiple subtasks as well as unseen composite tasks such as multilingual summarization.
  arXiv  Detail & Related papers  (2024-02-18T10:10:40Z)
• From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning [63.63840740526497]
  We investigate how instruction tuning adjusts pre-trained models with a focus on intrinsic changes. The impact of instruction tuning is then studied by comparing the explanations derived from the pre-trained and instruction-tuned models. Our findings reveal three significant impacts of instruction tuning.
  arXiv  Detail & Related papers  (2023-09-30T21:16:05Z)
• Compositional Program Generation for Few-Shot Systematic Generalization [59.57656559816271]
  This study on a neuro-symbolic architecture called the Compositional Program Generator (CPG) CPG has three key features: textitmodularity, textitcomposition, and textitabstraction, in the form of grammar rules. It perfect achieves generalization on both the SCAN and COGS benchmarks using just 14 examples for SCAN and 22 examples for COGS.
  arXiv  Detail & Related papers  (2023-09-28T14:33:20Z)
• Learning Label Modular Prompts for Text Classification in the Wild [56.66187728534808]
  We propose text classification in-the-wild, which introduces different non-stationary training/testing stages. Decomposing a complex task into modular components can enable robust generalisation under such non-stationary environment. We propose MODULARPROMPT, a label-modular prompt tuning framework for text classification tasks.
  arXiv  Detail & Related papers  (2022-11-30T16:26:38Z)
• Compositional Generalization in Grounded Language Learning via Induced Model Sparsity [81.38804205212425]
  We consider simple language-conditioned navigation problems in a grid world environment with disentangled observations. We design an agent that encourages sparse correlations between words in the instruction and attributes of objects, composing them together to find the goal. Our agent maintains a high level of performance on goals containing novel combinations of properties even when learning from a handful of demonstrations.
  arXiv  Detail & Related papers  (2022-07-06T08:46:27Z)
• Neuro-Symbolic Hierarchical Rule Induction [12.610497441047395]
  We propose an efficient interpretable neuro-symbolic model to solve Inductive Logic Programming (ILP) problems. In this model, which is built from a set of meta-rules organised in a hierarchical structure, first-order rules are invented by learning embeddings to match facts and body predicates of a meta-rule. We empirically validate our model on various tasks (ILP, visual genome, reinforcement learning) against several state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-26T17:02:14Z)
• Towards Evaluating Plan Generation Approaches with Instructional Texts [0.0]
  We introduce a dataset consisting of 83 textual instructions in English language, their refinement in a more structured form as well as manually developed plans for each of the instructions.
  arXiv  Detail & Related papers  (2020-01-13T12:35:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.