How Modular Should Neural Module Networks Be for Systematic Generalization?

• URL: http://arxiv.org/abs/2106.08170v1
• Date: Tue, 15 Jun 2021 14:13:47 GMT
• Title: How Modular Should Neural Module Networks Be for Systematic Generalization?
• Authors: Vanessa D'Amario, Tomotake Sasaki, Xavier Boix
• Abstract summary: NMNs aim at Visual Question Answering (VQA) via composition of modules that tackle a sub-task. In this paper, we demonstrate that the stage and the degree at which modularity is defined has large influence on systematic generalization.
• Score: 4.533408938245526
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural Module Networks (NMNs) aim at Visual Question Answering (VQA) via composition of modules that tackle a sub-task. NMNs are a promising strategy to achieve systematic generalization, i.e. overcoming biasing factors in the training distribution. However, the aspects of NMNs that facilitate systematic generalization are not fully understood. In this paper, we demonstrate that the stage and the degree at which modularity is defined has large influence on systematic generalization. In a series of experiments on three VQA datasets (MNIST with multiple attributes, SQOOP, and CLEVR-CoGenT), our results reveal that tuning the degree of modularity in the network, especially at the image encoder stage, reaches substantially higher systematic generalization. These findings lead to new NMN architectures that outperform previous ones in terms of systematic generalization.

Related papers

• Breaking Neural Network Scaling Laws with Modularity [8.482423139660153]
  We show how the amount of training data required to generalize varies with the intrinsic dimensionality of a task's input. We then develop a novel learning rule for modular networks to exploit this advantage.
  arXiv  Detail & Related papers  (2024-09-09T16:43:09Z)
• Generalization and Estimation Error Bounds for Model-based Neural Networks [78.88759757988761]
  We show that the generalization abilities of model-based networks for sparse recovery outperform those of regular ReLU networks. We derive practical design rules that allow to construct model-based networks with guaranteed high generalization.
  arXiv  Detail & Related papers  (2023-04-19T16:39:44Z)
• Neural Attentive Circuits [93.95502541529115]
  We introduce a general purpose, yet modular neural architecture called Neural Attentive Circuits (NACs) NACs learn the parameterization and a sparse connectivity of neural modules without using domain knowledge. NACs achieve an 8x speedup at inference time while losing less than 3% performance.
  arXiv  Detail & Related papers  (2022-10-14T18:00:07Z)
• Transformer Module Networks for Systematic Generalization in Visual Question Answering [4.169829151981242]
  Transformer Module Network (TMN) dynamically composes modules into a question-specific Transformer network. TMNs achieve state-of-the-art systematic generalization performance in three VQA datasets.
  arXiv  Detail & Related papers  (2022-01-27T04:22:25Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• Weakly Supervised Neuro-Symbolic Module Networks for Numerical Reasoning [44.5641465035393]
  We propose Weakly-Supervised Neuro-Symbolic Module Network (WNSMN) trained with answers as the sole supervision for numerical reasoning based. It learns to execute a noisy MRC program obtained from the dependency parsing of the query, as discrete actions over both neural and symbolic reasoning modules and trains it end-to-end in a reinforcement learning framework with discrete reward from answer matching. This showcases the effectiveness and generalizability of modular networks that can handle explicit discrete reasoning over noisy programs in an end-to-end manner.
  arXiv  Detail & Related papers  (2021-01-28T03:36:09Z)
• Neural Function Modules with Sparse Arguments: A Dynamic Approach to Integrating Information across Layers [84.57980167400513]
  Neural Function Modules (NFM) aims to introduce the same structural capability into deep learning. Most of the work in the context of feed-forward networks combining top-down and bottom-up feedback is limited to classification problems. The key contribution of our work is to combine attention, sparsity, top-down and bottom-up feedback, in a flexible algorithm.
  arXiv  Detail & Related papers  (2020-10-15T20:43:17Z)
• Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks [10.0444013205203]
  Understanding if and how NNs are modular could provide insights into how to improve them. Current inspection methods, however, fail to link modules to their functionality.
  arXiv  Detail & Related papers  (2020-10-05T15:04:11Z)
• Neural Complexity Measures [96.06344259626127]
  We propose Neural Complexity (NC), a meta-learning framework for predicting generalization. Our model learns a scalar complexity measure through interactions with many heterogeneous tasks in a data-driven way.
  arXiv  Detail & Related papers  (2020-08-07T02:12:10Z)
• Obtaining Faithful Interpretations from Compositional Neural Networks [72.41100663462191]
  We evaluate the intermediate outputs of NMNs on NLVR2 and DROP datasets. We find that the intermediate outputs differ from the expected output, illustrating that the network structure does not provide a faithful explanation of model behaviour.
  arXiv  Detail & Related papers  (2020-05-02T06:50:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.