Teaching Neural Module Networks to Do Arithmetic

• URL: http://arxiv.org/abs/2210.02703v1
• Date: Thu, 6 Oct 2022 06:38:04 GMT
• Title: Teaching Neural Module Networks to Do Arithmetic
• Authors: Jiayi Chen and Xiao-Yu Guo and Yuan-Fang Li and Gholamreza Haffari
• Abstract summary: We up-grade NMNs by bridging the gap between its interpreter and the complex questions. We introduce addition and subtraction modules that perform numerical reasoning over numbers. On a subset of DROP, experimental results show that our proposed methods enhance NMNs' numerical reasoning skills by 17.7% improvement of F1 score.
• Score: 54.06832128723388
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Answering complex questions that require multi-step multi-type reasoning over raw text is challenging, especially when conducting numerical reasoning. Neural Module Networks(NMNs), follow the programmer-interpreter framework and design trainable modules to learn different reasoning skills. However, NMNs only have limited reasoning abilities, and lack numerical reasoning capability. We up-grade NMNs by: (a) bridging the gap between its interpreter and the complex questions; (b) introducing addition and subtraction modules that perform numerical reasoning over numbers. On a subset of DROP, experimental results show that our proposed methods enhance NMNs' numerical reasoning skills by 17.7% improvement of F1 score and significantly outperform previous state-of-the-art models.

Related papers

• Multimodal Representations for Teacher-Guided Compositional Visual Reasoning [0.0]
  NMNs provide enhanced explainability compared to integrated models. We propose to exploit features obtained by a large-scale cross-modal encoder. We introduce an NMN learning strategy involving scheduled teacher guidance.
  arXiv  Detail & Related papers  (2023-10-24T07:51:08Z)
• Disentangling Reasoning Capabilities from Language Models with Compositional Reasoning Transformers [72.04044221898059]
  ReasonFormer is a unified reasoning framework for mirroring the modular and compositional reasoning process of humans. The representation module (automatic thinking) and reasoning modules (controlled thinking) are disentangled to capture different levels of cognition. The unified reasoning framework solves multiple tasks with a single model,and is trained and inferred in an end-to-end manner.
  arXiv  Detail & Related papers  (2022-10-20T13:39:55Z)
• Improving Numerical Reasoning Skills in the Modular Approach for Complex Question Answering on Text [39.22253030039486]
  A successful approach to complex question answering (CQA) on text, Neural Module Networks (NMNs), follows the programmer-interpreter paradigm. We propose effective techniques to improve NMNs' numerical reasoning capabilities by making the interpreter question-aware.
  arXiv  Detail & Related papers  (2021-09-06T08:34:31Z)
• Weakly Supervised Neuro-Symbolic Module Networks for Numerical Reasoning [44.5641465035393]
  We propose Weakly-Supervised Neuro-Symbolic Module Network (WNSMN) trained with answers as the sole supervision for numerical reasoning based. It learns to execute a noisy MRC program obtained from the dependency parsing of the query, as discrete actions over both neural and symbolic reasoning modules and trains it end-to-end in a reinforcement learning framework with discrete reward from answer matching. This showcases the effectiveness and generalizability of modular networks that can handle explicit discrete reasoning over noisy programs in an end-to-end manner.
  arXiv  Detail & Related papers  (2021-01-28T03:36:09Z)
• A Primer for Neural Arithmetic Logic Modules [2.4278445972594525]
  This paper is the first in discussing the current state of progress of this field. Focusing on the shortcomings of the NALU, we provide an in-depth analysis to reason about design choices of recent modules. To alleviate the existing inconsistencies, we create a benchmark which compares all existing arithmetic NALMs.
  arXiv  Detail & Related papers  (2021-01-23T16:09:16Z)
• Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models [61.480085460269514]
  We propose a framework for building interpretable systems that learn to solve complex tasks by decomposing them into simpler ones solvable by existing models. We use this framework to build ModularQA, a system that can answer multi-hop reasoning questions by decomposing them into sub-questions answerable by a neural factoid single-span QA model and a symbolic calculator.
  arXiv  Detail & Related papers  (2020-09-01T23:45:42Z)
• Obtaining Faithful Interpretations from Compositional Neural Networks [72.41100663462191]
  We evaluate the intermediate outputs of NMNs on NLVR2 and DROP datasets. We find that the intermediate outputs differ from the expected output, illustrating that the network structure does not provide a faithful explanation of model behaviour.
  arXiv  Detail & Related papers  (2020-05-02T06:50:35Z)
• iNALU: Improved Neural Arithmetic Logic Unit [2.331160520377439]
  The recently proposed Neural Arithmetic Logic Unit (NALU) is a novel neural architecture which is able to explicitly represent the mathematical relationships by the units of the network to learn operations such as summation, subtraction or multiplication. We show that our model solves stability issues and outperforms the original NALU model in means of arithmetic precision and convergence.
  arXiv  Detail & Related papers  (2020-03-17T10:37:22Z)
• Neural Arithmetic Units [84.65228064780744]
  Neural networks can approximate complex functions, but they struggle to perform exact arithmetic operations over real numbers. We present two new neural network components: the Neural Addition Unit (NAU), which can learn exact addition and subtraction, and the Neural multiplication Unit (NMU), which can multiply subsets of a vector. Our proposed units NAU and NMU, compared with previous neural units, converge more consistently, have fewer parameters, learn faster, can converge for larger hidden sizes, obtain sparse and meaningful weights, and can extrapolate to negative and small values.
  arXiv  Detail & Related papers  (2020-01-14T19:35:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.