Deep Distilling: automated code generation using explainable deep learning

• URL: http://arxiv.org/abs/2111.08275v1
• Date: Tue, 16 Nov 2021 07:45:41 GMT
• Title: Deep Distilling: automated code generation using explainable deep learning
• Authors: Paul J. Blazek, Kesavan Venkatesh, Milo M. Lin
• Abstract summary: We introduce deep distilling, a machine learning method that learns patterns from data using explainable deep learning. We show that deep distilling generates concise code that generalizes out-of-distribution to solve problems. Our approach demonstrates that unassisted machine intelligence can build generalizable and intuitive rules.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Human reasoning can distill principles from observed patterns and generalize them to explain and solve novel problems. The most powerful artificial intelligence systems lack explainability and symbolic reasoning ability, and have therefore not achieved supremacy in domains requiring human understanding, such as science or common sense reasoning. Here we introduce deep distilling, a machine learning method that learns patterns from data using explainable deep learning and then condenses it into concise, executable computer code. The code, which can contain loops, nested logical statements, and useful intermediate variables, is equivalent to the neural network but is generally orders of magnitude more compact and human-comprehensible. On a diverse set of problems involving arithmetic, computer vision, and optimization, we show that deep distilling generates concise code that generalizes out-of-distribution to solve problems orders-of-magnitude larger and more complex than the training data. For problems with a known ground-truth rule set, deep distilling discovers the rule set exactly with scalable guarantees. For problems that are ambiguous or computationally intractable, the distilled rules are similar to existing human-derived algorithms and perform at par or better. Our approach demonstrates that unassisted machine intelligence can build generalizable and intuitive rules explaining patterns in large datasets that would otherwise overwhelm human reasoning.

Related papers

• Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning [89.89857766491475]
  We propose a complex reasoning schema over KG upon large language models (LLMs) We augment the arbitrary first-order logical queries via binary tree decomposition to stimulate the reasoning capability of LLMs. Experiments across widely used datasets demonstrate that LACT has substantial improvements(brings an average +5.5% MRR score) over advanced methods.
  arXiv  Detail & Related papers  (2024-05-02T18:12:08Z)
• When Do Program-of-Thoughts Work for Reasoning? [51.2699797837818]
  We propose complexity-impacted reasoning score (CIRS) to measure correlation between code and reasoning abilities. Specifically, we use the abstract syntax tree to encode the structural information and calculate logical complexity. Code will be integrated into the EasyInstruct framework at https://github.com/zjunlp/EasyInstruct.
  arXiv  Detail & Related papers  (2023-08-29T17:22:39Z)
• Transgressing the boundaries: towards a rigorous understanding of deep learning and its (non-)robustness [3.1219977244201056]
  The recent advances in machine learning in various fields of applications can be attributed to the rise of deep learning (DL) methods and architectures. Despite being a key technology behind autonomous cars, image processing, speech recognition, etc., a notorious problem remains the lack of theoretical understanding of DL.
  arXiv  Detail & Related papers  (2023-07-05T17:27:17Z)
• Chaining Simultaneous Thoughts for Numerical Reasoning [92.2007997126144]
  numerical reasoning over text should be an essential skill of AI systems. Previous work focused on modeling the structures of equations, and has proposed various structured decoders. We propose CANTOR, a numerical reasoner that models reasoning steps using a directed acyclic graph.
  arXiv  Detail & Related papers  (2022-11-29T18:52:06Z)
• On Explainability in AI-Solutions: A Cross-Domain Survey [4.394025678691688]
  In automatically deriving a system model, AI algorithms learn relations in data that are not detectable for humans. The more complex a model, the more difficult it is for a human to understand the reasoning for the decisions. This work provides an extensive survey of literature on this topic, which, to a large part, consists of other surveys.
  arXiv  Detail & Related papers  (2022-10-11T06:21:47Z)
• Transformers discover an elementary calculation system exploiting local attention and grid-like problem representation [0.424243593213882]
  We show that universal transformers equipped with local attention and adaptive halting mechanisms can learn to exploit an external, grid-like memory to carry out multi-digit addition. The proposed model achieves remarkable accuracy even when tested with problems requiring extrapolation outside the training distribution.
  arXiv  Detail & Related papers  (2022-07-06T09:29:56Z)
• End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking [52.05847268235338]
  We show how machine learning systems can perform logical extrapolation without overthinking problems. We propose a recall architecture that keeps an explicit copy of the problem instance in memory so that it cannot be forgotten. We also employ a progressive training routine that prevents the model from learning behaviors that are specific to number and instead pushes it to learn behaviors that can be repeated indefinitely.
  arXiv  Detail & Related papers  (2022-02-11T18:43:28Z)
• Emergence of Machine Language: Towards Symbolic Intelligence with Neural Networks [73.94290462239061]
  We propose to combine symbolism and connectionism principles by using neural networks to derive a discrete representation. By designing an interactive environment and task, we demonstrated that machines could generate a spontaneous, flexible, and semantic language.
  arXiv  Detail & Related papers  (2022-01-14T14:54:58Z)
• An Empirical Investigation into Deep and Shallow Rule Learning [0.0]
  In this paper, we empirically compare deep and shallow rule learning with a uniform general algorithm. Our experiments on both artificial and real-world benchmark data indicate that deep rule networks outperform shallow networks.
  arXiv  Detail & Related papers  (2021-06-18T17:43:17Z)
• Inductive Biases for Deep Learning of Higher-Level Cognition [108.89281493851358]
  A fascinating hypothesis is that human and animal intelligence could be explained by a few principles. This work considers a larger list, focusing on those which concern mostly higher-level and sequential conscious processing. The objective of clarifying these particular principles is that they could potentially help us build AI systems benefiting from humans' abilities.
  arXiv  Detail & Related papers  (2020-11-30T18:29:25Z)
• Self-explaining AI as an alternative to interpretable AI [0.0]
  Double descent indicates that deep neural networks operate by smoothly interpolating between data points. Neural networks trained on complex real world data are inherently hard to interpret and prone to failure if asked to extrapolate. Self-explaining AIs are capable of providing a human-understandable explanation along with confidence levels for both the decision and explanation.
  arXiv  Detail & Related papers  (2020-02-12T18:50:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.