ExAIS: Executable AI Semantics
- URL: http://arxiv.org/abs/2202.09868v1
- Date: Sun, 20 Feb 2022 17:33:34 GMT
- Title: ExAIS: Executable AI Semantics
- Authors: Richard Schumi, Jun Sun
- Abstract summary: Neural networks can be regarded as a new programming paradigm, i.e., instead of building ever-more complex programs through (often informal) logical reasoning in the programmers' mind, complex 'AI' systems are built by optimising generic neural network models with big data.
In this new paradigm, AI frameworks such as PyTorch play a key role, which is as essential as the compiler for traditional programs.
It is known that the lack of a proper semantics for programming languages (such as C), i.e., a correctness specification for compilers, has contributed to many problematic program behaviours and security issues
- Score: 4.092001692194709
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Neural networks can be regarded as a new programming paradigm, i.e., instead
of building ever-more complex programs through (often informal) logical
reasoning in the programmers' mind, complex 'AI' systems are built by
optimising generic neural network models with big data. In this new paradigm,
AI frameworks such as TensorFlow and PyTorch play a key role, which is as
essential as the compiler for traditional programs. It is known that the lack
of a proper semantics for programming languages (such as C), i.e., a
correctness specification for compilers, has contributed to many problematic
program behaviours and security issues. While it is in general hard to have a
correctness specification for compilers due to the high complexity of
programming languages and their rapid evolution, we have a unique opportunity
to do it right this time for neural networks (which have a limited set of
functions, and most of them have stable semantics). In this work, we report our
effort on providing a correctness specification of neural network frameworks
such as TensorFlow. We specify the semantics of almost all TensorFlow layers in
the logical programming language Prolog. We demonstrate the usefulness of the
semantics through two applications. One is a fuzzing engine for TensorFlow,
which features a strong oracle and a systematic way of generating valid neural
networks. The other is a model validation approach which enables consistent bug
reporting for TensorFlow models.
Related papers
- pyhgf: A neural network library for predictive coding [0.2150989251218736]
texttpyhgf is a Python package for creating, manipulating and sampling dynamic networks for predictive coding.
We improve over other frameworks by enclosing the network components as transparent, modular and malleable variables in the message-passing steps.
The transparency of core variables can also translate into inference processes that leverage self-organisation principles.
arXiv Detail & Related papers (2024-10-11T19:21:38Z) - KHNNs: hypercomplex neural networks computations via Keras using TensorFlow and PyTorch [0.0]
We propose a library applications integrated with Keras that can do computations within Dense and PyTorch.
It provides Dense and Convolutional 1D, 2D, and 3D layers architectures.
arXiv Detail & Related papers (2024-06-29T14:36:37Z) - Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning [89.89857766491475]
We propose a complex reasoning schema over KG upon large language models (LLMs)
We augment the arbitrary first-order logical queries via binary tree decomposition to stimulate the reasoning capability of LLMs.
Experiments across widely used datasets demonstrate that LACT has substantial improvements(brings an average +5.5% MRR score) over advanced methods.
arXiv Detail & Related papers (2024-05-02T18:12:08Z) - Nondeterministic Stacks in Neural Networks [0.456877715768796]
We develop a differentiable data structure that efficiently simulates a nondeterministic pushdown automaton.
We show that this raises their formal recognition power to arbitrary context-free languages.
We also show that an RNN augmented with a nondeterministic stack is capable of surprisingly powerful behavior.
arXiv Detail & Related papers (2023-04-25T16:00:40Z) - A Neural Lambda Calculus: Neurosymbolic AI meets the foundations of
computing and functional programming [0.0]
We will analyze the ability of neural networks to learn how to execute programs as a whole.
We will introduce the use of integrated neural learning and calculi formalization.
arXiv Detail & Related papers (2023-04-18T20:30:16Z) - PDSketch: Integrated Planning Domain Programming and Learning [86.07442931141637]
We present a new domain definition language, named PDSketch.
It allows users to flexibly define high-level structures in the transition models.
Details of the transition model will be filled in by trainable neural networks.
arXiv Detail & Related papers (2023-03-09T18:54:12Z) - Dynamic Inference with Neural Interpreters [72.90231306252007]
We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules.
inputs to the model are routed through a sequence of functions in a way that is end-to-end learned.
We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
arXiv Detail & Related papers (2021-10-12T23:22:45Z) - Quantized Neural Networks via {-1, +1} Encoding Decomposition and
Acceleration [83.84684675841167]
We propose a novel encoding scheme using -1, +1 to decompose quantized neural networks (QNNs) into multi-branch binary networks.
We validate the effectiveness of our method on large-scale image classification, object detection, and semantic segmentation tasks.
arXiv Detail & Related papers (2021-06-18T03:11:15Z) - Learning to Execute Programs with Instruction Pointer Attention Graph
Neural Networks [55.98291376393561]
Graph neural networks (GNNs) have emerged as a powerful tool for learning software engineering tasks.
Recurrent neural networks (RNNs) are well-suited to long sequential chains of reasoning, but they do not naturally incorporate program structure.
We introduce a novel GNN architecture, the Instruction Pointer Attention Graph Neural Networks (IPA-GNN), which improves systematic generalization on the task of learning to execute programs.
arXiv Detail & Related papers (2020-10-23T19:12:30Z) - Recognizing Long Grammatical Sequences Using Recurrent Networks
Augmented With An External Differentiable Stack [73.48927855855219]
Recurrent neural networks (RNNs) are a widely used deep architecture for sequence modeling, generation, and prediction.
RNNs generalize poorly over very long sequences, which limits their applicability to many important temporal processing and time series forecasting problems.
One way to address these shortcomings is to couple an RNN with an external, differentiable memory structure, such as a stack.
In this paper, we improve the memory-augmented RNN with important architectural and state updating mechanisms.
arXiv Detail & Related papers (2020-04-04T14:19:15Z) - TF-Coder: Program Synthesis for Tensor Manipulations [29.46838583290554]
We present a tool called TF-Coder for programming by example in pruning.
We train models to predict operations from features of the input and output tensors and natural language descriptions of tasks.
TF-Coder solves 63 of 70 real-world tasks within 5 minutes, sometimes finding simpler solutions in less time compared to experienced human programmers.
arXiv Detail & Related papers (2020-03-19T22:53:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.