High-performance symbolic-numerics via multiple dispatch

• URL: http://arxiv.org/abs/2105.03949v2
• Date: Wed, 12 May 2021 17:02:31 GMT
• Title: High-performance symbolic-numerics via multiple dispatch
• Authors: Shashi Gowda, Yingbo Ma, Alessandro Cheli, Maja Gwozdz, Viral B. Shah, Alan Edelman, Christopher Rackauckas
• Abstract summary: Symbolics.jl is an extendable symbolic system which uses dynamic multiple dispatch to change behavior depending on the domain needs. We show that by formalizing a generic API on actions independent of implementation, we can retroactively add optimized data structures to our system. We demonstrate the ability to swap between classical term-rewriting simplifiers and e-graph-based term-rewriting simplifiers.
• Score: 52.77024349608834
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: As mathematical computing becomes more democratized in high-level languages, high-performance symbolic-numeric systems are necessary for domain scientists and engineers to get the best performance out of their machine without deep knowledge of code optimization. Naturally, users need different term types either to have different algebraic properties for them, or to use efficient data structures. To this end, we developed Symbolics.jl, an extendable symbolic system which uses dynamic multiple dispatch to change behavior depending on the domain needs. In this work we detail an underlying abstract term interface which allows for speed without sacrificing generality. We show that by formalizing a generic API on actions independent of implementation, we can retroactively add optimized data structures to our system without changing the pre-existing term rewriters. We showcase how this can be used to optimize term construction and give a 113x acceleration on general symbolic transformations. Further, we show that such a generic API allows for complementary term-rewriting implementations. We demonstrate the ability to swap between classical term-rewriting simplifiers and e-graph-based term-rewriting simplifiers. We showcase an e-graph ruleset which minimizes the number of CPU cycles during expression evaluation, and demonstrate how it simplifies a real-world reaction-network simulation to halve the runtime. Additionally, we show a reaction-diffusion partial differential equation solver which is able to be automatically converted into symbolic expressions via multiple dispatch tracing, which is subsequently accelerated and parallelized to give a 157x simulation speedup. Together, this presents Symbolics.jl as a next-generation symbolic-numeric computing environment geared towards modeling and simulation.

Related papers

• SymbolNet: Neural Symbolic Regression with Adaptive Dynamic Pruning [1.0356366043809717]
  We propose a neural network approach to symbolic regression in a novel framework that allows dynamic pruning of model weights, input features, and mathematical operators in a single training process. Our approach enables symbolic regression to achieve fast inference with nanosecond-scale latency on FPGAs for high-dimensional datasets in environments with stringent computational resource constraints.
  arXiv  Detail & Related papers  (2024-01-18T12:51:38Z)
• Tractable Bounding of Counterfactual Queries by Knowledge Compilation [51.47174989680976]
  We discuss the problem of bounding partially identifiable queries, such as counterfactuals, in Pearlian structural causal models. A recently proposed iterated EM scheme yields an inner approximation of those bounds by sampling the initialisation parameters. We show how a single symbolic knowledge compilation allows us to obtain the circuit structure with symbolic parameters to be replaced by their actual values.
  arXiv  Detail & Related papers  (2023-10-05T07:10:40Z)
• Performance Embeddings: A Similarity-based Approach to Automatic Performance Optimization [71.69092462147292]
  Performance embeddings enable knowledge transfer of performance tuning between applications. We demonstrate this transfer tuning approach on case studies in deep neural networks, dense and sparse linear algebra compositions, and numerical weather prediction stencils.
  arXiv  Detail & Related papers  (2023-03-14T15:51:35Z)
• Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networks [88.77951448313486]
  We present a new approach for model acceleration by exploiting spatial sparsity in visual data. We propose a dynamic token sparsification framework to prune redundant tokens. We extend our method to hierarchical models including CNNs and hierarchical vision Transformers.
  arXiv  Detail & Related papers  (2022-07-04T17:00:51Z)
• SymFormer: End-to-end symbolic regression using transformer-based architecture [2.2049183478692584]
  We propose a transformer-based approach called SymFormer, which predicts the formula by outputting the individual symbols and the corresponding constants simultaneously. We show on a set of benchmarks that SymFormer outperforms two state-of-the-art methods while having faster inference.
  arXiv  Detail & Related papers  (2022-05-31T13:01:50Z)
• Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learning [85.95599675484341]
  Recurrent neural networks have a strong inductive bias towards learning temporally compressed representations. Transformers have little inductive bias towards learning temporally compressed representations.
  arXiv  Detail & Related papers  (2022-05-30T00:12:33Z)
• SymForce: Symbolic Computation and Code Generation for Robotics [6.741275581634116]
  We present SymForce, a fast symbolic computation and code generation library for robotics applications. SymForce combines the development speed and flexibility of symbolic mathematics with the performance of autogenerated, highly optimized code in C++.
  arXiv  Detail & Related papers  (2022-04-17T00:15:10Z)
• Compiler-Driven Simulation of Reconfigurable Hardware Accelerators [0.8807375890824978]
  Existing simulators tend to two extremes: low-level and general approaches, such as RTL simulation, that can model any hardware but require substantial effort and long execution times. This work proposes a compiler-driven simulation workflow that can model hardware accelerator.
  arXiv  Detail & Related papers  (2022-02-01T20:31:04Z)
• Addressing Some Limitations of Transformers with Feedback Memory [51.94640029417114]
  Transformers have been successfully applied to sequential, auto-regressive tasks despite being feedforward networks. We propose the Feedback Transformer architecture that exposes all previous representations to all future representations. We demonstrate on a variety of benchmarks in language modeling, machine translation, and reinforcement learning that the increased representation capacity can create small, shallow models with much stronger performance than comparable Transformers.
  arXiv  Detail & Related papers  (2020-02-21T16:37:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.