Distilling Wikipedia mathematical knowledge into neural network models
- URL: http://arxiv.org/abs/2104.05930v1
- Date: Tue, 13 Apr 2021 04:16:50 GMT
- Title: Distilling Wikipedia mathematical knowledge into neural network models
- Authors: Joanne T. Kim, Mikel Landajuela Larma, Brenden K. Petersen
- Abstract summary: We introduce a pipeline for distilling mathematical expressions embedded in Wikipedia into symbolic encodings to be used in downstream machine learning tasks.
We demonstrate that a $textitmathematical$ $textitlanguage$ $textitmodel$ trained on this "corpus" of expressions can be used as a prior to improve the performance of neural-guided search for the task of symbolic regression.
- Score: 4.874780144224057
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Machine learning applications to symbolic mathematics are becoming
increasingly popular, yet there lacks a centralized source of real-world
symbolic expressions to be used as training data. In contrast, the field of
natural language processing leverages resources like Wikipedia that provide
enormous amounts of real-world textual data. Adopting the philosophy of
"mathematics as language," we bridge this gap by introducing a pipeline for
distilling mathematical expressions embedded in Wikipedia into symbolic
encodings to be used in downstream machine learning tasks. We demonstrate that
a $\textit{mathematical}$ $\textit{language}$ $\textit{model}$ trained on this
"corpus" of expressions can be used as a prior to improve the performance of
neural-guided search for the task of symbolic regression.
Related papers
- MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code [38.127313175508746]
We introduce a novel method for generating mathematical code accompanied with corresponding reasoning steps for continued pretraining.
Our approach begins with the construction of a high-quality mathematical continued pretraining dataset.
Appending the generated code to each reasoning step results in data consisting of paired natural language reasoning steps and their corresponding code.
arXiv Detail & Related papers (2024-10-10T17:58:40Z) - The $μ\mathcal{G}$ Language for Programming Graph Neural Networks [0.542060671850508]
$mumathcalG$ is an original domain-specific language for the specification of graph neural networks.
We show how $mumathcalG$ can be used to define some of the most popular graph neural network models.
arXiv Detail & Related papers (2024-07-12T17:27:43Z) - Symbolic Learning Enables Self-Evolving Agents [55.625275970720374]
We introduce agent symbolic learning, a systematic framework that enables language agents to optimize themselves on their own.
Agent symbolic learning is designed to optimize the symbolic network within language agents by mimicking two fundamental algorithms in connectionist learning.
We conduct proof-of-concept experiments on both standard benchmarks and complex real-world tasks.
arXiv Detail & Related papers (2024-06-26T17:59:18Z) - Mathematical Entities: Corpora and Benchmarks [0.8766411351797883]
There has been relatively little research on natural language processing for mathematical texts.
We provide annotated corpora that can be used to study the language of mathematics in different contexts.
arXiv Detail & Related papers (2024-06-17T14:11:00Z) - Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning [84.12154024070024]
We propose natural language embedded programs (NLEP) as a unifying framework for addressing math/symbolic reasoning, natural language understanding, and instruction following tasks.
Our approach prompts a language model to generate full Python programs that define functions over data structures which contain natural language representations of structured knowledge.
A Python interpreter then executes the generated code and prints the output.
arXiv Detail & Related papers (2023-09-19T17:54:21Z) - Neural Machine Translation for Mathematical Formulae [8.608288231153304]
We tackle the problem of neural machine translation of mathematical formulae between ambiguous presentation languages and unambiguous content languages.
We find that convolutional sequence-to-sequence networks achieve 95.1% and 90.7% exact matches, respectively.
arXiv Detail & Related papers (2023-05-25T19:15:06Z) - Tree-Based Representation and Generation of Natural and Mathematical
Language [77.34726150561087]
Mathematical language in scientific communications and educational scenarios is important yet relatively understudied.
Recent works on mathematical language focus either on representing stand-alone mathematical expressions, or mathematical reasoning in pre-trained natural language models.
We propose a series of modifications to existing language models to jointly represent and generate text and math.
arXiv Detail & Related papers (2023-02-15T22:38:34Z) - Math-KG: Construction and Applications of Mathematical Knowledge Graph [2.1828601975620257]
We propose a mathematical knowledge graph named Math-KG, which automatically constructed by the pipeline method with the natural language processing technology to integrate the resources of the mathematics.
We implement a simple application system to validate the proposed Math-KG can make contributions on a series of scenes, including faults analysis and semantic search.
arXiv Detail & Related papers (2022-05-08T03:39:07Z) - Linking Emergent and Natural Languages via Corpus Transfer [98.98724497178247]
We propose a novel way to establish a link by corpus transfer between emergent languages and natural languages.
Our approach showcases non-trivial transfer benefits for two different tasks -- language modeling and image captioning.
We also introduce a novel metric to predict the transferability of an emergent language by translating emergent messages to natural language captions grounded on the same images.
arXiv Detail & Related papers (2022-03-24T21:24:54Z) - Emergence of Machine Language: Towards Symbolic Intelligence with Neural
Networks [73.94290462239061]
We propose to combine symbolism and connectionism principles by using neural networks to derive a discrete representation.
By designing an interactive environment and task, we demonstrated that machines could generate a spontaneous, flexible, and semantic language.
arXiv Detail & Related papers (2022-01-14T14:54:58Z) - On The Ingredients of an Effective Zero-shot Semantic Parser [95.01623036661468]
We analyze zero-shot learning by paraphrasing training examples of canonical utterances and programs from a grammar.
We propose bridging these gaps using improved grammars, stronger paraphrasers, and efficient learning methods.
Our model achieves strong performance on two semantic parsing benchmarks (Scholar, Geo) with zero labeled data.
arXiv Detail & Related papers (2021-10-15T21:41:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.