Beam Tree Recursive Cells

• URL: http://arxiv.org/abs/2305.19999v3
• Date: Tue, 20 Jun 2023 11:50:33 GMT
• Title: Beam Tree Recursive Cells
• Abstract summary: We propose Beam Tree Recursive Cell (BT-Cell) to extend Recursive Neural Networks (RvNNs) with beam search for latent structure induction. We evaluate our proposed models in different out-of-distribution splits in both synthetic and realistic data.
• Score: 54.958581892688095
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose Beam Tree Recursive Cell (BT-Cell) - a backpropagation-friendly framework to extend Recursive Neural Networks (RvNNs) with beam search for latent structure induction. We further extend this framework by proposing a relaxation of the hard top-k operators in beam search for better propagation of gradient signals. We evaluate our proposed models in different out-of-distribution splits in both synthetic and realistic data. Our experiments show that BTCell achieves near-perfect performance on several challenging structure-sensitive synthetic tasks like ListOps and logical inference while maintaining comparable performance in realistic data against other RvNN-based models. Additionally, we identify a previously unknown failure case for neural models in generalization to unseen number of arguments in ListOps. The code is available at: https://github.com/JRC1995/BeamTreeRecursiveCells.

Related papers

• LTBs-KAN: Linear-Time B-splines Kolmogorov-Arnold Networks [2.224556533789044]
  Kolmogorov-Arnold Networks (KANs) are a recent neural network architecture offering an alternative to Multilayer Perceptrons (MLPs)<n>This work proposes a novel base-spline Linear-Time B-splines Kolmogorov-Arnold Network (LTBs-KAN) with linear complexity.
  arXiv  Detail & Related papers  (2026-04-23T19:47:13Z)
• MAT-Cell: A Multi-Agent Tree-Structured Reasoning Framework for Batch-Level Single-Cell Annotation [81.82645086402366]
  We propose MAT-Cell, a neuro-symbolic reasoning framework that reframes single-cell analysis into constructive, verifiable proof generation.<n> MAT-Cell injects symbolic constraints through adaptive Retrieval-Augmented Generation (RAG) to ground neural reasoning in biological axioms and reduce transcriptomic noise.<n>It significantly outperforms state-of-the-art (SOTA) models and maintains robust per-formance in challenging scenarios.
  arXiv  Detail & Related papers  (2026-04-07T04:56:06Z)
• Linearized Bregman Iterations for Sparse Spiking Neural Networks [0.1274452325287335]
  Spiking Neural Networks (SNNs) offer an energy efficient alternative to conventional Artificial Neural Networks (ANNs)<n>This work introduces Linearized Bregman Iterations (LBI) as an optimization for training SNNs.<n>To improve convergence and generalization, we employ the AdaBregeration, a momentum and bias corrected Bregman variant of Adam.
  arXiv  Detail & Related papers  (2026-03-17T12:48:31Z)
• Improving Set Function Approximation with Quasi-Arithmetic Neural Networks [23.73257235603082]
  We propose quasi-arithmetic neural networks (QUANNs)<n>QUANNs are universal approximators for a broad class of common set-function decompositions.<n>We provide a theoretical analysis showing that, QUANNs are universal approximators for a broad class of common set-function decompositions.
  arXiv  Detail & Related papers  (2026-02-04T18:36:31Z)
• RS-ORT: A Reduced-Space Branch-and-Bound Algorithm for Optimal Regression Trees [2.612627266839037]
  Mixed-integer programming (MIP) has emerged as a powerful framework for learning optimal decision trees.<n>Naively binarizing continuous features sacrifices global optimality and often yields needlessly deep trees.<n>We recast the optimal regression-tree training as a two-stage optimization problem and propose Reduced-Space Optimal Regression Trees (RS-ORT)<n> RS-ORT is a specialized branch-and-bound (BB) algorithm that branches exclusively on tree-structural variables.
  arXiv  Detail & Related papers  (2025-10-27T22:17:09Z)
• NRGBoost: Energy-Based Generative Boosted Trees [1.0878040851638]
  We propose an energy-based generative boosting algorithm that is analogous to the second-order boosting implemented in popular libraries like XGBoost. We show that, despite producing a generative model capable of handling inference tasks over any input variable, our proposed algorithm can achieve similar discriminative performance to GBDT. At the same time, we show that it is also competitive with neural-network-based models for sampling.
  arXiv  Detail & Related papers  (2024-10-04T15:54:02Z)
• Fast, accurate and lightweight sequential simulation-based inference using Gaussian locally linear mappings [0.820217860574125]
  We propose an alternative to "simulation-based inference" ( SBI) that provides both approximations to the likelihood and the posterior distribution. Our approach produces accurate posterior inference when compared to state-of-the-art NN-based SBI methods, even for multimodal posteriors. We illustrate our results on several benchmark models from the SBI literature and on a biological model of the translation kinetics after mRNA transfection.
  arXiv  Detail & Related papers  (2024-03-12T09:48:17Z)
• DORE: Document Ordered Relation Extraction based on Generative Framework [56.537386636819626]
  This paper investigates the root cause of the underwhelming performance of the existing generative DocRE models. We propose to generate a symbolic and ordered sequence from the relation matrix which is deterministic and easier for model to learn. Experimental results on four datasets show that our proposed method can improve the performance of the generative DocRE models.
  arXiv  Detail & Related papers  (2022-10-28T11:18:10Z)
• VQ-T: RNN Transducers using Vector-Quantized Prediction Network States [52.48566999668521]
  We propose to use vector-quantized long short-term memory units in the prediction network of RNN transducers. By training the discrete representation jointly with the ASR network, hypotheses can be actively merged for lattice generation. Our experiments on the Switchboard corpus show that the proposed VQ RNN transducers improve ASR performance over transducers with regular prediction networks.
  arXiv  Detail & Related papers  (2022-08-03T02:45:52Z)
• Generative Trees: Adversarial and Copycat [26.09279398946235]
  We exploit decades-old understanding of the supervised task's best components for DT induction. We introduce tree-based generative models, textitgenerative trees (GTs) We test our algorithms on tasks including fake/real distinction, training from fake data and missing data imputation.
  arXiv  Detail & Related papers  (2022-01-26T22:02:43Z)
• Convergent Boosted Smoothing for Modeling Graph Data with Tabular Node Features [46.052312251801]
  We propose a framework for iterating boosting with graph propagation steps. Our approach is anchored in a principled meta loss function. Across a variety of non-iid graph datasets, our method achieves comparable or superior performance.
  arXiv  Detail & Related papers  (2021-10-26T04:53:12Z)
• Boost then Convolve: Gradient Boosting Meets Graph Neural Networks [6.888700669980625]
  We show that gradient boosted decision trees (GBDT) often outperform other machine learning methods when faced with heterogeneous data. We propose a novel architecture that trains GBDT and GNN jointly to get the best of both worlds. Our model benefits from end-to-end optimization by allowing new trees to fit the gradient updates of GNN.
  arXiv  Detail & Related papers  (2021-01-21T10:46:41Z)
• Stretchable Cells Help DARTS Search Better [70.52254306274092]
  Differentiable neural architecture search (DARTS) has gained much success in discovering flexible and diverse cell types. Current DARTS methods are prone to wide and shallow cells, and this topology collapse induces sub-optimal searched cells. In this paper, we endowing the cells with explicit stretchability, so the search can be directly implemented on our stretchable cells.
  arXiv  Detail & Related papers  (2020-11-18T14:15:51Z)
• Exploiting Heterogeneity in Operational Neural Networks by Synaptic Plasticity [87.32169414230822]
  Recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) In this study the focus is drawn on searching the best-possible operator set(s) for the hidden neurons of the network based on the Synaptic Plasticity paradigm that poses the essential learning theory in biological neurons. Experimental results over highly challenging problems demonstrate that the elite ONNs even with few neurons and layers can achieve a superior learning performance than GIS-based ONNs.
  arXiv  Detail & Related papers  (2020-08-21T19:03:23Z)
• Linguistically Driven Graph Capsule Network for Visual Question Reasoning [153.76012414126643]
  We propose a hierarchical compositional reasoning model called the "Linguistically driven Graph Capsule Network" The compositional process is guided by the linguistic parse tree. Specifically, we bind each capsule in the lowest layer to bridge the linguistic embedding of a single word in the original question with visual evidence. Experiments on the CLEVR dataset, CLEVR compositional generation test, and FigureQA dataset demonstrate the effectiveness and composition generalization ability of our end-to-end model.
  arXiv  Detail & Related papers  (2020-03-23T03:34:25Z)
• Recurrent Dirichlet Belief Networks for Interpretable Dynamic Relational Data Modelling [29.138765941440145]
  Dirichlet Belief Network(DirBN) has been recently proposed as a promising approach in learning interpretable deep latent representations for objects. We propose a deep dynamic probabilistic framework -- the Recurrent Dirichlet Belief Network(Recurrent-DBN) -- to study interpretable hidden structures from dynamic relational data.
  arXiv  Detail & Related papers  (2020-02-24T13:40:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.