Autoregressive Generation of Static and Growing Trees

• URL: http://arxiv.org/abs/2502.04762v1
• Date: Fri, 07 Feb 2025 08:51:14 GMT
• Title: Autoregressive Generation of Static and Growing Trees
• Authors: Hanxiao Wang, Biao Zhang, Jonathan Klein, Dominik L. Michels, Dongming Yan, Peter Wonka,
• Abstract summary: We propose a transformer architecture and training strategy for tree generation. The architecture processes data at multiple resolutions and has an hourglass shape, with middle layers processing fewer tokens than outer layers. We extend this approach to perform image-to-tree and point-cloud-to-tree conditional generation and to simulate the tree growth processes, generating 4D trees.
• Score: 49.93294993975928
• License:
• Abstract: We propose a transformer architecture and training strategy for tree generation. The architecture processes data at multiple resolutions and has an hourglass shape, with middle layers processing fewer tokens than outer layers. Similar to convolutional networks, we introduce longer range skip connections to completent this multi-resolution approach. The key advantage of this architecture is the faster processing speed and lower memory consumption. We are therefore able to process more complex trees than would be possible with a vanilla transformer architecture. Furthermore, we extend this approach to perform image-to-tree and point-cloud-to-tree conditional generation and to simulate the tree growth processes, generating 4D trees. Empirical results validate our approach in terms of speed, memory consumption, and generation quality.

Related papers

• Parallelized Autoregressive Visual Generation [65.9579525736345]
  We propose a simple yet effective approach for parallelized autoregressive visual generation. Our method achieves a 3.6x speedup with comparable quality and up to 9.5x speedup with minimal quality degradation across both image and video generation tasks.
  arXiv  Detail & Related papers  (2024-12-19T17:59:54Z)
• Terminating Differentiable Tree Experts [77.2443883991608]
  We propose a neuro-symbolic Differentiable Tree Machine that learns tree operations using a combination of transformers and Representation Products. We first remove a series of different transformer layers that are used in every step by introducing a mixture of experts. We additionally propose a new termination algorithm to provide the model the power to choose how many steps to make automatically.
  arXiv  Detail & Related papers  (2024-07-02T08:45:38Z)
• Tree Variational Autoencoders [5.992683455757179]
  We propose a new generative hierarchical clustering model that learns a flexible tree-based posterior distribution over latent variables. TreeVAE hierarchically divides samples according to their intrinsic characteristics, shedding light on hidden structures in the data.
  arXiv  Detail & Related papers  (2023-06-15T09:25:04Z)
• Hierarchical clustering with dot products recovers hidden tree structure [53.68551192799585]
  In this paper we offer a new perspective on the well established agglomerative clustering algorithm, focusing on recovery of hierarchical structure. We recommend a simple variant of the standard algorithm, in which clusters are merged by maximum average dot product and not, for example, by minimum distance or within-cluster variance. We demonstrate that the tree output by this algorithm provides a bona fide estimate of generative hierarchical structure in data, under a generic probabilistic graphical model.
  arXiv  Detail & Related papers  (2023-05-24T11:05:12Z)
• DeepTree: Modeling Trees with Situated Latents [8.372189962601073]
  We propose a novel method for modeling trees based on learning developmental rules for branching structures instead of manually defining them. We call our deep neural model situated latent because its behavior is determined by the intrinsic state. Our method enables generating a wide variety of tree shapes without the need to define intricate parameters.
  arXiv  Detail & Related papers  (2023-05-09T03:33:14Z)
• A Tree-structured Transformer for Program Representation Learning [27.31416015946351]
  Long-term/global dependencies widely exist in programs, and most neural networks fail to capture these dependencies. In this paper, we propose Tree-Transformer, a novel tree-structured neural network which aims to overcome the above limitations. By combining bottom-up and top-down propagation, Tree-Transformer can learn both global contexts and meaningful node features.
  arXiv  Detail & Related papers  (2022-08-18T05:42:01Z)
• ForestPrune: Compact Depth-Controlled Tree Ensembles [7.538482310185135]
  We present ForestPrune, a novel framework to post-process tree ensembles by pruning depth layers from individual trees. We develop a specialized optimization algorithm to efficiently obtain high-quality solutions to problems under ForestPrune. Our experiments demonstrate that ForestPrune produces parsimonious models that outperform models extracted by existing post-processing algorithms.
  arXiv  Detail & Related papers  (2022-05-31T22:04:18Z)
• Point Cloud Compression with Sibling Context and Surface Priors [47.96018990521301]
  We present a novel octree-based multi-level framework for large-scale point cloud compression. In this framework, we propose a new entropy model that explores the hierarchical dependency in an octree. We locally fit surfaces with a voxel-based geometry-aware module to provide geometric priors in entropy encoding.
  arXiv  Detail & Related papers  (2022-05-02T09:13:26Z)
• Structural Optimization Makes Graph Classification Simpler and Better [5.770986723520119]
  We investigate the feasibility of improving graph classification performance while simplifying the model learning process. Inspired by progress in structural information assessment, we optimize the given data sample from graphs to encoding trees. We present an implementation of the scheme in a tree kernel and a convolutional network to perform graph classification.
  arXiv  Detail & Related papers  (2021-09-05T08:54:38Z)
• Growing Deep Forests Efficiently with Soft Routing and Learned Connectivity [79.83903179393164]
  This paper further extends the deep forest idea in several important aspects. We employ a probabilistic tree whose nodes make probabilistic routing decisions, a.k.a., soft routing, rather than hard binary decisions. Experiments on the MNIST dataset demonstrate that our empowered deep forests can achieve better or comparable performance than [1],[3].
  arXiv  Detail & Related papers  (2020-12-29T18:05:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.