Recursive Function Definitions in Static Dataflow Graphs and their Implementation in TensorFlow

• URL: http://arxiv.org/abs/2410.20225v1
• Date: Sat, 26 Oct 2024 16:40:24 GMT
• Title: Recursive Function Definitions in Static Dataflow Graphs and their Implementation in TensorFlow
• Authors: Kelly Kostopoulou, Angelos Charalambidis, Panos Rondogiannis,
• Abstract summary: We propose an efficient technique for supporting function definitions in dataflow-based systems. We make heavy use of the idea of tagging, which was one of the cornerstones of dataflow systems since their inception.
• Score: 0.8368470115534696
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern machine learning systems represent their computations as dataflow graphs. The increasingly complex neural network architectures crave for more powerful yet efficient programming abstractions. In this paper we propose an efficient technique for supporting recursive function definitions in dataflow-based systems such as TensorFlow. The proposed approach transforms the given recursive definitions into a static dataflow graph that is enriched with two simple yet powerful dataflow operations. Since static graphs do not change during execution, they can be easily partitioned and executed efficiently in distributed and heterogeneous environments. The proposed technique makes heavy use of the idea of tagging, which was one of the cornerstones of dataflow systems since their inception. We demonstrate that our technique is compatible with the idea of automatic differentiation, a notion that is crucial for dataflow systems that focus on deep learning applications. We describe the principles of an actual implementation of the technique in the TensorFlow framework, and present experimental results that demonstrate that the use of tagging is of paramount importance for developing efficient high-level abstractions for modern dataflow systems.

Related papers

• When Bayesian Tensor Completion Meets Multioutput Gaussian Processes: Functional Universality and Rank Learning [53.17227599983122]
  Functional tensor decomposition can analyze multi-dimensional data with real-valued indices.<n>We propose a rank-revealing functional low-rank tensor completion (RR-F) method.<n>We establish the universal approximation property of the model for continuous multi-dimensional signals.
  arXiv  Detail & Related papers  (2025-12-25T03:15:52Z)
• FlowFeat: Pixel-Dense Embedding of Motion Profiles [51.933932255458906]
  FlowFeat is a high-resolution and multi-task feature representation.<n>Our work takes a step forward towards reliable and versatile dense image representations.
  arXiv  Detail & Related papers  (2025-11-10T23:39:53Z)
• FlowUnits: Extending Dataflow for the Edge-to-Cloud Computing Continuum [41.94295877935867]
  FlowUnits organizes processing operators into cohesive, independently manageable components that can be transparently replicated across different regions. Our approach maintains the simplicity of dataflow while enabling seamless integration of edge and cloud resources into unified data processing pipelines.
  arXiv  Detail & Related papers  (2025-04-15T17:14:08Z)
• Efficient and Effective Implicit Dynamic Graph Neural Network [42.49148111696576]
  We present Implicit Dynamic Graph Neural Network (IDGNN) a novel implicit neural network for dynamic graphs. A key characteristic of IDGNN is that it demonstrably is well-posed, i.e., it is theoretically guaranteed to have a fixed-point representation.
  arXiv  Detail & Related papers  (2024-06-25T19:07:21Z)
• FlowerFormer: Empowering Neural Architecture Encoding using a Flow-aware Graph Transformer [22.99322654701362]
  We introduce FlowerFormer, a powerful graph transformer that incorporates the information flows within a neural architecture. Our experiments demonstrate the superiority of FlowerFormer over existing neural encoding methods.
  arXiv  Detail & Related papers  (2024-03-19T15:21:10Z)
• GNNFlow: A Distributed Framework for Continuous Temporal GNN Learning on Dynamic Graphs [11.302970701867844]
  We introduce GNNFlow, a distributed framework for efficient continuous temporal graph representation learning. GNNFlow supports distributed training across multiple machines with static scheduling to ensure load balance. Our experimental results show that GNNFlow provides up to 21.1x faster continuous learning than existing systems.
  arXiv  Detail & Related papers  (2023-11-29T07:30:32Z)
• Dynamic Perceiver for Efficient Visual Recognition [87.08210214417309]
  We propose Dynamic Perceiver (Dyn-Perceiver) to decouple the feature extraction procedure and the early classification task. A feature branch serves to extract image features, while a classification branch processes a latent code assigned for classification tasks. Early exits are placed exclusively within the classification branch, thus eliminating the need for linear separability in low-level features.
  arXiv  Detail & Related papers  (2023-06-20T03:00:22Z)
• Bures-Wasserstein Means of Graphs [60.42414991820453]
  We propose a novel framework for defining a graph mean via embeddings in the space of smooth graph signal distributions. By finding a mean in this embedding space, we can recover a mean graph that preserves structural information. We establish the existence and uniqueness of the novel graph mean, and provide an iterative algorithm for computing it.
  arXiv  Detail & Related papers  (2023-05-31T11:04:53Z)
• Generative Flow Networks for Precise Reward-Oriented Active Learning on Graphs [34.76241250013461]
  We formulate the graph active learning problem as a generative process, named GFlowGNN, which generates various samples through sequential actions. We show that the proposed approach has good exploration capability and transferability, outperforming various state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-04-24T10:47:08Z)
• FlowNAS: Neural Architecture Search for Optical Flow Estimation [65.44079917247369]
  We propose a neural architecture search method named FlowNAS to automatically find the better encoder architecture for flow estimation task. Experimental results show that the discovered architecture with the weights inherited from the super-network achieves 4.67% F1-all error on KITTI.
  arXiv  Detail & Related papers  (2022-07-04T09:05:25Z)
• FlowX: Towards Explainable Graph Neural Networks via Message Flows [59.025023020402365]
  We investigate the explainability of graph neural networks (GNNs) as a step toward elucidating their working mechanisms. We propose a novel method here, known as FlowX, to explain GNNs by identifying important message flows. We then propose an information-controlled learning algorithm to train flow scores toward diverse explanation targets.
  arXiv  Detail & Related papers  (2022-06-26T22:48:15Z)
• Automated Evolutionary Approach for the Design of Composite Machine Learning Pipelines [48.7576911714538]
  The proposed approach is aimed to automate the design of composite machine learning pipelines. It designs the pipelines with a customizable graph-based structure, analyzes the obtained results, and reproduces them. The software implementation on this approach is presented as an open-source framework.
  arXiv  Detail & Related papers  (2021-06-26T23:19:06Z)
• Self Normalizing Flows [65.73510214694987]
  We propose a flexible framework for training normalizing flows by replacing expensive terms in the gradient by learned approximate inverses at each layer. This reduces the computational complexity of each layer's exact update from $mathcalO(D3)$ to $mathcalO(D2)$. We show experimentally that such models are remarkably stable and optimize to similar data likelihood values as their exact gradient counterparts.
  arXiv  Detail & Related papers  (2020-11-14T09:51:51Z)
• Active Learning on Attributed Graphs via Graph Cognizant Logistic Regression and Preemptive Query Generation [37.742218733235084]
  We propose a novel graph-based active learning algorithm for the task of node classification in attributed graphs. Our algorithm uses graph cognizant logistic regression, equivalent to a linearized graph convolutional neural network (GCN) for the prediction phase and maximizes the expected error reduction in the query phase. We conduct experiments on five public benchmark datasets, demonstrating a significant improvement over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-07-09T18:00:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.