On the Practical Estimation and Interpretation of Rényi Transfer Entropy

• URL: http://arxiv.org/abs/2601.01497v1
• Date: Sun, 04 Jan 2026 11:53:09 GMT
• Title: On the Practical Estimation and Interpretation of Rényi Transfer Entropy
• Abstract summary: Rényi transfer entropy (RTE) is a generalization of classical transfer entropy with Rényi's information measure.<n>RTE shows strong potential for analyzing causal relations in complex, non-Gaussian systems.<n>We show that when the reliability conditions are met and the parameters are calibrated accordingly, the resulting effective RTE estimates accurately capture directional information flow.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Rényi transfer entropy (RTE) is a generalization of classical transfer entropy that replaces Shannon's entropy with Rényi's information measure. This, in turn, introduces a new tunable parameter $α$, which accounts for sensitivity to low- or high-probability events. Although RTE shows strong potential for analyzing causal relations in complex, non-Gaussian systems, its practical use is limited, primarily due to challenges related to its accurate estimation and interpretation. These difficulties are especially pronounced when working with finite, high-dimensional, or heterogeneous datasets. In this paper, we systematically study the performance of a k-nearest neighbor estimator for both Rényi entropy (RE) and RTE using various synthetic data sets with clear cause-and-effect relationships inherent to their construction. We test the estimator across a broad range of parameters, including sample size, dimensionality, memory length, and Rényi order $α$. In particular, we apply the estimator to a set of simulated processes with increasing structural complexity, ranging from linear dynamics to nonlinear systems with multi-source couplings. To address interpretational challenges arising from potentially negative RE and RTE values, we introduce three reliability conditions and formulate practical guidelines for tuning the estimator parameters. We show that when the reliability conditions are met and the parameters are calibrated accordingly, the resulting effective RTE estimates accurately capture directional information flow across a broad range of scenarios. Results obtained show that the explanatory power of RTE depends sensitively on the choice of the Rényi parameter $α$. This highlights the usefulness of the RTE framework for identifying the drivers of extreme behavior in complex systems.

Related papers

• Equation Recast for Canonical Operator Learning Across Parametric PDEs [59.44573033543284]
  We introduce equation recast, which reformulates parametric operator learning as the learning of a single canonical operator.<n>In high-fidelity tokamak simulations for nuclear fusion, the framework unifies electron-temperature data across four device geometries.
  arXiv  Detail & Related papers  (2026-09-02T13:15:50Z)
• Neural network surrogates with uncertainty quantification for inverse problems in partial differential equations [11.776563892451817]
  We introduce DeepGaLA, a neural-network surrogate for differential equation solvers that provides uncertainty-aware predictions.<n>Across a range of numerical experiments, DeepGaLA delivers forward-model approximations with accuracy comparable to established Gaussian-process surrogates.
  arXiv  Detail & Related papers  (2026-06-18T16:02:20Z)
• Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA [61.12136997430116]
  Decentralized federated learning (DFL) based on low-rank adaptation (LoRA) enables mobile devices with multi-task datasets to collaboratively fine-tune a large language model (LLM) by exchanging locally updated parameters with a subset of neighboring devices via wireless connections for knowledge integration.<n> directly aggregating parameters fine-tuned on heterogeneous datasets induces three primary issues across the DFL life-cycle: (i) catastrophic knowledge forgetting during fine-tuning process, arising from conflicting update directions caused by data heterogeneity; (ii) textitinefficient communication and convergence during model aggregation process,
  arXiv  Detail & Related papers  (2026-02-24T02:45:32Z)
• Nonparametric Identification and Inference for Counterfactual Distributions with Confounding [6.997978440999076]
  We propose nonparametric identification and semiparametric estimation of joint potential outcome in the presence of confounding.<n>By bridging classical semiparametric theory with modern representation learning, this work provides a robust statistical foundation for distributional and counterfactual inference in complex causal systems.
  arXiv  Detail & Related papers  (2026-02-17T05:00:13Z)
• Data-Driven Deep MIMO Detection:Network Architectures and Generalization Analysis [50.20709408241935]
  This paper proposes inspecting the fully data-driven DeepSIC detection within a Network-of-MLPs architecture.<n>Within such an architecture, DeepSIC can be upgraded as a graph-based message-passing process using Graph Neural Networks (GNNs)<n>GNNSIC achieves excellent expressivity comparable to DeepSIC with substantially fewer trainable parameters.
  arXiv  Detail & Related papers  (2026-02-13T04:38:51Z)
• Twinning Complex Networked Systems: Data-Driven Calibration of the mABCD Synthetic Graph Generator [2.6776012440607784]
  We propose a method for estimating matching configurations and for quantifying the associated error.<n>Our results demonstrate that this task is non-trivial, as strong interdependencies between configuration parameters weaken independent estimation and instead favour a joint-prediction approach.
  arXiv  Detail & Related papers  (2026-02-02T12:40:19Z)
• VIKING: Deep variational inference with stochastic projections [48.946143517489496]
  Variational mean field approximations tend to struggle with contemporary overparametrized deep neural networks.<n>We propose a simple variational family that considers two independent linear subspaces of the parameter space.<n>This allows us to build a fully-correlated approximate posterior reflecting the overparametrization.
  arXiv  Detail & Related papers  (2025-10-27T15:38:35Z)
• Unlocking Symbol-Level Precoding Efficiency Through Tensor Equivariant Neural Network [84.22115118596741]
  We propose an end-to-end deep learning (DL) framework with low inference complexity for symbol-level precoding.<n>We show that the proposed framework captures substantial performance gains of optimal SLP, while achieving an approximately 80-times speedup over conventional methods.
  arXiv  Detail & Related papers  (2025-10-02T15:15:50Z)
• Canonical Bayesian Linear System Identification [2.9250839865200846]
  We introduce canonical forms of LTI systems within the Bayesian framework.<n>We rigorously establish that inference in these minimal parameterizations fully captures all invariant system dynamics.<n>This approach unlocks the use of meaningful structure-aware priors.
  arXiv  Detail & Related papers  (2025-07-15T17:58:55Z)
• The Sample Complexity of Online Reinforcement Learning: A Multi-model Perspective [55.15192437680943]
  We study the sample complexity of online reinforcement learning in the general setting of nonlinear dynamical systems with continuous state and action spaces.<n>Our algorithm achieves a policy regret of $mathcalO(N epsilon2 + mathrmln(m(epsilon)/epsilon2)$, where $epsilon$ is the time horizon.<n>In the special case where the dynamics are parametrized by a compact and real-valued set of parameters, we prove a policy regret of $mathcalO(sqrt
  arXiv  Detail & Related papers  (2025-01-27T10:01:28Z)
• Bounds in Wasserstein Distance for Locally Stationary Processes [0.29771206318712146]
  We introduce a novel conditional probability distribution estimator specifically tailored for locally stationary (LSP) data.<n>We rigorously establish convergence rates for the NW-based conditional probability estimator under the Wasserstein metric.<n>We conduct extensive numerical simulations on synthetic datasets and provide empirical validations using real-world data.
  arXiv  Detail & Related papers  (2024-12-04T15:51:22Z)
• Addressing Misspecification in Simulation-based Inference through Data-driven Calibration [43.811367860375825]
  This work introduces robust posterior estimation(RoPE), a framework that overcomes model misspecification with a small real-world calibration set of ground-truth parameter measurements.<n>Results on four synthetic tasks and two real-world problems with ground-truth labels demonstrate that RoPE outperforms baselines and consistently returns informative and calibrated credible intervals.
  arXiv  Detail & Related papers  (2024-05-14T16:04:39Z)
• Efficient CNN-LSTM based Parameter Estimation of Levy Driven Stochastic Differential Equations [0.0]
  This study addresses the challenges in parameter estimation of differential equations driven by non-Gaussian noises. Previous research highlighted the potential of LSTM networks in estimating parameters of alpha stable Levy driven SDEs. We introduce the PEnet, a novel CNN-LSTM-based three-stage model that offers an end to end approach with superior accuracy and adaptability to varying data structures.
  arXiv  Detail & Related papers  (2024-03-07T06:07:31Z)
• Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts [40.24720443257405]
  We provide a convergence analysis for maximum likelihood estimation (MLE) in the Gaussian-gated MoE model. Our findings reveal that the MLE has distinct behaviors under two complement settings of location parameters of the Gaussian gating functions. Notably, these behaviors can be characterized by the solvability of two different systems of equations.
  arXiv  Detail & Related papers  (2023-05-12T16:02:19Z)
• Joint Sensing, Communication, and AI: A Trifecta for Resilient THz User Experiences [118.91584633024907]
  A novel joint sensing, communication, and artificial intelligence (AI) framework is proposed so as to optimize extended reality (XR) experiences over terahertz (THz) wireless systems.
  arXiv  Detail & Related papers  (2023-04-29T00:39:50Z)
• Online Parameter Estimation for Safety-Critical Systems with Gaussian Processes [6.122161391301866]
  We present a Bayesian optimization framework based on Gaussian processes (GPs) for online parameter estimation. It uses an efficient search strategy over a response surface in the parameter space for finding the global optima with minimal function evaluations. We demonstrate our technique on an actuated planar pendulum and safety-critical quadrotor in simulation with changing parameters.
  arXiv  Detail & Related papers  (2020-02-18T20:38:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.