Entropic Efficiency of Bayesian Inference Protocols

• URL: http://arxiv.org/abs/2601.17282v1
• Date: Sat, 24 Jan 2026 03:43:38 GMT
• Title: Entropic Efficiency of Bayesian Inference Protocols
• Abstract summary: Inference is a versatile tool that underlies scientific discovery, machine learning, and everyday decision-making.<n>We define an inferential efficiency as the ratio of information gain to cumulative memory erasure cost.<n>We benchmark two limiting measurement paradigms: sequential, in which the same memory is exploited iteratively, and parallel, in which many memories are exploited simultaneously.
• Score: 0.8594140167290097
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inference is a versatile tool that underlies scientific discovery, machine learning, and everyday decision-making: it describes how an agent updates a probability distribution as partial information is acquired from multiple measurements, reducing ignorance about a system's latent state. We define an inferential efficiency as the ratio of information gain to cumulative memory erasure cost, with inefficiency arising from unexploited correlations between the measured system and memories, and/or between memories and environment (noise). Using this efficiency, we benchmark two limiting measurement paradigms: sequential, in which the same memory is exploited iteratively, and parallel, in which many memories are exploited simultaneously. In both cases, the minimal erasure cost reflects correlations across memories: temporal in sequential, spatial in parallel. Remarkably, when all system-memory correlations are exploited for inference, both paradigms attain the same minimal erasure cost, even in the presence of noise. Conversely, the parallel paradigm performs better in the presence of unexploited correlations, stemming from hidden memories' degrees of freedom. This approach provides a quantitative, physically grounded criterion to compare inference strategies, determine their efficiency, and link target information gains to their minimal entropic cost.

Related papers

• Bridging Inference-Time Scaling and Episodic Memory with Action-Centric Graphs [48.754678397305355]
  We propose a novel framework, textitGAMER(Graph-based Action-centric Memory with Episodic Reasoning), that bridges the gap between inference scaling and episodic memory.<n>Our approach models historical reasoning as a dynamic textitAction-Centric Graph.<n> Experiments on multiple benchmarks demonstrate that textitGAMER achieves superior performance by textbf20.81%/6.17% for success/progress rate compared to vanilla baselines.
  arXiv  Detail & Related papers  (2026-07-29T19:31:50Z)
• Distinguishing quantum processes with bounded coherent memory [0.0]
  Distinguishing multi-time quantum processes is a fundamental task.<n>Standard benchmark for distinguishing two processes is the strategy-norm distance.<n>We introduce machines for autonomous distinction.
  arXiv  Detail & Related papers  (2026-06-17T18:52:19Z)
• Social Hippocampus Memory Learning [54.381608978482184]
  SoHip is a memory-centric social machine learning framework.<n>It enables collaboration among heterogeneous agents via memory sharing rather than model sharing.<n>Experiments on two benchmark datasets with seven baselines demonstrate that SoHip consistently outperforms existing methods.
  arXiv  Detail & Related papers  (2026-03-26T16:31:00Z)
• Evoking User Memory: Personalizing LLM via Recollection-Familiarity Adaptive Retrieval [59.295767860331004]
  RF-Mem is a familiarity uncertainty-guided dual-path memory retriever.<n>It embeds human-like dual-process recognition into the retriever.<n>It consistently outperforms one-shot retrieval and full-context reasoning under fixed budget and latency constraints.
  arXiv  Detail & Related papers  (2026-03-10T06:31:44Z)
• MemRec: Collaborative Memory-Augmented Agentic Recommender System [57.548438733740504]
  We propose MemRec, a framework that architecturally decouples reasoning from memory management.<n>MemRec introduces a dedicated LM_Mem to manage a dynamic collaborative memory graph.<n>It achieves state-of-the-art performance on four benchmarks.
  arXiv  Detail & Related papers  (2026-01-13T18:51:16Z)
• FlashMem: Distilling Intrinsic Latent Memory via Computation Reuse [4.210760734549566]
  FlashMem is a framework that distills intrinsic memory directly from transient reasoning states via computation reuse.<n>Experiments demonstrate that FlashMem matches the performance of heavy baselines while reducing inference latency by 5 times.
  arXiv  Detail & Related papers  (2026-01-09T03:27:43Z)
• SimpleMem: Efficient Lifelong Memory for LLM Agents [73.74399447715052]
  We introduce SimpleMem, an efficient memory framework based on semantic lossless compression.<n>We propose a three-stage pipeline designed to maximize information density and token utilization.<n> Experiments on benchmark datasets show that our method consistently outperforms baseline approaches in accuracy, retrieval efficiency, and inference cost.
  arXiv  Detail & Related papers  (2026-01-05T21:02:49Z)
• Provable Separations between Memorization and Generalization in Diffusion Models [31.32606216124959]
  We develop a dual-separation result via statistical estimation and network approximation.<n>We show that the ground-truth score function does not minimize the empirical denoising loss, creating a separation that drives memorization.
  arXiv  Detail & Related papers  (2025-11-05T05:37:57Z)
• PBFT-Backed Semantic Voting for Multi-Agent Memory Pruning [0.0]
  This paper introduces the Co-Forgetting Protocol to enable synchronized memory pruning in multi-agent systems (MAS)<n>The protocol integrates three key components: (1) context-aware voting, where agents utilize a lightweight DistilBERT model to assess the relevance of memory items based on their content and the current operational context; (2) multi-scale temporal decay functions, which assign diminishing importance to memories based on their age and access frequency across different time horizons; and (3) a Practical Byzantine Fault Tolerance (PBFT)-based consensus mechanism, ensuring that decisions to retain or discard memory items are agreed upon by a qualified and fault-tolerant majority of agents
  arXiv  Detail & Related papers  (2025-06-19T08:28:29Z)
• REPAIR: Rank Correlation and Noisy Pair Half-replacing with Memory for Noisy Correspondence [36.274879585424635]
  The presence of noise in acquired data invariably leads to performance degradation in crossmodal matching. We propose a framework as Rank corrElation and noisy hAlf wIth memoRy to tackle the mismatched data pair issue.
  arXiv  Detail & Related papers  (2024-03-13T04:01:20Z)
• Unknown measurement statistics cannot be redundantly copied using finite resources [0.0]
  Measurements can be viewed as interactions between a measured system and a pointer system that imprint information about the system on the pointer.<n>We consider the problem of unitarily distributing the outcome statistics to several pointers or quantum memories.<n>We show that the accuracy of this process is limited by thermodynamic restrictions on preparing the memories in pure states.
  arXiv  Detail & Related papers  (2024-03-12T13:50:01Z)
• Deep Ensembles Meets Quantile Regression: Uncertainty-aware Imputation for Time Series [45.76310830281876]
  We propose Quantile Sub-Ensembles, a novel method to estimate uncertainty with ensemble of quantile-regression-based task networks. Our method not only produces accurate imputations that is robust to high missing rates, but also is computationally efficient due to the fast training of its non-generative model.
  arXiv  Detail & Related papers  (2023-12-03T05:52:30Z)
• Hybrid Predictive Coding: Inferring, Fast and Slow [62.997667081978825]
  We propose a hybrid predictive coding network that combines both iterative and amortized inference in a principled manner. We demonstrate that our model is inherently sensitive to its uncertainty and adaptively balances balances to obtain accurate beliefs using minimum computational expense.
  arXiv  Detail & Related papers  (2022-04-05T12:52:45Z)
• Efficient Global-Local Memory for Real-time Instrument Segmentation of Robotic Surgical Video [53.14186293442669]
  We identify two important clues for surgical instrument perception, including local temporal dependency from adjacent frames and global semantic correlation in long-range duration. We propose a novel dual-memory network (DMNet) to relate both global and local-temporal knowledge. Our method largely outperforms the state-of-the-art works on segmentation accuracy while maintaining a real-time speed.
  arXiv  Detail & Related papers  (2021-09-28T10:10:14Z)
• Unsupervised Feature Learning for Event Data: Direct vs Inverse Problem Formulation [53.850686395708905]
  Event-based cameras record an asynchronous stream of per-pixel brightness changes. In this paper, we focus on single-layer architectures for representation learning from event data. We show improvements of up to 9 % in the recognition accuracy compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-09-23T10:40:03Z)
• Towards Certified Robustness of Distance Metric Learning [53.96113074344632]
  We advocate imposing an adversarial margin in the input space so as to improve the generalization and robustness of metric learning algorithms. We show that the enlarged margin is beneficial to the generalization ability by using the theoretical technique of algorithmic robustness.
  arXiv  Detail & Related papers  (2020-06-10T16:51:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.