Characterizing the Representational Capacity of Neural Processes

• URL: http://arxiv.org/abs/2605.24210v1
• Date: Fri, 22 May 2026 20:49:53 GMT
• Title: Characterizing the Representational Capacity of Neural Processes
• Abstract summary: Conditional Neural Processes (CNPs), Attentive Neural Processes (ANPs), Transformer Neural Processes (TNPs) and their latent variants are studied.<n>CNP-representable functions are exactly those depending on finitely many expected features of the context distribution.<n>For latent NPs, we show finite-dimensional latents provide coherent sampling but do not circumvent encoder limitations.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: What functions can Neural Processes represent? We analyze the representational capacity of popular NP architectures: Conditional Neural Processes (CNPs), Attentive Neural Processes (ANPs), Transformer Neural Processes (TNPs), and their latent variants. We prove these architectures form a strict hierarchy. CNP-representable functions are exactly those depending on finitely many expected features of the context distribution. ANPs strictly generalize CNPs via query-dependent reweighting, enabling kernel smoothers. ConvCNPs and ANPs are incomparable; each contains functions outside the other, separated by stationarity versus translation equivariance. TNPs with $L$ self-attention layers capture $L$-hop context interactions. For latent NPs, we show finite-dimensional latents provide coherent sampling but do not circumvent encoder limitations; matching GP posterior distributions requires latent dimension scaling with context size. These results provide a theoretical foundation for architecture selection based on task structure.

Related papers

• Exploring Pseudo-Token Approaches in Transformer Neural Processes [0.0]
  We introduce the Induced Set Attentive Neural Processes (ISANPs)<n>ISANPs perform competitively with Transformer Neural Processes (TNPs) and often surpass state-of-the-art models in 1D regression, image completion, contextual bandits, and Bayesian optimization.<n>ISANPs offer a tunable balance between performance and computational complexity, which scale well to larger datasets.
  arXiv  Detail & Related papers  (2025-04-19T22:47:59Z)
• In-Context In-Context Learning with Transformer Neural Processes [50.57807892496024]
  We develop the in-context in-context learning pseudo-token TNP (ICICL-TNP) The ICICL-TNP is capable of conditioning on both sets of datapoints and sets of datasets, enabling it to perform in-context in-context learning. We demonstrate the importance of in-context in-context learning and the effectiveness of the ICICL-TNP in a number of experiments.
  arXiv  Detail & Related papers  (2024-06-19T12:26:36Z)
• Spectral Convolutional Conditional Neural Processes [4.52069311861025]
  Conditional Neural Processes (CNPs) constitute a family of probabilistic models that harness the flexibility of neural networks to parameterize processes. We propose Spectral Convolutional Conditional Neural Processes (SConvCNPs), a new addition to the NPs family that allows for more efficient representation of functions in the frequency domain.
  arXiv  Detail & Related papers  (2024-04-19T21:13:18Z)
• Autoencoding Conditional Neural Processes for Representation Learning [31.63717849083666]
  Conditional neural processes (CNPs) are a flexible and efficient family of models that learn to learn from data. We develop the Partial Pixel Space Variational Autoencoder (PPS-VAE), an amortised variational framework that casts CNP context as latent variables learnt simultaneously with the CNP.
  arXiv  Detail & Related papers  (2023-05-29T12:10:18Z)
• Versatile Neural Processes for Learning Implicit Neural Representations [57.090658265140384]
  We propose Versatile Neural Processes (VNP), which largely increases the capability of approximating functions. Specifically, we introduce a bottleneck encoder that produces fewer and informative context tokens, relieving the high computational cost. We demonstrate the effectiveness of the proposed VNP on a variety of tasks involving 1D, 2D and 3D signals.
  arXiv  Detail & Related papers  (2023-01-21T04:08:46Z)
• Latent Bottlenecked Attentive Neural Processes [71.18817592128207]
  We present Latent Bottlenecked Attentive Neural Processes (LBANPs) LBANPs have a querying computational complexity independent of the number of context datapoints. We show LBANPs achieve results competitive with the state-of-the-art on meta-regression, image completion, and contextual multi-armed bandits.
  arXiv  Detail & Related papers  (2022-11-15T19:21:41Z)
• Signal Processing for Implicit Neural Representations [80.38097216996164]
  Implicit Neural Representations (INRs) encode continuous multi-media data via multi-layer perceptrons. Existing works manipulate such continuous representations via processing on their discretized instance. We propose an implicit neural signal processing network, dubbed INSP-Net, via differential operators on INR.
  arXiv  Detail & Related papers  (2022-10-17T06:29:07Z)
• Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling [26.377099481072992]
  We propose Transformer Neural Processes (TNPs) for uncertainty-aware meta learning. We learn TNPs via an autoregressive likelihood-based objective and instantiate it with a novel transformer-based architecture. We show that TNPs achieve state-of-the-art performance on various benchmark problems.
  arXiv  Detail & Related papers  (2022-07-09T02:28:58Z)
• Message Passing Neural Processes [3.0969191504482247]
  We introduce Message Passing Neural Processes (MPNPs), which explicitly makes use of relational structure within the model. MPNPs thrive at lower sampling rates, on existing benchmarks and newly-proposed CA and Cora-Branched tasks. We report strong generalisation over density-based CA rulesets and significant gains in challenging arbitrary-labelling and few-shot learning setups.
  arXiv  Detail & Related papers  (2020-09-29T09:40:09Z)
• Bootstrapping Neural Processes [114.97111530885093]
  Neural Processes (NPs) implicitly define a broad class of processes with neural networks. NPs still rely on an assumption that uncertainty in processes is modeled by a single latent variable. We propose the Boostrapping Neural Process (BNP), a novel extension of the NP family using the bootstrap.
  arXiv  Detail & Related papers  (2020-08-07T02:23:34Z)
• NP-PROV: Neural Processes with Position-Relevant-Only Variances [113.20013269514327]
  We present a new member named Neural Processes with Position-Relevant-Only Variances (NP-PROV) NP-PROV hypothesizes that a target point close to a context point has small uncertainty, regardless of the function value at that position. Our evaluation on synthetic and real-world datasets reveals that NP-PROV can achieve state-of-the-art likelihood while retaining a bounded variance.
  arXiv  Detail & Related papers  (2020-06-15T06:11:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.