Actively Inferring Optimal Measurement Sequences

• URL: http://arxiv.org/abs/2502.18142v1
• Date: Tue, 25 Feb 2025 12:10:28 GMT
• Title: Actively Inferring Optimal Measurement Sequences
• Authors: Catherine F. Higham, Paul Henderson, Roderick Murray-Smith,
• Abstract summary: We develop an active sequential inference algorithm that uses the low dimensional representational latent space to choose which measurement to make next.<n>The algorithm is illustrated using the Fashion MNIST dataset and a novel convolutional Hadamard pattern measurement basis.
• Score: 8.229775890542967
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Measurement of a physical quantity such as light intensity is an integral part of many reconstruction and decision scenarios but can be costly in terms of acquisition time, invasion of or damage to the environment and storage. Data minimisation and compliance with data protection laws is also an important consideration. Where there are a range of measurements that can be made, some may be more informative and compliant with the overall measurement objective than others. We develop an active sequential inference algorithm that uses the low dimensional representational latent space from a variational autoencoder (VAE) to choose which measurement to make next. Our aim is to recover high dimensional data by making as few measurements as possible. We adapt the VAE encoder to map partial data measurements on to the latent space of the complete data. The algorithm draws samples from this latent space and uses the VAE decoder to generate data conditional on the partial measurements. Estimated measurements are made on the generated data and fed back through the partial VAE encoder to the latent space where they can be evaluated prior to making a measurement. Starting from no measurements and a normal prior on the latent space, we consider alternative strategies for choosing the next measurement and updating the predictive posterior prior for the next step. The algorithm is illustrated using the Fashion MNIST dataset and a novel convolutional Hadamard pattern measurement basis. We see that useful patterns are chosen within 10 steps, leading to the convergence of the guiding generative images. Compared with using stochastic variational inference to infer the parameters of the posterior distribution for each generated data point individually, the partial VAE framework can efficiently process batches of generated data and obtains superior results with minimal measurements.

Related papers

• Reinforced Disentanglers on Random Unitary Circuits [0.10923877073891444]
  We search for efficient disentanglers on random Clifford circuits of two-qubit gates arranged in a brick-wall pattern. Disentanglers are defined as a set of projective measurements inserted between consecutive entangling layers.
  arXiv  Detail & Related papers  (2024-11-14T19:51:26Z)
• Minimally Supervised Learning using Topological Projections in Self-Organizing Maps [55.31182147885694]
  We introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs) Our proposed method first trains SOMs on unlabeled data and then a minimal number of available labeled data points are assigned to key best matching units (BMU) Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques.
  arXiv  Detail & Related papers  (2024-01-12T22:51:48Z)
• Data Selection for Language Models via Importance Resampling [90.9263039747723]
  We formalize the problem of selecting a subset of a large raw unlabeled dataset to match a desired target distribution. We extend the classic importance resampling approach used in low-dimensions for LM data selection. We instantiate the DSIR framework with hashed n-gram features for efficiency, enabling the selection of 100M documents in 4.5 hours.
  arXiv  Detail & Related papers  (2023-02-06T23:57:56Z)
• FONDUE: an algorithm to find the optimal dimensionality of the latent representations of variational autoencoders [2.969705152497174]
  In this paper, we explore the intrinsic dimension estimation (IDE) of the data and latent representations learned by VAEs. We show that the discrepancies between theIDE of the mean and sampled representations of a VAE after only a few steps of training reveal the presence of passive variables in the latent space. We propose FONDUE: an algorithm which quickly finds the number of latent dimensions after which the mean and sampled representations start to diverge.
  arXiv  Detail & Related papers  (2022-09-26T15:59:54Z)
• Information Entropy Initialized Concrete Autoencoder for Optimal Sensor Placement and Reconstruction of Geophysical Fields [58.720142291102135]
  We propose a new approach to the optimal placement of sensors for reconstructing geophysical fields from sparse measurements. We demonstrate our method on the two examples: (a) temperature and (b) salinity fields around the Barents Sea and the Svalbard group of islands. We find out that the obtained optimal sensor locations have clear physical interpretation and correspond to the boundaries between sea currents.
  arXiv  Detail & Related papers  (2022-06-28T12:43:38Z)
• RENs: Relevance Encoding Networks [0.0]
  This paper proposes relevance encoding networks (RENs): a novel probabilistic VAE-based framework that uses the automatic relevance determination (ARD) prior in the latent space to learn the data-specific bottleneck dimensionality. We show that the proposed model learns the relevant latent bottleneck dimensionality without compromising the representation and generation quality of the samples.
  arXiv  Detail & Related papers  (2022-05-25T21:53:48Z)
• Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty [58.144520501201995]
  Bi-Lipschitz regularization of neural network layers preserve relative distances between data instances in the feature spaces of each layer. With the use of an attentive set encoder, we propose to meta learn either diagonal or diagonal plus low-rank factors to efficiently construct task specific covariance matrices. We also propose an inference procedure which utilizes scaled energy to achieve a final predictive distribution.
  arXiv  Detail & Related papers  (2021-10-12T22:04:19Z)
• SreaMRAK a Streaming Multi-Resolution Adaptive Kernel Algorithm [60.61943386819384]
  Existing implementations of KRR require that all the data is stored in the main memory. We propose StreaMRAK - a streaming version of KRR. We present a showcase study on two synthetic problems and the prediction of the trajectory of a double pendulum.
  arXiv  Detail & Related papers  (2021-08-23T21:03:09Z)
• Data Generation in Low Sample Size Setting Using Manifold Sampling and a Geometry-Aware VAE [0.0]
  We develop two non emphprior-dependent generation procedures based on the geometry of the latent space. The latter method is used to perform data augmentation in a small sample size setting and is validated across various standard and emphreal-life data sets.
  arXiv  Detail & Related papers  (2021-03-25T11:07:10Z)
• Representation Learning for Sequence Data with Deep Autoencoding Predictive Components [96.42805872177067]
  We propose a self-supervised representation learning method for sequence data, based on the intuition that useful representations of sequence data should exhibit a simple structure in the latent space. We encourage this latent structure by maximizing an estimate of predictive information of latent feature sequences, which is the mutual information between past and future windows at each time step. We demonstrate that our method recovers the latent space of noisy dynamical systems, extracts predictive features for forecasting tasks, and improves automatic speech recognition when used to pretrain the encoder on large amounts of unlabeled data.
  arXiv  Detail & Related papers  (2020-10-07T03:34:01Z)
• Evaluating representations by the complexity of learning low-loss predictors [55.94170724668857]
  We consider the problem of evaluating representations of data for use in solving a downstream task. We propose to measure the quality of a representation by the complexity of learning a predictor on top of the representation that achieves low loss on a task of interest.
  arXiv  Detail & Related papers  (2020-09-15T22:06:58Z)
• Performance Analysis of Semi-supervised Learning in the Small-data Regime using VAEs [0.261072980439312]
  In this work, we applied an existing algorithm that pre-trains a latent space representation of the data to capture the features in a lower-dimension for the small-data regime input. The fine-tuned latent space provides constant weights that are useful for classification. Here we will present the performance analysis of the VAE algorithm with different latent space sizes in the semi-supervised learning.
  arXiv  Detail & Related papers  (2020-02-26T16:19:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.