Associative Memories in the Feature Space

• URL: http://arxiv.org/abs/2402.10814v1
• Date: Fri, 16 Feb 2024 16:37:48 GMT
• Title: Associative Memories in the Feature Space
• Authors: Tommaso Salvatori, Beren Millidge, Yuhang Song, Rafal Bogacz, Thomas Lukasiewicz
• Abstract summary: We propose a class of memory models that only stores low-dimensional semantic embeddings, and uses them to retrieve similar, but not identical, memories. We demonstrate a proof of concept of this method on a simple task on the MNIST dataset.
• Score: 68.1903319310263
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: An autoassociative memory model is a function that, given a set of data points, takes as input an arbitrary vector and outputs the most similar data point from the memorized set. However, popular memory models fail to retrieve images even when the corruption is mild and easy to detect for a human evaluator. This is because similarities are evaluated in the raw pixel space, which does not contain any semantic information about the images. This problem can be easily solved by computing \emph{similarities} in an embedding space instead of the pixel space. We show that an effective way of computing such embeddings is via a network pretrained with a contrastive loss. As the dimension of embedding spaces is often significantly smaller than the pixel space, we also have a faster computation of similarity scores. We test this method on complex datasets such as CIFAR10 and STL10. An additional drawback of current models is the need of storing the whole dataset in the pixel space, which is often extremely large. We relax this condition and propose a class of memory models that only stores low-dimensional semantic embeddings, and uses them to retrieve similar, but not identical, memories. We demonstrate a proof of concept of this method on a simple task on the MNIST dataset.

Related papers

• Entropic Hetero-Associative Memory [0.8287206589886881]
  The Entropic Associative Memory holds objects in a 2D relation or memory plane'' using a finite table as the medium. Stored objects are overlapped'' on the medium, hence the memory is indeterminate and has an entropy value at each state.
  arXiv  Detail & Related papers  (2024-11-02T08:30:57Z)
• Improving Image Recognition by Retrieving from Web-Scale Image-Text Data [68.63453336523318]
  We introduce an attention-based memory module, which learns the importance of each retrieved example from the memory. Compared to existing approaches, our method removes the influence of the irrelevant retrieved examples, and retains those that are beneficial to the input query. We show that it achieves state-of-the-art accuracies in ImageNet-LT, Places-LT and Webvision datasets.
  arXiv  Detail & Related papers  (2023-04-11T12:12:05Z)
• SeiT: Storage-Efficient Vision Training with Tokens Using 1% of Pixel Storage [52.317406324182215]
  We propose a storage-efficient training strategy for vision classifiers for large-scale datasets. Our token storage only needs 1% of the original JPEG-compressed raw pixels. Our experimental results on ImageNet-1k show that our method significantly outperforms other storage-efficient training methods with a large gap.
  arXiv  Detail & Related papers  (2023-03-20T13:55:35Z)
• STAIR: Learning Sparse Text and Image Representation in Grounded Tokens [84.14528645941128]
  We show that it is possible to build a sparse semantic representation that is as powerful as, or even better than, dense presentations. We extend the CLIP model and build a sparse text and image representation (STAIR), where the image and text are mapped to a sparse token space. It significantly outperforms a CLIP model with +$4.9%$ and +$4.3%$ absolute Recall@1 improvement.
  arXiv  Detail & Related papers  (2023-01-30T17:21:30Z)
• Classification and Generation of real-world data with an Associative Memory Model [0.0]
  We extend the capabilities of the basic Associative Memory Model by using a Multiple-Modality framework. By storing both the images and labels as modalities, a single Memory can be used to retrieve and complete patterns.
  arXiv  Detail & Related papers  (2022-07-11T12:51:27Z)
• A model of semantic completion in generative episodic memory [0.6690874707758508]
  We propose a computational model for generative episodic memory. The model is able to complete missing parts of a memory trace in a semantically plausible way. We also model an episodic memory experiment and can reproduce that semantically congruent contexts are always recalled better than incongruent ones.
  arXiv  Detail & Related papers  (2021-11-26T15:14:17Z)
• Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation [68.45737688496654]
  We establish correspondences directly between frames without re-encoding the mask features for every object. With the correspondences, every node in the current query frame is inferred by aggregating features from the past in an associative fashion. We validated that every memory node now has a chance to contribute, and experimentally showed that such diversified voting is beneficial to both memory efficiency and inference accuracy.
  arXiv  Detail & Related papers  (2021-06-09T16:50:57Z)
• Efficient Classification of Very Large Images with Tiny Objects [15.822654320750054]
  We present an end-to-end CNN model termed Zoom-In network for classification of large images with tiny objects. We evaluate our method on two large-image datasets and one gigapixel dataset.
  arXiv  Detail & Related papers  (2021-06-04T20:13:04Z)
• Kanerva++: extending The Kanerva Machine with differentiable, locally block allocated latent memory [75.65949969000596]
  Episodic and semantic memory are critical components of the human memory model. We develop a new principled Bayesian memory allocation scheme that bridges the gap between episodic and semantic memory. We demonstrate that this allocation scheme improves performance in memory conditional image generation.
  arXiv  Detail & Related papers  (2021-02-20T18:40:40Z)
• CNN with large memory layers [2.368995563245609]
  This work is centred around the recently proposed product key memory structure citelarge_memory, implemented for a number of computer vision applications. The memory structure can be regarded as a simple computation primitive suitable to be augmented to nearly all neural network architectures.
  arXiv  Detail & Related papers  (2021-01-27T20:58:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.