Map of Encoders -- Mapping Sentence Encoders using Quantum Relative Entropy

• URL: http://arxiv.org/abs/2602.08740v1
• Date: Mon, 09 Feb 2026 14:41:46 GMT
• Title: Map of Encoders -- Mapping Sentence Encoders using Quantum Relative Entropy
• Authors: Gaifan Zhang, Danushka Bollegala,
• Abstract summary: We construct a map of encoders covering 1101 publicly available sentence encoders.<n>Our map accurately reflects various relationships between encoders, where encoders with similar attributes are proximally located on the map.
• Score: 23.18985488473778
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a method to compare and visualise sentence encoders at scale by creating a map of encoders where each sentence encoder is represented in relation to the other sentence encoders. Specifically, we first represent a sentence encoder using an embedding matrix of a sentence set, where each row corresponds to the embedding of a sentence. Next, we compute the Pairwise Inner Product (PIP) matrix for a sentence encoder using its embedding matrix. Finally, we create a feature vector for each sentence encoder reflecting its Quantum Relative Entropy (QRE) with respect to a unit base encoder. We construct a map of encoders covering 1101 publicly available sentence encoders, providing a new perspective of the landscape of the pre-trained sentence encoders. Our map accurately reflects various relationships between encoders, where encoders with similar attributes are proximally located on the map. Moreover, our encoder feature vectors can be used to accurately infer downstream task performance of the encoders, such as in retrieval and clustering tasks, demonstrating the faithfulness of our map.

Related papers

• A blockBP decoder for the surface code [0.0]
  We present a new decoder for the surface code, which combines the accuracy of the tensor-network decoders with the efficiency and parallelism of the belief-propagation algorithm. Our decoder is therefore a belief-propagation decoder that works in the degenerate maximal likelihood decoding framework.
  arXiv  Detail & Related papers  (2024-02-07T13:32:32Z)
• DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers [6.405360669408265]
  We propose a simple, new method to analyze encoder-decoder Transformers: DecoderLens. Inspired by the LogitLens (for decoder-only Transformers), this method involves allowing the decoder to cross-attend representations of intermediate encoder layers. We report results from the DecoderLens applied to models trained on question answering, logical reasoning, speech recognition and machine translation.
  arXiv  Detail & Related papers  (2023-10-05T17:04:59Z)
• Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving [74.28510044056706]
  Existing methods usually adopt the decoupled encoder-decoder paradigm. In this work, we aim to alleviate the problem by two principles. We first predict a coarse-grained future position and action based on the encoder features. Then, conditioned on the position and action, the future scene is imagined to check the ramification if we drive accordingly.
  arXiv  Detail & Related papers  (2023-05-10T15:22:02Z)
• Is an encoder within reach? [3.9548535445908928]
  We introduce the idea of using the reach of the manifold spanned by the decoder to determine if an optimal encoder exists for a given dataset and decoder. We demonstrate that this allows us to determine which observations can be expected to have a unique, and thereby trustworthy, latent representation.
  arXiv  Detail & Related papers  (2022-06-03T13:06:22Z)
• MaskOCR: Text Recognition with Masked Encoder-Decoder Pretraining [68.05105411320842]
  We propose a novel approach MaskOCR to unify vision and language pre-training in the classical encoder-decoder recognition framework. We adopt the masked image modeling approach to pre-train the feature encoder using a large set of unlabeled real text images. We transform text data into synthesized text images to unify the data modalities of vision and language, and enhance the language modeling capability of the sequence decoder.
  arXiv  Detail & Related papers  (2022-06-01T08:27:19Z)
• StolenEncoder: Stealing Pre-trained Encoders [62.02156378126672]
  We propose the first attack called StolenEncoder to steal pre-trained image encoders. Our results show that the encoders stolen by StolenEncoder have similar functionality with the target encoders.
  arXiv  Detail & Related papers  (2022-01-15T17:04:38Z)
• Dense Coding with Locality Restriction for Decoder: Quantum Encoders vs. Super-Quantum Encoders [67.12391801199688]
  We investigate dense coding by imposing various locality restrictions to our decoder. In this task, the sender Alice and the receiver Bob share an entangled state.
  arXiv  Detail & Related papers  (2021-09-26T07:29:54Z)
• Small Lesion Segmentation in Brain MRIs with Subpixel Embedding [105.1223735549524]
  We present a method to segment MRI scans of the human brain into ischemic stroke lesion and normal tissues. We propose a neural network architecture in the form of a standard encoder-decoder where predictions are guided by a spatial expansion embedding network.
  arXiv  Detail & Related papers  (2021-09-18T00:21:17Z)
• Dynamic Neural Representational Decoders for High-Resolution Semantic Segmentation [98.05643473345474]
  We propose a novel decoder, termed dynamic neural representational decoder (NRD) As each location on the encoder's output corresponds to a local patch of the semantic labels, in this work, we represent these local patches of labels with compact neural networks. This neural representation enables our decoder to leverage the smoothness prior in the semantic label space, and thus makes our decoder more efficient.
  arXiv  Detail & Related papers  (2021-07-30T04:50:56Z)
• On Sparsifying Encoder Outputs in Sequence-to-Sequence Models [90.58793284654692]
  We take Transformer as the testbed and introduce a layer of gates in-between the encoder and the decoder. The gates are regularized using the expected value of the sparsity-inducing L0penalty. We investigate the effects of this sparsification on two machine translation and two summarization tasks.
  arXiv  Detail & Related papers  (2020-04-24T16:57:52Z)
• Encoder blind combinatorial compressed sensing [5.177947445379688]
  We consider the problem of designing a decoder to recover a set of sparse codes from their linear measurements alone. The contribution of this paper is a computationally efficient decoding algorithm, Decoder-Expander Based Factorisation.
  arXiv  Detail & Related papers  (2020-04-10T16:26:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.