Fast Machine Learning Method with Vector Embedding on Orthonormal Basis
and Spectral Transform
- URL: http://arxiv.org/abs/2310.18424v2
- Date: Mon, 13 Nov 2023 16:48:01 GMT
- Title: Fast Machine Learning Method with Vector Embedding on Orthonormal Basis
and Spectral Transform
- Authors: Louis Yu Lu
- Abstract summary: The paper provides examples of word embedding, text chunk embedding, and image embedding, implemented in Julia language with a vector database.
It also investigates unsupervised learning and supervised learning using this method, along with strategies for handling large data volumes.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: This paper presents a novel fast machine learning method that leverages two
techniques: Vector Embedding on Orthonormal Basis (VEOB) and Spectral Transform
(ST). The VEOB converts the original data encoding into a vector embedding with
coordinates projected onto orthonormal bases. The Singular Value Decomposition
(SVD) technique is used to calculate the vector basis and projection
coordinates, leading to an enhanced distance measurement in the embedding space
and facilitating data compression by preserving the projection vectors
associated with the largest singular values. On the other hand, ST transforms
sequence of vector data into spectral space. By applying the Discrete Cosine
Transform (DCT) and selecting the most significant components, it streamlines
the handling of lengthy vector sequences. The paper provides examples of word
embedding, text chunk embedding, and image embedding, implemented in Julia
language with a vector database. It also investigates unsupervised learning and
supervised learning using this method, along with strategies for handling large
data volumes.
Related papers
- Knowledge Composition using Task Vectors with Learned Anisotropic Scaling [51.4661186662329]
We introduce aTLAS, an algorithm that linearly combines parameter blocks with different learned coefficients, resulting in anisotropic scaling at the task vector level.
We show that such linear combinations explicitly exploit the low intrinsicity of pre-trained models, with only a few coefficients being the learnable parameters.
We demonstrate the effectiveness of our method in task arithmetic, few-shot recognition and test-time adaptation, with supervised or unsupervised objectives.
arXiv Detail & Related papers (2024-07-03T07:54:08Z) - An Intrinsic Vector Heat Network [64.55434397799728]
This paper introduces a novel neural network architecture for learning tangent vector fields embedded in 3D.
We introduce a trainable vector heat diffusion module to spatially propagate vector-valued feature data across the surface.
We also demonstrate the effectiveness of our method on the useful industrial application of quadrilateral mesh generation.
arXiv Detail & Related papers (2024-06-14T00:40:31Z) - Layered Image Vectorization via Semantic Simplification [46.23779847614095]
This work presents a novel progressive image vectorization technique aimed at generating layered vectors that represent the original image from coarse to fine detail levels.
Our approach introduces semantic simplification, which combines Score Distillation Sampling and semantic segmentation to iteratively simplify the input image.
Our method provides robust optimization, which avoids local minima and enables adjustable detail levels in the final output.
arXiv Detail & Related papers (2024-06-08T08:54:35Z) - In-Context Learning State Vector with Inner and Momentum Optimization [23.33921300777915]
Large Language Models (LLMs) have exhibited an impressive ability to perform In-Context Learning (ICL) from only a few examples.
Recent works have indicated that the functions learned by ICL can be represented through compressed vectors derived from the transformer.
This paper presents a comprehensive analysis of these compressed vectors, drawing parallels to the parameters trained with gradient descent, and the concept of state vector.
arXiv Detail & Related papers (2024-04-17T10:19:15Z) - Banach-Tarski Embeddings and Transformers [0.0]
We introduce a new construction of embeddings of arbitrary data structures into high dimensional vectors.
These embeddings provide an interpretable model for the latent state vectors of transformers.
We show that these embeddings can be decoded to the original data structure when the embedding dimension is sufficiently large.
arXiv Detail & Related papers (2023-11-15T21:30:26Z) - Weakly Supervised Change Detection Using Guided Anisotropic Difusion [97.43170678509478]
We propose original ideas that help us to leverage such datasets in the context of change detection.
First, we propose the guided anisotropic diffusion (GAD) algorithm, which improves semantic segmentation results.
We then show its potential in two weakly-supervised learning strategies tailored for change detection.
arXiv Detail & Related papers (2021-12-31T10:03:47Z) - TransCMD: Cross-Modal Decoder Equipped with Transformer for RGB-D
Salient Object Detection [86.94578023985677]
In this work, we rethink this task from the perspective of global information alignment and transformation.
Specifically, the proposed method (TransCMD) cascades several cross-modal integration units to construct a top-down transformer-based information propagation path.
Experimental results on seven RGB-D SOD benchmark datasets demonstrate that a simple two-stream encoder-decoder framework can surpass the state-of-the-art purely CNN-based methods.
arXiv Detail & Related papers (2021-12-04T15:45:34Z) - Dynamic Texture Recognition using PDV Hashing and Dictionary Learning on
Multi-scale Volume Local Binary Pattern [11.497810572868396]
We propose a method for dynamic texture recognition using PDV hashing and dictionary learning on multi-scale volume local binary pattern (PHD-MVLBP)
Instead of forming very high-dimensional LBP histogram features, it first uses hash functions to map the pixel difference vectors (PDVs) to binary vectors, then forms a dictionary using the derived binary vector, and encodes them using the derived dictionary.
arXiv Detail & Related papers (2021-11-24T07:57:14Z) - A Tensor SVD-based Classification Algorithm Applied to fMRI Data [0.0]
We use a projection-based classification algorithm using the t-SVDM, a tensor analog of the matrix SVD.
Our numerical experiments demonstrate that there exists a superior tensor-based approach to fMRI classification than the best possible equivalent matrix-based approach.
arXiv Detail & Related papers (2021-10-31T20:39:23Z) - Anchor & Transform: Learning Sparse Embeddings for Large Vocabularies [60.285091454321055]
We design a simple and efficient embedding algorithm that learns a small set of anchor embeddings and a sparse transformation matrix.
On text classification, language modeling, and movie recommendation benchmarks, we show that ANT is particularly suitable for large vocabulary sizes.
arXiv Detail & Related papers (2020-03-18T13:07:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.