Enhancing Deep Learning Models through Tensorization: A Comprehensive Survey and Framework

• URL: http://arxiv.org/abs/2309.02428v3
• Date: Mon, 9 Oct 2023 11:14:41 GMT
• Title: Enhancing Deep Learning Models through Tensorization: A Comprehensive Survey and Framework
• Authors: Manal Helal
• Abstract summary: This paper explores the steps involved in multidimensional data sources, various multiway analysis methods employed, and the benefits of these approaches. A small example of Blind Source Separation (BSS) is presented comparing 2-dimensional algorithms and a multiway algorithm in Python. Results indicate that multiway analysis is more expressive.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The burgeoning growth of public domain data and the increasing complexity of deep learning model architectures have underscored the need for more efficient data representation and analysis techniques. This paper is motivated by the work of (Helal, 2023) and aims to present a comprehensive overview of tensorization. This transformative approach bridges the gap between the inherently multidimensional nature of data and the simplified 2-dimensional matrices commonly used in linear algebra-based machine learning algorithms. This paper explores the steps involved in tensorization, multidimensional data sources, various multiway analysis methods employed, and the benefits of these approaches. A small example of Blind Source Separation (BSS) is presented comparing 2-dimensional algorithms and a multiway algorithm in Python. Results indicate that multiway analysis is more expressive. Contrary to the intuition of the dimensionality curse, utilising multidimensional datasets in their native form and applying multiway analysis methods grounded in multilinear algebra reveal a profound capacity to capture intricate interrelationships among various dimensions while, surprisingly, reducing the number of model parameters and accelerating processing. A survey of the multi-away analysis methods and integration with various Deep Neural Networks models is presented using case studies in different application domains.

Related papers

• You Only Scan Once: Efficient Multi-dimension Sequential Modeling with LightNet [47.48142221329556]
  We develop an efficient multi-dimensional sequential modeling framework called LightNet based on the new recurrence. We present two new multi-dimensional linear relative positional encoding methods, MD-TPE and MD-LRPE, to enhance the model's ability to discern positional information in multi-dimensional scenarios.
  arXiv  Detail & Related papers  (2024-05-31T17:09:16Z)
• Revealing Multimodal Contrastive Representation Learning through Latent Partial Causal Models [85.67870425656368]
  We introduce a unified causal model specifically designed for multimodal data. We show that multimodal contrastive representation learning excels at identifying latent coupled variables. Experiments demonstrate the robustness of our findings, even when the assumptions are violated.
  arXiv  Detail & Related papers  (2024-02-09T07:18:06Z)
• MixUp-MIL: A Study on Linear & Multilinear Interpolation-Based Data Augmentation for Whole Slide Image Classification [1.5810132476010594]
  We investigate a data augmentation technique for classifying digital whole slide images. The results show an extraordinarily high variability in the effect of the method. We identify several interesting aspects to bring light into the darkness and identified novel promising fields of research.
  arXiv  Detail & Related papers  (2023-11-06T12:00:53Z)
• Dynamic Latent Separation for Deep Learning [67.62190501599176]
  A core problem in machine learning is to learn expressive latent variables for model prediction on complex data. Here, we develop an approach that improves expressiveness, provides partial interpretation, and is not restricted to specific applications.
  arXiv  Detail & Related papers  (2022-10-07T17:56:53Z)
• Understanding High Dimensional Spaces through Visual Means Employing Multidimensional Projections [0.0]
  Two of the relevant algorithms in the data visualisation field are t-distributed neighbourhood embedding (t-SNE) and Least-Square Projection (LSP) These algorithms can be used to understand several ranges of mathematical functions including their impact on datasets. We illustrate ways of employing the visual results of multidimensional projection algorithms to understand and fine-tune the parameters of their mathematical framework.
  arXiv  Detail & Related papers  (2022-07-12T20:30:33Z)
• Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization [101.32332941117271]
  Decision making algorithms are used in a multitude of different applications. Deep learning approaches that use highly parametric architectures tuned from data without relying on mathematical models are becoming increasingly popular. Model-based optimization and data-centric deep learning are often considered to be distinct disciplines.
  arXiv  Detail & Related papers  (2022-05-05T13:40:08Z)
• Exploring Dimensionality Reduction Techniques in Multilingual Transformers [64.78260098263489]
  This paper gives a comprehensive account of the impact of dimensional reduction techniques on the performance of state-of-the-art multilingual Siamese Transformers. It shows that it is possible to achieve an average reduction in the number of dimensions of $91.58% pm 2.59%$ and $54.65% pm 32.20%$, respectively.
  arXiv  Detail & Related papers  (2022-04-18T17:20:55Z)
• Geometric Multimodal Deep Learning with Multi-Scaled Graph Wavelet Convolutional Network [21.06669693699965]
  Multimodal data provide information of a natural phenomenon by integrating data from various domains with very different statistical properties. Capturing the intra-modality and cross-modality information of multimodal data is the essential capability of multimodal learning methods. Generalizing deep learning methods to the non-Euclidean domains is an emerging research field.
  arXiv  Detail & Related papers  (2021-11-26T08:41:51Z)
• Multimodal Data Fusion in High-Dimensional Heterogeneous Datasets via Generative Models [16.436293069942312]
  We are interested in learning probabilistic generative models from high-dimensional heterogeneous data in an unsupervised fashion. We propose a general framework that combines disparate data types through the exponential family of distributions. The proposed algorithm is presented in detail for the commonly encountered heterogeneous datasets with real-valued (Gaussian) and categorical (multinomial) features.
  arXiv  Detail & Related papers  (2021-08-27T18:10:31Z)
• Deep Representational Similarity Learning for analyzing neural signatures in task-based fMRI dataset [81.02949933048332]
  This paper develops Deep Representational Similarity Learning (DRSL), a deep extension of Representational Similarity Analysis (RSA) DRSL is appropriate for analyzing similarities between various cognitive tasks in fMRI datasets with a large number of subjects.
  arXiv  Detail & Related papers  (2020-09-28T18:30:14Z)
• Multi-Objective Genetic Programming for Manifold Learning: Balancing Quality and Dimensionality [4.4181317696554325]
  State-of-the-art manifold learning algorithms are opaque in how they perform this transformation. We introduce a multi-objective approach that automatically balances the competing objectives of manifold quality and dimensionality. Our proposed approach is competitive with a range of baseline and state-of-the-art manifold learning methods.
  arXiv  Detail & Related papers  (2020-01-05T23:24:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.