Golden Ratio-Based Sufficient Dimension Reduction

• URL: http://arxiv.org/abs/2410.19300v1
• Date: Fri, 25 Oct 2024 04:15:15 GMT
• Title: Golden Ratio-Based Sufficient Dimension Reduction
• Authors: Wenjing Yang, Yuhong Yang,
• Abstract summary: We propose a neural network based sufficient dimension reduction method. It identifies the structural dimension effectively and estimates the central space well. It takes advantages of approximation capabilities of neural networks for functions in Barron classes and leads to reduced computation cost.
• Score: 6.184279198087624
• License:
• Abstract: Many machine learning applications deal with high dimensional data. To make computations feasible and learning more efficient, it is often desirable to reduce the dimensionality of the input variables by finding linear combinations of the predictors that can retain as much original information as possible in the relationship between the response and the original predictors. We propose a neural network based sufficient dimension reduction method that not only identifies the structural dimension effectively, but also estimates the central space well. It takes advantages of approximation capabilities of neural networks for functions in Barron classes and leads to reduced computation cost compared to other dimension reduction methods in the literature. Additionally, the framework can be extended to fit practical dimension reduction, making the methodology more applicable in practical settings.

Related papers

• Approximated Likelihood Ratio: A Forward-Only and Parallel Framework for Boosting Neural Network Training [30.452060061499523]
  We introduce an approximation technique for the likelihood ratio (LR) method to alleviate computational and memory demands in gradient estimation. Experiments demonstrate the effectiveness of the approximation technique in neural network training.
  arXiv  Detail & Related papers  (2024-03-18T23:23:50Z)
• Nonlinear functional regression by functional deep neural network with kernel embedding [20.306390874610635]
  We propose a functional deep neural network with an efficient and fully data-dependent dimension reduction method. The architecture of our functional net consists of a kernel embedding step, a projection step, and a deep ReLU neural network for the prediction. The utilization of smooth kernel embedding enables our functional net to be discretization invariant, efficient, and robust to noisy observations.
  arXiv  Detail & Related papers  (2024-01-05T16:43:39Z)
• Large-Scale OD Matrix Estimation with A Deep Learning Method [70.78575952309023]
  The proposed method integrates deep learning and numerical optimization algorithms to infer matrix structure and guide numerical optimization. We conducted tests to demonstrate the good generalization performance of our method on a large-scale synthetic dataset.
  arXiv  Detail & Related papers  (2023-10-09T14:30:06Z)
• Towards a Better Theoretical Understanding of Independent Subnetwork Training [56.24689348875711]
  We take a closer theoretical look at Independent Subnetwork Training (IST) IST is a recently proposed and highly effective technique for solving the aforementioned problems. We identify fundamental differences between IST and alternative approaches, such as distributed methods with compressed communication.
  arXiv  Detail & Related papers  (2023-06-28T18:14:22Z)
• DimenFix: A novel meta-dimensionality reduction method for feature preservation [64.0476282000118]
  We propose a novel meta-method, DimenFix, which can be operated upon any base dimensionality reduction method that involves a gradient-descent-like process. By allowing users to define the importance of different features, which is considered in dimensionality reduction, DimenFix creates new possibilities to visualize and understand a given dataset.
  arXiv  Detail & Related papers  (2022-11-30T05:35:22Z)
• Nonlinear Sufficient Dimension Reduction with a Stochastic Neural Network [9.173528450234906]
  We propose a new type of neural network for sufficient dimension reduction for large-scale data. The proposed neural network is trained using an adaptive gradient Markov chain Monte Carlo algorithm. We show that the proposed method compares favorably with the existing state-of-the-art sufficient dimension reduction methods.
  arXiv  Detail & Related papers  (2022-10-09T20:56:57Z)
• Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning [53.17258888552998]
  This work proposes an exploration variant of the basic $Q$-learning protocol with linear function approximation. We show that the performance of the algorithm degrades very gracefully under a novel and more permissive notion of approximation error.
  arXiv  Detail & Related papers  (2022-06-01T23:26:51Z)
• Scalable computation of prediction intervals for neural networks via matrix sketching [79.44177623781043]
  Existing algorithms for uncertainty estimation require modifying the model architecture and training procedure. This work proposes a new algorithm that can be applied to a given trained neural network and produces approximate prediction intervals.
  arXiv  Detail & Related papers  (2022-05-06T13:18:31Z)
• A Local Similarity-Preserving Framework for Nonlinear Dimensionality Reduction with Neural Networks [56.068488417457935]
  We propose a novel local nonlinear approach named Vec2vec for general purpose dimensionality reduction. To train the neural network, we build the neighborhood similarity graph of a matrix and define the context of data points. Experiments of data classification and clustering on eight real datasets show that Vec2vec is better than several classical dimensionality reduction methods in the statistical hypothesis test.
  arXiv  Detail & Related papers  (2021-03-10T23:10:47Z)
• Joint Dimensionality Reduction for Separable Embedding Estimation [43.22422640265388]
  Low-dimensional embeddings for data from disparate sources play critical roles in machine learning, multimedia information retrieval, and bioinformatics. We propose a supervised dimensionality reduction method that learns linear embeddings jointly for two feature vectors representing data of different modalities or data from distinct types of entities. Our approach compares favorably against other dimensionality reduction methods, and against a state-of-the-art method of bilinear regression for predicting gene-disease associations.
  arXiv  Detail & Related papers  (2021-01-14T08:48:37Z)
• Relative gradient optimization of the Jacobian term in unsupervised deep learning [9.385902422987677]
  Learning expressive probabilistic models correctly describing the data is a ubiquitous problem in machine learning. Deep density models have been widely used for this task, but their maximum likelihood based training requires estimating the log-determinant of the Jacobian. We propose a new approach for exact training of such neural networks.
  arXiv  Detail & Related papers  (2020-06-26T16:41:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.