Generalized Factor Neural Network Model for High-dimensional Regression

• URL: http://arxiv.org/abs/2502.11310v1
• Date: Sun, 16 Feb 2025 23:13:55 GMT
• Title: Generalized Factor Neural Network Model for High-dimensional Regression
• Authors: Zichuan Guo, Mihai Cucuringu, Alexander Y. Shestopaloff,
• Abstract summary: We tackle the challenges of modeling high-dimensional data sets with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships. Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression.
• Score: 50.554377879576066
• License:
• Abstract: We tackle the challenges of modeling high-dimensional data sets, particularly those with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships. Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression. Our approach introduces PCA and Soft PCA layers, which can be embedded at any stage of a neural network architecture, allowing the model to alternate between factor modeling and non-linear transformations. This flexibility makes our method especially effective for processing hierarchical compositional data. We explore ours and other techniques for imposing low-rank structures on neural networks and examine how architectural design impacts model performance. The effectiveness of our method is demonstrated through simulation studies, as well as applications to forecasting future price movements of equity ETF indices and nowcasting with macroeconomic data.

Related papers

• A Survey of Model Architectures in Information Retrieval [64.75808744228067]
  We focus on two key aspects: backbone models for feature extraction and end-to-end system architectures for relevance estimation. We trace the development from traditional term-based methods to modern neural approaches, particularly highlighting the impact of transformer-based models and subsequent large language models (LLMs) We conclude by discussing emerging challenges and future directions, including architectural optimizations for performance and scalability, handling of multimodal, multilingual data, and adaptation to novel application domains beyond traditional search paradigms.
  arXiv  Detail & Related papers  (2025-02-20T18:42:58Z)
• A Riemannian Framework for Learning Reduced-order Lagrangian Dynamics [18.151022395233152]
  We propose a novel geometric network architecture to learn physically-consistent reduced-order dynamic parameters. Our approach enables accurate long-term predictions of the high-dimensional dynamics of rigid and deformable systems.
  arXiv  Detail & Related papers  (2024-10-24T15:53:21Z)
• Physics-Informed Machine Learning for Seismic Response Prediction OF Nonlinear Steel Moment Resisting Frame Structures [6.483318568088176]
  PiML method integrates scientific principles and physical laws into deep neural networks to model seismic responses of nonlinear structures. Manipulating the equation of motion helps learn system nonlinearities and confines solutions within physically interpretable results. Result handles complex data better than existing physics-guided LSTM models and outperforms other non-physics data-driven networks.
  arXiv  Detail & Related papers  (2024-02-28T02:16:03Z)
• Self Expanding Convolutional Neural Networks [1.4330085996657045]
  We present a novel method for dynamically expanding Convolutional Neural Networks (CNNs) during training. We employ a strategy where a single model is dynamically expanded, facilitating the extraction of checkpoints at various complexity levels.
  arXiv  Detail & Related papers  (2024-01-11T06:22:40Z)
• Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective [64.04617968947697]
  We introduce a novel data-model co-design perspective: to promote superior weight sparsity. Specifically, customized Visual Prompts are mounted to upgrade neural Network sparsification in our proposed VPNs framework.
  arXiv  Detail & Related papers  (2023-12-03T13:50:24Z)
• Recurrent neural networks and transfer learning for elasto-plasticity in woven composites [0.0]
  This article presents Recurrent Neural Network (RNN) models as a surrogate for computationally intensive meso-scale simulation of woven composites. A mean-field model generates a comprehensive data set representing elasto-plastic behavior. In simulations, arbitrary six-dimensional strain histories are used to predict stresses under random walking as the source task and cyclic loading conditions as the target task.
  arXiv  Detail & Related papers  (2023-11-22T14:47:54Z)
• Scaling Pre-trained Language Models to Deeper via Parameter-efficient Architecture [68.13678918660872]
  We design a more capable parameter-sharing architecture based on matrix product operator (MPO) MPO decomposition can reorganize and factorize the information of a parameter matrix into two parts. Our architecture shares the central tensor across all layers for reducing the model size.
  arXiv  Detail & Related papers  (2023-03-27T02:34:09Z)
• Application of Clustering Algorithms for Dimensionality Reduction in Infrastructure Resilience Prediction Models [4.350783459690612]
  We present a clustering-based method that simultaneously minimizes the problem of high-dimensionality and improves the prediction accuracy of machine learning models. The proposed method can be used to develop decision-support tools for post-disaster recovery of infrastructure networks.
  arXiv  Detail & Related papers  (2022-05-06T15:51:05Z)
• Sparse Flows: Pruning Continuous-depth Models [107.98191032466544]
  We show that pruning improves generalization for neural ODEs in generative modeling. We also show that pruning finds minimal and efficient neural ODE representations with up to 98% less parameters compared to the original network, without loss of accuracy.
  arXiv  Detail & Related papers  (2021-06-24T01:40:17Z)
• An Ode to an ODE [78.97367880223254]
  We present a new paradigm for Neural ODE algorithms, called ODEtoODE, where time-dependent parameters of the main flow evolve according to a matrix flow on the group O(d) This nested system of two flows provides stability and effectiveness of training and provably solves the gradient vanishing-explosion problem.
  arXiv  Detail & Related papers  (2020-06-19T22:05:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.