Kernel Ridge Regression Using Importance Sampling with Application to Seismic Response Prediction

• URL: http://arxiv.org/abs/2009.09136v1
• Date: Sat, 19 Sep 2020 01:44:56 GMT
• Title: Kernel Ridge Regression Using Importance Sampling with Application to Seismic Response Prediction
• Authors: Farhad Pourkamali-Anaraki, Mohammad Amin Hariri-Ardebili, Lydia Morawiec
• Abstract summary: We propose a novel landmark selection method that promotes diversity using an efficient two-step approach. We also investigate the performance of several landmark selection techniques using a novel application of kernel methods for predicting structural responses due to earthquake load and material uncertainties.
• Score: 1.4180331276028657
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scalable kernel methods, including kernel ridge regression, often rely on low-rank matrix approximations using the Nystrom method, which involves selecting landmark points from large data sets. The existing approaches to selecting landmarks are typically computationally demanding as they require manipulating and performing computations with large matrices in the input or feature space. In this paper, our contribution is twofold. The first contribution is to propose a novel landmark selection method that promotes diversity using an efficient two-step approach. Our landmark selection technique follows a coarse to fine strategy, where the first step computes importance scores with a single pass over the whole data. The second step performs K-means clustering on the constructed coreset to use the obtained centroids as landmarks. Hence, the introduced method provides tunable trade-offs between accuracy and efficiency. Our second contribution is to investigate the performance of several landmark selection techniques using a novel application of kernel methods for predicting structural responses due to earthquake load and material uncertainties. Our experiments exhibit the merits of our proposed landmark selection scheme against baselines.

Related papers

• MIK: Modified Isolation Kernel for Biological Sequence Visualization, Classification, and Clustering [3.9146761527401424]
  This research proposes a novel approach called the Modified Isolation Kernel (MIK) as an alternative to the Gaussian kernel. MIK uses adaptive density estimation to capture local structures more accurately and integrates robustness measures. It exhibits improved preservation of the local and global structure and enables better visualization of clusters and subclusters in the embedded space.
  arXiv  Detail & Related papers  (2024-10-21T06:57:09Z)
• Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Models [57.52124921268249]
  We propose a Trust Sequential Quadratic Programming method to find both first and second-order stationary points. To converge to first-order stationary points, our method computes a gradient step in each iteration defined by minimizing a approximation of the objective subject. To converge to second-order stationary points, our method additionally computes an eigen step to explore the negative curvature the reduced Hessian matrix.
  arXiv  Detail & Related papers  (2024-09-24T04:39:47Z)
• Optimal Kernel Choice for Score Function-based Causal Discovery [92.65034439889872]
  We propose a kernel selection method within the generalized score function that automatically selects the optimal kernel that best fits the data. We conduct experiments on both synthetic data and real-world benchmarks, and the results demonstrate that our proposed method outperforms kernel selection methods.
  arXiv  Detail & Related papers  (2024-07-14T09:32:20Z)
• Column and row subset selection using nuclear scores: algorithms and theory for Nyström approximation, CUR decomposition, and graph Laplacian reduction [0.0]
  We develop unified methodologies for fast, efficient, and theoretically guaranteed column selection. First, we derive and implement a sparsity-exploiting deterministic algorithm applicable to tasks including kernel approximation and CUR decomposition. Next, we develop a matrix-free formalism relying on a randomization scheme satisfying guaranteed concentration bounds.
  arXiv  Detail & Related papers  (2024-07-01T18:10:19Z)
• A Weighted K-Center Algorithm for Data Subset Selection [70.49696246526199]
  Subset selection is a fundamental problem that can play a key role in identifying smaller portions of the training data. We develop a novel factor 3-approximation algorithm to compute subsets based on the weighted sum of both k-center and uncertainty sampling objective functions.
  arXiv  Detail & Related papers  (2023-12-17T04:41:07Z)
• A Framework and Benchmark for Deep Batch Active Learning for Regression [2.093287944284448]
  We study active learning methods that adaptively select batches of unlabeled data for labeling. We present a framework for constructing such methods out of (network-dependent) base kernels, kernel transformations, and selection methods. Our proposed method outperforms the state-of-the-art on our benchmark, scales to large data sets, and works out-of-the-box without adjusting the network architecture or training code.
  arXiv  Detail & Related papers  (2022-03-17T16:11:36Z)
• On the Benefits of Large Learning Rates for Kernel Methods [110.03020563291788]
  We show that a phenomenon can be precisely characterized in the context of kernel methods. We consider the minimization of a quadratic objective in a separable Hilbert space, and show that with early stopping, the choice of learning rate influences the spectral decomposition of the obtained solution.
  arXiv  Detail & Related papers  (2022-02-28T13:01:04Z)
• Gradient Based Clustering [72.15857783681658]
  We propose a general approach for distance based clustering, using the gradient of the cost function that measures clustering quality. The approach is an iterative two step procedure (alternating between cluster assignment and cluster center updates) and is applicable to a wide range of functions.
  arXiv  Detail & Related papers  (2022-02-01T19:31:15Z)
• Unsupervised learning of disentangled representations in deep restricted kernel machines with orthogonality constraints [15.296955630621566]
  Constr-DRKM is a deep kernel method for the unsupervised learning of disentangled data representations. We quantitatively evaluate the proposed method's effectiveness in disentangled feature learning.
  arXiv  Detail & Related papers  (2020-11-25T11:40:10Z)
• Holistically-Attracted Wireframe Parsing [123.58263152571952]
  This paper presents a fast and parsimonious parsing method to detect a vectorized wireframe in an input image with a single forward pass. The proposed method is end-to-end trainable, consisting of three components: (i) line segment and junction proposal generation, (ii) line segment and junction matching, and (iii) line segment and junction verification.
  arXiv  Detail & Related papers  (2020-03-03T17:43:57Z)
• Diversity sampling is an implicit regularization for kernel methods [13.136143245702915]
  We show that Nystr"om kernel regression with diverse landmarks increases the accuracy of the regression in sparser regions of the dataset. A greedy is also proposed to select diverse samples of significant size within large datasets when exact DPP sampling is not practically feasible.
  arXiv  Detail & Related papers  (2020-02-20T08:24:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.