Decentralized Inference for Spatial Data Using Low-Rank Models

• URL: http://arxiv.org/abs/2502.00309v2
• Date: Mon, 10 Feb 2025 06:56:04 GMT
• Title: Decentralized Inference for Spatial Data Using Low-Rank Models
• Authors: Jianwei Shi, Sameh Abdulah, Ying Sun, Marc G. Genton,
• Abstract summary: This paper presents a decentralized framework tailored for parameter inference in spatial low-rank models. A key obstacle arises from the spatial dependence among observations, which prevents the log-likelihood from being expressed as a summation. Our approach employs a block descent method integrated with multi-consensus and dynamic consensus averaging for effective parameter optimization.
• Score: 4.168323530566095
• License:
• Abstract: Advancements in information technology have enabled the creation of massive spatial datasets, driving the need for scalable and efficient computational methodologies. While offering viable solutions, centralized frameworks are limited by vulnerabilities such as single-point failures and communication bottlenecks. This paper presents a decentralized framework tailored for parameter inference in spatial low-rank models to address these challenges. A key obstacle arises from the spatial dependence among observations, which prevents the log-likelihood from being expressed as a summation-a critical requirement for decentralized optimization approaches. To overcome this challenge, we propose a novel objective function leveraging the evidence lower bound, which facilitates the use of decentralized optimization techniques. Our approach employs a block descent method integrated with multi-consensus and dynamic consensus averaging for effective parameter optimization. We prove the convexity of the new objective function in the vicinity of the true parameters, ensuring the convergence of the proposed method. Additionally, we present the first theoretical results establishing the consistency and asymptotic normality of the estimator within the context of spatial low-rank models. Extensive simulations and real-world data experiments corroborate these theoretical findings, showcasing the robustness and scalability of the framework.

Related papers

• Efficient Text-driven Motion Generation via Latent Consistency Training [21.348658259929053]
  We propose a motion latent consistency training framework (MLCT) to solve nonlinear reverse diffusion trajectories. By combining these enhancements, we achieve stable and consistency training in non-pixel modality and latent representation spaces.
  arXiv  Detail & Related papers  (2024-05-05T02:11:57Z)
• Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints [42.47298301874283]
  We propose to perform optimization within the data manifold using diffusion models. Depending on the differentiability of the objective function, we propose two different sampling methods. Our method achieves better or comparable performance with previous state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-28T03:09:12Z)
• Distributed Markov Chain Monte Carlo Sampling based on the Alternating Direction Method of Multipliers [143.6249073384419]
  In this paper, we propose a distributed sampling scheme based on the alternating direction method of multipliers. We provide both theoretical guarantees of our algorithm's convergence and experimental evidence of its superiority to the state-of-the-art. In simulation, we deploy our algorithm on linear and logistic regression tasks and illustrate its fast convergence compared to existing gradient-based methods.
  arXiv  Detail & Related papers  (2024-01-29T02:08:40Z)
• Distributionally Robust Model-based Reinforcement Learning with Large State Spaces [55.14361269378122]
  Three major challenges in reinforcement learning are the complex dynamical systems with large state spaces, the costly data acquisition processes, and the deviation of real-world dynamics from the training environment deployment. We study distributionally robust Markov decision processes with continuous state spaces under the widely used Kullback-Leibler, chi-square, and total variation uncertainty sets. We propose a model-based approach that utilizes Gaussian Processes and the maximum variance reduction algorithm to efficiently learn multi-output nominal transition dynamics.
  arXiv  Detail & Related papers  (2023-09-05T13:42:11Z)
• Robust Networked Federated Learning for Localization [7.332862402432447]
  This paper addresses the problem of approximation, which is non-smooth in a federated setting where the data is distributed across a multitude of devices. We propose a method that adopts an $L_$-norm that a robust formulation within a distributed subgradient framework, explicitly designed to handle these obstacles.
  arXiv  Detail & Related papers  (2023-08-31T13:54:37Z)
• Efficient Model-Free Exploration in Low-Rank MDPs [76.87340323826945]
  Low-Rank Markov Decision Processes offer a simple, yet expressive framework for RL with function approximation. Existing algorithms are either (1) computationally intractable, or (2) reliant upon restrictive statistical assumptions. We propose the first provably sample-efficient algorithm for exploration in Low-Rank MDPs.
  arXiv  Detail & Related papers  (2023-07-08T15:41:48Z)
• Information Theoretical Importance Sampling Clustering [18.248246885248733]
  A current assumption of most clustering methods is that the training data and future data are taken from the same distribution. We propose an information theoretical importance sampling based approach for clustering problems (ITISC) Experiment results on synthetic datasets and a real-world load forecasting problem validate the effectiveness of the proposed model.
  arXiv  Detail & Related papers  (2023-02-09T03:18:53Z)
• Decentralized Stochastic Optimization with Inherent Privacy Protection [103.62463469366557]
  Decentralized optimization is the basic building block of modern collaborative machine learning, distributed estimation and control, and large-scale sensing. Since involved data, privacy protection has become an increasingly pressing need in the implementation of decentralized optimization algorithms.
  arXiv  Detail & Related papers  (2022-05-08T14:38:23Z)
• Decentralized Personalized Federated Learning for Min-Max Problems [79.61785798152529]
  This paper is the first to study PFL for saddle point problems encompassing a broader range of optimization problems. We propose new algorithms to address this problem and provide a theoretical analysis of the smooth (strongly) convex-(strongly) concave saddle point problems. Numerical experiments for bilinear problems and neural networks with adversarial noise demonstrate the effectiveness of the proposed methods.
  arXiv  Detail & Related papers  (2021-06-14T10:36:25Z)
• Learning while Respecting Privacy and Robustness to Distributional Uncertainties and Adversarial Data [66.78671826743884]
  The distributionally robust optimization framework is considered for training a parametric model. The objective is to endow the trained model with robustness against adversarially manipulated input data. Proposed algorithms offer robustness with little overhead.
  arXiv  Detail & Related papers  (2020-07-07T18:25:25Z)
• Gradient tracking and variance reduction for decentralized optimization and machine learning [19.54092620537586]
  Decentralized methods to solve finite-sum problems are important in many signal processing and machine learning tasks. We provide a unified algorithmic framework that combines variance-reduction with gradient tracking to achieve robust performance.
  arXiv  Detail & Related papers  (2020-02-13T07:17:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.