Learning Multi-Attribute Differential Graphs with Non-Convex Penalties

• URL: http://arxiv.org/abs/2505.09748v1
• Date: Wed, 14 May 2025 19:19:09 GMT
• Title: Learning Multi-Attribute Differential Graphs with Non-Convex Penalties
• Authors: Jitendra K Tugnait,
• Abstract summary: Two multi-attribute Gaussian graphical models (GGMs) are known to have similar D-trace loss function with non-dimensional penalties.<n>We consider the problem of estimating differences in two multi-attribute graphical models (GGMs) which are known to have similar D-trace loss function with non-dimensional penalties.
• Score: 12.94486861344922
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider the problem of estimating differences in two multi-attribute Gaussian graphical models (GGMs) which are known to have similar structure, using a penalized D-trace loss function with non-convex penalties. The GGM structure is encoded in its precision (inverse covariance) matrix. Existing methods for multi-attribute differential graph estimation are based on a group lasso penalized loss function. In this paper, we consider a penalized D-trace loss function with non-convex (log-sum and smoothly clipped absolute deviation (SCAD)) penalties. Two proximal gradient descent methods are presented to optimize the objective function. Theoretical analysis establishing sufficient conditions for consistency in support recovery, convexity and estimation in high-dimensional settings is provided. We illustrate our approaches with numerical examples based on synthetic and real data.

Related papers

• Multi-Attribute Graph Estimation with Sparse-Group Non-Convex Penalties [12.94486861344922]
  We consider the problem of inferring the conditional independence graph (CIG) vectors from multi-attribute data.<n>Most existing methods for graph estimation are based on a single scalar random variable with each node.
  arXiv  Detail & Related papers  (2025-05-17T12:35:28Z)
• A Mirror Descent-Based Algorithm for Corruption-Tolerant Distributed Gradient Descent [57.64826450787237]
  We show how to analyze the behavior of distributed gradient descent algorithms in the presence of adversarial corruptions.<n>We show how to use ideas from (lazy) mirror descent to design a corruption-tolerant distributed optimization algorithm.<n> Experiments based on linear regression, support vector classification, and softmax classification on the MNIST dataset corroborate our theoretical findings.
  arXiv  Detail & Related papers  (2024-07-19T08:29:12Z)
• Analysis of Corrected Graph Convolutions [10.991475575578855]
  State-of-the-art machine learning models often use multiple graph convolutions on the data.<n>We show that too many graph convolutions can degrade performance significantly, a phenomenon known as oversmoothing.<n>For exact classification, we show that the separability threshold can be improved exponentially up to $O(logn/loglogn)$ corrected convolutions.
  arXiv  Detail & Related papers  (2024-05-22T20:50:17Z)
• Learning High-Dimensional Differential Graphs From Multi-Attribute Data [12.94486861344922]
  We consider the problem of estimating differences in two Gaussian graphical models (GGMs) which are known to have similar structure. Existing methods for differential graph estimation are based on single-attribute (SA) models. In this paper, we analyze a group lasso penalized D-trace loss function approach for differential graph learning from multi-attribute data.
  arXiv  Detail & Related papers  (2023-12-05T18:54:46Z)
• Learning Graphical Factor Models with Riemannian Optimization [70.13748170371889]
  This paper proposes a flexible algorithmic framework for graph learning under low-rank structural constraints. The problem is expressed as penalized maximum likelihood estimation of an elliptical distribution. We leverage geometries of positive definite matrices and positive semi-definite matrices of fixed rank that are well suited to elliptical models.
  arXiv  Detail & Related papers  (2022-10-21T13:19:45Z)
• Sparse Graph Learning Under Laplacian-Related Constraints [23.503820266772504]
  We focus on graph Laplacian-related constraints on the sparse precision matrix that encodes conditional dependence between random variables. We investigate modifications to widely used penalized log-likelihood approaches to enforce total positivity but not the Laplacian structure. An alternating direction method of multipliers (ADMM) algorithm is presented and analyzed for constrained optimization.
  arXiv  Detail & Related papers  (2021-11-16T00:50:36Z)
• Learning Sparse Graph with Minimax Concave Penalty under Gaussian Markov Random Fields [51.07460861448716]
  This paper presents a convex-analytic framework to learn from data. We show that a triangular convexity decomposition is guaranteed by a transform of the corresponding to its upper part.
  arXiv  Detail & Related papers  (2021-09-17T17:46:12Z)
• Differentiable Annealed Importance Sampling and the Perils of Gradient Noise [68.44523807580438]
  Annealed importance sampling (AIS) and related algorithms are highly effective tools for marginal likelihood estimation. Differentiability is a desirable property as it would admit the possibility of optimizing marginal likelihood as an objective. We propose a differentiable algorithm by abandoning Metropolis-Hastings steps, which further unlocks mini-batch computation.
  arXiv  Detail & Related papers  (2021-07-21T17:10:14Z)
• Shaping Deep Feature Space towards Gaussian Mixture for Visual Classification [74.48695037007306]
  We propose a Gaussian mixture (GM) loss function for deep neural networks for visual classification. With a classification margin and a likelihood regularization, the GM loss facilitates both high classification performance and accurate modeling of the feature distribution. The proposed model can be implemented easily and efficiently without using extra trainable parameters.
  arXiv  Detail & Related papers  (2020-11-18T03:32:27Z)
• Learning Graph Laplacian with MCP [8.409980020848167]
  We consider the problem of learning a graph under the semilacian with a non-smooth penalty constraint. We apply an efficient Newton method to subproblems of the DCA.
  arXiv  Detail & Related papers  (2020-10-22T09:33:49Z)
• Piecewise Linear Regression via a Difference of Convex Functions [50.89452535187813]
  We present a new piecewise linear regression methodology that utilizes fitting a difference of convex functions (DC functions) to the data. We empirically validate the method, showing it to be practically implementable, and to have comparable performance to existing regression/classification methods on real-world datasets.
  arXiv  Detail & Related papers  (2020-07-05T18:58:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.