Triple Component Matrix Factorization: Untangling Global, Local, and Noisy Components

• URL: http://arxiv.org/abs/2404.07955v2
• Date: Fri, 08 Nov 2024 17:04:45 GMT
• Title: Triple Component Matrix Factorization: Untangling Global, Local, and Noisy Components
• Authors: Naichen Shi, Salar Fattahi, Raed Al Kontar,
• Abstract summary: We solve the problem of common and unique feature extraction from noisy data. Despite the intricate nature of the problem, we provide a Taylor series characterization by solving the corresponding KarushKuhn-Tucker algorithm. Numerical experiments in video segmentation and anomaly detection highlight the superior feature extraction abilities of TCMF.
• Score: 13.989390077752232
• License:
• Abstract: In this work, we study the problem of common and unique feature extraction from noisy data. When we have N observation matrices from N different and associated sources corrupted by sparse and potentially gross noise, can we recover the common and unique components from these noisy observations? This is a challenging task as the number of parameters to estimate is approximately thrice the number of observations. Despite the difficulty, we propose an intuitive alternating minimization algorithm called triple component matrix factorization (TCMF) to recover the three components exactly. TCMF is distinguished from existing works in literature thanks to two salient features. First, TCMF is a principled method to separate the three components given noisy observations provably. Second, the bulk of the computation in TCMF can be distributed. On the technical side, we formulate the problem as a constrained nonconvex nonsmooth optimization problem. Despite the intricate nature of the problem, we provide a Taylor series characterization of its solution by solving the corresponding Karush-Kuhn-Tucker conditions. Using this characterization, we can show that the alternating minimization algorithm makes significant progress at each iteration and converges into the ground truth at a linear rate. Numerical experiments in video segmentation and anomaly detection highlight the superior feature extraction abilities of TCMF.

Related papers

• CWF: Consolidating Weak Features in High-quality Mesh Simplification [50.634070540791555]
  We propose a smooth functional that simultaneously considers all of these requirements. The functional comprises a normal anisotropy term and a Centroidal Voronoi Tessellation (CVT) energy term.
  arXiv  Detail & Related papers  (2024-04-24T05:37:17Z)
• Stochastic Nested Compositional Bi-level Optimization for Robust Feature Learning [11.236838268731804]
  We develop and analyze algorithms for solving nested bi-level optimization problems. Our proposed algorithm does not rely on matrix complexity or mini-batches.
  arXiv  Detail & Related papers  (2023-07-11T15:52:04Z)
• D4FT: A Deep Learning Approach to Kohn-Sham Density Functional Theory [79.50644650795012]
  We propose a deep learning approach to solve Kohn-Sham Density Functional Theory (KS-DFT) We prove that such an approach has the same expressivity as the SCF method, yet reduces the computational complexity. In addition, we show that our approach enables us to explore more complex neural-based wave functions.
  arXiv  Detail & Related papers  (2023-03-01T10:38:10Z)
• Robust Manifold Nonnegative Tucker Factorization for Tensor Data Representation [44.845291873747335]
  Nonnegative Tucker Factorization (NTF) minimizes the euclidean distance or Kullback-Leibler divergence between the original data and its low-rank approximation. NTF suffers from rotational ambiguity, whose solutions with and without rotation transformations are equally in the sense of yielding the maximum likelihood. We propose three Robust Manifold NTF algorithms to handle outliers by incorporating structural knowledge about the outliers.
  arXiv  Detail & Related papers  (2022-11-08T01:16:21Z)
• Orthogonal Matrix Retrieval with Spatial Consensus for 3D Unknown-View Tomography [58.60249163402822]
  Unknown-view tomography (UVT) reconstructs a 3D density map from its 2D projections at unknown, random orientations. The proposed OMR is more robust and performs significantly better than the previous state-of-the-art OMR approach.
  arXiv  Detail & Related papers  (2022-07-06T21:40:59Z)
• Large-Scale Learning with Fourier Features and Tensor Decompositions [3.6930948691311007]
  We exploit the tensor product structure of deterministic Fourier features, which enables us to represent the model parameters as a low-rank tensor decomposition. We demonstrate by means of numerical experiments how our low-rank tensor approach obtains the same performance of the corresponding nonparametric model.
  arXiv  Detail & Related papers  (2021-09-03T14:12:53Z)
• Regularization by Denoising Sub-sampled Newton Method for Spectral CT Multi-Material Decomposition [78.37855832568569]
  We propose to solve a model-based maximum-a-posterior problem to reconstruct multi-materials images with application to spectral CT. In particular, we propose to solve a regularized optimization problem based on a plug-in image-denoising function. We show numerical and experimental results for spectral CT materials decomposition.
  arXiv  Detail & Related papers  (2021-03-25T15:20:10Z)
• Feature Weighted Non-negative Matrix Factorization [92.45013716097753]
  We propose the Feature weighted Non-negative Matrix Factorization (FNMF) in this paper. FNMF learns the weights of features adaptively according to their importances. It can be solved efficiently with the suggested optimization algorithm.
  arXiv  Detail & Related papers  (2021-03-24T21:17:17Z)
• Probabilistic Simplex Component Analysis [66.30587591100566]
  PRISM is a probabilistic simplex component analysis approach to identifying the vertices of a data-circumscribing simplex from data. The problem has a rich variety of applications, the most notable being hyperspectral unmixing in remote sensing and non-negative matrix factorization in machine learning.
  arXiv  Detail & Related papers  (2021-03-18T05:39:00Z)
• Recovering Joint Probability of Discrete Random Variables from Pairwise Marginals [22.77704627076251]
  Learning the joint probability of random variables (RVs) is the cornerstone of statistical signal processing and machine learning. Recent work has proposed to recover the joint probability mass function (PMF) of an arbitrary number of RVs from three-dimensional marginals. accurately estimating three-dimensional marginals can still be costly in terms of sample complexity. This work puts forth a new framework for learning the joint PMF using only pairwise marginals.
  arXiv  Detail & Related papers  (2020-06-30T15:43:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.