SinBasis Networks: Matrix-Equivalent Feature Extraction for Wave-Like Optical Spectrograms

• URL: http://arxiv.org/abs/2505.06275v2
• Date: Thu, 31 Jul 2025 14:24:03 GMT
• Title: SinBasis Networks: Matrix-Equivalent Feature Extraction for Wave-Like Optical Spectrograms
• Authors: Yuzhou Zhu, Zheng Zhang, Ruyi Zhang, Liang Zhou,
• Abstract summary: We propose a unified, matrix-equivalent framework that reinterprets convolution and attention as linear transforms on flattened inputs.<n> Embedding these transforms into CNN, ViT and Capsule architectures yields Sin-Basis Networks with heightened sensitivity to periodic motifs.
• Score: 8.37266944852829
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Wave-like images-from attosecond streaking spectrograms to optical spectra, audio mel-spectrograms and periodic video frames-encode critical harmonic structures that elude conventional feature extractors. We propose a unified, matrix-equivalent framework that reinterprets convolution and attention as linear transforms on flattened inputs, revealing filter weights as basis vectors spanning latent feature subspaces. To infuse spectral priors we apply elementwise $\sin(\cdot)$ mappings to each weight matrix. Embedding these transforms into CNN, ViT and Capsule architectures yields Sin-Basis Networks with heightened sensitivity to periodic motifs and built-in invariance to spatial shifts. Experiments on a diverse collection of wave-like image datasets-including 80,000 synthetic attosecond streaking spectrograms, thousands of Raman, photoluminescence and FTIR spectra, mel-spectrograms from AudioSet and cycle-pattern frames from Kinetics-demonstrate substantial gains in reconstruction accuracy, translational robustness and zero-shot cross-domain transfer. Theoretical analysis via matrix isomorphism and Mercer-kernel truncation quantifies how sinusoidal reparametrization enriches expressivity while preserving stability in data-scarce regimes. Sin-Basis Networks thus offer a lightweight, physics-informed approach to deep learning across all wave-form imaging modalities.

Related papers

• CARL: Camera-Agnostic Representation Learning for Spectral Image Analysis [75.25966323298003]
  Spectral imaging offers promising applications across diverse domains, including medicine and urban scene understanding.<n> variability in channel dimensionality and captured wavelengths among spectral cameras impede the development of AI-driven methodologies.<n>We introduce $textbfCARL$, a model for $textbfC$amera-$textbfA$gnostic $textbfR$esupervised $textbfL$ across RGB, multispectral, and hyperspectral imaging modalities.
  arXiv  Detail & Related papers  (2025-04-27T13:06:40Z)
• Spectral Dictionary Learning for Generative Image Modeling [0.0]
  We propose a novel spectral generative model for image synthesis.<n>Images are reconstructed as linear combinations of a set of learned spectral basis functions.<n>We show that our approach achieves competitive performance in terms of reconstruction quality and perceptual fidelity.
  arXiv  Detail & Related papers  (2025-04-21T01:11:17Z)
• Spectral and Rhythm Features for Audio Classification with Deep Convolutional Neural Networks [0.0]
  Convolutional neural networks (CNNs) are widely used in computer vision. They can be used to represent spectral and rhythm features extracted from digital imagery for the acoustic classification of sounds. Different spectral and rhythm feature representations like mel-scaled spectrograms, mel-frequency cepstral coefficients (MFCCs) are investigated.
  arXiv  Detail & Related papers  (2024-10-09T14:21:59Z)
• Neural Spectral Decomposition for Dataset Distillation [48.59372086450124]
  We propose Neural Spectrum Decomposition, a generic decomposition framework for dataset distillation. We aim to discover the low-rank representation of the entire dataset and perform distillation efficiently. Our results demonstrate that our approach achieves state-of-the-art performance on benchmarks, including CIFAR10, CIFAR100, Tiny Imagenet, and ImageNet Subset.
  arXiv  Detail & Related papers  (2024-08-29T03:26:14Z)
• FCDM: A Physics-Guided Bidirectional Frequency Aware Convolution and Diffusion-Based Model for Sinogram Inpainting [14.043383277622874]
  Full-view sinograms require high radiation dose and long scan times.<n>Sparse-view CT alleviates this burden but yields incomplete sinograms with structured signal loss.<n>We proposemodelname, a diffusion-based framework tailored for sinograms.
  arXiv  Detail & Related papers  (2024-08-26T12:31:38Z)
• A Differential Smoothness-based Compact-Dynamic Graph Convolutional Network for Spatiotemporal Signal Recovery [9.369246678101048]
  This paper proposes a Compact-fold Con Graphal Network (CDCN) fortemporal signal recovery. Experiments on real-world datasets show that CDCN significantly outperforms the state-of-the-art models fortemporal signal recovery.
  arXiv  Detail & Related papers  (2024-08-06T06:42:53Z)
• Spectrum Translation for Refinement of Image Generation (STIG) Based on Contrastive Learning and Spectral Filter Profile [15.5188527312094]
  We propose a framework to mitigate the disparity in frequency domain of the generated images. This is realized by spectrum translation for the refinement of image generation (STIG) based on contrastive learning. We evaluate our framework across eight fake image datasets and various cutting-edge models to demonstrate the effectiveness of STIG.
  arXiv  Detail & Related papers  (2024-03-08T06:39:24Z)
• SpectralNeRF: Physically Based Spectral Rendering with Neural Radiance Field [70.15900280156262]
  We propose an end-to-end Neural Radiance Field (NeRF)-based architecture for high-quality physically based rendering from a novel spectral perspective. SpectralNeRF is superior to recent NeRF-based methods when synthesizing new views on synthetic and real datasets.
  arXiv  Detail & Related papers  (2023-12-14T07:19:31Z)
• HoloNets: Spectral Convolutions do extend to Directed Graphs [59.851175771106625]
  Conventional wisdom dictates that spectral convolutional networks may only be deployed on undirected graphs. Here we show this traditional reliance on the graph Fourier transform to be superfluous. We provide a frequency-response interpretation of newly developed filters, investigate the influence of the basis used to express filters and discuss the interplay with characteristic operators on which networks are based.
  arXiv  Detail & Related papers  (2023-10-03T17:42:09Z)
• Speed Limits for Deep Learning [67.69149326107103]
  Recent advancement in thermodynamics allows bounding the speed at which one can go from the initial weight distribution to the final distribution of the fully trained network. We provide analytical expressions for these speed limits for linear and linearizable neural networks. Remarkably, given some plausible scaling assumptions on the NTK spectra and spectral decomposition of the labels -- learning is optimal in a scaling sense.
  arXiv  Detail & Related papers  (2023-07-27T06:59:46Z)
• Universal Scaling Laws of Absorbing Phase Transitions in Artificial Deep Neural Networks [0.8932296777085644]
  Conventional artificial deep neural networks operating near the phase boundary of the signal propagation dynamics, also known as the edge of chaos, exhibit universal scaling laws of absorbing phase transitions.<n>We exploit the fully deterministic nature of the propagation dynamics to elucidate an analogy between a signal collapse in the neural networks and an absorbing state.
  arXiv  Detail & Related papers  (2023-07-05T13:39:02Z)
• Fast and Robust State Estimation and Tracking via Hierarchical Learning [9.341558827016332]
  We aim to speed up the convergence and enhance the resilience of state estimation and tracking for large-scale networks. We numerically validate our algorithms through simulation studies of underwater acoustic networks and large-scale synthetic networks.
  arXiv  Detail & Related papers  (2023-06-29T19:07:17Z)
• Neuromorphic Optical Flow and Real-time Implementation with Event Cameras [47.11134388304464]
  We build on the latest developments in event-based vision and spiking neural networks. We propose a new network architecture that improves the state-of-the-art self-supervised optical flow accuracy. We demonstrate high speed optical flow prediction with almost two orders of magnitude reduced complexity.
  arXiv  Detail & Related papers  (2023-04-14T14:03:35Z)
• Correlating sparse sensing for large-scale traffic speed estimation: A Laplacian-enhanced low-rank tensor kriging approach [76.45949280328838]
  We propose a Laplacian enhanced low-rank tensor (LETC) framework featuring both lowrankness and multi-temporal correlations for large-scale traffic speed kriging. We then design an efficient solution algorithm via several effective numeric techniques to scale up the proposed model to network-wide kriging.
  arXiv  Detail & Related papers  (2022-10-21T07:25:57Z)
• Unsupervised inter-frame motion correction for whole-body dynamic PET using convolutional long short-term memory in a convolutional neural network [9.349668170221975]
  We develop an unsupervised deep learning-based framework to correct inter-frame body motion. The motion estimation network is a convolutional neural network with a combined convolutional long short-term memory layer. Once trained, the motion estimation inference time of our proposed network was around 460 times faster than the conventional registration baseline.
  arXiv  Detail & Related papers  (2022-06-13T17:38:16Z)
• Discrete-time Temporal Network Embedding via Implicit Hierarchical Learning in Hyperbolic Space [43.280123606888395]
  We propose a hyperbolic temporal graph network (HTGN) that takes advantage of the exponential capacity and hierarchical awareness of hyperbolic geometry. HTGN maps the temporal graph into hyperbolic space, and incorporates hyperbolic graph neural network and hyperbolic gated recurrent neural network. Experimental results on multiple real-world datasets demonstrate the superiority of HTGN for temporal graph embedding.
  arXiv  Detail & Related papers  (2021-07-08T11:24:59Z)
• SpectralFormer: Rethinking Hyperspectral Image Classification with Transformers [91.09957836250209]
  Hyperspectral (HS) images are characterized by approximately contiguous spectral information. CNNs have been proven to be a powerful feature extractor in HS image classification. We propose a novel backbone network called ulSpectralFormer for HS image classification.
  arXiv  Detail & Related papers  (2021-07-07T02:59:21Z)
• Spectrally-Encoded Single-Pixel Machine Vision Using Diffractive Networks [6.610893384480686]
  3D engineering of matter has opened up new avenues for designing systems that can perform various computational tasks through light-matter interaction. Here, we demonstrate the design of optical networks in the form of multiple diffractive layers that are trained using deep learning to transform and encode the spatial information of objects into the power spectrum of the diffracted light. We experimentally validated this machine vision framework at terahertz spectrum to optically classify the images of handwritten digits by detecting the spectral power of the diffracted light at ten distinct wavelengths.
  arXiv  Detail & Related papers  (2020-05-15T09:18:21Z)
• Residual-Sparse Fuzzy $C$-Means Clustering Incorporating Morphological Reconstruction and Wavelet frames [146.63177174491082]
  Fuzzy $C$-Means (FCM) algorithm incorporates a morphological reconstruction operation and a tight wavelet frame transform. We present an improved FCM algorithm by imposing an $ell_0$ regularization term on the residual between the feature set and its ideal value. Experimental results reported for synthetic, medical, and color images show that the proposed algorithm is effective and efficient, and outperforms other algorithms.
  arXiv  Detail & Related papers  (2020-02-14T10:00:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.