Rethinking Parameter Sharing as Graph Coloring for Structured Compression

• URL: http://arxiv.org/abs/2511.06786v1
• Date: Mon, 10 Nov 2025 07:21:35 GMT
• Title: Rethinking Parameter Sharing as Graph Coloring for Structured Compression
• Abstract summary: Geo-Sharing is a form of structured compression that reduces redundancy in deep models.<n>We recast parameter sharing from a group-theoretic perspective as structural symmetries in the model's parameter space.<n>By projecting perturbations onto the Hessian's low-curvature eigensubspace, the criterion provides an analytic rule for selecting sharing groups.
• Score: 4.890400460873276
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Modern deep models have massive parameter sizes, leading to high inference-time memory usage that limits practical deployment. Parameter sharing, a form of structured compression, effectively reduces redundancy, but existing approaches remain heuristic-restricted to adjacent layers and lacking a systematic analysis for cross-layer sharing. However, extending sharing across multiple layers leads to an exponentially expanding configuration space, making exhaustive search computationally infeasible and forming a critical bottleneck for parameter sharing. We recast parameter sharing from a group-theoretic perspective as introducing structural symmetries in the model's parameter space. A sharing configuration can be described by a coloring function $\alpha:L\rightarrow C$ (L: layer indices and C: sharing classes), which determines inter-layer sharing groups while preserving structural symmetry. To determine the coloring function, we propose a second-order geometric criterion based on Taylor expansion and the Hessian spectrum. By projecting perturbations onto the Hessian's low-curvature eigensubspace, the criterion provides an analytic rule for selecting sharing groups that minimize performance impact, yielding a principled and scalable configuration procedure. Across diverse architectures and tasks, Geo-Sharing consistently outperforms state-of-the-art heuristic sharing strategies, achieving higher compression ratios with smaller accuracy degradation.

Related papers

• Diffusion learning reveals viable parameter manifolds and compensation geometry in biological dynamical systems [0.8373151777137791]
  We formalize compatible parameter sets as emphviable parameter manifold: the inverse images of a system's target behaviors under a parameter-to-feature map.<n>We train conditional score-based diffusion models on simulated parameter--feature pairs and use them as amortized samplers of prior-weighted viable sets.<n>In this view, robustness, compensation, and hidden parameter dependencies are organized as inverse geometry, with diffusion models providing practical tools for sampling, visualizing, and interrogating that geometry.
  arXiv  Detail & Related papers  (2026-07-04T02:51:11Z)
• Learning Sparse Compositional Functions with Norm-Constrained Neural Networks [8.967327976469653]
  We show that deep networks can exploit the compositional structure of the target functions, effectively avoiding the curse of dimensionality (CoD)<n>Our results have broad applicability since every function that is efficiently Turing computable admits sparse compositional representations.
  arXiv  Detail & Related papers  (2026-05-25T09:02:13Z)
• A renormalization-group inspired lattice-based framework for piecewise generalized linear models [0.6599344783327052]
  We introduce a class of models inspired by renormalization group (RG) theory.<n>They are built on additive hierarchical expansions analogous to those appearing in functional ANOVA and mixed-effects models.<n>We use techniques from statistical physics -- specifically replica analysis -- to study their generalization properties.
  arXiv  Detail & Related papers  (2026-05-06T22:27:50Z)
• LG-HCC: Local Geometry-Aware Hierarchical Context Compression for 3D Gaussian Splatting [77.81227097905865]
  Anchor-based 3DGS compression schemes reduce gaussian redundancy through some advanced context models.<n>We propose a Local Geometry-aware Hierarchical Context Compression framework for 3DGS that incorporates inter-anchor geometric correlations into anchor pruning and entropy coding for compact representation.<n>Experiments show that LG-HCC effectively alleviates structural preservation issues,achieving superior geometric integrity and rendering fidelity while reducing storage by up to 30.85x compared to the Scaffold-GS baseline on the Mip-NeRF360 dataset.
  arXiv  Detail & Related papers  (2026-03-30T13:39:35Z)
• Unsupervised Behavioral Compression: Learning Low-Dimensional Policy Manifolds through State-Occupancy Matching [42.26405543045843]
  Action-based Policy Compression (APC) is widely recognized as sample-inefficient.<n>We introduce Occupancy-based Policy Compression (OPC), which enhances APC by shifting behavior representation from immediate action-matching to long-horizon state-space coverage.<n>These modifications force the generative model to organize the latent space around true functional similarity, promoting a latent representation that generalizes over a broad spectrum of behaviors.
  arXiv  Detail & Related papers  (2026-03-27T23:16:27Z)
• CSMCIR: CoT-Enhanced Symmetric Alignment with Memory Bank for Composed Image Retrieval [54.15776146365823]
  Composed Image Retrieval (CIR) enables users to search for target images using both a reference image and manipulation text.<n>We propose CSMCIR, a unified representation framework that achieves efficient query-target alignment through three synergistic components.
  arXiv  Detail & Related papers  (2026-01-07T09:21:38Z)
• GS-KAN: Parameter-Efficient Kolmogorov-Arnold Networks via Sprecher-Type Shared Basis Functions [0.0]
  We propose GS-KAN (Generalized Sprecher-KAN), a lightweight architecture inspired by David Sprecher's refinement of the superposition theorem.<n> GS-KAN constructs unique edge functions by applying learnable linear transformations to a single learnable, shared parent function per layer.<n>Our results demonstrate that GS-KAN outperforms both approximations and standard KAN baselines on continuous function tasks while maintaining superior parameter efficiency.
  arXiv  Detail & Related papers  (2025-12-09T19:56:36Z)
• Lighter-X: An Efficient and Plug-and-play Strategy for Graph-based Recommendation through Decoupled Propagation [49.865020394064096]
  We propose textbfLighter-X, an efficient and modular framework that can be seamlessly integrated with existing GNN-based recommender architectures.<n>Our approach substantially reduces both parameter size and computational complexity while preserving the theoretical guarantees and empirical performance of the base models.<n>Experiments demonstrate that Lighter-X achieves comparable performance to baseline models with significantly fewer parameters.
  arXiv  Detail & Related papers  (2025-10-11T08:33:08Z)
• ISCS: Parameter-Guided Channel Ordering and Grouping for Learned Image Compression [11.645256028223272]
  We propose a dataset-agnostic method to identify and organize important channels in pretrained LIC models.<n>Our method effectively reduces computation while maintaining reconstruction quality, providing a practical and modular enhancement to existing learned compression frameworks.
  arXiv  Detail & Related papers  (2025-09-21T00:44:15Z)
• Enhancing point cloud analysis via neighbor aggregation correction based on cross-stage structure correlation [22.48120946682699]
  Point cloud analysis is a cornerstone of many downstream tasks, among which aggregating local structures is the basis for understanding point cloud data.<n>We propose the Point Distribution Set Abstraction module (PDSA) that utilizes the correlation in the high-dimensional space to correct the feature distribution during aggregation.<n>PDSA distinguishes the point correlation based on a lightweight cross-stage structural descriptor, and enhances structural homogeneity.
  arXiv  Detail & Related papers  (2025-06-18T06:08:17Z)
• A General Error-Theoretical Analysis Framework for Constructing Compression Strategies [3.1316260533944007]
  We propose a Compression Error Theory (CET) framework to determine the optimal compression level for each layer.<n>Specifically, on the ResNet-34 model, CET achieves nearly 11$times$ parameter compression while even surpassing performance comparable to the original model.
  arXiv  Detail & Related papers  (2025-02-19T06:12:43Z)
• Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks [17.487761710665968]
  We present a framework to define a large class of neural networks for which, by construction, training by gradient flow provably reaches arbitrarily low loss when the number of parameters grows.
  arXiv  Detail & Related papers  (2025-01-10T12:52:00Z)
• RGM: A Robust Generalizable Matching Model [49.60975442871967]
  We propose a deep model for sparse and dense matching, termed RGM (Robust Generalist Matching) To narrow the gap between synthetic training samples and real-world scenarios, we build a new, large-scale dataset with sparse correspondence ground truth. We are able to mix up various dense and sparse matching datasets, significantly improving the training diversity.
  arXiv  Detail & Related papers  (2023-10-18T07:30:08Z)
• Beyond Learned Metadata-based Raw Image Reconstruction [86.1667769209103]
  Raw images have distinct advantages over sRGB images, e.g., linearity and fine-grained quantization levels. They are not widely adopted by general users due to their substantial storage requirements. We propose a novel framework that learns a compact representation in the latent space, serving as metadata.
  arXiv  Detail & Related papers  (2023-06-21T06:59:07Z)
• DepGraph: Towards Any Structural Pruning [68.40343338847664]
  We study general structural pruning of arbitrary architecture like CNNs, RNNs, GNNs and Transformers. We propose a general and fully automatic method, emphDependency Graph (DepGraph), to explicitly model the dependency between layers and comprehensively group parameters for pruning. In this work, we extensively evaluate our method on several architectures and tasks, including ResNe(X)t, DenseNet, MobileNet and Vision transformer for images, GAT for graph, DGCNN for 3D point cloud, alongside LSTM for language, and demonstrate that, even with a
  arXiv  Detail & Related papers  (2023-01-30T14:02:33Z)
• DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution [136.7261709896713]
  We propose a data-driven approach that generates the appropriate convolution kernels to apply in response to the nature of the instances. The proposed method achieves promising results on both ScanetNetV2 and S3DIS. It also improves inference speed by more than 25% over the current state-of-the-art.
  arXiv  Detail & Related papers  (2020-11-26T14:56:57Z)
• Generalising Recursive Neural Models by Tensor Decomposition [12.069862650316262]
  We introduce a general approach to model aggregation of structural context leveraging a tensor-based formulation. We show how the exponential growth in the size of the parameter space can be controlled through an approximation based on the Tucker decomposition. By this means, we can effectively regulate the trade-off between expressivity of the encoding, controlled by the hidden size, computational complexity and model generalisation.
  arXiv  Detail & Related papers  (2020-06-17T17:28:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.