OSDTW: Optimal Shared Depth and Task Weighting for Long-Tailed Recognition

• URL: http://arxiv.org/abs/2605.24969v1
• Date: Sun, 24 May 2026 09:48:02 GMT
• Title: OSDTW: Optimal Shared Depth and Task Weighting for Long-Tailed Recognition
• Abstract summary: Long-tailed recognition suffers from a persistent head--tail trade-off.<n>We propose a principled task-decomposition framework that partitions the original single-label recognition problem into a head task and a tail task.<n>We show that the resulting Kullback-Leibler divergence-based generalization error can be written as the sum of task-wise terms up to an additive constant.
• Score: 12.869661291857843
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-tailed recognition suffers from a persistent head--tail trade-off: improving tail performance often degrades head accuracy and can increase training instability. Despite strong empirical results from re-weighting, decoupled training, and multi-expert methods, key design choices about representation sharing between head and tail classes and supervision weighting across class groups remain largely heuristic. In this work, we propose OSDTW, a principled task-decomposition framework that partitions the original single-label recognition problem into a head task and a tail task, implemented with a shared encoder and task-specific decoders. To handle the mutual exclusivity and statistical dependence between the two label groups, we introduce a factorized model and show that the resulting Kullback--Leibler divergence-based generalization error can be written as the sum of task-wise terms up to an additive constant, yielding a well-defined task-wise objective. We further develop a three-stage training pipeline: independent task training to estimate task-wise optima and the Fisher information matrix, weighted joint training to learn a shared encoder, and branch assembly to construct the final decoupled model. Under a block-diagonal Fisher approximation, we derive a computable second-order expansion of the expected generalization error, decomposing it into encoder variance, encoder bias, and decoder variance. This bias--variance decomposition provides a computable proxy to select the shared depth and task weights, enabling efficient hyper-parameter search. Experiments on standard long-tailed benchmarks demonstrate the effectiveness of the proposed approach over strong baselines.

Related papers

• Multi-stage neural operator learning with application for convolutions [4.1727004663626905]
  This paper introduces two general multi-stage neural operator learning frameworks.<n>The first, Deep Collocation Neural Operator (DCNO), is a supervised approach that iteratively refines the operator approximation by learning residuals from input-output data pairs.<n>The second, Deep Galerkin Neural Operator (DGNO), is an unsupervised framework applicable when the target operator can be represented by a PDE, leveraging the weak form of the PDE residual for training.
  arXiv  Detail & Related papers  (2026-08-19T12:23:20Z)
• PACT: Preserving Anchored Cores in Task-vectors for Model Merging [68.52455853496585]
  Model merging aims to combine multiple task-specific fine-tuned models into a single multi-task model.<n>Most existing model merging approaches follow the Task Arithmetic paradigm.<n>We propose PACT, which preserves the anchored task-specific cores (i.e., LBW dimensions) within task vectors by aligning their complements with the subspace of the pre-trained weights.
  arXiv  Detail & Related papers  (2026-06-17T02:48:35Z)
• Variational Task Vector Composition [53.476598858325985]
  We propose variational task vector composition, where composition coefficients are taken as latent variables and estimated in a Bayesian inference framework.<n>Motivated by the observation of structural redundancy in task vectors, we introduce a Spike-and-Slab prior that promotes sparsity.<n>We develop a gated sampling mechanism that constructs a controllable posterior by filtering the composition coefficients based on both uncertainty and importance.
  arXiv  Detail & Related papers  (2025-09-21T02:46:02Z)
• AdaRank: Adaptive Rank Pruning for Enhanced Model Merging [23.649762835129167]
  Model merging has emerged as a promising approach for unifying independently fine-tuned models into an integrated framework.<n>We propose AdaRank, a novel model merging framework that adaptively selects the most beneficial singular directions of task vectors to merge multiple models.<n>AdaRank consistently achieves state-of-the-art performance with various backbones and number of tasks, reducing the performance gap between fine-tuned models to nearly 1%.
  arXiv  Detail & Related papers  (2025-03-28T06:49:06Z)
• Open Domain Generalization with a Single Network by Regularization Exploiting Pre-trained Features [37.518025833882334]
  Open Domain Generalization (ODG) is a challenging task as it deals with distribution shifts and category shifts. Previous work has used multiple source-specific networks, which involve a high cost. This paper proposes a method that can handle ODG using only a single network.
  arXiv  Detail & Related papers  (2023-12-08T16:22:10Z)
• Object Segmentation by Mining Cross-Modal Semantics [68.88086621181628]
  We propose a novel approach by mining the Cross-Modal Semantics to guide the fusion and decoding of multimodal features. Specifically, we propose a novel network, termed XMSNet, consisting of (1) all-round attentive fusion (AF), (2) coarse-to-fine decoder (CFD), and (3) cross-layer self-supervision.
  arXiv  Detail & Related papers  (2023-05-17T14:30:11Z)
• Robust Outlier Rejection for 3D Registration with Variational Bayes [70.98659381852787]
  We develop a novel variational non-local network-based outlier rejection framework for robust alignment. We propose a voting-based inlier searching strategy to cluster the high-quality hypothetical inliers for transformation estimation.
  arXiv  Detail & Related papers  (2023-04-04T03:48:56Z)
• Exposing and Addressing Cross-Task Inconsistency in Unified Vision-Language Models [80.23791222509644]
  Inconsistent AI models are considered brittle and untrustworthy by human users. We find that state-of-the-art vision-language models suffer from a surprisingly high degree of inconsistent behavior across tasks. We propose a rank correlation-based auxiliary training objective, computed over large automatically created cross-task contrast sets.
  arXiv  Detail & Related papers  (2023-03-28T16:57:12Z)
• Relieving Long-tailed Instance Segmentation via Pairwise Class Balance [85.53585498649252]
  Long-tailed instance segmentation is a challenging task due to the extreme imbalance of training samples among classes. It causes severe biases of the head classes (with majority samples) against the tailed ones. We propose a novel Pairwise Class Balance (PCB) method, built upon a confusion matrix which is updated during training to accumulate the ongoing prediction preferences.
  arXiv  Detail & Related papers  (2022-01-08T07:48:36Z)
• Robust Learning Through Cross-Task Consistency [92.42534246652062]
  We propose a broadly applicable and fully computational method for augmenting learning with Cross-Task Consistency. We observe that learning with cross-task consistency leads to more accurate predictions and better generalization to out-of-distribution inputs.
  arXiv  Detail & Related papers  (2020-06-07T09:24:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.