Beyond Argmax: A Mechanistic Study of Semantic Retention in Frozen Foundation-Model Composition for Generalized Few-Shot 3D Segmentation

• URL: http://arxiv.org/abs/2609.12099v1
• Date: Thu, 10 Sep 2026 18:25:53 GMT
• Title: Beyond Argmax: A Mechanistic Study of Semantic Retention in Frozen Foundation-Model Composition for Generalized Few-Shot 3D Segmentation
• Abstract summary: We investigate how much useful semantic information is lost when heterogeneous sources are collapsed to a single class before they can interact.<n>We find that most information is recovered by retaining a compact set of plausible alternatives.<n>The contribution is a controlled diagnosis of premature semantic collapse as a repeatable information bottleneck in heterogeneous frozen-model composition.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Classical classifier-combination work distinguishes score-level fusion from hard decision-level voting. We revisit this distinction where independently pretrained, frozen foundation models are composed at inference time for generalized few-shot 3D segmentation. We ask: how much useful semantic information is lost when heterogeneous sources are collapsed to a single class before they can interact? We answer with a same-input semantic-retention intervention. Dense RegionPLC and sparse cross-view SAM3 evidence, model weights, masks, geometry, vocabularies, and fusion rules are frozen; only the number of semantic alternatives retained before interaction is varied via a matched top-k ladder. On 156 held-out ScanNet200 scenes, top-1 reaches 28.47 harmonic-mean (HM) IoU while full distribution fusion reaches 34.87 HM (+6.40, 95% CI [+5.24,+7.64]). The pattern replicates on 50 ScanNet++ scenes: 23.02 vs. 26.50 HM (+3.48, 95% CI [+1.64,+5.93]). The conclusion is robust: full-distribution HM is stable across sparse-source weights 0.3--0.7; alternative operators (max, geometric pooling) also outperform top-1; and a GroundingDINO--SAM2.1 source-replacement diagnostic shows monotonic HM increase from 14.77 to 18.75 with full retention. Calibration diagnostics reveal opposite miscalibration of the two sources, yet correcting calibration does not eliminate the retention advantage. Across datasets and source stacks, most information is recovered by retaining a compact set of plausible alternatives. The contribution is a controlled diagnosis of premature semantic collapse as a repeatable information bottleneck in heterogeneous frozen-model composition.

Related papers

• Visual Anchoring in Diffusion: Multimodal Zero-Shot Skeleton Action Recognition [4.970910262474302]
  Zero-shot Skeleton Action Recognition (ZSAR) remains ambiguous when unseen actions share similar skeleton joint dynamics but differ in objects or scene context.<n>Existing multimodal methods typically maintain independent skeleton and RGB scoring branches and fuse their outputs.<n>We propose a text-conditioned denoising Transformer with a non-diffused RGB condition token that serves as a stable visual anchor during skeleton data reconstruction.
  arXiv  Detail & Related papers  (2026-08-05T09:28:00Z)
• DepthART: Scaling Foundation Monocular Depth to Tiny Models [51.10174763031444]
  DepthART is a compact MDE model for on-device deployment across diverse scenes.<n>We show how DepthART consistently surpasses previous tiny baselines in both zero-shot generalization and metric accuracy.<n>We also provide a scalable model family, with DepthART-S reaching 347/245 FPS (strict FP32) on an GTX A6000 at $2242/4482$, 102 FPS (TF32) on a Orin NX 8GB, and over 15 FPS (FP32) on a Jetson Nano 4GB.
  arXiv  Detail & Related papers  (2026-07-19T06:54:52Z)
• Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems [47.027290803102666]
  Normal behaviour is the union of many imbalanced, curved, thin-fringed operating regimes.<n>We model the normal law with a jointly learned latent representation plus explicit Gaussian-mixture mode clustering.<n>We evaluate under a deliberately fair protocol: raw point-wise metrics with no point adjustment, a trivial-detector difficulty split, prevalence-matched F1, and train-normal-only calibration.
  arXiv  Detail & Related papers  (2026-07-07T10:10:00Z)
• Measuring and Improving Behavioral Consistency in Large Language Models through Fact-Heuristic-Emotion State Enforcement [4.226475360842309]
  Large language models (LLMs) can give different answers to the same decision problem across runs, and reverse a decision when their own prior answer returns as context.<n>We ask whether this instability can be measured and partially reduced without changing model weights.<n>We test the Cognitive Kernel Model (CKM), a prompt-level state-enforcement layer.
  arXiv  Detail & Related papers  (2026-06-05T12:36:36Z)
• Striding Across Reynolds Numbers: Representation Geometry in Neural PDE Generalisation [0.5451632647096261]
  Cross-Reynolds generalisation in neural PDE solvers remains poorly characterised.<n>We test this hypothesis through ConvAE-Relay, which matches states in a source-trained convolutional autoencoder latent space.<n>Experiments confirm that source-regime dynamics directions remain transferable (cosine similarity 0.84) when matching stays on-manifold.
  arXiv  Detail & Related papers  (2026-05-28T15:49:26Z)
• More Is Not Always Better: Cross-Component Interference in LLM Agent Scaffolding [4.738949927143789]
  LLM agent systems are built by stacking components (planning, tools, memory, self-reflection, retrieval)<n>We run a full factorial experiment over all 25=32 subsets of five components on HotpotQA and GSM8K with Llama-3.1-8B/70B conditions.<n>The All-In system is consistently suboptimal: on HotpotQA, a single-tool agent surpasses All-In by 32% (F1 0.233 vs 0.177, p23)<n>Our findings suggest maximally-equipped agent should be replaced by subset selection via interaction-aware analysis.
  arXiv  Detail & Related papers  (2026-05-07T06:01:43Z)
• PRISM: Differentiable Analysis-by-Synthesis for Fixel Recovery in Diffusion MRI [1.3063136334195184]
  This work introduces, a differentiable analysis-by-synthesis framework that fits an explicit multi-compartment forward model end-to-end cerebrospinal model.<n> PRISM supports fast MSE objective and a Rician negative log-ze-likelihood (NLL) that jointly learns without information.<n> PRISM achieves 3.5 degrees best-match angular error with 95% recall is 1.9x lower than best baseline.
  arXiv  Detail & Related papers  (2026-03-31T21:22:49Z)
• Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching [66.39914384073145]
  We propose a self-consistency framework that turns cheap diffusion-sampled reasoning into a reusable pool of step-level candidates.<n>We find that step-level recombination is most beneficial on harder problems.<n>Our training-free framework improves average accuracy by up to 2 across six math and coding tasks.
  arXiv  Detail & Related papers  (2026-02-26T11:08:39Z)
• Brain Tumor Classification on MRI in Light of Molecular Markers [56.99710477905796]
  Co-deletion of the 1p/19q gene is associated with clinical outcomes in low-grade gliomas.<n>This study aims to utilize a specially MRI-based convolutional neural network for brain cancer detection.
  arXiv  Detail & Related papers  (2024-09-29T07:04:26Z)
• Diffusion Soup: Model Merging for Text-to-Image Diffusion Models [90.01635703779183]
  We present Diffusion Soup, a compartmentalization method for Text-to-Image Generation that averages the weights of diffusion models trained on sharded data. By construction, our approach enables training-free continual learning and unlearning with no additional memory or inference costs.
  arXiv  Detail & Related papers  (2024-06-12T17:16:16Z)
• MTC: Multiresolution Tensor Completion from Partial and Coarse Observations [49.931849672492305]
  Existing completion formulation mostly relies on partial observations from a single tensor. We propose an efficient Multi-resolution Completion model (MTC) to solve the problem.
  arXiv  Detail & Related papers  (2021-06-14T02:20:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.