Revitalizing Dense Material Segmentation: Stabilized Vision Transformers and the Generalization Paradox

• URL: http://arxiv.org/abs/2605.23747v1
• Date: Fri, 22 May 2026 15:20:27 GMT
• Title: Revitalizing Dense Material Segmentation: Stabilized Vision Transformers and the Generalization Paradox
• Abstract summary: We revive the Apple-DMS benchmark to establish a modern Vision Transformer baseline.<n>We introduce a stabilized training recipe featuring High-Fidelity Logit Projection, Query Entropy Regularization, and a domain-specific, physics-compliant augmentation pipeline.<n>Our optimized SegFormer-B5 achieves a new State-of-the-Art (SOTA) of 0.4572 mIoU on the original dataset split.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Material segmentation, the pixel-wise classification of physical surface properties, remains a challenging problem in computer vision, requiring physicochemical understanding distinct from object-centric parsing. Despite the introduction of the rigorous Apple Dense Material Segmentation (DMS) dataset, the benchmark has suffered from attrition and stagnation, increasingly overshadowed by geometry-biased foundation models. In this paper, we revive the Apple-DMS benchmark to establish a modern Vision Transformer baseline. We conduct an exhaustive evaluation of SegFormer and Mask2Former architectures, revealing that standard training paradigms fail on amorphous texture fields due to high-variance gradients. To address this, we introduce a stabilized training recipe featuring High-Fidelity Logit Projection, Query Entropy Regularization, and a domain-specific, physics-compliant augmentation pipeline. Our optimized SegFormer-B5 achieves a new State-of-the-Art (SOTA) of 0.4572 mIoU on the original dataset split, significantly surpassing the prior convolutional baseline. Furthermore, we identify a critical "Generalization Paradox": while re-partitioning the dataset into a data-rich 80/10/10 split inflates the metric to 0.5276 mIoU, expert qualitative analysis reveals this induces distributional homogenization, severely degrading real-world, out-of-distribution performance. By releasing our recovered dataset index and robust training framework, we demonstrate that material perception is far from solved and urge the community to leverage the rigorous original split to drive genuine progress in physically grounded artificial intelligence.

Related papers

• Physics-Informed Coarsening for Multigrid Graph Neural Surrogates [0.769672852567215]
  We introduce a graph neural network for solid mechanics that couples an encoder-processor-decoder backbone with a physics-informed coarsening strategy.<n>Our results highlight the importance of physics-informed coarsening for scalable surrogate modeling in solid mechanics.
  arXiv  Detail & Related papers  (2026-05-29T08:47:04Z)
• MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scale [92.09717763663873]
  We present MinerU2.5-Pro, which advances the state of the art purely through data engineering and training strategy design.<n>At its core is a Data Engine co-designed around coverage, informativeness, and annotation accuracy.<n>We show that MinerU2.5-Pro achieves 95.69 on OmniDocBench v1.6, improving over the same-architecture baseline by 2.71 points and surpassing all existing methods.
  arXiv  Detail & Related papers  (2026-04-06T15:44:18Z)
• VFM-Recon: Unlocking Cross-Domain Scene-Level Neural Reconstruction with Scale-Aligned Foundation Priors [49.39553550491549]
  We present VFMRecon, the first attempt to bridge transferable VFM priors with scaleconsistent requirements in scene-level neural reconstruction.<n> Specifically, we first introduce a lightweight scale alignment stage that restores multiview scale coherence.<n>We then integrate pretrained VFM features into the neural volumetric reconstruction pipeline via lightweight task-specific adapters.
  arXiv  Detail & Related papers  (2026-03-13T05:00:44Z)
• Fake It Right: Injecting Anatomical Logic into Synthetic Supervised Pre-training for Medical Segmentation [21.75204301463342]
  Vision Transformers (ViTs) excel in 3D medical segmentation but require massive datasets.<n>Formula-Driven Supervised Learning (F) offers a privacy-preserving alternative by pre-training on synthetic mathematical primitives.<n>We propose an annotated-Informed Synthetic Anatomy Supervised Pre-training framework unifying F's infinite scalability with anatomical realism.
  arXiv  Detail & Related papers  (2026-03-01T08:15:18Z)
• Understanding Degradation with Vision Language Model [56.09241449206817]
  Understanding visual degradations is a critical yet challenging problem in computer vision.<n>We introduce DU-VLM, a multimodal chain-of-thought model trained with supervised fine-tuning and reinforcement learning.<n>We also introduce textbfDU-110k, a large-scale dataset comprising 110,000 clean-degraded pairs with grounded physical annotations.
  arXiv  Detail & Related papers  (2026-02-04T13:51:15Z)
• Simplicity Prevails: The Emergence of Generalizable AIGI Detection in Visual Foundation Models [15.709482146201283]
  A simple linear classifier trained on the frozen features of modern Vision Foundation Models establishes a new state-of-the-art.<n>We show that this baseline matches specialized detectors on standard benchmarks but also decisively outperforms them on in-the-wild datasets.<n>We conclude by advocating for a paradigm shift in AI forensics, moving from overfitting on static benchmarks to harnessing the evolving world knowledge of foundation models for real-world reliability.
  arXiv  Detail & Related papers  (2026-02-02T07:20:02Z)
• SIGMA: Scalable Spectral Insights for LLM Collapse [51.863164847253366]
  We introduce SIGMA (Spectral Inequalities for Gram Matrix Analysis), a unified framework for model collapse.<n>By utilizing benchmarks that deriving and deterministic bounds on the matrix's spectrum, SIGMA provides a mathematically grounded metric to track the contraction of the representation space.<n>We demonstrate that SIGMA effectively captures the transition towards states, offering both theoretical insights into the mechanics of collapse.
  arXiv  Detail & Related papers  (2026-01-06T19:47:11Z)
• UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Pass [83.7071371474926]
  UniSH is a unified, feed-forward framework for joint metric-scale 3D scene and human reconstruction.<n>Our framework bridges strong, disparate priors from scene reconstruction and HMR.<n>Our model achieves state-of-the-art performance on human-centric scene reconstruction.
  arXiv  Detail & Related papers  (2026-01-03T16:06:27Z)
• No-rank Tensor Decomposition Using Metric Learning [0.0]
  This paper introduces a no-rank tensor decomposition framework grounded in metric learning.<n>We provide theoretical guarantees for the framework's convergence and establish bounds on its metric properties.<n>Our approach achieves superior performance with smaller training datasets compared to transformer-based methods.
  arXiv  Detail & Related papers  (2025-11-03T18:21:53Z)
• Migration as a Probe: A Generalizable Benchmark Framework for Specialist vs. Generalist Machine-Learned Force Fields [1.572216094651749]
  Machine-learned force fields (MLFFs) are transforming computational materials science by enabling ab initio-level accuracy at molecular dynamics scales.<n>Yet their rapid rise raises a key question: should researchers train specialist models from scratch, fine-tune generalist foundation models, or use hybrid approaches?<n>We introduce a benchmarking framework using defect migration pathways, evaluated through elastic band trajectories, as diagnostic probes.<n>Fine-tuned models substantially outperform from-scratch and zero-shot approaches for kinetic properties but show partial loss of long-range physics.
  arXiv  Detail & Related papers  (2025-08-27T13:24:41Z)
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection [84.78366452133514]
  Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support.<n>We introduce the first benchmark for HOI detection, evaluating model resilience under diverse challenges.<n>Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric.
  arXiv  Detail & Related papers  (2025-07-12T01:58:04Z)
• SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation [81.36747103102459]
  Expressive human pose and shape estimation (EHPS) unifies body, hands, and face motion capture with numerous applications.<n>Current state-of-the-art methods focus on training innovative architectural designs on confined datasets.<n>We investigate the impact of scaling up EHPS towards a family of generalist foundation models.
  arXiv  Detail & Related papers  (2025-01-16T18:59:46Z)
• DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation [20.676510832922016]
  We propose a probabilistic model that relies on diffusion to estimate dense canonical maps crucial for recovering partial object shapes. We introduce critical components to enhance performance by leveraging the strength of the diffusion models with multi-modal input representations. Despite being trained solely on our generated synthetic data, our approach achieves state-of-the-art performance and unprecedented generalization qualities.
  arXiv  Detail & Related papers  (2024-02-20T01:48:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.