DealMaTe: Multi-Dimensional Material Transfer via Diffusion Transformer

• URL: http://arxiv.org/abs/2605.15681v1
• Date: Fri, 15 May 2026 07:06:39 GMT
• Title: DealMaTe: Multi-Dimensional Material Transfer via Diffusion Transformer
• Abstract summary: DealMaTe is a diffusion framework that eliminates text guidance and reference networks.<n>We show that DealMaTe achieves remarkable high-fidelity material transfer under arbitrary input materials.
• Score: 45.232470509013815
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, diffusion-based material transfer methods rely on image fine-tuning or complex architectures with auxiliary networks but face challenges such as text dependency, additional computational costs, and feature misalignment. To address these limitations, we propose \textbf{DealMaTe}, using \underline{\textbf{de}}pth, norm\underline{\textbf{a}}l, and \underline{\textbf{l}}ighting images for \underline{\textbf{ma}}terial \underline{\textbf{t}}ransf\underline{\textbf{e}}r. DealMaTe is a simplified diffusion framework that eliminates text guidance and reference networks. We design a lightweight 3D information injection method, Multi-Dim 3D Shader LoRA, which, without modifying the base model weights, enables compatible control conditions and achieves harmonious and stable results. Additionally, we optimize the attention mechanism with Shader Causal Mutual Attention and key-value (KV) caching to reduce inference latency caused by multiple conditions, improve computational efficiency, and achieve high-quality material transfer results with low architectural complexity. Extensive experiments covering a wide variety of objects and lighting conditions consistently demonstrate that DealMaTe achieves remarkable high-fidelity material transfer under arbitrary input materials. The code is available at https://github.com/haha-lisa/DealMaTe.

Related papers

• $K$-NeAS: Scalable Multi-Material CT Reconstruction Using Neural SDFs [0.0]
  Implicit scene representations (ISRs) address this by recovering continuous volumetric attenuation fields directly from sparse projections.<n>This paper proposes $K$-NeAS, a unified and scalable architecture for automated, multi-material surface reconstruction.
  arXiv  Detail & Related papers  (2026-07-15T23:08:45Z)
• SAB-LVLM: Significance-Aware Binarization for Large Vision-Language Models [75.75091459417304]
  Large Vision-Language Models (LVLMs) have achieved remarkable progress in multimodal understanding.<n>LVLMs incur substantial memory and latency overhead, severely limiting real-world deployment on resource-constrained devices.<n>We propose a binarization method to drastically reduce storage and computational costs.
  arXiv  Detail & Related papers  (2026-07-02T08:30:00Z)
• MaTe: Images Are All You Need for Material Transfer via Diffusion Transformer [21.547583825573255]
  MaTe is a streamlined diffusion framework that eliminates textual guidance and reference networks.<n>It integrates input images at the token level, enabling unified processing via multi-modal attention in a shared latent space.<n>It outperforms state-of-the-art methods in both visual quality and efficiency while preserving precise detail alignment.
  arXiv  Detail & Related papers  (2026-05-15T06:31:41Z)
• Unleashing Degradation-Carrying Features in Symmetric U-Net: Simpler and Stronger Baselines for All-in-One Image Restoration [52.82397287366076]
  All-in-one image restoration aims to handle diverse degradations (e.g., noise, blur, adverse weather) within a unified framework.<n>In this work, we reveal a critical insight: well-crafted feature extraction inherently encodes degradation-carrying information.<n>Our symmetric design preserves intrinsic degradation signals robustly, rendering simple additive fusion in skip connections.
  arXiv  Detail & Related papers  (2025-12-11T12:20:31Z)
• IMG: Calibrating Diffusion Models via Implicit Multimodal Guidance [74.89810064807142]
  Implicit Multimodal Guidance (IMG) is a novel re-generation-based multimodal alignment framework.<n>IMG identifies misalignments and formulates the re-alignment goal into a trainable objective.<n>IMG acts as a flexible plug-and-play adapter, seamlessly enhancing prior finetuning-based alignment methods.
  arXiv  Detail & Related papers  (2025-09-30T13:27:03Z)
• Rethinking Cross-Modal Interaction in Multimodal Diffusion Transformers [79.94246924019984]
  Multimodal Diffusion Transformers (MM-DiTs) have achieved remarkable progress in text-driven visual generation.<n>We propose textbfTemperature-Adjusted Cross-modal Attention (TACA), a parameter-efficient method that dynamically rebalances multimodal interactions.<n>Our findings highlight the importance of balancing cross-modal attention in improving semantic fidelity in text-to-image diffusion models.
  arXiv  Detail & Related papers  (2025-06-09T17:54:04Z)
• RelitLRM: Generative Relightable Radiance for Large Reconstruction Models [52.672706620003765]
  We propose RelitLRM for generating high-quality Gaussian splatting representations of 3D objects under novel illuminations. Unlike prior inverse rendering methods requiring dense captures and slow optimization, RelitLRM adopts a feed-forward transformer-based model. We show our sparse-view feed-forward RelitLRM offers competitive relighting results to state-of-the-art dense-view optimization-based baselines.
  arXiv  Detail & Related papers  (2024-10-08T17:40:01Z)
• Magnet: We Never Know How Text-to-Image Diffusion Models Work, Until We Learn How Vision-Language Models Function [13.588643982359413]
  We critically examine the limitations of the CLIP text encoder in understanding attributes and investigate how this affects diffusion models. We propose textbf Magnet, a novel training-free approach to tackle the attribute binding problem.
  arXiv  Detail & Related papers  (2024-09-30T05:36:24Z)
• GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Clouds [72.60362979456035]
  Masked Autoencoders (MAE) are challenging to explore in large-scale 3D point clouds. We propose a textbfGenerative textbfDecoder for MAE (GD-MAE) to automatically merges the surrounding context. We demonstrate the efficacy of the proposed method on several large-scale benchmarks: KITTI, and ONCE.
  arXiv  Detail & Related papers  (2022-12-06T14:32:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.