HcNet: Image Modeling with Heat Conduction Equation

• URL: http://arxiv.org/abs/2408.05901v2
• Date: Tue, 13 Aug 2024 02:23:45 GMT
• Title: HcNet: Image Modeling with Heat Conduction Equation
• Authors: Zhemin Zhang, Xun Gong,
• Abstract summary: This paper aims to integrate the overall architectural design of the model into the heat conduction theory framework. Our Heat Conduction Network (HcNet) still shows competitive performance.
• Score: 6.582336726258388
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Foundation models, such as CNNs and ViTs, have powered the development of image modeling. However, general guidance to model architecture design is still missing. The design of many modern model architectures, such as residual structures, multiplicative gating signal, and feed-forward networks, can be interpreted in terms of the heat conduction equation. This finding inspired us to model images by the heat conduction equation, where the essential idea is to conceptualize image features as temperatures and model their information interaction as the diffusion of thermal energy. We can take advantage of the rich knowledge in the heat conduction equation to guide us in designing new and more interpretable models. As an example, we propose Heat Conduction Layer and Refine Approximation Layer inspired by solving the heat conduction equation using Finite Difference Method and Fourier series, respectively. This paper does not aim to present a state-of-the-art model; instead, it seeks to integrate the overall architectural design of the model into the heat conduction theory framework. Nevertheless, our Heat Conduction Network (HcNet) still shows competitive performance. Code available at \url{https://github.com/ZheminZhang1/HcNet}.

Related papers

• RS-vHeat: Heat Conduction Guided Efficient Remote Sensing Foundation Model [59.37279559684668]
  We introduce RS-vHeat, an efficient multi-modal remote sensing foundation model.<n>Specifically, RS-vHeat applies the Heat Conduction Operator (HCO) with a complexity of $O(N1.5)$ and a global receptive field.<n>Compared to attention-based remote sensing foundation models, we reduce memory usage by 84%, FLOPs by 24% and improves throughput by 2.7 times.
  arXiv  Detail & Related papers  (2024-11-27T01:43:38Z)
• Physics Informed Distillation for Diffusion Models [21.173298037358954]
  We introduce Physics Informed Distillation (PID), which employs a student model to represent the solution of the ODE system corresponding to the teacher diffusion model. We observe that PID performance achieves comparable to recent distillation methods.
  arXiv  Detail & Related papers  (2024-11-13T07:03:47Z)
• IFH: a Diffusion Framework for Flexible Design of Graph Generative Models [53.219279193440734]
  Graph generative models can be classified into two prominent families: one-shot models, which generate a graph in one go, and sequential models, which generate a graph by successive additions of nodes and edges. This paper proposes a graph generative model, called Insert-Fill-Halt (IFH), that supports the specification of a sequentiality degree.
  arXiv  Detail & Related papers  (2024-08-23T16:24:40Z)
• Finite-temperature properties of string-net models [0.0]
  We compute the partition function of the string-net model and investigate several thermodynamical quantities. In the thermodynamic limit, we show that the partition function is dominated by the contribution of special particles, dubbed pure fluxons. We also analyze the behavior of Wegner-Wilson loops associated to excitations and show that they obey an area law.
  arXiv  Detail & Related papers  (2024-06-28T07:51:58Z)
• vHeat: Building Vision Models upon Heat Conduction [63.00030330898876]
  vHeat is a novel vision backbone model that simultaneously achieves both high computational efficiency and global receptive field. The essential idea is to conceptualize image patches as heat sources and model the calculation of their correlations as the diffusion of thermal energy.
  arXiv  Detail & Related papers  (2024-05-26T12:58:04Z)
• Deep generative modelling of canonical ensemble with differentiable thermal properties [0.9421843976231371]
  We propose a variational modelling method with differentiable temperature for canonical ensembles. Using a deep generative model, the free energy is estimated and minimized simultaneously in a continuous temperature range. The training process requires no dataset, and works with arbitrary explicit density generative models.
  arXiv  Detail & Related papers  (2024-04-29T03:41:49Z)
• Thermal Face Image Classification using Deep Learning Techniques [0.0]
  This work uses a convolutional neural network (CNN) architecture, specifically ResNet-50 and VGGNet-19, to extract features from thermal images. The experimental results demonstrate the effectiveness of the proposed approach in terms of accuracy and efficiency.
  arXiv  Detail & Related papers  (2023-11-04T03:56:40Z)
• FluxGAN: A Physics-Aware Generative Adversarial Network Model for Generating Microstructures That Maintain Target Heat Flux [0.0]
  We propose a physics-aware generative adversarial network model, FluxGAN, capable of simultaneously generating high-quality images of large microstructures. The model is capable of generating coating microstructures and physical processes in three-dimensional (3D) domain after being trained on two-dimensional (2D) examples. Our approach has the potential to transform the design and optimization of thermal sprayed coatings for various applications.
  arXiv  Detail & Related papers  (2023-10-06T23:13:40Z)
• Learning Structure-Guided Diffusion Model for 2D Human Pose Estimation [71.24808323646167]
  We propose textbfDiffusionPose, a new scheme for learning keypoints heatmaps by a neural network. During training, the keypoints are diffused to random distribution by adding noises and the diffusion model learns to recover ground-truth heatmaps from noised heatmaps. Experiments show the prowess of our scheme with improvements of 1.6, 1.2, and 1.2 mAP on widely-used COCO, CrowdPose, and AI Challenge datasets.
  arXiv  Detail & Related papers  (2023-06-29T16:24:32Z)
• Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance [95.12230117950232]
  We show that a common latent space emerges from two diffusion models trained independently on related domains. Applying CycleDiffusion to text-to-image diffusion models, we show that large-scale text-to-image diffusion models can be used as zero-shot image-to-image editors.
  arXiv  Detail & Related papers  (2022-10-11T15:53:52Z)
• Does Thermal Really Always Matter for RGB-T Salient Object Detection? [153.17156598262656]
  This paper proposes a network named TNet to solve the RGB-T salient object detection (SOD) task. In this paper, we introduce a global illumination estimation module to predict the global illuminance score of the image. On the other hand, we introduce a two-stage localization and complementation module in the decoding phase to transfer object localization cue and internal integrity cue in thermal features to the RGB modality.
  arXiv  Detail & Related papers  (2022-10-09T13:50:12Z)
• Adaptive Convolutional Dictionary Network for CT Metal Artifact Reduction [62.691996239590125]
  We propose an adaptive convolutional dictionary network (ACDNet) for metal artifact reduction. Our ACDNet can automatically learn the prior for artifact-free CT images via training data and adaptively adjust the representation kernels for each input CT image. Our method inherits the clear interpretability of model-based methods and maintains the powerful representation ability of learning-based methods.
  arXiv  Detail & Related papers  (2022-05-16T06:49:36Z)
• InvGAN: Invertible GANs [88.58338626299837]
  InvGAN, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model. This allows us to perform image inpainting, merging, and online data augmentation.
  arXiv  Detail & Related papers  (2021-12-08T21:39:00Z)
• Sparse Flows: Pruning Continuous-depth Models [107.98191032466544]
  We show that pruning improves generalization for neural ODEs in generative modeling. We also show that pruning finds minimal and efficient neural ODE representations with up to 98% less parameters compared to the original network, without loss of accuracy.
  arXiv  Detail & Related papers  (2021-06-24T01:40:17Z)
• Learning Manifold Implicitly via Explicit Heat-Kernel Learning [63.354671267760516]
  We propose the concept of implicit manifold learning, where manifold information is implicitly obtained by learning the associated heat kernel. The learned heat kernel can be applied to various kernel-based machine learning models, including deep generative models (DGM) for data generation and Stein Variational Gradient Descent for Bayesian inference.
  arXiv  Detail & Related papers  (2020-10-05T03:39:58Z)
• An unsupervised learning approach to solving heat equations on chip based on Auto Encoder and Image Gradient [0.43512163406551996]
  Solving heat transfer equations on chip becomes very critical in the upcoming 5G and AI chip-package-systems. Data driven methods are data hungry, to address this, Physics Informed Neural Networks (PINN) have been proposed. This paper investigates an unsupervised learning approach for solving heat transfer equations on chip without using data.
  arXiv  Detail & Related papers  (2020-07-19T15:01:01Z)
• Efficient and Model-Based Infrared and Visible Image Fusion Via Algorithm Unrolling [24.83209572888164]
  Infrared and visible image fusion (IVIF) expects to obtain images that retain thermal radiation information from infrared images and texture details from visible images. A model-based convolutional neural network (CNN) model is proposed to overcome the shortcomings of traditional CNN-based IVIF models.
  arXiv  Detail & Related papers  (2020-05-12T16:15:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.