Boundary Attention: Learning to Localize Boundaries under High Noise
- URL: http://arxiv.org/abs/2401.00935v2
- Date: Mon, 18 Mar 2024 23:41:41 GMT
- Title: Boundary Attention: Learning to Localize Boundaries under High Noise
- Authors: Mia Gaia Polansky, Charles Herrmann, Junhwa Hur, Deqing Sun, Dor Verbin, Todd Zickler,
- Abstract summary: We present a differentiable model that infers explicit boundaries, including curves, corners and junctions, using a mechanism that we call boundary attention.
Boundary attention is a boundary-aware local attention operation that, when applied densely and repeatedly, progressively refines a field of variables.
We find that our method generalizes to natural images corrupted by real sensor noise, and predicts consistent boundaries under increasingly noisy conditions.
- Score: 23.467103272604906
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present a differentiable model that infers explicit boundaries, including curves, corners and junctions, using a mechanism that we call boundary attention. Boundary attention is a boundary-aware local attention operation that, when applied densely and repeatedly, progressively refines a field of variables that specify an unrasterized description of the local boundary structure in every overlapping patch within an image. It operates in a bottom-up fashion, similar to classical methods for sub-pixel edge localization and edge-linking, but with a higher-dimensional description of local boundary structure, a notion of spatial consistency that is learned instead of designed, and a sequence of operations that is end-to-end differentiable. We train our model using simple synthetic data and then evaluate it using photographs that were captured under low-light conditions with variable amounts of noise. We find that our method generalizes to natural images corrupted by real sensor noise, and predicts consistent boundaries under increasingly noisy conditions where other state-of-the-art methods fail.
Related papers
- Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration [64.84134880709625]
We show that it is possible to perform domain adaptation via the noise-space using diffusion models.
In particular, by leveraging the unique property of how the multi-step denoising process is influenced by auxiliary conditional inputs, we obtain meaningful gradients from noise prediction.
To prevent shortcuts during training, we present useful techniques such as channel shuffling and residual-swapping contrastive learning.
arXiv Detail & Related papers (2024-06-26T17:40:30Z) - RecDiffusion: Rectangling for Image Stitching with Diffusion Models [53.824503710254206]
We introduce a novel diffusion-based learning framework, textbfRecDiffusion, for image stitching rectangling.
This framework combines Motion Diffusion Models (MDM) to generate motion fields, effectively transitioning from the stitched image's irregular borders to a geometrically corrected intermediary.
arXiv Detail & Related papers (2024-03-28T06:22:45Z) - CT-Bound: Robust Boundary Detection From Noisy Images Via Hybrid Convolution and Transformer Neural Networks [10.622511683372815]
We present CT-Bound, a robust and fast boundary detection method for very noisy images using a hybrid Convolution and Transformer neural network.
During the local detection, the model uses a convolutional architecture to predict the boundary structure of each image patch.
Then, it uses a feed-forward transformer architecture to globally refine the boundary structures of each patch to generate an edge map and a smoothed color map simultaneously.
arXiv Detail & Related papers (2024-03-25T07:22:22Z) - NoiseDiffusion: Correcting Noise for Image Interpolation with Diffusion Models beyond Spherical Linear Interpolation [86.7260950382448]
We propose a novel approach to correct noise for image validity, NoiseDiffusion.
NoiseDiffusion performs within the noisy image space and injects raw images into these noisy counterparts to address the challenge of information loss.
arXiv Detail & Related papers (2024-03-13T12:32:25Z) - Boundary-RL: Reinforcement Learning for Weakly-Supervised Prostate
Segmentation in TRUS Images [8.057488225592605]
We propose Boundary-RL, a novel weakly supervised segmentation method.
We envision the segmentation as a boundary detection problem, rather than a pixel-level classification as in previous works.
Particularly of interest, ultrasound images, where intensity values represent acoustic impedance differences between boundaries, may also benefit from the boundary delineation approach.
arXiv Detail & Related papers (2023-08-22T12:02:05Z) - Towards Coherent Image Inpainting Using Denoising Diffusion Implicit
Models [43.83732051916894]
We propose COPAINT, which can coherently inpaint the whole image without introducing mismatches.
COPAINT also uses the Bayesian framework to jointly modify both revealed and unrevealed regions.
Our experiments verify that COPAINT can outperform the existing diffusion-based methods under both objective and subjective metrics.
arXiv Detail & Related papers (2023-04-06T18:35:13Z) - A Random Point Initialization Approach to Image Segmentation with
Variational Level-sets [0.0]
We propose a modification to the variational level set image segmentation method that can quickly detect object boundaries.
We demonstrate the efficacy of our approach by comparing the performance of our method on real images to that of the prominent Canny Method.
arXiv Detail & Related papers (2021-12-23T04:37:44Z) - BoundarySqueeze: Image Segmentation as Boundary Squeezing [104.43159799559464]
We propose a novel method for fine-grained high-quality image segmentation of both objects and scenes.
Inspired by dilation and erosion from morphological image processing techniques, we treat the pixel level segmentation problems as squeezing object boundary.
Our method yields large gains on COCO, Cityscapes, for both instance and semantic segmentation and outperforms previous state-of-the-art PointRend in both accuracy and speed under the same setting.
arXiv Detail & Related papers (2021-05-25T04:58:51Z) - Field of Junctions: Extracting Boundary Structure at Low SNR [5.584060970507507]
We introduce a bottom-up detector for simultaneously finding many boundary elements in an image, including contours, corners and junctions.
Notably, its analysis of contours, corners, junctions and uniform regions allows it to succeed at high noise levels, where other methods for boundary detection fail.
arXiv Detail & Related papers (2020-11-27T17:46:08Z) - Contrastive Rendering for Ultrasound Image Segmentation [59.23915581079123]
The lack of sharp boundaries in US images remains an inherent challenge for segmentation.
We propose a novel and effective framework to improve boundary estimation in US images.
Our proposed method outperforms state-of-the-art methods and has the potential to be used in clinical practice.
arXiv Detail & Related papers (2020-10-10T07:14:03Z) - Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed
Scenes [54.836331922449666]
We propose a Semantic Guidance and Evaluation Network (SGE-Net) to update the structural priors and the inpainted image.
It utilizes semantic segmentation map as guidance in each scale of inpainting, under which location-dependent inferences are re-evaluated.
Experiments on real-world images of mixed scenes demonstrated the superiority of our proposed method over state-of-the-art approaches.
arXiv Detail & Related papers (2020-03-15T17:49:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.