Err on the Side of Texture: Texture Bias on Real Data

• URL: http://arxiv.org/abs/2412.10597v2
• Date: Mon, 10 Feb 2025 21:53:37 GMT
• Title: Err on the Side of Texture: Texture Bias on Real Data
• Authors: Blaine Hoak, Ryan Sheatsley, Patrick McDaniel,
• Abstract summary: We introduce the Texture Association Value (TAV), a novel metric that quantifies how strongly models rely on the presence of specific textures when classifying objects. Our results show that texture bias explains the existence of natural adversarial examples, where over 90% of these samples contain textures that are misaligned with the learned texture of their true label.
• Score: 3.5990273573803058
• License:
• Abstract: Bias significantly undermines both the accuracy and trustworthiness of machine learning models. To date, one of the strongest biases observed in image classification models is texture bias-where models overly rely on texture information rather than shape information. Yet, existing approaches for measuring and mitigating texture bias have not been able to capture how textures impact model robustness in real-world settings. In this work, we introduce the Texture Association Value (TAV), a novel metric that quantifies how strongly models rely on the presence of specific textures when classifying objects. Leveraging TAV, we demonstrate that model accuracy and robustness are heavily influenced by texture. Our results show that texture bias explains the existence of natural adversarial examples, where over 90% of these samples contain textures that are misaligned with the learned texture of their true label, resulting in confident mispredictions.

Related papers

• Real-time Free-view Human Rendering from Sparse-view RGB Videos using Double Unprojected Textures [87.80984588545589]
  Real-time free-view human rendering from sparse-view RGB inputs is a challenging task due to the sensor scarcity and the tight time budget. Recent methods leverage 2D CNNs operating in texture space to learn rendering primitives. We present Double Unprojected Textures, which at the core disentangles coarse geometric deformation estimation from appearance synthesis.
  arXiv  Detail & Related papers  (2024-12-17T18:57:38Z)
• On Synthetic Texture Datasets: Challenges, Creation, and Curation [1.9567015559455132]
  We create a dataset of 362,880 texture images that span 56 textures. During the process of generating images, we find that NSFW safety filters in image generation pipelines are highly sensitive to texture.
  arXiv  Detail & Related papers  (2024-09-16T14:02:18Z)
• Deep Shape-Texture Statistics for Completely Blind Image Quality Evaluation [48.278380421089764]
  Deep features as visual descriptors have advanced IQA in recent research, but they are discovered to be highly texture-biased and lack of shape-bias. We find out that image shape and texture cues respond differently towards distortions, and the absence of either one results in an incomplete image representation. To formulate a well-round statistical description for images, we utilize the shapebiased and texture-biased deep features produced by Deep Neural Networks (DNNs) simultaneously.
  arXiv  Detail & Related papers  (2024-01-16T04:28:09Z)
• Prompt-Propose-Verify: A Reliable Hand-Object-Interaction Data Generation Framework using Foundational Models [0.0]
  Diffusion models when conditioned on text prompts, generate realistic-looking images with intricate details. But most of these pre-trained models fail to generate accurate images when it comes to human features like hands, teeth, etc.
  arXiv  Detail & Related papers  (2023-12-23T12:59:22Z)
• Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction [49.15931834209624]
  We present the first benchmark that simulates the evaluation of open information extraction models in the real world. We design and annotate a large-scale testbed in which each example is a knowledge-invariant clique. By further elaborating the robustness metric, a model is judged to be robust if its performance is consistently accurate on the overall cliques.
  arXiv  Detail & Related papers  (2023-05-23T12:05:09Z)
• TexPose: Neural Texture Learning for Self-Supervised 6D Object Pose Estimation [55.94900327396771]
  We introduce neural texture learning for 6D object pose estimation from synthetic data. We learn to predict realistic texture of objects from real image collections. We learn pose estimation from pixel-perfect synthetic data.
  arXiv  Detail & Related papers  (2022-12-25T13:36:32Z)
• Data Generation using Texture Co-occurrence and Spatial Self-Similarity for Debiasing [6.976822832216875]
  We propose a novel de-biasing approach that explicitly generates additional images using texture representations of oppositely labeled images. Every new generated image contains similar spatial information from a source image while transferring textures from a target image of opposite label. Our model integrates a texture co-occurrence loss that determines whether a generated image's texture is similar to that of the target, and a spatial self-similarity loss that determines whether the spatial details between the generated and source images are well preserved.
  arXiv  Detail & Related papers  (2021-10-15T08:04:59Z)
• Texture Generation with Neural Cellular Automata [64.70093734012121]
  We learn a texture generator from a single template image. We make claims that the behaviour exhibited by the NCA model is a learned, distributed, local algorithm to generate a texture.
  arXiv  Detail & Related papers  (2021-05-15T22:05:46Z)
• Identifying Invariant Texture Violation for Robust Deepfake Detection [17.306386179823576]
  We propose the Invariant Texture Learning framework, which only accesses the published dataset with low visual quality. Our method is based on the prior that the microscopic facial texture of the source face is inevitably violated by the texture transferred from the target person.
  arXiv  Detail & Related papers  (2020-12-19T03:02:15Z)
• Informative Dropout for Robust Representation Learning: A Shape-bias Perspective [84.30946377024297]
  We propose a light-weight model-agnostic method, namely Informative Dropout (InfoDrop), to improve interpretability and reduce texture bias. Specifically, we discriminate texture from shape based on local self-information in an image, and adopt a Dropout-like algorithm to decorrelate the model output from the local texture.
  arXiv  Detail & Related papers  (2020-08-10T16:52:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.