Addressing Weak Decision Boundaries in Image Classification by Leveraging Web Search and Generative Models

• URL: http://arxiv.org/abs/2310.19986v1
• Date: Mon, 30 Oct 2023 20:04:50 GMT
• Title: Addressing Weak Decision Boundaries in Image Classification by Leveraging Web Search and Generative Models
• Authors: Preetam Prabhu Srikar Dammu, Yunhe Feng, Chirag Shah
• Abstract summary: One major issue among many is that machine learning models do not perform equally well for underrepresented groups. We propose an approach that leverages the power of web search and generative models to alleviate some of the shortcomings of discriminative models. Although we showcase our method on vulnerable populations in this study, the proposed technique is extendable to a wide range of problems and domains.
• Score: 14.732229124148596
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning (ML) technologies are known to be riddled with ethical and operational problems, however, we are witnessing an increasing thrust by businesses to deploy them in sensitive applications. One major issue among many is that ML models do not perform equally well for underrepresented groups. This puts vulnerable populations in an even disadvantaged and unfavorable position. We propose an approach that leverages the power of web search and generative models to alleviate some of the shortcomings of discriminative models. We demonstrate our method on an image classification problem using ImageNet's People Subtree subset, and show that it is effective in enhancing robustness and mitigating bias in certain classes that represent vulnerable populations (e.g., female doctor of color). Our new method is able to (1) identify weak decision boundaries for such classes; (2) construct search queries for Google as well as text for generating images through DALL-E 2 and Stable Diffusion; and (3) show how these newly captured training samples could alleviate population bias issue. While still improving the model's overall performance considerably, we achieve a significant reduction (77.30\%) in the model's gender accuracy disparity. In addition to these improvements, we observed a notable enhancement in the classifier's decision boundary, as it is characterized by fewer weakspots and an increased separation between classes. Although we showcase our method on vulnerable populations in this study, the proposed technique is extendable to a wide range of problems and domains.

Related papers

• Extracting Human Attention through Crowdsourced Patch Labeling [18.947126675569667]
  In image classification, a significant problem arises from bias in the datasets. One approach to mitigate such biases is to direct the model's attention toward the target object's location. We propose a novel patch-labeling method that integrates AI assistance with crowdsourcing to capture human attention from images.
  arXiv  Detail & Related papers  (2024-03-22T07:57:27Z)
• Classes Are Not Equal: An Empirical Study on Image Recognition Fairness [100.36114135663836]
  We experimentally demonstrate that classes are not equal and the fairness issue is prevalent for image classification models across various datasets. Our findings reveal that models tend to exhibit greater prediction biases for classes that are more challenging to recognize. Data augmentation and representation learning algorithms improve overall performance by promoting fairness to some degree in image classification.
  arXiv  Detail & Related papers  (2024-02-28T07:54:50Z)
• Detecting and Mitigating Algorithmic Bias in Binary Classification using Causal Modeling [0.0]
  We show that gender bias in the prediction model is statistically significant at the 0.05 level. We demonstrate the effectiveness of the causal model in mitigating gender bias by cross-validation. Our novel approach is intuitive, easy-to-use, and can be implemented using existing statistical software tools such as "lavaan" in R.
  arXiv  Detail & Related papers  (2023-10-19T02:21:04Z)
• Counterfactual Image Generation for adversarially robust and interpretable Classifiers [1.3859669037499769]
  We propose a unified framework leveraging image-to-image translation Generative Adrial Networks (GANs) to produce counterfactual samples. This is achieved by combining the classifier and discriminator into a single model that attributes real images to their respective classes and flags generated images as "fake" We show how the model exhibits improved robustness to adversarial attacks, and we show how the discriminator's "fakeness" value serves as an uncertainty measure of the predictions.
  arXiv  Detail & Related papers  (2023-10-01T18:50:29Z)
• Your Diffusion Model is Secretly a Zero-Shot Classifier [90.40799216880342]
  We show that density estimates from large-scale text-to-image diffusion models can be leveraged to perform zero-shot classification. Our generative approach to classification attains strong results on a variety of benchmarks. Our results are a step toward using generative over discriminative models for downstream tasks.
  arXiv  Detail & Related papers  (2023-03-28T17:59:56Z)
• Spuriosity Rankings: Sorting Data to Measure and Mitigate Biases [62.54519787811138]
  We present a simple but effective method to measure and mitigate model biases caused by reliance on spurious cues. We rank images within their classes based on spuriosity, proxied via deep neural features of an interpretable network. Our results suggest that model bias due to spurious feature reliance is influenced far more by what the model is trained on than how it is trained.
  arXiv  Detail & Related papers  (2022-12-05T23:15:43Z)
• Few-shot Open-set Recognition Using Background as Unknowns [58.04165813493666]
  Few-shot open-set recognition aims to classify both seen and novel images given only limited training data of seen classes. Our proposed method not only outperforms multiple baselines but also sets new results on three popular benchmarks.
  arXiv  Detail & Related papers  (2022-07-19T04:19:29Z)
• Subverting Fair Image Search with Generative Adversarial Perturbations [14.669429931620689]
  We present a case study in which we attack a state-of-the-art, fairness-aware image search engine. These perturbations attempt to cause the fair re-ranking algorithm to unfairly boost the rank of images containing people from an adversary-selected subpopulation. We demonstrate that our attacks are robust across a number of variables, that they have close to zero impact on the relevance of search results, and that they succeed under a strict threat model.
  arXiv  Detail & Related papers  (2022-05-05T03:05:34Z)
• Few-shot Forgery Detection via Guided Adversarial Interpolation [56.59499187594308]
  Existing forgery detection methods suffer from significant performance drops when applied to unseen novel forgery approaches. We propose Guided Adversarial Interpolation (GAI) to overcome the few-shot forgery detection problem. Our method is validated to be robust to choices of majority and minority forgery approaches.
  arXiv  Detail & Related papers  (2022-04-12T16:05:10Z)
• Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domains [80.11169390071869]
  Adversarial examples have posed a severe threat to deep neural networks due to their transferable nature. We propose a Beyond ImageNet Attack (BIA) to investigate the transferability towards black-box domains. Our methods outperform state-of-the-art approaches by up to 7.71% (towards coarse-grained domains) and 25.91% (towards fine-grained domains) on average.
  arXiv  Detail & Related papers  (2022-01-27T14:04:27Z)
• Knowledge Transfer Based Fine-grained Visual Classification [19.233180617535492]
  Fine-grained visual classification (FGVC) aims to distinguish the sub-classes of the same category. Its essential solution is to mine the subtle and discriminative regions. CNNs, which employ the cross entropy loss (CE-loss) as the loss function, show poor performance.
  arXiv  Detail & Related papers  (2020-12-21T14:41:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.