Harnessing Synthetic Datasets: The Role of Shape Bias in Deep Neural Network Generalization

• URL: http://arxiv.org/abs/2311.06224v1
• Date: Fri, 10 Nov 2023 18:25:44 GMT
• Title: Harnessing Synthetic Datasets: The Role of Shape Bias in Deep Neural Network Generalization
• Authors: Elior Benarous, Sotiris Anagnostidis, Luca Biggio, Thomas Hofmann
• Abstract summary: We investigate how neural networks exhibit shape bias during training on synthetic datasets. Shape bias varies across network architectures and types of supervision. We propose a novel interpretation of shape bias as a tool for estimating the diversity of samples within a dataset.
• Score: 27.39922946288783
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advancements in deep learning have been primarily driven by the use of large models trained on increasingly vast datasets. While neural scaling laws have emerged to predict network performance given a specific level of computational resources, the growing demand for expansive datasets raises concerns. To address this, a new research direction has emerged, focusing on the creation of synthetic data as a substitute. In this study, we investigate how neural networks exhibit shape bias during training on synthetic datasets, serving as an indicator of the synthetic data quality. Specifically, our findings indicate three key points: (1) Shape bias varies across network architectures and types of supervision, casting doubt on its reliability as a predictor for generalization and its ability to explain differences in model recognition compared to human capabilities. (2) Relying solely on shape bias to estimate generalization is unreliable, as it is entangled with diversity and naturalism. (3) We propose a novel interpretation of shape bias as a tool for estimating the diversity of samples within a dataset. Our research aims to clarify the implications of using synthetic data and its associated shape bias in deep learning, addressing concerns regarding generalization and dataset quality.

Related papers

• Cross-Dataset Generalization in Deep Learning [4.706219235601874]
  Deep learning has been extensively used in various fields, such as phase imaging, 3D imaging reconstruction, phase unwrapping, and laser speckle reduction. Its data-driven nature allows for implicit construction of mathematical relationships within the network through training with abundant data. A critical challenge in practical applications is the generalization issue, where a network trained on one dataset struggles to recognize an unknown target from a different dataset.
  arXiv  Detail & Related papers  (2024-10-15T02:48:21Z)
• Will the Inclusion of Generated Data Amplify Bias Across Generations in Future Image Classification Models? [29.71939692883025]
  We investigate the effects of generated data on image classification tasks, with a specific focus on bias. Hundreds of experiments are conducted on Colorized MNIST, CIFAR-20/100, and Hard ImageNet datasets. Our findings contribute to the ongoing debate on the implications of synthetic data for fairness in real-world applications.
  arXiv  Detail & Related papers  (2024-10-14T05:07:06Z)
• Do Deep Neural Networks Always Perform Better When Eating More Data? [82.6459747000664]
  We design experiments from Identically Independent Distribution(IID) and Out of Distribution(OOD) Under IID condition, the amount of information determines the effectivity of each sample, the contribution of samples and difference between classes determine the amount of class information. Under OOD condition, the cross-domain degree of samples determine the contributions, and the bias-fitting caused by irrelevant elements is a significant factor of cross-domain.
  arXiv  Detail & Related papers  (2022-05-30T15:40:33Z)
• Exploring the Trade-off between Plausibility, Change Intensity and Adversarial Power in Counterfactual Explanations using Multi-objective Optimization [73.89239820192894]
  We argue that automated counterfactual generation should regard several aspects of the produced adversarial instances. We present a novel framework for the generation of counterfactual examples.
  arXiv  Detail & Related papers  (2022-05-20T15:02:53Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• General Greedy De-bias Learning [163.65789778416172]
  We propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model like gradient descent in functional space. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
  arXiv  Detail & Related papers  (2021-12-20T14:47:32Z)
• Transitioning from Real to Synthetic data: Quantifying the bias in model [1.6134566438137665]
  This study aims to establish a trade-off between bias and fairness in the models trained using synthetic data. We demonstrate there exist a varying levels of bias impact on models trained using synthetic data.
  arXiv  Detail & Related papers  (2021-05-10T06:57:14Z)
• Synthesizing Irreproducibility in Deep Networks [2.28438857884398]
  Modern day deep networks suffer from irreproducibility (also referred to as nondeterminism or underspecification) We show that even with a single nonlinearity and for very simple data and models, irreproducibility occurs. Model complexity and the choice of nonlinearity also play significant roles in making deep models irreproducible.
  arXiv  Detail & Related papers  (2021-02-21T21:51:28Z)
• Vulnerability Under Adversarial Machine Learning: Bias or Variance? [77.30759061082085]
  We investigate the effect of adversarial machine learning on the bias and variance of a trained deep neural network. Our analysis sheds light on why the deep neural networks have poor performance under adversarial perturbation. We introduce a new adversarial machine learning algorithm with lower computational complexity than well-known adversarial machine learning strategies.
  arXiv  Detail & Related papers  (2020-08-01T00:58:54Z)
• Learning from Failure: Training Debiased Classifier from Biased Classifier [76.52804102765931]
  We show that neural networks learn to rely on spurious correlation only when it is "easier" to learn than the desired knowledge. We propose a failure-based debiasing scheme by training a pair of neural networks simultaneously. Our method significantly improves the training of the network against various types of biases in both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2020-07-06T07:20:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.