Quantifying Overfitting: Introducing the Overfitting Index
- URL: http://arxiv.org/abs/2308.08682v1
- Date: Wed, 16 Aug 2023 21:32:57 GMT
- Title: Quantifying Overfitting: Introducing the Overfitting Index
- Authors: Sanad Aburass
- Abstract summary: Overfitting is where a model exhibits superior performance on training data but falters on unseen data.
This paper introduces the Overfitting Index (OI), a novel metric devised to quantitatively assess a model's tendency to overfit.
Our results underscore the variable overfitting behaviors across architectures and highlight the mitigative impact of data augmentation.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In the rapidly evolving domain of machine learning, ensuring model
generalizability remains a quintessential challenge. Overfitting, where a model
exhibits superior performance on training data but falters on unseen data, is a
recurrent concern. This paper introduces the Overfitting Index (OI), a novel
metric devised to quantitatively assess a model's tendency to overfit. Through
extensive experiments on the Breast Ultrasound Images Dataset (BUS) and the
MNIST dataset using architectures such as MobileNet, U-Net, ResNet, Darknet,
and ViT-32, we illustrate the utility and discernment of the OI. Our results
underscore the variable overfitting behaviors across architectures and
highlight the mitigative impact of data augmentation, especially on smaller and
more specialized datasets. The ViT-32's performance on MNIST further emphasizes
the robustness of certain models and the dataset's comprehensive nature. By
providing an objective lens to gauge overfitting, the OI offers a promising
avenue to advance model optimization and ensure real-world efficacy.
Related papers
- DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception [78.26734070960886]
Current perceptive models heavily depend on resource-intensive datasets.
We introduce perception-aware loss (P.A. loss) through segmentation, improving both quality and controllability.
Our method customizes data augmentation by extracting and utilizing perception-aware attribute (P.A. Attr) during generation.
arXiv Detail & Related papers (2024-03-20T04:58:03Z) - Towards In-Vehicle Multi-Task Facial Attribute Recognition:
Investigating Synthetic Data and Vision Foundation Models [8.54530542456452]
We investigate the utility of synthetic datasets for training complex multi-task models that recognize facial attributes of passengers of a vehicle.
Our study unveils counter-intuitive findings, notably the superior performance of ResNet over ViTs in our specific multi-task context.
arXiv Detail & Related papers (2024-03-10T04:17:54Z) - Data-efficient Large Vision Models through Sequential Autoregression [58.26179273091461]
We develop an efficient, autoregression-based vision model on a limited dataset.
We demonstrate how this model achieves proficiency in a spectrum of visual tasks spanning both high-level and low-level semantic understanding.
Our empirical evaluations underscore the model's agility in adapting to various tasks, heralding a significant reduction in the parameter footprint.
arXiv Detail & Related papers (2024-02-07T13:41:53Z) - AttributionScanner: A Visual Analytics System for Model Validation with Metadata-Free Slice Finding [29.07617945233152]
Data slice finding is an emerging technique for validating machine learning (ML) models by identifying and analyzing subgroups in a dataset that exhibit poor performance.
This approach faces significant challenges, including the laborious and costly requirement for additional metadata.
We introduce AttributionScanner, an innovative human-in-the-loop Visual Analytics (VA) system, designed for metadata-free data slice finding.
Our system identifies interpretable data slices that involve common model behaviors and visualizes these patterns through an Attribution Mosaic design.
arXiv Detail & Related papers (2024-01-12T09:17:32Z) - Visual Prompting Upgrades Neural Network Sparsification: A Data-Model
Perspective [67.25782152459851]
We introduce a novel data-model co-design perspective: to promote superior weight sparsity.
Specifically, customized Visual Prompts are mounted to upgrade neural Network sparsification in our proposed VPNs framework.
arXiv Detail & Related papers (2023-12-03T13:50:24Z) - StableLLaVA: Enhanced Visual Instruction Tuning with Synthesized
Image-Dialogue Data [129.92449761766025]
We propose a novel data collection methodology that synchronously synthesizes images and dialogues for visual instruction tuning.
This approach harnesses the power of generative models, marrying the abilities of ChatGPT and text-to-image generative models.
Our research includes comprehensive experiments conducted on various datasets.
arXiv Detail & Related papers (2023-08-20T12:43:52Z) - Exploring the Effects of Data Augmentation for Drivable Area
Segmentation [0.0]
We focus on investigating the benefits of data augmentation by analyzing pre-existing image datasets.
Our results show that the performance and robustness of existing state of the art (or SOTA) models can be increased dramatically.
arXiv Detail & Related papers (2022-08-06T03:39:37Z) - MAIN: Multihead-Attention Imputation Networks [4.427447378048202]
We propose a novel mechanism based on multi-head attention which can be applied effortlessly in any model.
Our method inductively models patterns of missingness in the input data in order to increase the performance of the downstream task.
arXiv Detail & Related papers (2021-02-10T13:50:02Z) - Improving Zero and Few-Shot Abstractive Summarization with Intermediate
Fine-tuning and Data Augmentation [101.26235068460551]
Models pretrained with self-supervised objectives on large text corpora achieve state-of-the-art performance on English text summarization tasks.
Models are typically fine-tuned on hundreds of thousands of data points, an infeasible requirement when applying summarization to new, niche domains.
We introduce a novel and generalizable method, called WikiTransfer, for fine-tuning pretrained models for summarization in an unsupervised, dataset-specific manner.
arXiv Detail & Related papers (2020-10-24T08:36:49Z) - Adversarial Filters of Dataset Biases [96.090959788952]
Large neural models have demonstrated human-level performance on language and vision benchmarks.
Their performance degrades considerably on adversarial or out-of-distribution samples.
We propose AFLite, which adversarially filters such dataset biases.
arXiv Detail & Related papers (2020-02-10T21:59:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.