A Comparative Performance Analysis of Classification and Segmentation Models on Bangladeshi Pothole Dataset

• URL: http://arxiv.org/abs/2501.06602v1
• Date: Sat, 11 Jan 2025 18:03:46 GMT
• Title: A Comparative Performance Analysis of Classification and Segmentation Models on Bangladeshi Pothole Dataset
• Authors: Antara Firoz Parsa, S. M. Abdullah, Anika Hasan Talukder, Md. Asif Shahidullah Kabbya, Shakib Al Hasan, Md. Farhadul Islam, Jannatun Noor,
• Abstract summary: The study involves a comprehensive performance analysis of popular classification and segmentation models, applied over a Bangladeshi pothole dataset. This custom dataset of 824 samples, collected from the streets of Dhaka and Bogura performs competitively against the existing industrial and custom datasets. The experimental results exhibit that, our dataset performs on par or outperforms the similar classification models utilized in the existing literature, reaching accuracy and f1-scores over 99%.
• Score: 0.953693516244499
• License:
• Abstract: The study involves a comprehensive performance analysis of popular classification and segmentation models, applied over a Bangladeshi pothole dataset, being developed by the authors of this research. This custom dataset of 824 samples, collected from the streets of Dhaka and Bogura performs competitively against the existing industrial and custom datasets utilized in the present literature. The dataset was further augmented four-fold for segmentation and ten-fold for classification evaluation. We tested nine classification models (CCT, CNN, INN, Swin Transformer, ConvMixer, VGG16, ResNet50, DenseNet201, and Xception) and four segmentation models (U-Net, ResU-Net, U-Net++, and Attention-Unet) over both the datasets. Among the classification models, lightweight models namely CCT, CNN, INN, Swin Transformer, and ConvMixer were emphasized due to their low computational requirements and faster prediction times. The lightweight models performed respectfully, oftentimes equating to the performance of heavyweight models. In addition, augmentation was found to enhance the performance of all the tested models. The experimental results exhibit that, our dataset performs on par or outperforms the similar classification models utilized in the existing literature, reaching accuracy and f1-scores over 99%. The dataset also performed on par with the existing datasets for segmentation, achieving model Dice Similarity Coefficient up to 67.54% and IoU scores up to 59.39%.

Related papers

• DataDream: Few-shot Guided Dataset Generation [90.09164461462365]
  We propose a framework for synthesizing classification datasets that more faithfully represents the real data distribution. DataDream fine-tunes LoRA weights for the image generation model on the few real images before generating the training data using the adapted model. We then fine-tune LoRA weights for CLIP using the synthetic data to improve downstream image classification over previous approaches on a large variety of datasets.
  arXiv  Detail & Related papers  (2024-07-15T17:10:31Z)
• A Lightweight Measure of Classification Difficulty from Application Dataset Characteristics [4.220363193932374]
  We propose an efficient cosine similarity-based classification difficulty measure S. It is calculated from the number of classes and intra- and inter-class similarity metrics of the dataset. We show how a practitioner can use this measure to help select an efficient model 6 to 29x faster than through repeated training and testing.
  arXiv  Detail & Related papers  (2024-04-09T03:27:09Z)
• No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance [68.18779562801762]
  multimodal models require exponentially more data to achieve linear improvements in downstream "zero-shot" performance. Our study reveals an exponential need for training data which implies that the key to "zero-shot" generalization capabilities under large-scale training paradigms remains to be found.
  arXiv  Detail & Related papers  (2024-04-04T17:58:02Z)
• Feedback-guided Data Synthesis for Imbalanced Classification [10.836265321046561]
  We introduce a framework for augmenting static datasets with useful synthetic samples. We find that the samples must be close to the support of the real data of the task at hand, and be sufficiently diverse. On ImageNet-LT, we achieve state-of-the-art results, with over 4 percent improvement on underrepresented classes.
  arXiv  Detail & Related papers  (2023-09-29T21:47:57Z)
• A Meta-Learning Approach to Predicting Performance and Data Requirements [163.4412093478316]
  We propose an approach to estimate the number of samples required for a model to reach a target performance. We find that the power law, the de facto principle to estimate model performance, leads to large error when using a small dataset. We introduce a novel piecewise power law (PPL) that handles the two data differently.
  arXiv  Detail & Related papers  (2023-03-02T21:48:22Z)
• Part-Based Models Improve Adversarial Robustness [57.699029966800644]
  We show that combining human prior knowledge with end-to-end learning can improve the robustness of deep neural networks. Our model combines a part segmentation model with a tiny classifier and is trained end-to-end to simultaneously segment objects into parts. Our experiments indicate that these models also reduce texture bias and yield better robustness against common corruptions and spurious correlations.
  arXiv  Detail & Related papers  (2022-09-15T15:41:47Z)
• Machine Learning Models in Stock Market Prediction [0.0]
  The paper focuses on predicting the Nifty 50 Index by using 8 Supervised Machine Learning Models. Experiments are based on historical data of Nifty 50 Index of Indian Stock Market from 22nd April, 1996 to 16th April, 2021.
  arXiv  Detail & Related papers  (2022-02-06T10:33:42Z)
• MSeg: A Composite Dataset for Multi-domain Semantic Segmentation [100.17755160696939]
  We present MSeg, a composite dataset that unifies semantic segmentation datasets from different domains. We reconcile the generalization and bring the pixel-level annotations into alignment by relabeling more than 220,000 object masks in more than 80,000 images. A model trained on MSeg ranks first on the WildDash-v1 leaderboard for robust semantic segmentation, with no exposure to WildDash data during training.
  arXiv  Detail & Related papers  (2021-12-27T16:16:35Z)
• Comparing Test Sets with Item Response Theory [53.755064720563]
  We evaluate 29 datasets using predictions from 18 pretrained Transformer models on individual test examples. We find that Quoref, HellaSwag, and MC-TACO are best suited for distinguishing among state-of-the-art models. We also observe span selection task format, which is used for QA datasets like QAMR or SQuAD2.0, is effective in differentiating between strong and weak models.
  arXiv  Detail & Related papers  (2021-06-01T22:33:53Z)
• A pipeline for fair comparison of graph neural networks in node classification tasks [4.418753792543564]
  Graph neural networks (GNNs) have been investigated for potential applicability in multiple fields that employ graph data. There are no standard training settings to ensure fair comparisons among new methods. We introduce a standard, reproducible benchmark to which the same training settings can be applied for node classification.
  arXiv  Detail & Related papers  (2020-12-19T07:43:05Z)
• A Simple and Efficient Ensemble Classifier Combining Multiple Neural Network Models on Social Media Datasets in Vietnamese [2.7528170226206443]
  This study aims to classify Vietnamese texts on social media from three different Vietnamese benchmark datasets. Advanced deep learning models are used and optimized in this study, including CNN, LSTM, and their variants. Our ensemble model achieves the best performance on all three datasets.
  arXiv  Detail & Related papers  (2020-09-28T04:28:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.