Foundation Model or Finetune? Evaluation of few-shot semantic segmentation for river pollution

• URL: http://arxiv.org/abs/2409.03754v1
• Date: Thu, 5 Sep 2024 17:59:32 GMT
• Title: Foundation Model or Finetune? Evaluation of few-shot semantic segmentation for river pollution
• Authors: Marga Don, Stijn Pinson, Blanca Guillen Cebrian, Yuki M. Asano,
• Abstract summary: Foundation models (FMs) are a popular topic of research in AI. In this work, we compare the performance of FMs to finetuned pre-trained supervised models in the task of semantic segmentation. We see that finetuned models consistently outperform the FMs tested, even in cases were data is scarce.
• Score: 16.272314073324626
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundation models (FMs) are a popular topic of research in AI. Their ability to generalize to new tasks and datasets without retraining or needing an abundance of data makes them an appealing candidate for applications on specialist datasets. In this work, we compare the performance of FMs to finetuned pre-trained supervised models in the task of semantic segmentation on an entirely new dataset. We see that finetuned models consistently outperform the FMs tested, even in cases were data is scarce. We release the code and dataset for this work on GitHub.

Related papers

• Learning with Noisy Foundation Models [95.50968225050012]
  This paper is the first work to comprehensively understand and analyze the nature of noise in pre-training datasets. We propose a tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise and improve generalization.
  arXiv  Detail & Related papers  (2024-03-11T16:22:41Z)
• Learning Defect Prediction from Unrealistic Data [57.53586547895278]
  Pretrained models of code have become popular choices for code understanding and generation tasks. Such models tend to be large and require commensurate volumes of training data. It has become popular to train models with far larger but less realistic datasets, such as functions with artificially injected bugs. Models trained on such data tend to only perform well on similar data, while underperforming on real world programs.
  arXiv  Detail & Related papers  (2023-11-02T01:51:43Z)
• Quality > Quantity: Synthetic Corpora from Foundation Models for Closed-Domain Extractive Question Answering [35.38140071573828]
  We study extractive question answering within closed domains and introduce the concept of targeted pre-training. Our proposed framework uses Galactica to generate synthetic, targeted'' corpora that align with specific writing styles and topics.
  arXiv  Detail & Related papers  (2023-10-25T20:48:16Z)
• Learning from aggregated data with a maximum entropy model [73.63512438583375]
  We show how a new model, similar to a logistic regression, may be learned from aggregated data only by approximating the unobserved feature distribution with a maximum entropy hypothesis. We present empirical evidence on several public datasets that the model learned this way can achieve performances comparable to those of a logistic model trained with the full unaggregated data.
  arXiv  Detail & Related papers  (2022-10-05T09:17:27Z)
• Can Foundation Models Wrangle Your Data? [13.11923018654058]
  Foundation Models (FMs) are models trained on large corpora of data that can generalize to new tasks without task-specific finetuning. This paper aims to understand an underexplored area of FMs: classical data tasks like cleaning and integration. We find that large FMs generalize and achieve SoTA performance on data cleaning and integration tasks, even though they are not trained for these data tasks.
  arXiv  Detail & Related papers  (2022-05-20T00:53:43Z)
• InPars: Data Augmentation for Information Retrieval using Large Language Models [5.851846467503597]
  In this work, we harness the few-shot capabilities of large pretrained language models as synthetic data generators for information retrieval tasks. We show that models finetuned solely on our unsupervised dataset outperform strong baselines such as BM25. retrievers finetuned on both supervised and our synthetic data achieve better zero-shot transfer than models finetuned only on supervised data.
  arXiv  Detail & Related papers  (2022-02-10T16:52:45Z)
• Comparing Test Sets with Item Response Theory [53.755064720563]
  We evaluate 29 datasets using predictions from 18 pretrained Transformer models on individual test examples. We find that Quoref, HellaSwag, and MC-TACO are best suited for distinguishing among state-of-the-art models. We also observe span selection task format, which is used for QA datasets like QAMR or SQuAD2.0, is effective in differentiating between strong and weak models.
  arXiv  Detail & Related papers  (2021-06-01T22:33:53Z)
• Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics [118.75207687144817]
  We introduce Data Maps, a model-based tool to characterize and diagnose datasets. We leverage a largely ignored source of information: the behavior of the model on individual instances during training. Our results indicate that a shift in focus from quantity to quality of data could lead to robust models and improved out-of-distribution generalization.
  arXiv  Detail & Related papers  (2020-09-22T20:19:41Z)
• Adversarial Filters of Dataset Biases [96.090959788952]
  Large neural models have demonstrated human-level performance on language and vision benchmarks. Their performance degrades considerably on adversarial or out-of-distribution samples. We propose AFLite, which adversarially filters such dataset biases.
  arXiv  Detail & Related papers  (2020-02-10T21:59:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.