Assessing Annotation Accuracy in Ice Sheets Using Quantitative Metrics
- URL: http://arxiv.org/abs/2407.09535v1
- Date: Wed, 26 Jun 2024 04:43:51 GMT
- Title: Assessing Annotation Accuracy in Ice Sheets Using Quantitative Metrics
- Authors: Bayu Adhi Tama, Vandana Janeja, Sanjay Purushotham,
- Abstract summary: This study addresses the need for accurate ice sheet data interpretation by introducing a suite of quantitative metrics designed to validate ice sheet annotation techniques.
Our methodology incorporates several computer vision metrics, traditionally underutilized in glaciological research, to evaluate the continuity and connectivity of ice layer annotations.
- Score: 10.770434484584342
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The increasing threat of sea level rise due to climate change necessitates a deeper understanding of ice sheet structures. This study addresses the need for accurate ice sheet data interpretation by introducing a suite of quantitative metrics designed to validate ice sheet annotation techniques. Focusing on both manual and automated methods, including ARESELP and its modified version, MARESELP, we assess their accuracy against expert annotations. Our methodology incorporates several computer vision metrics, traditionally underutilized in glaciological research, to evaluate the continuity and connectivity of ice layer annotations. The results demonstrate that while manual annotations provide invaluable expert insights, automated methods, particularly MARESELP, improve layer continuity and alignment with expert labels.
Related papers
- Partial Label Learning with Focal Loss for Sea Ice Classification Based on Ice Charts [2.0270474485799017]
We present a novel GeoAI approach to training sea ice classification by formalizing it as a partial label learning task with explicit confidence scores.
We treat the polygon-level labels as candidate partial labels, assign the corresponding ice concentrations as confidence scores to each label, and integrate them with focal loss to train a Convolutional Neural Network (CNN)
Our proposed approach leads to enhanced performance for sea ice classification in Sentinel-1 dual-polarized SAR images, improving classification accuracy (from 87% to 92%) and weighted average F-1 score (from 90% to 93%) compared to the conventional training approach.
arXiv Detail & Related papers (2024-06-05T22:49:30Z) - Global-Local Detail Guided Transformer for Sea Ice Recognition in Optical Remote Sensing Images [4.540236408836131]
A Global-Local Detail Guided Transformer (GDGT) method is proposed for sea ice recognition in optical remote sensing images.
In GDGT, a global-local feature fusiont mechanism is designed to fuse global structural correlation features and local spatial detail features.
A detail-guided decoder is developed to retain more high-resolution detail information during feature reconstruction for improving the performance of sea ice recognition.
arXiv Detail & Related papers (2024-05-21T21:02:20Z) - Region-level labels in ice charts can produce pixel-level segmentation for Sea Ice types [12.480532138980834]
We present a weakly supervised learning method for sea ice classification with lower-resolution labels from expert-annotated ice charts.
Our method outperforms the fully supervised U-Net benchmark in both mapping resolution and class-wise accuracy.
arXiv Detail & Related papers (2024-05-16T21:54:33Z) - Online Feature Updates Improve Online (Generalized) Label Shift
Adaptation [54.3888105557787]
This paper addresses the prevalent issue of label shift in an online setting with missing labels.
We explore the untapped potential of enhancing feature representations using unlabeled data at test-time.
Our method, Online Label Shift adaptation with Online Feature Updates (OLS-OFU), leverages self-supervised learning to refine the feature extraction process.
arXiv Detail & Related papers (2024-02-05T22:03:25Z) - Enhancing Post-Hoc Explanation Benchmark Reliability for Image
Classification [0.0]
Empirical evaluations demonstrate significant improvements in benchmark reliability across metrics, datasets, and post-hoc methods.
This pioneering work establishes a foundation for more reliable evaluation practices in the realm of post-hoc explanation methods.
arXiv Detail & Related papers (2023-11-29T18:21:24Z) - Goodhart's Law Applies to NLP's Explanation Benchmarks [57.26445915212884]
We critically examine two sets of metrics: the ERASER metrics (comprehensiveness and sufficiency) and the EVAL-X metrics.
We show that we can inflate a model's comprehensiveness and sufficiency scores dramatically without altering its predictions or explanations on in-distribution test inputs.
Our results raise doubts about the ability of current metrics to guide explainability research, underscoring the need for a broader reassessment of what precisely these metrics are intended to capture.
arXiv Detail & Related papers (2023-08-28T03:03:03Z) - An Experimental Investigation into the Evaluation of Explainability
Methods [60.54170260771932]
This work compares 14 different metrics when applied to nine state-of-the-art XAI methods and three dummy methods (e.g., random saliency maps) used as references.
Experimental results show which of these metrics produces highly correlated results, indicating potential redundancy.
arXiv Detail & Related papers (2023-05-25T08:07:07Z) - ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning [63.77667876176978]
Large language models show improved downstream task interpretability when prompted to generate step-by-step reasoning to justify their final answers.
These reasoning steps greatly improve model interpretability and verification, but objectively studying their correctness is difficult.
We present ROS, a suite of interpretable, unsupervised automatic scores that improve and extend previous text generation evaluation metrics.
arXiv Detail & Related papers (2022-12-15T15:52:39Z) - FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality
Assessment [93.09267863425492]
We argue that understanding both high-level semantics and internal temporal structures of actions in competitive sports videos is the key to making predictions accurate and interpretable.
We construct a new fine-grained dataset, called FineDiving, developed on diverse diving events with detailed annotations on action procedures.
arXiv Detail & Related papers (2022-04-07T17:59:32Z) - Simulating surface height and terminus position for marine outlet
glaciers using a level set method with data assimilation [0.0]
We implement a data assimilation framework for integrating ice surface and terminus position observations into a numerical ice-flow model.
The model is also applied to simulate Helheim Glacier, a major tidewater-terminating glacier of the Greenland Ice Sheet.
arXiv Detail & Related papers (2022-01-28T16:45:37Z) - Deep Semi-supervised Knowledge Distillation for Overlapping Cervical
Cell Instance Segmentation [54.49894381464853]
We propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation.
We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining.
Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only.
arXiv Detail & Related papers (2020-07-21T13:27:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.