ShiftedBronzes: Benchmarking and Analysis of Domain Fine-Grained Classification in Open-World Settings

• URL: http://arxiv.org/abs/2412.12683v1
• Date: Tue, 17 Dec 2024 08:56:59 GMT
• Title: ShiftedBronzes: Benchmarking and Analysis of Domain Fine-Grained Classification in Open-World Settings
• Authors: Rixin Zhou, Honglin Pang, Qian Zhang, Ruihua Qi, Xi Yang, Chuntao Li,
• Abstract summary: ShiftedBronzes is a benchmark dataset for fine-grained bronze ware dating.<n> ShiftedBronzes incorporates two types of bronze ware data and seven types of OOD data.<n>We conduct experiments on ShiftedBronzes and five commonly used general OOD datasets.
• Score: 5.144974645567005
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In real-world applications across specialized domains, addressing complex out-of-distribution (OOD) challenges is a common and significant concern. In this study, we concentrate on the task of fine-grained bronze ware dating, a critical aspect in the study of ancient Chinese history, and developed a benchmark dataset named ShiftedBronzes. By extensively expanding the bronze Ding dataset, ShiftedBronzes incorporates two types of bronze ware data and seven types of OOD data, which exhibit distribution shifts commonly encountered in bronze ware dating scenarios. We conduct benchmarking experiments on ShiftedBronzes and five commonly used general OOD datasets, employing a variety of widely adopted post-hoc, pre-trained Vision Large Model (VLM)-based and generation-based OOD detection methods. Through analysis of the experimental results, we validate previous conclusions regarding post-hoc, VLM-based, and generation-based methods, while also highlighting their distinct behaviors on specialized datasets. These findings underscore the unique challenges of applying general OOD detection methods to domain-specific tasks such as bronze ware dating. We hope that the ShiftedBronzes benchmark provides valuable insights into both the field of bronze ware dating and the and the development of OOD detection methods. The dataset and associated code will be available later.

Related papers

• Graph Out-of-Distribution Detection via Test-Time Calibration with Dual Dynamic Dictionaries [15.43092254877282]
  Key challenge in graph out-of-distribution (OOD) detection lies in the absence of ground-truth OOD samples during training.<n>We propose a novel test-time graph OOD detection method, termed BaCa, that calibrates OOD scores using dual dynamically updated dictionaries.<n>BaCa significantly outperforms existing state-of-the-art methods in OOD detection.
  arXiv  Detail & Related papers  (2025-11-17T16:14:48Z)
• WeiPer: OOD Detection using Weight Perturbations of Class Projections [11.130659240045544]
  We introduce perturbations of the class projections in the final fully connected layer which creates a richer representation of the input. We achieve state-of-the-art OOD detection results across multiple benchmarks of the OpenOOD framework.
  arXiv  Detail & Related papers  (2024-05-27T13:38:28Z)
• EAT: Towards Long-Tailed Out-of-Distribution Detection [55.380390767978554]
  This paper addresses the challenging task of long-tailed OOD detection. The main difficulty lies in distinguishing OOD data from samples belonging to the tail classes. We propose two simple ideas: (1) Expanding the in-distribution class space by introducing multiple abstention classes, and (2) Augmenting the context-limited tail classes by overlaying images onto the context-rich OOD data.
  arXiv  Detail & Related papers  (2023-12-14T13:47:13Z)
• ExCeL : Combined Extreme and Collective Logit Information for Enhancing Out-of-Distribution Detection [9.689089164964484]
  ExCeL combines extreme and collective information within the output layer for enhanced accuracy in OOD detection. We show that ExCeL consistently is among the five top-performing methods out of twenty-one existing post-hoc baselines.
  arXiv  Detail & Related papers  (2023-11-23T14:16:03Z)
• DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization [58.704753031608625]
  Time series is one of the most challenging modalities in machine learning research. OOD detection and generalization on time series tend to suffer due to its non-stationary property. We propose DIVERSIFY, a framework for OOD detection and generalization on dynamic distributions of time series.
  arXiv  Detail & Related papers  (2023-08-04T12:27:11Z)
• General-Purpose Multi-Modal OOD Detection Framework [5.287829685181842]
  Out-of-distribution (OOD) detection identifies test samples that differ from the training data, which is critical to ensuring the safety and reliability of machine learning (ML) systems. We propose a general-purpose weakly-supervised OOD detection framework, called WOOD, that combines a binary classifier and a contrastive learning component. We evaluate the proposed WOOD model on multiple real-world datasets, and the experimental results demonstrate that the WOOD model outperforms the state-of-the-art methods for multi-modal OOD detection.
  arXiv  Detail & Related papers  (2023-07-24T18:50:49Z)
• Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations [111.88727295707454]
  This paper reexamines the research on out-of-distribution (OOD) robustness in the field of NLP. We propose a benchmark construction protocol that ensures clear differentiation and challenging distribution shifts. We conduct experiments on pre-trained language models for analysis and evaluation of OOD robustness.
  arXiv  Detail & Related papers  (2023-06-07T17:47:03Z)
• A Survey on Out-of-Distribution Detection in NLP [119.80687868012393]
  Out-of-distribution (OOD) detection is essential for the reliable and safe deployment of machine learning systems in the real world. This paper presents the first review of recent advances in OOD detection with a particular focus on natural language processing approaches.
  arXiv  Detail & Related papers  (2023-05-05T01:38:49Z)
• Multi-Granularity Archaeological Dating of Chinese Bronze Dings Based on a Knowledge-Guided Relation Graph [5.359415272318481]
  Current archaeology depends on trained experts to carry out bronze dating. We propose a learning-based approach to integrate advanced deep learning techniques and archaeological knowledge.
  arXiv  Detail & Related papers  (2023-03-27T14:54:50Z)
• Pseudo-OOD training for robust language models [78.15712542481859]
  OOD detection is a key component of a reliable machine-learning model for any industry-scale application. We propose POORE - POsthoc pseudo-Ood REgularization, that generates pseudo-OOD samples using in-distribution (IND) data. We extensively evaluate our framework on three real-world dialogue systems, achieving new state-of-the-art in OOD detection.
  arXiv  Detail & Related papers  (2022-10-17T14:32:02Z)
• OpenOOD: Benchmarking Generalized Out-of-Distribution Detection [60.13300701826931]
  Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications. The field currently lacks a unified, strictly formulated, and comprehensive benchmark. We build a unified, well-structured called OpenOOD, which implements over 30 methods developed in relevant fields.
  arXiv  Detail & Related papers  (2022-10-13T17:59:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.