Entity6K: A Large Open-Domain Evaluation Dataset for Real-World Entity Recognition

• URL: http://arxiv.org/abs/2403.12339v1
• Date: Tue, 19 Mar 2024 01:07:53 GMT
• Title: Entity6K: A Large Open-Domain Evaluation Dataset for Real-World Entity Recognition
• Authors: Jielin Qiu, William Han, Winfred Wang, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Christos Faloutsos, Lei Li, Lijuan Wang,
• Abstract summary: We introduce Entity6K, a comprehensive dataset for real-world entity recognition. It features 5,700 entities across 26 categories, each supported by 5 human-verified images with annotations.
• Score: 100.39728263079736
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open-domain real-world entity recognition is essential yet challenging, involving identifying various entities in diverse environments. The lack of a suitable evaluation dataset has been a major obstacle in this field due to the vast number of entities and the extensive human effort required for data curation. We introduce Entity6K, a comprehensive dataset for real-world entity recognition, featuring 5,700 entities across 26 categories, each supported by 5 human-verified images with annotations. Entity6K offers a diverse range of entity names and categorizations, addressing a gap in existing datasets. We conducted benchmarks with existing models on tasks like image captioning, object detection, zero-shot classification, and dense captioning to demonstrate Entity6K's effectiveness in evaluating models' entity recognition capabilities. We believe Entity6K will be a valuable resource for advancing accurate entity recognition in open-domain settings.

Related papers

• Web-Scale Visual Entity Recognition: An LLM-Driven Data Approach [56.55633052479446]
  Web-scale visual entity recognition presents significant challenges due to the lack of clean, large-scale training data. We propose a novel methodology to curate such a dataset, leveraging a multimodal large language model (LLM) for label verification, metadata generation, and rationale explanation. Experiments demonstrate that models trained on this automatically curated data achieve state-of-the-art performance on web-scale visual entity recognition tasks.
  arXiv  Detail & Related papers  (2024-10-31T06:55:24Z)
• LLM-DER:A Named Entity Recognition Method Based on Large Language Models for Chinese Coal Chemical Domain [4.639851504108679]
  We propose a Large Language Models (LLMs)-based entity recognition framework LLM-DER for the domain-specific entity recognition problem in Chinese. LLMs-DER generates a list of relationships containing entity types through LLMs, and designs a plausibility and consistency evaluation method to remove misrecognized entities. The experimental results of this paper on the Resume dataset and the self-constructed coal chemical dataset Coal show that LLM-DER performs outstandingly in domain-specific entity recognition.
  arXiv  Detail & Related papers  (2024-09-16T08:28:05Z)
• An Attribute-Enriched Dataset and Auto-Annotated Pipeline for Open Detection [7.531866919805308]
  We introduce the Objects365-Attr dataset, an extension of the existing Objects365 dataset, distinguished by its attribute annotations. This dataset reduces inconsistencies in object detection by integrating a broad spectrum of attributes, including color, material, state, texture and tone. It contains an extensive collection of 5.6M object-level attribute descriptions, meticulously annotated across 1.4M bounding boxes.
  arXiv  Detail & Related papers  (2024-09-10T07:53:32Z)
• Pedestrian Attribute Recognition: A New Benchmark Dataset and A Large Language Model Augmented Framework [15.991114464911844]
  In the past five years, no large-scale dataset has been opened to the public. This paper proposes a new large-scale, cross-domain pedestrian attribute recognition dataset, MSP60K. It consists of 60,122 images and 57 attribute annotations across eight scenarios.
  arXiv  Detail & Related papers  (2024-08-19T06:19:31Z)
• Bayesian Detector Combination for Object Detection with Crowdsourced Annotations [49.43709660948812]
  Acquiring fine-grained object detection annotations in unconstrained images is time-consuming, expensive, and prone to noise. We propose a novel Bayesian Detector Combination (BDC) framework to more effectively train object detectors with noisy crowdsourced annotations. BDC is model-agnostic, requires no prior knowledge of the annotators' skill level, and seamlessly integrates with existing object detection models.
  arXiv  Detail & Related papers  (2024-07-10T18:00:54Z)
• Cross-domain Recommender Systems via Multimodal Domain Adaptation [2.306402684958048]
  Collaborative Filtering (CF) has emerged as one of the most prominent implementation strategies for building recommender systems. Cross-domain CF alleviates the problem of data sparsity by finding a common set of entities (users or items) across the domains. This paper introduces a domain adaptation technique to align the embeddings of entities across domains.
  arXiv  Detail & Related papers  (2023-06-24T07:27:43Z)
• Modeling Entities as Semantic Points for Visual Information Extraction in the Wild [55.91783742370978]
  We propose an alternative approach to precisely and robustly extract key information from document images. We explicitly model entities as semantic points, i.e., center points of entities are enriched with semantic information describing the attributes and relationships of different entities. The proposed method can achieve significantly enhanced performance on entity labeling and linking, compared with previous state-of-the-art models.
  arXiv  Detail & Related papers  (2023-03-23T08:21:16Z)
• Inferring Latent Domains for Unsupervised Deep Domain Adaptation [54.963823285456925]
  Unsupervised Domain Adaptation (UDA) refers to the problem of learning a model in a target domain where labeled data are not available. This paper introduces a novel deep architecture which addresses the problem of UDA by automatically discovering latent domains in visual datasets. We evaluate our approach on publicly available benchmarks, showing that it outperforms state-of-the-art domain adaptation methods.
  arXiv  Detail & Related papers  (2021-03-25T14:33:33Z)
• Entity-Switched Datasets: An Approach to Auditing the In-Domain Robustness of Named Entity Recognition Models [49.878051587667244]
  We propose a method for auditing the in-domain robustness of systems, focusing specifically on differences in performance due to the national origin of entities. We create entity-switched datasets, in which named entities in the original texts are replaced by plausible named entities of the same type but of different national origin. We find that state-of-the-art systems' performance vary widely even in-domain: In the same context, entities from certain origins are more reliably recognized than entities from elsewhere.
  arXiv  Detail & Related papers  (2020-04-08T17:11:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.