Towards Visual Taxonomy Expansion

• URL: http://arxiv.org/abs/2309.06105v1
• Date: Tue, 12 Sep 2023 10:17:28 GMT
• Title: Towards Visual Taxonomy Expansion
• Authors: Tinghui Zhu, Jingping Liu, Jiaqing Liang, Haiyun Jiang, Yanghua Xiao, Zongyu Wang, Rui Xie, Yunsen Xian
• Abstract summary: We propose Visual Taxonomy Expansion (VTE), introducing visual features into the taxonomy expansion task. We propose a textual hypernymy learning task and a visual prototype learning task to cluster textual and visual semantics. Our method is evaluated on two datasets, where we obtain compelling results.
• Score: 50.462998483087915
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Taxonomy expansion task is essential in organizing the ever-increasing volume of new concepts into existing taxonomies. Most existing methods focus exclusively on using textual semantics, leading to an inability to generalize to unseen terms and the "Prototypical Hypernym Problem." In this paper, we propose Visual Taxonomy Expansion (VTE), introducing visual features into the taxonomy expansion task. We propose a textual hypernymy learning task and a visual prototype learning task to cluster textual and visual semantics. In addition to the tasks on respective modalities, we introduce a hyper-proto constraint that integrates textual and visual semantics to produce fine-grained visual semantics. Our method is evaluated on two datasets, where we obtain compelling results. Specifically, on the Chinese taxonomy dataset, our method significantly improves accuracy by 8.75 %. Additionally, our approach performs better than ChatGPT on the Chinese taxonomy dataset.

Related papers

• CodeTaxo: Enhancing Taxonomy Expansion with Limited Examples via Code Language Prompts [40.52605902842168]
  textscCodeTaxo is a novel approach that leverages large language models through code language prompts to capture the taxonomic structure. Experiments on five real-world benchmarks from different domains demonstrate that textscCodeTaxo consistently achieves superior performance across all evaluation metrics.
  arXiv  Detail & Related papers  (2024-08-17T02:15:07Z)
• Boosting Video-Text Retrieval with Explicit High-Level Semantics [115.66219386097295]
  We propose a novel visual-linguistic aligning model named HiSE for VTR. It improves the cross-modal representation by incorporating explicit high-level semantics. Our method achieves the superior performance over state-of-the-art methods on three benchmark datasets.
  arXiv  Detail & Related papers  (2022-08-08T15:39:54Z)
• TaxoEnrich: Self-Supervised Taxonomy Completion via Structure-Semantic Representations [28.65753036636082]
  We propose a new taxonomy completion framework, which effectively leverages both semantic features and structural information in the existing taxonomy. TaxoEnrich consists of four components: (1) taxonomy-contextualized embedding which incorporates both semantic meanings of concept and taxonomic relations based on powerful pretrained language models; (2) a taxonomy-aware sequential encoder which learns candidate position representations by encoding the structural information of taxonomy. Experiments on four large real-world datasets from different domains show that TaxoEnrich achieves the best performance among all evaluation metrics and outperforms previous state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2022-02-10T08:10:43Z)
• Taxonomy Enrichment with Text and Graph Vector Representations [61.814256012166794]
  We address the problem of taxonomy enrichment which aims at adding new words to the existing taxonomy. We present a new method that allows achieving high results on this task with little effort. We achieve state-of-the-art results across different datasets and provide an in-depth error analysis of mistakes.
  arXiv  Detail & Related papers  (2022-01-21T09:01:12Z)
• TaxoCom: Topic Taxonomy Completion with Hierarchical Discovery of Novel Topic Clusters [57.59286394188025]
  We propose a novel framework for topic taxonomy completion, named TaxoCom. TaxoCom discovers novel sub-topic clusters of terms and documents. Our comprehensive experiments on two real-world datasets demonstrate that TaxoCom not only generates the high-quality topic taxonomy in terms of term coherency and topic coverage.
  arXiv  Detail & Related papers  (2022-01-18T07:07:38Z)
• Who Should Go First? A Self-Supervised Concept Sorting Model for Improving Taxonomy Expansion [50.794640012673064]
  As data and business scope grow in real applications, existing need to be expanded to incorporate new concepts. Previous works on taxonomy expansion process the new concepts independently and simultaneously, ignoring the potential relationships among them and the appropriate order of inserting operations. We propose TaxoOrder, a novel self-supervised framework that simultaneously discovers the local hypernym-hyponym structure among new concepts and decides the order of insertion.
  arXiv  Detail & Related papers  (2021-04-08T11:00:43Z)
• Can Taxonomy Help? Improving Semantic Question Matching using Question Taxonomy [37.57300969050908]
  We propose a hybrid technique for semantic question matching. It uses our proposed two-layered taxonomy for English questions by augmenting state-of-the-art deep learning models with question classes obtained from a deep learning based question.
  arXiv  Detail & Related papers  (2021-01-20T16:23:04Z)
• Taxonomy Completion via Triplet Matching Network [18.37146040410778]
  We formulate a new task, "taxonomy completion", by discovering both the hypernym and hyponym concepts for a query. We propose Triplet Matching Network (TMN), to find the appropriate hypernym, hyponym> pairs for a given query concept. TMN achieves the best performance on both taxonomy completion task and the previous taxonomy expansion task, outperforming existing methods.
  arXiv  Detail & Related papers  (2021-01-06T07:19:55Z)
• TaxoExpan: Self-supervised Taxonomy Expansion with Position-Enhanced Graph Neural Network [62.12557274257303]
  Taxonomies consist of machine-interpretable semantics and provide valuable knowledge for many web applications. We propose a novel self-supervised framework, named TaxoExpan, which automatically generates a set of query concept, anchor concept> pairs from the existing taxonomy as training data. We develop two innovative techniques in TaxoExpan: (1) a position-enhanced graph neural network that encodes the local structure of an anchor concept in the existing taxonomy, and (2) a noise-robust training objective that enables the learned model to be insensitive to the label noise in the self-supervision data.
  arXiv  Detail & Related papers  (2020-01-26T21:30:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.