How to Understand Named Entities: Using Common Sense for News Captioning

• URL: http://arxiv.org/abs/2403.06520v1
• Date: Mon, 11 Mar 2024 08:52:52 GMT
• Title: How to Understand Named Entities: Using Common Sense for News Captioning
• Authors: Ning Xu, Yanhui Wang, Tingting Zhang, Hongshuo Tian, Mohan Kankanhalli, An-An Liu
• Abstract summary: News captioning aims to describe an image with its news article body as input. This paper exploits commonsense knowledge to understand named entities for news captioning.
• Score: 34.10048889674029
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: News captioning aims to describe an image with its news article body as input. It greatly relies on a set of detected named entities, including real-world people, organizations, and places. This paper exploits commonsense knowledge to understand named entities for news captioning. By ``understand'', we mean correlating the news content with common sense in the wild, which helps an agent to 1) distinguish semantically similar named entities and 2) describe named entities using words outside of training corpora. Our approach consists of three modules: (a) Filter Module aims to clarify the common sense concerning a named entity from two aspects: what does it mean? and what is it related to?, which divide the common sense into explanatory knowledge and relevant knowledge, respectively. (b) Distinguish Module aggregates explanatory knowledge from node-degree, dependency, and distinguish three aspects to distinguish semantically similar named entities. (c) Enrich Module attaches relevant knowledge to named entities to enrich the entity description by commonsense information (e.g., identity and social position). Finally, the probability distributions from both modules are integrated to generate the news captions. Extensive experiments on two challenging datasets (i.e., GoodNews and NYTimes) demonstrate the superiority of our method. Ablation studies and visualization further validate its effectiveness in understanding named entities.

Related papers

• Nested Named Entity Recognition from Medical Texts: An Adaptive Shared Network Architecture with Attentive CRF [53.55504611255664]
  We propose a novel method, referred to as ASAC, to solve the dilemma caused by the nested phenomenon. The proposed method contains two key modules: the adaptive shared (AS) part and the attentive conditional random field (ACRF) module. Our model could learn better entity representations by capturing the implicit distinctions and relationships between different categories of entities.
  arXiv  Detail & Related papers  (2022-11-09T09:23:56Z)
• Generative Entity-to-Entity Stance Detection with Knowledge Graph Augmentation [7.857310305816312]
  Stance detection is typically framed as predicting the sentiment in a text towards a target entity. In this paper, we emphasize the need for studying interactions among entities when inferring stances. We first introduce a new task, entity-to-entity (E2E) stance detection, which primes models to identify entities in their canonical names and discern stances jointly.
  arXiv  Detail & Related papers  (2022-11-02T20:16:42Z)
• Ered: Enhanced Text Representations with Entities and Descriptions [5.977668609935748]
  External knowledge,e.g., entities and entity descriptions, can help humans understand texts. This paper aims to explicitly include both entities and entity descriptions in the fine-tuning stage. We conducted experiments on four knowledge-oriented tasks and two common tasks, and the results achieved new state-of-the-art on several datasets.
  arXiv  Detail & Related papers  (2022-08-18T16:51:16Z)
• Boosting Entity-aware Image Captioning with Multi-modal Knowledge Graph [96.95815946327079]
  It is difficult to learn the association between named entities and visual cues due to the long-tail distribution of named entities. We propose a novel approach that constructs a multi-modal knowledge graph to associate the visual objects with named entities.
  arXiv  Detail & Related papers  (2021-07-26T05:50:41Z)
• Improving Entity Linking through Semantic Reinforced Entity Embeddings [16.868791358905916]
  We propose a method to inject fine-grained semantic information into entity embeddings to reduce the distinctiveness and facilitate the learning of contextual commonality. Based on our entity embeddings, we achieved new sate-of-the-art performance on entity linking.
  arXiv  Detail & Related papers  (2021-06-16T00:27:56Z)
• KGSynNet: A Novel Entity Synonyms Discovery Framework with Knowledge Graph [23.053995137917994]
  We propose a novel entity synonyms discovery framework, named emphKGSynNet. Specifically, we pre-train subword embeddings for mentions and entities using a large-scale domain-specific corpus. We employ a specifically designed emphfusion gate to adaptively absorb the entities' knowledge information into their semantic features.
  arXiv  Detail & Related papers  (2021-03-16T07:32:33Z)
• Named Entity Recognition for Social Media Texts with Semantic Augmentation [70.44281443975554]
  Existing approaches for named entity recognition suffer from data sparsity problems when conducted on short and informal texts. We propose a neural-based approach to NER for social media texts where both local (from running text) and augmented semantics are taken into account.
  arXiv  Detail & Related papers  (2020-10-29T10:06:46Z)
• Edge Guided GANs with Contrastive Learning for Semantic Image Synthesis [194.1452124186117]
  We propose a novel ECGAN for the challenging semantic image synthesis task. Our ECGAN achieves significantly better results than state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-03-31T01:23:21Z)
• Symbiotic Attention with Privileged Information for Egocentric Action Recognition [71.0778513390334]
  We propose a novel Symbiotic Attention framework for egocentric video recognition. Our framework enables mutual communication among the verb branch, the noun branch, and the privileged information. Notably, it achieves the state-of-the-art on two large-scale egocentric video datasets.
  arXiv  Detail & Related papers  (2020-02-08T10:48:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.