Multi-Modal Knowledge Graph Construction and Application: A Survey

• URL: http://arxiv.org/abs/2202.05786v1
• Date: Fri, 11 Feb 2022 17:31:12 GMT
• Title: Multi-Modal Knowledge Graph Construction and Application: A Survey
• Authors: Xiangru Zhu, Zhixu Li, Xiaodan Wang, Xueyao Jiang, Penglei Sun, Xuwu Wang, Yanghua Xiao, Nicholas Jing Yuan
• Abstract summary: Multi-modalization of knowledge graphs is an inevitable key step towards the realization of human-level machine intelligence. We first give definitions of MMKGs constructed by texts and images, followed with the preliminaries on multi-modal tasks and techniques. We then systematically review the challenges, progresses and opportunities on the construction and application of MMKGs respectively, with detailed analyses of the strength and weakness of different solutions.
• Score: 17.203534055251435
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent years have witnessed the resurgence of knowledge engineering which is featured by the fast growth of knowledge graphs. However, most of existing knowledge graphs are represented with pure symbols, which hurts the machine's capability to understand the real world. The multi-modalization of knowledge graphs is an inevitable key step towards the realization of human-level machine intelligence. The results of this endeavor are Multi-modal Knowledge Graphs (MMKGs). In this survey on MMKGs constructed by texts and images, we first give definitions of MMKGs, followed with the preliminaries on multi-modal tasks and techniques. We then systematically review the challenges, progresses and opportunities on the construction and application of MMKGs respectively, with detailed analyses of the strength and weakness of different solutions. We finalize this survey with open research problems relevant to MMKGs.

Related papers

• Knowledge Editing with Dynamic Knowledge Graphs for Multi-Hop Question Answering [31.676874886966925]
  Multi-hop question answering (MHQA) poses a significant challenge for large language models (LLMs) Knowledge editing offers a potential solution for addressing MHQA challenges with LLMs. Most parameter-preserving editing methods are hindered by inaccurate retrieval and overlook secondary editing issues. This paper introduces KEDKG, a novel knowledge editing method that leverages a dynamic knowledge graph for MHQA.
  arXiv  Detail & Related papers  (2024-12-18T12:21:46Z)
• Neural-Symbolic Reasoning over Knowledge Graphs: A Survey from a Query Perspective [55.79507207292647]
  Knowledge graph reasoning is pivotal in various domains such as data mining, artificial intelligence, the Web, and social sciences. The rise of Neural AI marks a significant advancement, merging the robustness of deep learning with the precision of symbolic reasoning. The advent of large language models (LLMs) has opened new frontiers in knowledge graph reasoning.
  arXiv  Detail & Related papers  (2024-11-30T18:54:08Z)
• Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey [61.8716670402084]
  This survey focuses on KG-aware research in two principal aspects: KG-driven Multi-Modal (KG4MM) learning, and Multi-Modal Knowledge Graph (MM4KG) Our review includes two primary task categories: KG-aware multi-modal learning tasks, and intrinsic MMKG tasks. For most of these tasks, we provide definitions, evaluation benchmarks, and additionally outline essential insights for conducting relevant research.
  arXiv  Detail & Related papers  (2024-02-08T04:04:36Z)
• On the Evolution of Knowledge Graphs: A Survey and Perspective [11.061075842989817]
  Knowledge graphs (KGs) are structured representations of diversified knowledge. They are widely used in various intelligent applications. We provide a comprehensive survey on the evolution of various types of KGs and techniques for knowledge extraction and reasoning. We propose our perspective on the future directions of knowledge engineering.
  arXiv  Detail & Related papers  (2023-10-07T14:46:51Z)
• A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal [57.8455911689554]
  Knowledge graph reasoning (KGR) aims to deduce new facts from existing facts based on mined logic rules underlying knowledge graphs (KGs) It has been proven to significantly benefit the usage of KGs in many AI applications, such as question answering, recommendation systems, and etc.
  arXiv  Detail & Related papers  (2022-12-12T08:40:04Z)
• A Survey on Visual Transfer Learning using Knowledge Graphs [0.8701566919381223]
  This survey focuses on visual transfer learning approaches using knowledge graphs (KGs) KGs can represent auxiliary knowledge either in an underlying graph-structured schema or in a vector-based knowledge graph embedding. We provide a broad overview of knowledge graph embedding methods and describe several joint training objectives suitable to combine them with high dimensional visual embeddings.
  arXiv  Detail & Related papers  (2022-01-27T20:19:55Z)
• Dynamic Semantic Graph Construction and Reasoning for Explainable Multi-hop Science Question Answering [50.546622625151926]
  We propose a new framework to exploit more valid facts while obtaining explainability for multi-hop QA. Our framework contains three new ideas: (a) tt AMR-SG, an AMR-based Semantic Graph, constructed by candidate fact AMRs to uncover any hop relations among question, answer and multiple facts, (b) a novel path-based fact analytics approach exploiting tt AMR-SG to extract active facts from a large fact pool to answer questions, and (c) a fact-level relation modeling leveraging graph convolution network (GCN) to guide the reasoning process.
  arXiv  Detail & Related papers  (2021-05-25T09:14:55Z)
• KACC: A Multi-task Benchmark for Knowledge Abstraction, Concretization and Completion [99.47414073164656]
  A comprehensive knowledge graph (KG) contains an instance-level entity graph and an ontology-level concept graph. The two-view KG provides a testbed for models to "simulate" human's abilities on knowledge abstraction, concretization, and completion. We propose a unified KG benchmark by improving existing benchmarks in terms of dataset scale, task coverage, and difficulty.
  arXiv  Detail & Related papers  (2020-04-28T16:21:57Z)
• A Survey on Knowledge Graphs: Representation, Acquisition and Applications [89.78089494738002]
  We review research topics about 1) knowledge graph representation learning, 2) knowledge acquisition and completion, 3) temporal knowledge graph, and 4) knowledge-aware applications. For knowledge acquisition, especially knowledge graph completion, embedding methods, path inference, and logical rule reasoning, are reviewed. We explore several emerging topics, including meta learning, commonsense reasoning, and temporal knowledge graphs.
  arXiv  Detail & Related papers  (2020-02-02T13:17:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.