Building a Multimodal Dataset of Academic Paper for Keyword Extraction

• URL: http://arxiv.org/abs/2606.31069v1
• Date: Tue, 30 Jun 2026 02:57:23 GMT
• Title: Building a Multimodal Dataset of Academic Paper for Keyword Extraction
• Abstract summary: The aim is to investigate the differences in performance in keyword extraction task with respect to different modal information and the fusion of multimodal information.<n>The concatenation of paper text, image text and audio text can effectively enhance the keyword extraction performance of academic papers.
• Score: 16.902387570889335
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Up to this point, keyword extraction task typically relies solely on textual data. Neglecting visual details and audio features from image and audio modalities leads to deficiencies in information richness and overlooks potential correlations, thereby constraining the model's ability to learn representations of the data and the accuracy of model predictions. Furthermore, the currently available multimodal datasets for keyword extraction task are particularly scarce, further hindering the progress of research on multimodal keyword extraction task. Therefore, this study constructs a multimodal dataset of academic paper consisting of 1000 samples, with each sample containing paper text, images, audios and keywords. Based on unsupervised and supervised methods of keyword extraction, experiments are conducted using textual data from papers, as well as text extracted from images and audio. The aim is to investigate the differences in performance in keyword extraction task with respect to different modal information and the fusion of multimodal information. The experimental results indicate that text from different modalities exhibits distinct characteristics in the model. The concatenation of paper text, image text and audio text can effectively enhance the keyword extraction performance of academic papers.

Related papers

• ATIR: Towards Audio-Text Interleaved Contextual Retrieval [63.68521448682396]
  We introduce the Audio-Text Interleaved contextual Retrieval (ATIR) task, where queries can alternate between audio and text modalities.<n>This benchmark substantially addresses the limitations of existing audio retrieval datasets in semantic retrieval.
  arXiv  Detail & Related papers  (2026-04-22T07:11:58Z)
• Enhancing Unsupervised Keyword Extraction in Academic Papers through Integrating Highlights with Abstract [7.006270460190784]
  This paper focuses on the highlights section - a summary describing the key findings and contributions.<n>Our observations indicate that highlights contain valuable keyword information that can effectively complement the abstract.<n>Experiments conducted with four unsupervised models on Computer Science (CS), Library and Information Science (LIS) datasets.
  arXiv  Detail & Related papers  (2026-04-21T14:22:21Z)
• Towards Text-Image Interleaved Retrieval [49.96332254241075]
  We introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences.<n>We construct a TIIR benchmark based on naturally interleaved wikiHow tutorials, where a specific pipeline is designed to generate interleaved queries.<n>We propose a novel Matryoshka Multimodal Embedder (MME), which compresses the number of visual tokens at different granularity.
  arXiv  Detail & Related papers  (2025-02-18T12:00:47Z)
• OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models [122.27878464009181]
  We conducted a comprehensive evaluation of Large Multimodal Models, such as GPT4V and Gemini, in various text-related visual tasks. OCRBench contains 29 datasets, making it the most comprehensive OCR evaluation benchmark available.
  arXiv  Detail & Related papers  (2023-05-13T11:28:37Z)
• Improving Keyphrase Extraction with Data Augmentation and Information Filtering [67.43025048639333]
  Keyphrase extraction is one of the essential tasks for document understanding in NLP. We present a novel corpus and method for keyphrase extraction from the videos streamed on the Behance platform.
  arXiv  Detail & Related papers  (2022-09-11T22:38:02Z)
• TRIE++: Towards End-to-End Information Extraction from Visually Rich Documents [51.744527199305445]
  This paper proposes a unified end-to-end information extraction framework from visually rich documents. Text reading and information extraction can reinforce each other via a well-designed multi-modal context block. The framework can be trained in an end-to-end trainable manner, achieving global optimization.
  arXiv  Detail & Related papers  (2022-07-14T08:52:07Z)
• Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets [56.018551958004814]
  This paper addresses the task of generating fluent descriptions by training on a non-uniform combination of data sources. Large-scale datasets with noisy image-text pairs provide a sub-optimal source of supervision. We propose to leverage and separate semantics and descriptive style through the incorporation of a style token and keywords extracted through a retrieval component.
  arXiv  Detail & Related papers  (2021-11-24T19:00:05Z)
• Multi-Document Keyphrase Extraction: A Literature Review and the First Dataset [24.91326715164367]
  Multi-document keyphrase extraction has been infrequently studied, despite its utility for describing sets of documents. We present here the first literature review and the first dataset for the task, MK-DUC-01, which can serve as a new benchmark.
  arXiv  Detail & Related papers  (2021-10-03T19:10:28Z)
• A Joint Learning Approach based on Self-Distillation for Keyphrase Extraction from Scientific Documents [29.479331909227998]
  Keyphrase extraction is the task of extracting a small set of phrases that best describe a document. Most existing benchmark datasets for the task typically have limited numbers of annotated documents. We propose a simple and efficient joint learning approach based on the idea of self-distillation.
  arXiv  Detail & Related papers  (2020-10-22T18:36:31Z)
• TRIE: End-to-End Text Reading and Information Extraction for Document Understanding [56.1416883796342]
  We propose a unified end-to-end text reading and information extraction network. multimodal visual and textual features of text reading are fused for information extraction. Our proposed method significantly outperforms the state-of-the-art methods in both efficiency and accuracy.
  arXiv  Detail & Related papers  (2020-05-27T01:47:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.