Detecting Legend Items on Historical Maps Using GPT-4o with In-Context Learning

• URL: http://arxiv.org/abs/2510.08385v1
• Date: Thu, 09 Oct 2025 16:08:48 GMT
• Title: Detecting Legend Items on Historical Maps Using GPT-4o with In-Context Learning
• Abstract summary: We present a method that combines LayoutLMv3 for layout detection with GPT-4o using in-context learning.<n>Our experiments show that GPT-4 with structured prompts outperforms the baseline, achieving 88% F-1 and 85% IoU.<n>This approach supports scalable, layout-aware legend parsing and improves the indexing and searchability of historical maps across various visual styles.
• Score: 5.267675589648353
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Historical map legends are critical for interpreting cartographic symbols. However, their inconsistent layouts and unstructured formats make automatic extraction challenging. Prior work focuses primarily on segmentation or general optical character recognition (OCR), with few methods effectively matching legend symbols to their corresponding descriptions in a structured manner. We present a method that combines LayoutLMv3 for layout detection with GPT-4o using in-context learning to detect and link legend items and their descriptions via bounding box predictions. Our experiments show that GPT-4 with structured JSON prompts outperforms the baseline, achieving 88% F-1 and 85% IoU, and reveal how prompt design, example counts, and layout alignment affect performance. This approach supports scalable, layout-aware legend parsing and improves the indexing and searchability of historical maps across various visual styles.

Related papers

• ChartPoint: Guiding MLLMs with Grounding Reflection for Chart Reasoning [54.86473583610112]
  We propose PointCoT, which integrates reflective interaction into chain-of-thought reasoning in charts.<n>By prompting MLLMs to generate bounding boxes and re-render charts based on location annotations, we establish connections between textual reasoning steps and visual grounding regions.<n>We develop two instruction-tuned models, ChartPointQ2 and ChartPointQ2.5, which outperform state-of-the-art across several chart benchmarks.
  arXiv  Detail & Related papers  (2025-11-29T04:01:55Z)
• Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generation [120.23172120151821]
  We propose a novel approach for disentangling visual and semantic features from the backbones of pre-trained diffusion models.<n>We introduce an automated pipeline that constructs image pairs with annotated semantic and visual correspondences.<n>We propose a new metric, Visual Semantic Matching, that quantifies visual inconsistencies in subject-driven image generation.
  arXiv  Detail & Related papers  (2025-09-26T07:11:55Z)
• Automated Label Placement on Maps via Large Language Models [3.7553323195283697]
  We introduce a new paradigm for automatic label placement (ALP) that formulates the task as a data editing problem.<n>To support this direction, we curate MAPLE, the first known benchmarking dataset for evaluating ALP on real-world maps.<n>We evaluate four open-source LLMs on MAPLE, analyzing both overall performance and generalization across different types of landmarks.
  arXiv  Detail & Related papers  (2025-07-29T18:00:22Z)
• Hyper-Local Deformable Transformers for Text Spotting on Historical Maps [2.423679070137552]
  Text on historical maps contains valuable information providing georeferenced historical, political, and cultural contexts.<n>Previous approaches use ad-hoc steps tailored to only specific map styles.<n>Recent machine learning-based text spotters have the potential to solve these challenges.<n>This paper proposes PALETTE, an end-to-end text spotter for scanned historical maps.
  arXiv  Detail & Related papers  (2025-06-17T22:41:10Z)
• Semantic Segmentation for Sequential Historical Maps by Learning from Only One Map [0.4915744683251151]
  We propose an automated approach to digitization using deep-learning-based semantic segmentation.<n>A key challenge in this process is the lack of ground-truth annotations required for training deep neural networks.<n>We introduce a weakly-supervised age-tracing strategy for model fine-tuning.
  arXiv  Detail & Related papers  (2025-01-03T14:55:22Z)
• MapExplorer: New Content Generation from Low-Dimensional Visualizations [60.02149343347818]
  Low-dimensional visualizations, or "projection maps," are widely used to interpret large-scale and complex datasets.<n>These visualizations not only aid in understanding existing knowledge spaces but also implicitly guide exploration into unknown areas.<n>We introduce MapExplorer, a novel knowledge discovery task that translates coordinates within any projection map into coherent, contextually aligned textual content.
  arXiv  Detail & Related papers  (2024-12-24T20:16:13Z)
• Detecting Document-level Paraphrased Machine Generated Content: Mimicking Human Writing Style and Involving Discourse Features [57.34477506004105]
  Machine-generated content poses challenges such as academic plagiarism and the spread of misinformation.<n>We introduce novel methodologies and datasets to overcome these challenges.<n>We propose MhBART, an encoder-decoder model designed to emulate human writing style.<n>We also propose DTransformer, a model that integrates discourse analysis through PDTB preprocessing to encode structural features.
  arXiv  Detail & Related papers  (2024-12-17T08:47:41Z)
• An Efficient System for Automatic Map Storytelling -- A Case Study on Historical Maps [11.037615422309296]
  Historical maps provide valuable information and knowledge about the past. As they often feature non-standard projections, hand-drawn styles, and artistic elements, it is challenging for non-experts to identify and interpret them. Existing image captioning methods have achieved remarkable success on natural images, their performance on maps is suboptimal as maps are underrepresented in their pre-training process. Despite the recent advance of GPT-4 in text recognition and map captioning, it still has a limited understanding of maps, as its performance wanes when texts in maps are missing or inaccurate. We propose a novel decision tree architecture to only generate captions relevant
  arXiv  Detail & Related papers  (2024-10-21T08:45:26Z)
• RETTA: Retrieval-Enhanced Test-Time Adaptation for Zero-Shot Video Captioning [77.59074909960913]
  We propose a novel zero-shot video captioning framework named Retrieval-Enhanced Test-Time Adaptation (RETTA)<n>We bridge video and text using four key models: a general video-text retrieval model XCLIP, a general image-text matching model CLIP, a text alignment model AnglE, and a text generation model GPT-2.<n>To address this problem, we propose using learnable tokens as a communication medium among these four frozen models GPT-2, XCLIP, CLIP, and AnglE.
  arXiv  Detail & Related papers  (2024-05-11T16:22:00Z)
• Guided Interpretable Facial Expression Recognition via Spatial Action Unit Cues [55.97779732051921]
  A new learning strategy is proposed to explicitly incorporate au cues into classifier training. We show that our strategy can improve layer-wise interpretability without degrading classification performance.
  arXiv  Detail & Related papers  (2024-02-01T02:13:49Z)
• Instruction-Guided Scene Text Recognition [51.853730414264625]
  We propose a novel instruction-guided scene text recognition (IGTR) paradigm that formulates STR as an instruction learning problem.<n>IGTR first devises $left langle condition,question,answerright rangle$ instruction triplets, providing rich and diverse descriptions of character attributes.<n>To effectively learn these attributes through question-answering, IGTR develops a lightweight instruction encoder, a cross-modal feature fusion module and a multi-task answer head.
  arXiv  Detail & Related papers  (2024-01-31T14:13:01Z)
• Synthetic Map Generation to Provide Unlimited Training Data for Historical Map Text Detection [5.872532529455414]
  We propose a method to automatically generate an unlimited amount of annotated historical map images for training text detection models. We show that the state-of-the-art text detection models can benefit from the synthetic historical maps.
  arXiv  Detail & Related papers  (2021-12-12T00:27:03Z)
• Learning to Generate Scene Graph from Natural Language Supervision [52.18175340725455]
  We propose one of the first methods that learn from image-sentence pairs to extract a graphical representation of localized objects and their relationships within an image, known as scene graph. We leverage an off-the-shelf object detector to identify and localize object instances, match labels of detected regions to concepts parsed from captions, and thus create "pseudo" labels for learning scene graph.
  arXiv  Detail & Related papers  (2021-09-06T03:38:52Z)
• Graph Sampling Based Deep Metric Learning for Generalizable Person Re-Identification [114.56752624945142]
  We argue that the most popular random sampling method, the well-known PK sampler, is not informative and efficient for deep metric learning. We propose an efficient mini batch sampling method called Graph Sampling (GS) for large-scale metric learning.
  arXiv  Detail & Related papers  (2021-04-04T06:44:15Z)
• Weakly-Supervised Salient Object Detection via Scribble Annotations [54.40518383782725]
  We propose a weakly-supervised salient object detection model to learn saliency from scribble labels. We present a new metric, termed saliency structure measure, to measure the structure alignment of the predicted saliency maps. Our method not only outperforms existing weakly-supervised/unsupervised methods, but also is on par with several fully-supervised state-of-the-art models.
  arXiv  Detail & Related papers  (2020-03-17T12:59:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.