GeoRisk-RAG: A Hierarchy-Aware Risk Framework for Improving RAG Reliability through Selective Answering

• URL: http://arxiv.org/abs/2608.22634v1
• Date: Sun, 23 Aug 2026 22:39:21 GMT
• Title: GeoRisk-RAG: A Hierarchy-Aware Risk Framework for Improving RAG Reliability through Selective Answering
• Abstract summary: We present GeoRisk-RAG, a hierarchy-aware framework that addresses this geographic-validity gap through selective answering.<n>Experiments on a novel held-out wildfire-related question-answering dataset show that GeoRisk-RAG significantly reduces false confidence rates for location-dependent questions.<n>This work provides a more comprehensive assessment of end-to-end RAG pipelines by integrating geographic validity and selective-answering behavior for safer decision-making in geospatial domains.
• Score: 8.527283791363045
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current work on improving reliability in large language model (LLM)- generated answers has primarily leveraged Retrieval-Augmented Generation (RAG), knowledge-graph augmentation, and reinforcement learning. While these methods are adept at enhancing and measuring reliability through semantic similarity and faithfulness, they often struggle to distinguish semantic similarity from geographic validity. This is especially critical in natural hazard management domains where geographic granularity (i.e., town vs. city vs. state) is significant for decision-making, as responses valid in one municipality may not transfer to another. In such domains, a confidently wrong answer carries greater risk than abstaining. We present GeoRisk-RAG, a novel hierarchy-aware framework that addresses this geographic-validity gap through selective answering. This framework explicitly estimates geographic applicability using a Directed Acyclic Graph (DAG)-based distance for context retrieval before response generation. Experiments on a novel held-out wildfire-related question-answering (QA) dataset show that GeoRisk-RAG significantly reduces false confidence rates for location-dependent questions, lowering the rate to 0.009 compared with ~0.090 for standard semantic similarity and reranking baselines, while consistently achieving higher human preference alignment. This work provides a more comprehensive assessment of end-to-end RAG pipelines by integrating geographic validity and selective-answering behavior for safer decision-making in geospatial domains.

Related papers

• ReaGeo: Reasoning-Enhanced End-to-End Geocoding with LLMs [8.089023511080248]
  This paper proposes ReaGeo, an end-to-end geocoding framework based on large language models.<n>The method converts geographic coordinates into geohash sequences, reformulating the coordinate prediction task as a text generation problem.<n>Reinforcement learning with a distance-deviation-based reward is applied to optimize the generation accuracy.
  arXiv  Detail & Related papers  (2026-04-23T07:18:21Z)
• GeoAgent: Learning to Geolocate Everywhere with Reinforced Geographic Characteristics [91.17301794848025]
  This paper presents GeoAgent, a model capable of reasoning closely with humans and deriving fine-grained address conclusions.<n>Previous RL-based methods have achieved breakthroughs in performance and interpretability but still remain concerns because of their reliance on AI-generated chain-of-thought (CoT) data and training strategies.
  arXiv  Detail & Related papers  (2026-02-13T04:48:05Z)
• Vision-Language Reasoning for Geolocalization: A Reinforcement Learning Approach [41.001581773172695]
  We present Geo-R, a retrieval-free framework that uncovers structured reasoning paths from existing ground-truth coordinates.<n>We propose the Chain of Region, a rule-based hierarchical reasoning paradigm that generates precise, interpretable supervision.<n>Our approach bridges structured geographic reasoning with direct spatial supervision, yielding improved localization accuracy, stronger generalization, and more transparent inference.
  arXiv  Detail & Related papers  (2026-01-01T16:51:41Z)
• Geospatial Chain of Thought Reasoning for Enhanced Visual Question Answering on Satellite Imagery [1.9153836153207706]
  Geospatial chain of thought (CoT) reasoning is essential for advancing Visual Question Answering (VQA) on satellite imagery.<n>We propose a VQA framework that integrates CoT reasoning with Direct Preference Optimization (DPO) to improve interpretability, robustness, and accuracy.<n> Experiments show that CoT supervision improves accuracy by 34.9% over direct baselines, while DPO yields additional gains in accuracy and reasoning quality.
  arXiv  Detail & Related papers  (2025-11-14T11:54:05Z)
• Grounding Long-Context Reasoning with Contextual Normalization for Retrieval-Augmented Generation [57.97548022208733]
  We show that seemingly superficial choices in key-value extraction can induce shifts in accuracy and stability.<n>We introduce Contextual Normalization, a strategy that adaptively standardizes context representations before generation.
  arXiv  Detail & Related papers  (2025-10-15T06:28:25Z)
• Retrieval-Augmented Generation for Reliable Interpretation of Radio Regulations [49.671779378073886]
  We study question answering in the domain of radio regulations.<n>We propose a telecom-specific Retrieval-Augmented Generation (RAG) pipeline.<n>Our approach consistently improves generation accuracy across all tested models.
  arXiv  Detail & Related papers  (2025-09-11T17:43:42Z)
• GeoSR: Cognitive-Agentic Framework for Probing Geospatial Knowledge Boundaries via Iterative Self-Refinement [4.026524042818433]
  GeoSR is a self-refining agentic reasoning framework that embeds core geographic principles into an iterative prediction loop.<n>We validate GeoSR on tasks ranging from physical-world property estimation to socioeconomic prediction.
  arXiv  Detail & Related papers  (2025-08-06T04:45:34Z)
• Spatial-RAG: Spatial Retrieval Augmented Generation for Real-World Geospatial Reasoning Questions [5.053463027769152]
  Spatial-RAG is a Retrieval-Augmented Generation framework designed for geospatial question answering.<n>It integrates structured spatial databases with large language models (LLMs) via a hybrid spatial retriever.<n>It formulates the answering process as a multi-objective optimization over spatial and semantic relevance.
  arXiv  Detail & Related papers  (2025-02-04T01:30:06Z)
• SpatialRank: Urban Event Ranking with NDCG Optimization on Spatiotemporal Data [55.609946936979036]
  We propose a novel spatial event ranking approach named SpatialRank. We show that SpatialRank can effectively identify the top riskiest locations of crimes and traffic accidents.
  arXiv  Detail & Related papers  (2023-09-30T06:20:21Z)
• Geo-Encoder: A Chunk-Argument Bi-Encoder Framework for Chinese Geographic Re-Ranking [61.60169764507917]
  Chinese geographic re-ranking task aims to find the most relevant addresses among retrieved candidates. We propose an innovative framework, namely Geo-Encoder, to more effectively integrate Chinese geographical semantics into re-ranking pipelines.
  arXiv  Detail & Related papers  (2023-09-04T13:44:50Z)
• Scope Head for Accurate Localization in Object Detection [135.9979405835606]
  We propose a novel detector coined as ScopeNet, which models anchors of each location as a mutually dependent relationship. With our concise and effective design, the proposed ScopeNet achieves state-of-the-art results on COCO.
  arXiv  Detail & Related papers  (2020-05-11T04:00:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.