XHotpotQA: A Benchmark for Cross-Lingual Knowledge Composition in Multi-Hop Question Answering

• URL: http://arxiv.org/abs/2608.27481v2
• Date: Mon, 31 Aug 2026 15:00:22 GMT
• Title: XHotpotQA: A Benchmark for Cross-Lingual Knowledge Composition in Multi-Hop Question Answering
• Abstract summary: XHotpotQA is a benchmark for cross-lingual knowledge composition over mixed-language evidence.<n>The audited resource contains 15,661 training and 7,405 validation instances.<n>Full question-evidence mismatch is associated with 10.25 to 15.79 lower Unicode-aware answer F1 than partial alignment.
• Score: 4.102826164335728
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Knowledge-intensive multi-hop question answering requires systems to select evidence and compose dependent facts, yet multilingual benchmarks usually translate an entire example into one language. This hides failures at language boundaries inside the reasoning chain. We introduce XHotpotQA, a controlled benchmark for cross-lingual knowledge composition over mixed-language evidence. Each instance is modeled as an evidence-dependency graph whose question, bridge evidence, answer-bearing evidence, and distractors have explicit language assignments. The audited resource contains 15,661 training and 7,405 validation instances, with sentence-level support supervision and supplied distractors. In validation, 99.81% of items cross the question-to-gold-evidence language interface and 95.60% use gold paragraphs in different languages. Across three reader artifacts, full question-evidence mismatch is associated with 10.25 to 15.79 lower Unicode-aware answer F1 than partial alignment, and different-script evidence with deficits of 11.98 to 23.70 points; the corresponding adapted-selector contrasts are 1.71 and 1.78 points. Under this supplied-candidate design, the evaluated readers therefore show substantially larger condition-associated deficits than the selector. XHotpotQA provides role-aware diagnostics, modular evaluation, and an audited test bed for knowledge-based systems that must integrate evidence across languages.

Related papers

• Better Decomposition, Free Aggregation: A Synthesizer-Folding Framework for Multilingual Multi-Hop Question Answering [48.94279718429342]
  We introduce Syfer, a framework for multilingual multi-hop question answering.<n>Syfer defers translation rather than applying it by default.<n> Experiments across multiple languages show that Syfer attains competitive accuracy.
  arXiv  Detail & Related papers  (2026-08-13T12:25:59Z)
• LitTraceQA: A Benchmark for Multi-Stage Grounding and Verification in Scientific Question Answering [30.616598264585203]
  We present LitTraceQA, a benchmark for literature-grounded question answering over scientific papers.<n>Given a research question and a metadata pool of papers, a system must return three connected outputs.<n>We analyze a larger final annotation collection with 4,978 unique-question records over 4,859 unique gold papers.
  arXiv  Detail & Related papers  (2026-08-07T16:11:52Z)
• Language-Routed RAG and Direct Option Scoring for Multilingual Financial QA: DS@GT at FinMMEval [0.0]
  Financial certification exams such as the CFA, EFPA, and CPA demand structured domain reasoning.<n>We build a retrieval-augmented pipeline on LangGraph that detects query language and retrieves semantically relevant exemplars.<n>We show that effective multilingual financial reasoning requires language-aware retrieval, model routing, and deliberate scoring strategy selection.
  arXiv  Detail & Related papers  (2026-07-24T18:31:11Z)
• Overview of FinMMEval 2026 Task 1: Multilingual Financial Multiple-Choice Question Answering [59.67537254067838]
  FinMMEval 2026 Task 1 evaluates multilingual financial multiple-choice question answering in English, Chinese, Arabic, and Hindi.<n>The final-test set contains 800 questions, with 200 questions per language; gold answers were withheld during submission, and each language was ranked independently by accuracy.<n>Top accuracies range from 92.0% in Hindi to 97.5% in English and Arabic, with the same leading teams appearing near the top across all four languages.
  arXiv  Detail & Related papers  (2026-07-22T07:43:44Z)
• Beyond Monolingual Deep Research: Evaluating Agents and Retrievers with Cross-Lingual BrowseComp-Plus [35.39357302837359]
  XBCP (Cross-lingual BrowseComp-Plus) is a controlled benchmark that preserves the English question-and-answer space of BrowseComp-Plus but varies the languages of the supporting documents.<n>XBCP instantiates two complementary settings: in the cross-lingual setting, each query is paired with evidence in a single assigned language.<n>We evaluate four deep research agents using sparse and dense multilingual retrievers, measuring answer accuracy, evidence recall, search behavior, calibration, citation fidelity, and oracle retrieval.
  arXiv  Detail & Related papers  (2026-06-13T15:11:52Z)
• Self-Improving Multilingual Long Reasoning via Translation-Reasoning Integrated Training [50.177839592528294]
  Long reasoning models often struggle in multilingual settings.<n>We propose TRIT (Translation-Reasoning Integrated Training), a self-improving framework that integrates the training of translation into multilingual reasoning.
  arXiv  Detail & Related papers  (2026-02-05T17:55:09Z)
• Do Language Models Reason Across Languages? [19.660512783888016]
  We find that language models are more sensitive to language variation in answer-span documents than in those providing bridging information.<n>We propose a simple three-stage SUBQ prompting method to guide the multi-step reasoning with sub-questions.
  arXiv  Detail & Related papers  (2026-01-10T17:59:34Z)
• Evaluating and Modeling Attribution for Cross-Lingual Question Answering [80.4807682093432]
  This work is the first to study attribution for cross-lingual question answering. We collect data in 5 languages to assess the attribution level of a state-of-the-art cross-lingual QA system. We find that a substantial portion of the answers is not attributable to any retrieved passages.
  arXiv  Detail & Related papers  (2023-05-23T17:57:46Z)
• Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering [124.16250115608604]
  We present Science Question Answering (SQA), a new benchmark that consists of 21k multimodal multiple choice questions with a diverse set of science topics and annotations of their answers with corresponding lectures and explanations. We show that SQA improves the question answering performance by 1.20% in few-shot GPT-3 and 3.99% in fine-tuned UnifiedQA. Our analysis further shows that language models, similar to humans, benefit from explanations to learn from fewer data and achieve the same performance with just 40% of the data.
  arXiv  Detail & Related papers  (2022-09-20T07:04:24Z)
• Delving Deeper into Cross-lingual Visual Question Answering [115.16614806717341]
  We show that simple modifications to the standard training setup can substantially reduce the transfer gap to monolingual English performance. We analyze cross-lingual VQA across different question types of varying complexity for different multilingual multimodal Transformers.
  arXiv  Detail & Related papers  (2022-02-15T18:22:18Z)
• UPV at CheckThat! 2021: Mitigating Cultural Differences for Identifying Multilingual Check-worthy Claims [6.167830237917659]
  In this paper, we propose a language identification task as an auxiliary task to mitigate unintended bias. Our results show that joint training of language identification and check-worthy claim detection tasks can provide performance gains for some of the selected languages.
  arXiv  Detail & Related papers  (2021-09-19T21:46:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.