Language-Routed RAG and Direct Option Scoring for Multilingual Financial QA: DS@GT at FinMMEval

• URL: http://arxiv.org/abs/2607.22841v1
• Date: Fri, 24 Jul 2026 18:31:11 GMT
• Title: Language-Routed RAG and Direct Option Scoring for Multilingual Financial QA: DS@GT at FinMMEval
• Abstract summary: Financial certification exams such as the CFA, EFPA, and CPA demand structured domain reasoning.<n>We build a retrieval-augmented pipeline on LangGraph that detects query language and retrieves semantically relevant exemplars.<n>We show that effective multilingual financial reasoning requires language-aware retrieval, model routing, and deliberate scoring strategy selection.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present DS@GT's submission to FinMMEval 2026 Task 1, a multilingual financial exam question answering benchmark spanning English, Spanish, Greek, Chinese, and Hindi. Financial certification exams such as the CFA, EFPA, and CPA demand structured domain reasoning that standard NLP benchmarks do not capture, and this challenge compounds across languages where retrieval and representation infrastructure is underdeveloped. We build a retrieval-augmented pipeline on LangGraph that detects query language and retrieves semantically relevant exemplars from a 30,209-entry multilingual knowledge base using BGE-M3 embeddings and FAISS indexing. The system then scores answers via Retrieval-Augmented Direct Scoring (RADS), reading next-token log-probabilities over candidate option letters rather than generating free-form output. For low-resource languages, we fuse per-language and cross-lingual retrieval indices using weighted Reciprocal Rank Fusion. Model selection is language-routed: Qwen3-14B for Arabic, Chinese, and Hindi; Qwen2.5-14B for English; and Llama-3.1-8B for Greek, a routing derived from empirical ablations that reveal substantial language-asymmetric performance gaps. Notably, chain-of-thought prompting significantly degrades Greek accuracy (90.7% to 20.9%), and enabling Qwen3's default thinking mode collapses Arabic RADS performance to near-chance levels. Our results indicate that effective multilingual financial reasoning requires language-aware retrieval, model routing, and deliberate scoring strategy selection.

Related papers

• The Multilingual Curse at the Retrieval Layer: Evidence from Amharic [49.25758237235551]
  Strong zero-shot scores on multilingual benchmarks are often taken as evidence that current encoders transfer reliably across many languages.<n>We argue that this assumption breaks down for underrepresented, morphologically rich languages.<n>We compare zero-shot multilingual retrievers, Amharic-fine-tuned multilingual retrievers, and monolingual Amharic retrievers.
  arXiv  Detail & Related papers  (2026-05-23T12:44:30Z)
• DaPT: A Dual-Path Framework for Multilingual Multi-hop Question Answering [41.923993964253405]
  We propose a novel framework for retrieving across multilingual corpora and queries.<n>DaPT generates sub-question graphs in parallel for both the source-language query and its English translation counterpart.<n>DaPT consistently yields more accurate and concise answers compared to the baselines.
  arXiv  Detail & Related papers  (2026-03-19T16:23:04Z)
• PARSE: An Open-Domain Reasoning Question Answering Benchmark for Persian [23.04786342076179]
  We introduce PARSE, the first open-domain Persian reasoning QA benchmark.<n>It contains 10,800 questions across Boolean, multiple-choice, and factoid formats.<n>It is built via a controlled LLM-based generation pipeline and validated through human evaluation.
  arXiv  Detail & Related papers  (2026-02-01T14:11:51Z)
• One Instruction Does Not Fit All: How Well Do Embeddings Align Personas and Instructions in Low-Resource Indian Languages? [1.071318785217926]
  We present a benchmark spanning 12 Indian languages and four evaluation tasks.<n>E5-Large-Instruct achieves the highest Recall@1 of 27.4% on monolingual retrieval and 20.7% on cross-lingual transfer.<n>For classification, LaBSE attains 75.3% AUROC with strong calibration.
  arXiv  Detail & Related papers  (2026-01-15T09:10:14Z)
• SLRTP2025 Sign Language Production Challenge: Methodology, Results, and Future Work [87.9341538630949]
  The first Sign Language Production Challenge was held as part of the third SLRTP Workshop at CVPR 2025.<n>The competition's aims are to evaluate architectures that translate from spoken language sentences to a sequence of skeleton poses.<n>This paper presents the challenge design and the winning methodologies.
  arXiv  Detail & Related papers  (2025-08-09T11:57:33Z)
• From Multiple-Choice to Extractive QA: A Case Study for English and Arabic [51.13706104333848]
  We explore the feasibility of repurposing an existing multilingual dataset for a new NLP task.<n>We present annotation guidelines and a parallel EQA dataset for English and Modern Standard Arabic.<n>We aim to help others adapt our approach for the remaining 120 BELEBELE language variants, many of which are deemed under-resourced.
  arXiv  Detail & Related papers  (2024-04-26T11:46:05Z)
• Multilingual Answer Sentence Reranking via Automatically Translated Data [97.98885151955467]
  We present a study on the design of multilingual Answer Sentence Selection (AS2) models, which are a core component of modern Question Answering (QA) systems. The main idea is to transfer data, created from one resource rich language, e.g., English, to other languages, less rich in terms of resources.
  arXiv  Detail & Related papers  (2021-02-20T03:52:08Z)
• Pivot Through English: Reliably Answering Multilingual Questions without Document Retrieval [4.4973334555746]
  Existing methods for open-retrieval question answering in lower resource languages (LRLs) lag significantly behind English. We formulate a task setup more realistic to available resources, that circumvents document retrieval to reliably transfer knowledge from English to lower resource languages. Within this task setup we propose Reranked Maximal Inner Product Search (RM-MIPS), akin to semantic similarity retrieval over the English training set with reranking.
  arXiv  Detail & Related papers  (2020-12-28T04:38:45Z)
• GATE: Graph Attention Transformer Encoder for Cross-lingual Relation and Event Extraction [107.8262586956778]
  We introduce graph convolutional networks (GCNs) with universal dependency parses to learn language-agnostic sentence representations. GCNs struggle to model words with long-range dependencies or are not directly connected in the dependency tree. We propose to utilize the self-attention mechanism to learn the dependencies between words with different syntactic distances.
  arXiv  Detail & Related papers  (2020-10-06T20:30:35Z)
• Inducing Language-Agnostic Multilingual Representations [61.97381112847459]
  Cross-lingual representations have the potential to make NLP techniques available to the vast majority of languages in the world. We examine three approaches for this: (i) re-aligning the vector spaces of target languages to a pivot source language; (ii) removing language-specific means and variances, which yields better discriminativeness of embeddings as a by-product; and (iii) increasing input similarity across languages by removing morphological contractions and sentence reordering.
  arXiv  Detail & Related papers  (2020-08-20T17:58:56Z)
• LAReQA: Language-agnostic answer retrieval from a multilingual pool [29.553907688813347]
  LAReQA tests for "strong" cross-lingual alignment. We find that augmenting training data via machine translation is effective. This finding underscores our claim that languageagnostic retrieval is a substantively new kind of cross-lingual evaluation.
  arXiv  Detail & Related papers  (2020-04-11T20:51:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.