Overview of FinMMEval 2026 Task 1: Multilingual Financial Multiple-Choice Question Answering

• URL: http://arxiv.org/abs/2607.19856v1
• Date: Wed, 22 Jul 2026 07:43:44 GMT
• Title: Overview of FinMMEval 2026 Task 1: Multilingual Financial Multiple-Choice Question Answering
• Abstract summary: FinMMEval 2026 Task 1 evaluates multilingual financial multiple-choice question answering in English, Chinese, Arabic, and Hindi.<n>The final-test set contains 800 questions, with 200 questions per language; gold answers were withheld during submission, and each language was ranked independently by accuracy.<n>Top accuracies range from 92.0% in Hindi to 97.5% in English and Arabic, with the same leading teams appearing near the top across all four languages.
• Score: 59.67537254067838
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: FinMMEval 2026 Task 1 evaluates multilingual financial multiple-choice question answering in English, Chinese, Arabic, and Hindi. The task tests whether systems can select the correct answer to finance questions involving domain terminology, numerical interpretation, and conceptual financial reasoning across languages and scripts. The final-test set contains 800 questions, with 200 questions per language; gold answers were withheld during submission, and each language was ranked independently by accuracy. The final leaderboards contain 13 English, 11 Chinese, 11 Arabic, and 10 Hindi ranked submissions. Top accuracies range from 92.0% in Hindi to 97.5% in English and Arabic, with the same leading teams appearing near the top across all four languages. The documented systems used retrieval augmentation, direct answer-option scoring, language-specific prompting, selective self-consistency, confidence checks, and LLM-based review stages.

Related papers

• XHotpotQA: A Benchmark for Cross-Lingual Knowledge Composition in Multi-Hop Question Answering [4.102826164335728]
  XHotpotQA is a benchmark for cross-lingual knowledge composition over mixed-language evidence.<n>The audited resource contains 15,661 training and 7,405 validation instances.<n>Full question-evidence mismatch is associated with 10.25 to 15.79 lower Unicode-aware answer F1 than partial alignment.
  arXiv  Detail & Related papers  (2026-08-23T15:05:04Z)
• Overview of FinMMEval 2026 Task 2: Multilingual Financial Short-Answer Question Answering [59.6753725406784]
  FinMMEval 2026 Task 2 evaluates short-answer financial question answering over multilingual evidence.<n>The final-test set contains 256 items, split evenly between easy and expert tiers.<n>The strongest systems are closely clustered, with the top four separated by less than one percentage point in ROUGE-1 F1.
  arXiv  Detail & Related papers  (2026-07-22T07:54:17Z)
• SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures [93.36078244548077]
  The task data consist of an extended version of our manually constructed BLEnD benchmark (Myung et al. 2024), covering more than 30 language-culture pairs.<n>As the task is designed strictly for evaluation, participants were not permitted to use the data for training, fine-tuning, few-shot learning, or any other form of model modification.<n>We report the results and present an analysis of the best-performing systems and the most commonly adopted approaches.
  arXiv  Detail & Related papers  (2026-05-04T13:49:44Z)
• The CLEF-2026 FinMMEval Lab: Multilingual and Multimodal Evaluation of Financial AI Systems [54.12165004393043]
  FinMMEval 2026 offers three interconnected tasks that span financial understanding, reasoning, and decision-making.<n>The lab aims to promote the development of robust, transparent, and globally inclusive financial AI systems.
  arXiv  Detail & Related papers  (2026-02-11T14:14:06Z)
• Self-Improving Multilingual Long Reasoning via Translation-Reasoning Integrated Training [50.177839592528294]
  Long reasoning models often struggle in multilingual settings.<n>We propose TRIT (Translation-Reasoning Integrated Training), a self-improving framework that integrates the training of translation into multilingual reasoning.
  arXiv  Detail & Related papers  (2026-02-05T17:55:09Z)
• Multilingual Conversational AI for Financial Assistance: Bridging Language Barriers in Indian FinTech [0.429809467472829]
  We present a multilingual conversational AI system for a financial assistance use case that supports code-mixed languages like Hinglish.<n>Our system employs a multi-agent architecture with language classification, function management, and multilingual response generation.<n>This work contributes to bridging the language gap in digital financial services for emerging markets.
  arXiv  Detail & Related papers  (2025-12-01T09:23:13Z)
• ParamBench: A Graduate-Level Benchmark for Evaluating LLM Understanding on Indic Subjects [4.2155105586549535]
  We present ParamBench, consisting of more than 17K questions in the Hindi language, comprising questionnaires from 21 diverse subjects.<n>These questions are primarily derived from a nationwide graduate-level entrance examination covering topics such as history, music, instruments, yoga,ush, literature, philosophy, law, etc.<n>We evaluate the performance of more than 16 open source LLMs on this benchmark, observing that Gemma3-27B attains the highest overall accuracy of 56.4%.
  arXiv  Detail & Related papers  (2025-08-22T07:59:37Z)
• MIA 2022 Shared Task: Evaluating Cross-lingual Open-Retrieval Question Answering for 16 Diverse Languages [54.002969723086075]
  We evaluate cross-lingual open-retrieval question answering systems in 16 typologically diverse languages. The best system leveraging iteratively mined diverse negative examples achieves 32.2 F1, outperforming our baseline by 4.5 points. The second best system uses entity-aware contextualized representations for document retrieval, and achieves significant improvements in Tamil (20.8 F1), whereas most of the other systems yield nearly zero scores.
  arXiv  Detail & Related papers  (2022-07-02T06:54:10Z)
• The SIGMORPHON 2022 Shared Task on Morpheme Segmentation [39.44280269663147]
  The SIGMORPHON 2022 shared task on morpheme segmentation challenged systems to decompose a word into a sequence of morphemes. The best systems outperformed all three state-of-the-art subword tokenization methods (BPE, ULM, Morfessor2) by 30.71% absolute. To facilitate error analysis and support any type of future studies, we released all system predictions, the evaluation script, and all gold standard datasets.
  arXiv  Detail & Related papers  (2022-06-15T15:57:22Z)
• MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering [6.452012363895865]
  This dataset supplies the widest range of languages to-date for evaluating question answering. We benchmark a variety of state-of-the-art methods and baselines for generative and extractive question answering. Results indicate this dataset is challenging even in English, but especially in low-resource languages.
  arXiv  Detail & Related papers  (2020-07-30T03:33:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.