NMIXX: Domain-Adapted Neural Embeddings for Cross-Lingual eXploration of Finance

• URL: http://arxiv.org/abs/2507.09601v1
• Date: Sun, 13 Jul 2025 12:14:57 GMT
• Title: NMIXX: Domain-Adapted Neural Embeddings for Cross-Lingual eXploration of Finance
• Authors: Hanwool Lee, Sara Yu, Yewon Hwang, Jonghyun Choi, Heejae Ahn, Sungbum Jung, Youngjae Yu,
• Abstract summary: General-purpose sentence embedding models often struggle to capture specialized financial semantics.<n>We introduce NMIXX, a suite of cross-lingual embedding models fine-tuned with 18.8K high-confidence triplets.<n>We also release KorFinSTS, a benchmark spanning news, disclosures, research reports, and regulations.
• Score: 24.697311579081983
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: General-purpose sentence embedding models often struggle to capture specialized financial semantics, especially in low-resource languages like Korean, due to domain-specific jargon, temporal meaning shifts, and misaligned bilingual vocabularies. To address these gaps, we introduce NMIXX (Neural eMbeddings for Cross-lingual eXploration of Finance), a suite of cross-lingual embedding models fine-tuned with 18.8K high-confidence triplets that pair in-domain paraphrases, hard negatives derived from a semantic-shift typology, and exact Korean-English translations. Concurrently, we release KorFinSTS, a 1,921-pair Korean financial STS benchmark spanning news, disclosures, research reports, and regulations, designed to expose nuances that general benchmarks miss. When evaluated against seven open-license baselines, NMIXX's multilingual bge-m3 variant achieves Spearman's rho gains of +0.10 on English FinSTS and +0.22 on KorFinSTS, outperforming its pre-adaptation checkpoint and surpassing other models by the largest margin, while revealing a modest trade-off in general STS performance. Our analysis further shows that models with richer Korean token coverage adapt more effectively, underscoring the importance of tokenizer design in low-resource, cross-lingual settings. By making both models and the benchmark publicly available, we provide the community with robust tools for domain-adapted, multilingual representation learning in finance.

Related papers

• Investigating the Multilingual Calibration Effects of Language Model Instruction-Tuning [58.355275813623685]
  This work looks at a critical gap in the calibration of large language models (LLMs) within multilingual settings.<n>Even in low-resource languages, model confidence can increase significantly after instruction-tuning on high-resource language SFT datasets.<n>However, improvements in accuracy are marginal or non-existent, highlighting a critical shortcoming of standard SFT for multilingual languages.
  arXiv  Detail & Related papers  (2026-01-04T04:29:12Z)
• Low-Resource English-Tigrinya MT: Leveraging Multilingual Models, Custom Tokenizers, and Clean Evaluation Benchmarks [6.177998679139308]
  Despite advances in Neural Machine Translation (NMT), low-resource languages like Tigrinya remain underserved.<n>This paper investigates transfer learning techniques using multilingual pretrained models to enhance translation quality for morphologically rich, low-resource languages.
  arXiv  Detail & Related papers  (2025-09-24T15:02:57Z)
• False Friends Are Not Foes: Investigating Vocabulary Overlap in Multilingual Language Models [53.01170039144264]
  Subword tokenizers trained on multilingual corpora naturally produce overlapping tokens across languages.<n>Does token overlap facilitate cross-lingual transfer or instead introduce interference between languages?<n>We find that models with overlap outperform models with disjoint vocabularies.
  arXiv  Detail & Related papers  (2025-09-23T07:47:54Z)
• The Token Tax: Systematic Bias in Multilingual Tokenization [0.8820808252713569]
  Tokenization inefficiency imposes structural disadvantages on morphologically complex, low-resource languages.<n>We show that fertility (tokens/word) reliably predicts accuracy.<n>We also find that reasoning models (DeepSeek, o1) consistently outperform non-reasoning peers.
  arXiv  Detail & Related papers  (2025-09-05T20:20:51Z)
• Adapting Language Models to Indonesian Local Languages: An Empirical Study of Language Transferability on Zero-Shot Settings [1.1556013985948772]
  We evaluate transferability of pre-trained language models to low-resource Indonesian local languages.<n>We group the target languages into three categories: seen, partially seen, and unseen.<n> Multilingual models perform best on seen languages, moderately on partially seen ones, and poorly on unseen languages.<n>We find that MAD-X significantly improves performance, especially for seen and partially seen languages, without requiring labeled data in the target language.
  arXiv  Detail & Related papers  (2025-07-02T12:17:55Z)
• TWICE: What Advantages Can Low-Resource Domain-Specific Embedding Model Bring? -- A Case Study on Korea Financial Texts [0.8624680612413766]
  Existing benchmarks, such as FinMTEB, are primarily designed for high-resource languages.<n>We introduce KorFinMTEB, a novel benchmark for the Korean financial domain, specifically tailored to reflect its unique cultural characteristics.<n>Our experimental results reveal that while the models perform robustly on a translated version of FinMTEB, their performance on KorFinMTEB uncovers subtle yet critical discrepancies.
  arXiv  Detail & Related papers  (2025-02-10T23:49:39Z)
• Registering Source Tokens to Target Language Spaces in Multilingual Neural Machine Translation [26.737542701290153]
  We introduce registering, a novel method that enables a small MNMT-specific model to compete with large language models (LLMs)<n> Experiments on EC-40, a large-scale benchmark, show that our method advances the state-of-the-art of MNMT.<n>We open-source our models to facilitate further research and development in MNMT.
  arXiv  Detail & Related papers  (2025-01-06T12:42:54Z)
• Distributionally Robust Multilingual Machine Translation [94.51866646879337]
  We propose a new learning objective for Multilingual neural machine translation (MNMT) based on distributionally robust optimization. We show how to practically optimize this objective for large translation corpora using an iterated best response scheme. Our method consistently outperforms strong baseline methods in terms of average and per-language performance under both many-to-one and one-to-many translation settings.
  arXiv  Detail & Related papers  (2021-09-09T03:48:35Z)
• Cross-lingual Transferring of Pre-trained Contextualized Language Models [73.97131976850424]
  We propose a novel cross-lingual model transferring framework for PrLMs: TreLM. To handle the symbol order and sequence length differences between languages, we propose an intermediate TRILayer" structure. We show the proposed framework significantly outperforms language models trained from scratch with limited data in both performance and efficiency.
  arXiv  Detail & Related papers  (2021-07-27T06:51:13Z)
• Towards Multi-Sense Cross-Lingual Alignment of Contextual Embeddings [41.148892848434585]
  We propose a novel framework to align contextual embeddings at the sense level by leveraging cross-lingual signal from bilingual dictionaries only. We operationalize our framework by first proposing a novel sense-aware cross entropy loss to model word senses explicitly. We then propose a sense alignment objective on top of the sense-aware cross entropy loss for cross-lingual model pretraining, and pretrain cross-lingual models for several language pairs.
  arXiv  Detail & Related papers  (2021-03-11T04:55:35Z)
• Nearest Neighbor Machine Translation [113.96357168879548]
  We introduce $k$-nearest-neighbor machine translation ($k$NN-MT) It predicts tokens with a nearest neighbor classifier over a large datastore of cached examples. It consistently improves performance across many settings.
  arXiv  Detail & Related papers  (2020-10-01T22:24:46Z)
• Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages [48.28540903568198]
  We show that multilinguality is critical to making unsupervised systems practical for low-resource settings. We present a single model for 5 low-resource languages (Gujarati, Kazakh, Nepali, Sinhala, and Turkish) to and from English directions. We outperform all current state-of-the-art unsupervised baselines for these languages, achieving gains of up to 14.4 BLEU.
  arXiv  Detail & Related papers  (2020-09-23T15:07:33Z)
• Inducing Language-Agnostic Multilingual Representations [61.97381112847459]
  Cross-lingual representations have the potential to make NLP techniques available to the vast majority of languages in the world. We examine three approaches for this: (i) re-aligning the vector spaces of target languages to a pivot source language; (ii) removing language-specific means and variances, which yields better discriminativeness of embeddings as a by-product; and (iii) increasing input similarity across languages by removing morphological contractions and sentence reordering.
  arXiv  Detail & Related papers  (2020-08-20T17:58:56Z)
• Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation [81.7786241489002]
  Massively multilingual models for neural machine translation (NMT) are theoretically attractive, but often underperform bilingual models and deliver poor zero-shot translations. We argue that multilingual NMT requires stronger modeling capacity to support language pairs with varying typological characteristics. We propose random online backtranslation to enforce the translation of unseen training language pairs.
  arXiv  Detail & Related papers  (2020-04-24T17:21:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.