Lost without translation -- Can transformer (language models) understand mood states?

• URL: http://arxiv.org/abs/2512.00274v1
• Date: Sat, 29 Nov 2025 01:55:11 GMT
• Title: Lost without translation -- Can transformer (language models) understand mood states?
• Abstract summary: Methods: We collected 247 unique phrases for the four mood states across 11 Indic languages.<n>Performance was measured using a composite score based on Adjusted Rand Index, Normalised Mutual Information, Homogeneity and Completeness.
• Score: 3.640688858400333
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Background: Large Language Models show promise in psychiatry but are English-centric. Their ability to understand mood states in other languages is unclear, as different languages have their own idioms of distress. Aim: To quantify the ability of language models to faithfully represent phrases (idioms of distress) of four distinct mood states (depression, euthymia, euphoric mania, dysphoric mania) expressed in Indian languages. Methods: We collected 247 unique phrases for the four mood states across 11 Indic languages. We tested seven experimental conditions, comparing k-means clustering performance on: (a) direct embeddings of native and Romanised scripts (using multilingual and Indic-specific models) and (b) embeddings of phrases translated to English and Chinese. Performance was measured using a composite score based on Adjusted Rand Index, Normalised Mutual Information, Homogeneity and Completeness. Results: Direct embedding of Indic languages failed to cluster mood states (Composite Score = 0.002). All translation-based approaches showed significant improvement. High performance was achieved using Gemini-translated English (Composite=0.60) and human-translated English (Composite=0.61) embedded with gemini-001. Surprisingly, human-translated English, further translated into Chinese and embedded with a Chinese model, performed best (Composite = 0.67). Specialised Indic models (IndicBERT and Sarvam-M) performed poorly. Conclusion: Current models cannot meaningfully represent mood states directly from Indic languages, posing a fundamental barrier to their psychiatric application for diagnostic or therapeutic purposes in India. While high-quality translation bridges this gap, reliance on proprietary models or complex translation pipelines is unsustainable. Models must first be built to understand diverse local languages to be effective in global mental health.

Related papers

• VakyArth: Evaluating Pragmatic Competence in LLMs across Indic Languages [6.468608052089483]
  We introduce VakyArth, the first pragmatic benchmark for Indic languages.<n>VakyArth evaluates models across five phenomena: deixis, speech acts, implicature, social pragmatics, and coherence.<n>We show that automatic translation metrics can miss fluent but pragmatically unfaithful outputs.
  arXiv  Detail & Related papers  (2026-09-01T19:00:32Z)
• From Monolingual to Bilingual: Investigating Language Conditioning in Large Language Models for Psycholinguistic Tasks [9.837135712999471]
  Large Language Models (LLMs) exhibit strong linguistic capabilities, but little is known about how they encode psycholinguistic knowledge across languages.<n>We investigate whether and how LLMs exhibit human-like psycholinguistic responses under different linguistic identities.<n>We evaluate two models, Llama-3.3-70B-Instruct and Qwen2.5-72B-Instruct, under monolingual and bilingual prompting in English, Dutch, and Chinese.
  arXiv  Detail & Related papers  (2025-08-04T15:10:44Z)
• IndicSentEval: How Effectively do Multilingual Transformer Models encode Linguistic Properties for Indic Languages? [16.53746282142566]
  We investigate encoding capability and robustness for 8 linguistic properties across 13 different perturbations in 6 Indic languages.<n>Our probing analysis of surface, syntactic, and semantic properties reveals that while almost all multilingual models demonstrate consistent encoding performance for English.<n>Indic-specific multilingual models capture linguistic properties in Indic languages better than universal models.
  arXiv  Detail & Related papers  (2024-10-03T15:50:08Z)
• Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss Weighting [66.02718577386426]
  We provide a simple characterization of idiomatic translation and related issues. We conduct a synthetic experiment revealing a tipping point at which transformer-based machine translation models correctly default to idiomatic translations. To improve translation of natural idioms, we introduce two straightforward yet effective techniques.
  arXiv  Detail & Related papers  (2023-10-10T23:47:25Z)
• Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decoding [53.84948040596055]
  We introduce two related methods to mitigate failure cases with a modified decoding objective. Experiments on the massively multilingual models M2M-100 (418M) and SMaLL-100 show that these methods suppress hallucinations and off-target translations.
  arXiv  Detail & Related papers  (2023-09-13T17:15:27Z)
• No Language Left Behind: Scaling Human-Centered Machine Translation [69.28110770760506]
  We create datasets and models aimed at narrowing the performance gap between low and high-resource languages. We propose multiple architectural and training improvements to counteract overfitting while training on thousands of tasks. Our model achieves an improvement of 44% BLEU relative to the previous state-of-the-art.
  arXiv  Detail & Related papers  (2022-07-11T07:33:36Z)
• Do Multilingual Language Models Capture Differing Moral Norms? [71.52261949766101]
  Massively multilingual sentence representations are trained on large corpora of uncurated data. This may cause the models to grasp cultural values including moral judgments from the high-resource languages. The lack of data in certain languages can also lead to developing random and thus potentially harmful beliefs.
  arXiv  Detail & Related papers  (2022-03-18T12:26:37Z)
• IndicBART: A Pre-trained Model for Natural Language Generation of Indic Languages [24.638109544527104]
  IndicBART is a multilingual, sequence-to-sequence pre-trained model focusing on 11 Indic languages and English. We evaluate IndicBART on two NLG tasks: Neural Machine Translation (NMT) and extreme summarization.
  arXiv  Detail & Related papers  (2021-09-07T07:08:33Z)
• Attention based Sequence to Sequence Learning for Machine Translation of Low Resourced Indic Languages -- A case of Sanskrit to Hindi [0.0]
  The paper shows the construction of Sanskrit to Hindi bilingual parallel corpus with nearly 10K samples and having 178,000 tokens. The attention mechanism based neural translation has achieved 88% accuracy in human evaluation and a BLEU score of 0.92 on Sanskrit to Hindi translation.
  arXiv  Detail & Related papers  (2021-09-07T04:55:48Z)
• AM2iCo: Evaluating Word Meaning in Context across Low-ResourceLanguages with Adversarial Examples [51.048234591165155]
  We present AM2iCo, Adversarial and Multilingual Meaning in Context. It aims to faithfully assess the ability of state-of-the-art (SotA) representation models to understand the identity of word meaning in cross-lingual contexts. Results reveal that current SotA pretrained encoders substantially lag behind human performance.
  arXiv  Detail & Related papers  (2021-04-17T20:23:45Z)
• Vy\=akarana: A Colorless Green Benchmark for Syntactic Evaluation in Indic Languages [0.0]
  Indic languages have rich morphosyntax, grammatical genders, free linear word-order, and highly inflectional morphology. We introduce Vy=akarana: a benchmark of gender-balanced Colorless Green sentences in Indic languages for syntactic evaluation of multilingual language models. We use the datasets from the evaluation tasks to probe five multilingual language models of varying architectures for syntax in Indic languages.
  arXiv  Detail & Related papers  (2021-03-01T09:07:58Z)
• Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation [61.88012735215636]
  Unsupervised neural machine translation (UNMT) has recently achieved remarkable results for several language pairs. UNMT can only translate between a single language pair and cannot produce translation results for multiple language pairs at the same time. In this paper, we empirically introduce a simple method to translate between thirteen languages using a single encoder and a single decoder.
  arXiv  Detail & Related papers  (2020-04-21T17:26:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.