Vectorizing Classical Tamil: Representation Learning for Verse-Commentary Pairs

• URL: http://arxiv.org/abs/2609.04755v2
• Date: Mon, 07 Sep 2026 14:05:50 GMT
• Title: Vectorizing Classical Tamil: Representation Learning for Verse-Commentary Pairs
• Abstract summary: We construct a corpus of 1,262 verse--commentary (urai) pairs from five Classical Tamil source sections.<n>We train recurrent and Transformer encoders, a Siamese-style pair-matching network, an mBART-style encoder--decoder, and a decoder--only language model.
• Score: 0.21485350418225244
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We construct a corpus of 1,262 verse--commentary (urai) pairs from five Classical Tamil source sections, ranging from technical grammatical prose to modern paraphrase, and ask what information representation learning can recover. We train recurrent and Transformer encoders, a Siamese-style pair-matching network, an mBART-style encoder--decoder, and a decoder-only language model. Each analysis is interpreted against an appropriate control on the same data. TF-IDF provides a strong no-training lexical retrieval baseline, alongside representation analyses and generation controls for the learned models. A fixed string containing the 25 most frequent commentary words scores higher on generation overlap than the decoder-only model. Canonical correlation reaches 1.000 on Gaussian noise at these sample sizes, token-F1 spans only about 0.02--0.20 on this corpus, and the encoder--decoder continues to lower training loss for sixteen epochs after validation loss has begun to rise. One narrow result remains: the decoder-only model prefers authentic word order in 107 of 112 minimal-pair comparisons (95.5%), but does not reproduce held-out commentary content. We release the extraction and evaluation protocol; redistribution of the source commentaries remains subject to permission.

Related papers

• CodecArena: Codec Quality Assessment via Visual Reinforcement Learning [70.59130074737446]
  We propose CodecArena, the first vision-language framework for video coding quality assessment.<n>CodecArena casts evaluation as source-conditioned comparative reasoning between a reference and its reconstructions.<n>Its facet-anchored reward uses automatically derived facet directions as weak anchors, rather than human per-facet labels.
  arXiv  Detail & Related papers  (2026-08-10T05:37:06Z)
• SPOT: An Annotated French Corpus and Benchmark for Detecting Critical Interventions in Online Conversations [10.409447852574907]
  SPOT is the first annotated corpus translating the sociological concept of stopping point into a reproducible NLP task.<n>The corpus contains 43,305 manually annotated French Facebook comments linked to URLs flagged as false information by social media users.<n>We benchmark fine-tuned encoder models (CamemBERT) and instruction-tuned LLMs under various prompting strategies.
  arXiv  Detail & Related papers  (2025-11-10T18:54:40Z)
• Abjad AI at NADI 2025: CATT-Whisper: Multimodal Diacritic Restoration Using Text and Speech Representations [1.1391158217994781]
  We tackle the Diacritic Restoration (DR) task for Arabic dialectal sentences using a multimodal approach.<n>We propose a model that represents the text modality using an encoder extracted from our own pre-trained model named CATT.<n>Our experiments show that the proposed approach achieves a word error rate (WER) of 0.25 and a character error rate (CER) of 0.9 on the development set.
  arXiv  Detail & Related papers  (2025-10-28T09:58:18Z)
• SyllableLM: Learning Coarse Semantic Units for Speech Language Models [21.762112843104028]
  We introduce a controllable self-supervised technique to merge speech representations into coarser syllable-like units. Our method produces controllable-rate semantic units at as low as 5Hz and 60bps and SotA inc segmentation and clustering. SyllableLM achieves significant improvements in efficiency with a 30x reduction in training compute and a 4x wall-clock inference speedup.
  arXiv  Detail & Related papers  (2024-10-05T04:29:55Z)
• Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder [75.03283861464365]
  The seq2seq task aims at generating the target sequence based on the given input source sequence. Traditionally, most of the seq2seq task is resolved by an encoder to encode the source sequence and a decoder to generate the target text. Recently, a bunch of new approaches have emerged that apply decoder-only language models directly to the seq2seq task.
  arXiv  Detail & Related papers  (2023-04-08T15:44:29Z)
• Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages [58.43299730989809]
  We introduce Wav2Seq, the first self-supervised approach to pre-train both parts of encoder-decoder models for speech data. We induce a pseudo language as a compact discrete representation, and formulate a self-supervised pseudo speech recognition task. This process stands on its own, or can be applied as low-cost second-stage pre-training.
  arXiv  Detail & Related papers  (2022-05-02T17:59:02Z)
• ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference [70.36083572306839]
  This paper proposes a new training and inference paradigm for re-ranking. We finetune a pretrained encoder-decoder model using in the form of document to query generation. We show that this encoder-decoder architecture can be decomposed into a decoder-only language model during inference.
  arXiv  Detail & Related papers  (2022-04-25T06:26:29Z)
• Pre-Training Transformer Decoder for End-to-End ASR Model with Unpaired Speech Data [145.95460945321253]
  We introduce two pre-training tasks for the encoder-decoder network using acoustic units, i.e., pseudo codes. The proposed Speech2C can relatively reduce the word error rate (WER) by 19.2% over the method without decoder pre-training.
  arXiv  Detail & Related papers  (2022-03-31T15:33:56Z)
• Sentence Bottleneck Autoencoders from Transformer Language Models [53.350633961266375]
  We build a sentence-level autoencoder from a pretrained, frozen transformer language model. We adapt the masked language modeling objective as a generative, denoising one, while only training a sentence bottleneck and a single-layer modified transformer decoder. We demonstrate that the sentence representations discovered by our model achieve better quality than previous methods that extract representations from pretrained transformers on text similarity tasks, style transfer, and single-sentence classification tasks in the GLUE benchmark, while using fewer parameters than large pretrained models.
  arXiv  Detail & Related papers  (2021-08-31T19:39:55Z)
• Label-Synchronous Speech-to-Text Alignment for ASR Using Forward and Backward Transformers [49.403414751667135]
  This paper proposes a novel label-synchronous speech-to-text alignment technique for automatic speech recognition (ASR) The proposed method re-defines the speech-to-text alignment as a label-synchronous text mapping problem. Experiments using the corpus of spontaneous Japanese (CSJ) demonstrate that the proposed method provides an accurate utterance-wise alignment.
  arXiv  Detail & Related papers  (2021-04-21T03:05:12Z)
• Attentional Speech Recognition Models Misbehave on Out-of-domain Utterances [16.639133822656458]
  We decode audio from the British National Corpus with an attentional encoder-decoder model trained solely on the LibriSpeech corpus. We observe that there are many 5-second recordings that produce more than 500 characters of decoding output. A frame-synchronous hybrid (DNN-HMM) model trained on the same data does not produce these unusually long transcripts.
  arXiv  Detail & Related papers  (2020-02-12T18:53:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.