SCRIPT: A Subcharacter Compositional Representation Injection Module for Korean Pre-Trained Language Models

• URL: http://arxiv.org/abs/2604.12377v1
• Date: Tue, 14 Apr 2026 07:09:44 GMT
• Title: SCRIPT: A Subcharacter Compositional Representation Injection Module for Korean Pre-Trained Language Models
• Abstract summary: SCRIPT is a model-agnostic module that injects subcharacter compositional knowledge into Korean PLMs.<n>SCRIPT enhances all baselines across various Korean natural language understanding (NLU) and generation (NLG) tasks.
• Score: 11.111389249714533
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Korean is a morphologically rich language with a featural writing system in which each character is systematically composed of subcharacter units known as Jamo. These subcharacters not only determine the visual structure of Korean but also encode frequent and linguistically meaningful morphophonological processes. However, most current Korean language models (LMs) are based on subword tokenization schemes, which are not explicitly designed to capture the internal compositional structure of characters. To address this limitation, we propose SCRIPT, a model-agnostic module that injects subcharacter compositional knowledge into Korean PLMs. SCRIPT allows to enhance subword embeddings with structural granularity, without requiring architectural changes or additional pre-training. As a result, SCRIPT enhances all baselines across various Korean natural language understanding (NLU) and generation (NLG) tasks. Moreover, beyond performance gains, detailed linguistic analyses show that SCRIPT reshapes the embedding space in a way that better captures grammatical regularities and semantically cohesive variations. Our code is available at https://github.com/SungHo3268/SCRIPT.

Related papers

• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• KOMBO: Korean Character Representations Based on the Combination Rules of Subcharacters [20.335880983366508]
  The Korean writing system, textitHangeul, has a unique character representation rigidly following the invention principles recorded in textitHunminjeongeum.footnotetextitHunminjeongeum.<n>We introduce a novel framework for Korean pre-trained language models (PLMs) called KOMBO.<n>Our proposed method, KOMBO, exhibits notable experimental proficiency across diverse NLP tasks.
  arXiv  Detail & Related papers  (2026-04-27T01:53:52Z)
• Linguistically Informed Graph Model and Semantic Contrastive Learning for Korean Short Text Classification [2.4071330817126477]
  We propose LIGRAM, a hierarchical heterogeneous graph model for Korean short-text classification.<n>The proposed model constructs sub-graphs at the morpheme, part-of-speech, and named-entity levels and hierarchically integrates them to compensate for the limited contextual information in short texts.<n>We evaluate LIGRAM on four Korean short-text datasets, where it consistently outperforms existing baseline models.
  arXiv  Detail & Related papers  (2026-03-04T02:17:13Z)
• Enhanced Generative Structure Prior for Chinese Text Image Super-resolution [101.66745917380837]
  We introduce a text image framework designed to restore the precise strokes of low-resolution (LR) Chinese characters.<n>Our framework incorporates this structure prior within a StyleGAN model.<n>Our code and pre-trained models will be available at https://github.com/csi2016/MARCONetPlus.
  arXiv  Detail & Related papers  (2025-08-11T01:34:45Z)
• Splintering Nonconcatenative Languages for Better Tokenization [4.496923806879088]
  We present SPLINTER, a pre-processing step which rearranges text into a linear form.<n>We demonstrate its merit using both intrinsic measures evaluating token vocabularies in Hebrew, Arabic, and Malay.
  arXiv  Detail & Related papers  (2025-03-18T17:11:09Z)
• Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generator [55.94334001112357]
  We introduce a multilingual sign language model, Signs as Tokens (SOKE), which can generate 3D sign avatars autoregressively from text inputs.<n>We propose a retrieval-enhanced SLG approach, which incorporates external sign dictionaries to provide accurate word-level signs.
  arXiv  Detail & Related papers  (2024-11-26T18:28:09Z)
• Does Incomplete Syntax Influence Korean Language Model? Focusing on Word Order and Case Markers [7.275938266030414]
  Syntactic elements, such as word order and case markers, are fundamental in natural language processing. This study explores whether Korean language models can accurately capture this flexibility.
  arXiv  Detail & Related papers  (2024-07-12T11:33:41Z)
• MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modeling [70.34758460372629]
  We introduce a new paradigm that encodes the same information with segments of consistent size across diverse languages. MYTE produces shorter encodings for all 99 analyzed languages. This, in turn, improves multilingual LM performance and diminishes the perplexity gap throughout diverse languages.
  arXiv  Detail & Related papers  (2024-03-15T21:21:11Z)
• Word segmentation granularity in Korean [1.0619039878979954]
  There are multiple possible levels of word segmentation granularity in Korean. For specific language processing and corpus annotation tasks, several different granularity levels have been proposed and utilized. Interestingly, the granularity by separating only functional morphemes results in the optimal performance for phrase structure parsing.
  arXiv  Detail & Related papers  (2023-09-07T13:42:05Z)
• Automatic Extraction of Rules Governing Morphological Agreement [103.78033184221373]
  We develop an automated framework for extracting a first-pass grammatical specification from raw text. We focus on extracting rules describing agreement, a morphosyntactic phenomenon at the core of the grammars of many of the world's languages. We apply our framework to all languages included in the Universal Dependencies project, with promising results.
  arXiv  Detail & Related papers  (2020-10-02T18:31:45Z)
• Phonological Features for 0-shot Multilingual Speech Synthesis [50.591267188664666]
  We show that code-switching is possible for languages unseen during training, even within monolingual models. We generate intelligible, code-switched speech in a new language at test time, including the approximation of sounds never seen in training.
  arXiv  Detail & Related papers  (2020-08-06T18:25:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.