Kanbun-LM: Reading and Translating Classical Chinese in Japanese Methods by Language Models

• URL: http://arxiv.org/abs/2305.12759v2
• Date: Tue, 2 Jul 2024 13:39:31 GMT
• Title: Kanbun-LM: Reading and Translating Classical Chinese in Japanese Methods by Language Models
• Authors: Hao Wang, Hirofumi Shimizu, Daisuke Kawahara,
• Abstract summary: We construct the first Classical-Chinese-to-Kanbun dataset in the world. Character reordering and machine translation play a significant role in Kanbun comprehension. We release our code and dataset on GitHub.
• Score: 17.749113496737106
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent studies in natural language processing (NLP) have focused on modern languages and achieved state-of-the-art results in many tasks. Meanwhile, little attention has been paid to ancient texts and related tasks. Classical Chinese first came to Japan approximately 2,000 years ago. It was gradually adapted to a Japanese form called Kanbun-Kundoku (Kanbun) in Japanese reading and translating methods, which has significantly impacted Japanese literature. However, compared to the rich resources for ancient texts in mainland China, Kanbun resources remain scarce in Japan. To solve this problem, we construct the first Classical-Chinese-to-Kanbun dataset in the world. Furthermore, we introduce two tasks, character reordering and machine translation, both of which play a significant role in Kanbun comprehension. We also test the current language models on these tasks and discuss the best evaluation method by comparing the results with human scores. We release our code and dataset on GitHub.

Related papers

• When Does Classical Chinese Help? Quantifying Cross-Lingual Transfer in Hanja and Kanbun [48.07219104902607]
  We question the assumption of cross-lingual transferability from Classical Chinese to Hanja and Kanbun. Our experiments show minimal impact of Classical Chinese datasets on language model performance for ancient Korean documents written in Hanja.
  arXiv  Detail & Related papers  (2024-11-07T15:59:54Z)
• Puzzle Pieces Picker: Deciphering Ancient Chinese Characters with Radical Reconstruction [73.26364649572237]
  Oracle Bone Inscriptions is one of the oldest existing forms of writing in the world. A large number of Oracle Bone Inscriptions (OBI) remain undeciphered, making it one of the global challenges in paleography today. This paper introduces a novel approach, namely Puzzle Pieces Picker (P$3$), to decipher these enigmatic characters through radical reconstruction.
  arXiv  Detail & Related papers  (2024-06-05T07:34:39Z)
• Shuo Wen Jie Zi: Rethinking Dictionaries and Glyphs for Chinese Language Pre-training [50.100992353488174]
  We introduce CDBERT, a new learning paradigm that enhances the semantics understanding ability of the Chinese PLMs with dictionary knowledge and structure of Chinese characters. We name the two core modules of CDBERT as Shuowen and Jiezi, where Shuowen refers to the process of retrieving the most appropriate meaning from Chinese dictionaries. Our paradigm demonstrates consistent improvements on previous Chinese PLMs across all tasks.
  arXiv  Detail & Related papers  (2023-05-30T05:48:36Z)
• HUE: Pretrained Model and Dataset for Understanding Hanja Documents of Ancient Korea [59.35609710776603]
  We release the Hanja Understanding Evaluation dataset consisting of chronological attribution, topic classification, named entity recognition, and summary retrieval tasks. We also present BERT-based models continued training on the two major corpora from the 14th to the 19th centuries: the Annals of the Joseon Dynasty and Diaries of the Royal Secretariats.
  arXiv  Detail & Related papers  (2022-10-11T03:04:28Z)
• Translating Hanja Historical Documents to Contemporary Korean and English [52.625998002213585]
  Annals of Joseon Dynasty contain the daily records of the Kings of Joseon, the 500-year kingdom preceding the modern nation of Korea. The Annals were originally written in an archaic Korean writing system, Hanja', and were translated into Korean from 1968 to 1993. Since then, the records of only one king have been completed in a decade. We propose H2KE, a neural machine translation model, that translates historical documents in Hanja to more easily understandable Korean and to English.
  arXiv  Detail & Related papers  (2022-05-20T08:25:11Z)
• Native Chinese Reader: A Dataset Towards Native-Level Chinese Machine Reading Comprehension [9.66226932673554]
  Native Chinese Reader is a new machine reading comprehension dataset with particularly long articles in both modern and classical Chinese. NCR is collected from the exam questions for the Chinese course in China's high schools, which are designed to evaluate the language proficiency of native Chinese youth.
  arXiv  Detail & Related papers  (2021-12-13T09:11:38Z)
• Predicting the Ordering of Characters in Japanese Historical Documents [6.82324732276004]
  Change in Japanese writing system in 1900 made historical documents inaccessible for the general public. We explore a few approaches to the task of predicting the sequential ordering of the characters. Our best-performing system has an accuracy of 98.65% and has a perfect accuracy on 49% of the books in our dataset.
  arXiv  Detail & Related papers  (2021-06-12T14:39:20Z)
• Deep Learning for Text Style Transfer: A Survey [71.8870854396927]
  Text style transfer is an important task in natural language generation, which aims to control certain attributes in the generated text. We present a systematic survey of the research on neural text style transfer, spanning over 100 representative articles since the first neural text style transfer work in 2017. We discuss the task formulation, existing datasets and subtasks, evaluation, as well as the rich methodologies in the presence of parallel and non-parallel data.
  arXiv  Detail & Related papers  (2020-11-01T04:04:43Z)
• KINNEWS and KIRNEWS: Benchmarking Cross-Lingual Text Classification for Kinyarwanda and Kirundi [18.01565807026177]
  We introduce two news datasets for classification of news articles in Kinyarwanda and Kirundi, two low-resource African languages. We provide statistics, guidelines for preprocessing, and monolingual and cross-lingual baseline models. Our experiments show that training embeddings on the relatively higher-resourced Kinyarwanda yields successful cross-lingual transfer to Kirundi.
  arXiv  Detail & Related papers  (2020-10-23T05:37:42Z)
• AnchiBERT: A Pre-Trained Model for Ancient ChineseLanguage Understanding and Generation [22.08457469951396]
  AnchiBERT is a pre-trained language model based on the architecture of BERT. We evaluate AnchiBERT on both language understanding and generation tasks, including poem classification.
  arXiv  Detail & Related papers  (2020-09-24T03:41:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.