Enhancing Document-Level Machine Translation via Filtered Synthetic Corpora and Two-Stage LLM Adaptation

• URL: http://arxiv.org/abs/2603.22186v1
• Date: Mon, 23 Mar 2026 16:48:22 GMT
• Title: Enhancing Document-Level Machine Translation via Filtered Synthetic Corpora and Two-Stage LLM Adaptation
• Abstract summary: Large Language Models (LLMs) excel at modeling contextual information, making them a natural fit for document-level translation tasks.<n>We propose a two-stage fine-tuning strategy leveraging LLM-augmented document-level data.<n>We employ a two-stage fine-tuning strategy: first fine-tuning on the abundant sentence-level MT resources, and then on the filtered document-level corpus.
• Score: 4.0913617519821255
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In Machine Translation, Large Language Models (LLMs) have generally underperformed compared to conventional encoder-decoder systems and thus see limited adoption. However, LLMs excel at modeling contextual information, making them a natural fit for document-level translation tasks where coherence across sentences is crucial. Despite this potential, document-level MT with LLMs faces two key challenges: (1) the scarcity of large-scale, high-quality document-level parallel data; and (2) the propensity of LLMs to introduce hallucinations and omissions during generation. To address these challenges, we propose a two-stage fine-tuning strategy leveraging LLM-augmented document-level data. First, we augment data by converting summarization data into document-level parallel data using a LLM, and then filter it using multiple metrics, leveraging sacreBLEU, COMET, and LaBSE-based cosine similarity-to improve data quality. Finally, we employ a two-stage fine-tuning strategy: first fine-tuning on the abundant sentence-level MT resources, and then on the filtered document-level corpus.

Related papers

• URaG: Unified Retrieval and Generation in Multimodal LLMs for Efficient Long Document Understanding [55.45331924836242]
  We present URaG, a framework that Unifies Retrieval and Generation within a single MLLM.<n>We show that URaG achieves state-of-the-art performance while reducing computational overhead by 44-56%.
  arXiv  Detail & Related papers  (2025-11-13T17:54:09Z)
• Train a Unified Multimodal Data Quality Classifier with Synthetic Data [56.872668770081766]
  Multimodal Large Language Models (MLLMs) are continually pre-trained on a mixture of image-text caption data and interleaved document data.<n>We propose to train an efficient MLLM as a Unified Mulitmodal Data Quality to Filter both high-quality image-text caption and interleaved data.
  arXiv  Detail & Related papers  (2025-10-16T21:53:28Z)
• TIT: A Tree-Structured Instruction Tuning Approach for LLM-Based Code Translation [11.882496324328905]
  We propose TIT, a Tree-structured Instruction Tuning paradigm for LLM-based code translation.<n>To mitigate syntactic confusion, the syntactic information representation module integrates language-agnostic syntactic features.<n>To generate high-quality fine-grained parallel data, the fine-grained parallel dataset augmentation module aligns nodes with code segments.
  arXiv  Detail & Related papers  (2025-10-10T13:53:46Z)
• DaMoC: Efficiently Selecting the Optimal Large Language Model for Fine-tuning Domain Tasks Based on Data and Model Compression [7.1654056866441245]
  Large language models (LLMs) excel in general tasks but struggle with domain-specific ones, requiring fine-tuning with specific data.<n>We introduce a Data and Model Compression Framework (DaMoC) that addresses this challenge.<n>We show that we can select the optimal LLM while saving approximately 20-fold in training time.
  arXiv  Detail & Related papers  (2025-09-01T08:06:49Z)
• M-DocSum: Do LVLMs Genuinely Comprehend Interleaved Image-Text in Document Summarization? [49.53982792497275]
  We investigate whether Large Vision-Language Models (LVLMs) genuinely comprehend interleaved image-text in the document.<n>Existing document understanding benchmarks often assess LVLMs using question-answer formats.<n>We introduce a novel and challenging Multimodal Document Summarization Benchmark (M-DocSum-Bench)<n>M-DocSum-Bench comprises 500 high-quality arXiv papers, along with interleaved multimodal summaries aligned with human preferences.
  arXiv  Detail & Related papers  (2025-03-27T07:28:32Z)
• Integrating Planning into Single-Turn Long-Form Text Generation [66.08871753377055]
  We propose to use planning to generate long form content. Our main novelty lies in a single auxiliary task that does not require multiple rounds of prompting or planning. Our experiments demonstrate on two datasets from different domains, that LLMs fine-tuned with the auxiliary task generate higher quality documents.
  arXiv  Detail & Related papers  (2024-10-08T17:02:40Z)
• Why Not Transform Chat Large Language Models to Non-English? [60.200490339844634]
  The scarcity of non-English data limits the development of non-English large language models (LLMs)<n>TransLLM divides the transfer problem into some common sub-tasks with the translation chain-of-thought.<n>Our method, using only single-turn data, outperforms strong baselines and ChatGPT on multi-turn benchmark MT-bench.
  arXiv  Detail & Related papers  (2024-05-22T18:53:25Z)
• LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders [34.421335513040795]
  Large decoder-only language models (LLMs) are the state-of-the-art models on most of today's NLP tasks and benchmarks. We introduce LLM2Vec, a simple unsupervised approach that can transform any decoder-only LLM into a strong text encoder.
  arXiv  Detail & Related papers  (2024-04-09T02:51:05Z)
• Adapting Large Language Models for Document-Level Machine Translation [46.370862171452444]
  Large language models (LLMs) have significantly advanced various natural language processing (NLP) tasks. Recent research indicates that moderately-sized LLMs often outperform larger ones after task-specific fine-tuning. This study focuses on adapting LLMs for document-level machine translation (DocMT) for specific language pairs.
  arXiv  Detail & Related papers  (2024-01-12T09:29:13Z)
• Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimes [57.62036621319563]
  We introduce CLLM, which leverages the prior knowledge of Large Language Models (LLMs) for data augmentation in the low-data regime. We demonstrate the superior performance of CLLM in the low-data regime compared to conventional generators.
  arXiv  Detail & Related papers  (2023-12-19T12:34:46Z)
• Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes [54.13559879916708]
  EVAPORATE is a prototype system powered by large language models (LLMs)<n>Code synthesis is cheap, but far less accurate than directly processing each document with the LLM.<n>We propose an extended code implementation, EVAPORATE-CODE+, which achieves better quality than direct extraction.
  arXiv  Detail & Related papers  (2023-04-19T06:00:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.