Noisy-Channel Minimum Bayes Risk Decoding

• URL: http://arxiv.org/abs/2607.05198v1
• Date: Mon, 06 Jul 2026 15:14:13 GMT
• Title: Noisy-Channel Minimum Bayes Risk Decoding
• Abstract summary: Minimum Bayes Risk (MBR) decoding yields more robust and higher-quality text generation than maximum a posteriori (MAP) decoding.<n>There exists a discrepancy in the design: hypothesis selection calculates expected utility scores conditioned on given pseudo-references.<n>It is important to consider both hypothesis-to-reference and reference-to-hypothesis directional effects.
• Score: 58.87954687016989
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Minimum Bayes Risk (MBR) decoding yields more robust and higher-quality text generation than maximum a posteriori (MAP) decoding by selecting hypotheses that maximize expected utility over sampled pseudo-references. However, there exists a discrepancy in the design: hypothesis selection calculates expected utility scores conditioned on given pseudo-references, while commonly used evaluation metrics, e.g., BLEU and COMET, are asymmetric. Therefore, it is important to consider both hypothesis-to-reference and reference-to-hypothesis directional effects. In this study, we introduce a noisy channel decomposition of MBR decoding that naturally incorporates bidirectional effects to account for these asymmetries. We decompose MBR decoding into four interacting components: hypothesis-to-reference likelihood, reference-to-hypothesis likelihood, hypothesis prior, and reference prior. This decomposition provides a unified interpretation of existing MBR variants and enables metric- and task-specific interpretability by isolating the contribution of each channel. Our comprehensive analysis reveals that channel-wise contributions exhibit distinct characteristics across metrics while remaining consistent across tasks, and suggests that appropriate channel weighting may lead to improvements over original MBR decoding.

Related papers

• Overfitting Mitigation via Singular Value Decomposition in Minimum Bayes Risk Decoding [53.9009465845701]
  We introduce SVD-MBR, which frames the pairwise utility matrix as a noisy information signal.<n> Experiments demonstrate that SVD-MBR successfully regularizes decoding, yielding substantial gains across a range of generalized metrics.
  arXiv  Detail & Related papers  (2026-09-01T12:11:29Z)
• Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach [58.15427952222424]
  General-purpose text embeddings are widely deployed for such tasks, but broad semantic similarity can place semantically similar yet task-distinct examples.<n>We introduce Prototype-Guided Contrastive Learning (PGCL), a prototype-guided geometric regularization module built on top of frozen text embeddings.
  arXiv  Detail & Related papers  (2026-08-15T13:19:25Z)
• Minimum Bayes Risk Decoding for Error Span Detection in Reference-Free Automatic Machine Translation Evaluation [50.83502171176548]
  State-of-the-art generative ESD methods typically decode using Maximum a Posteriori (MAP)<n>We address this issue by applying Minimum Bayes Risk (MBR) decoding to generative ESD models.
  arXiv  Detail & Related papers  (2025-12-08T13:21:44Z)
• Measurement-Constrained Sampling for Text-Prompted Blind Face Restoration [60.45423400845294]
  Blind face restoration (BFR) may correspond to multiple plausible high-quality (HQ) reconstructions under extremely low-quality (LQ) inputs.<n>We propose a Measurement-Constrained Sampling (MCS) approach that enables diverse LQ face reconstructions conditioned on different textual prompts.
  arXiv  Detail & Related papers  (2025-11-18T07:44:54Z)
• Post-Hoc Split-Point Self-Consistency Verification for Efficient, Unified Quantification of Aleatoric and Epistemic Uncertainty in Deep Learning [5.996056764788456]
  Uncertainty quantification (UQ) is vital for trustworthy deep learning, yet existing methods are either computationally intensive or provide only partial, task-specific estimates.<n>We propose a post-hoc single-forward-pass framework that jointly captures aleatoric and epistemic uncertainty without modifying or retraining pretrained models.<n>Our method applies emphSplit-Point Analysis (SPA) to decompose predictive residuals into upper and lower subsets, computing emphMean Absolute Residuals (MARs) on each side.
  arXiv  Detail & Related papers  (2025-09-16T17:16:01Z)
• Diversity Explains Inference Scaling Laws: Through a Case Study of Minimum Bayes Risk Decoding [32.02732402635305]
  Inference methods play an important role in eliciting the performance of large language models (LLMs)<n>Currently, LLMs use inference methods utilizing generated multiple samples, which can be derived from Minimum Bayes Risk (MBR) Decoding.<n>Previous studies have conducted empirical analyses to clarify the improvements in generation performance achieved by MBR decoding.<n>We offer a new theoretical interpretation of MBR decoding from the perspective of bias-diversity decomposition.
  arXiv  Detail & Related papers  (2024-10-19T07:32:10Z)
• Unveiling the Power of Source: Source-based Minimum Bayes Risk Decoding for Neural Machine Translation [30.323103270892734]
  Minimum Bayes Risk (MBR) decoding offers an alternative by seeking hypotheses with the highest expected utility.<n>Our findings suggest that sMBR is a promising approach for NMT decoding.
  arXiv  Detail & Related papers  (2024-06-17T15:13:52Z)
• On the True Distribution Approximation of Minimum Bayes-Risk Decoding [3.409873726183299]
  Minimum Bayes-risk (MBR) decoding has recently gained renewed attention in text generation. Previous studies reported that the performance varies by sampling methods. This study uses anomaly detection to measure the degree of approximation.
  arXiv  Detail & Related papers  (2024-03-31T17:47:22Z)
• Machine Reading Comprehension using Case-based Reasoning [92.51061570746077]
  We present an accurate and interpretable method for answer extraction in machine reading comprehension. Our method builds upon the hypothesis that contextualized answers to similar questions share semantic similarities with each other.
  arXiv  Detail & Related papers  (2023-05-24T07:09:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.