Exploring Post-Training Alignment of Small Language Models for Biomedical Data-to-Text Generation: A Case Study of Medication Leaflet

• URL: http://arxiv.org/abs/2607.13430v1
• Date: Wed, 15 Jul 2026 04:22:25 GMT
• Title: Exploring Post-Training Alignment of Small Language Models for Biomedical Data-to-Text Generation: A Case Study of Medication Leaflet
• Abstract summary: This study presents a comparative analysis of training small language models (SLMs) in specialized biomedical datato-text generation tasks.<n>We explore widely adopted post-training methods including supervised fine-tuning (SFT), direct preference optimization (DPO), odds ratio preference optimization (ORPO)<n>To assess cross-dataset generalizability, we also curated drug label data from openFDA.
• Score: 21.43689940835954
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Translating complex biomedical data into patient-friendly narratives is central to modern biomedical informatics. This study presents a comparative analysis of training small language models (SLMs) in specialized biomedical datato-text generation tasks. We explore widely adopted post-training methods including supervised fine-tuning (SFT), direct preference optimization (DPO), odds ratio preference optimization (ORPO), and group relative policy optimization (GRPO) with Qwen-based SLMs on a medicine package leaflets dataset. To assess cross-dataset generalizability, we also curated drug label data from openFDA. We evaluate models using both standard lexical overlap metrics like ROUGE as well as semantic similarity measures. Across our experiments, the results show that (1) the aligned SLMs outperform proprietary models like GPT-5; (2) ORPO outperforms the SFTbaselines; (3) GRPO yields the most robust cross-dataset performance among the alignment methods tested as well as GPT-5.

Related papers

• Unlocking Biological Workflows for Robust Protein-Text Question Answering: A Dual-Dimensional RAG Framework [60.82334952881798]
  Protein-Text Question Answering (QA) is crucial for interpreting biological sequences through natural language.<n>We propose 2D-ProteinRAG, a novel framework that empowers Large Language Models (LLMs) to operate within the gold-standard biological research workflow.<n>We show that 2D-Protein consistently achieves state-of-the-art performance, outperforming fine-tuned baselines and other RAG methods.
  arXiv  Detail & Related papers  (2026-05-17T05:03:24Z)
• FL-MedSegBench: A Comprehensive Benchmark for Federated Learning on Medical Image Segmentation [57.930067699003764]
  Federated learning (FL) offers a privacy-preserving paradigm for collaborative medical image analysis without sharing raw data.<n>We introduce FL-MedSegBench, the first comprehensive benchmark for federated learning on medical image segmentation.<n>We systematically evaluate eight generic FL (gFL) and five personalized FL (pFL) methods across multiple dimensions.
  arXiv  Detail & Related papers  (2026-03-12T08:28:41Z)
• A Federated and Parameter-Efficient Framework for Large Language Model Training in Medicine [59.78991974851707]
  Large language models (LLMs) have demonstrated strong performance on medical benchmarks, including question answering and diagnosis.<n>Most medical LLMs are trained on data from a single institution, which faces limitations in generalizability and safety in heterogeneous systems.<n>We introduce the model-agnostic and parameter-efficient federated learning framework for adapting LLMs to medical applications.
  arXiv  Detail & Related papers  (2026-01-29T18:48:21Z)
• Plain language adaptations of biomedical text using LLMs: Comparision of evaluation metrics [1.4984469763984425]
  This study investigated the application of Large Language Models (LLMs) for simplifying biomedical texts to enhance health literacy.<n>We developed and evaluated several approaches, specifically a baseline approach using a prompt template, a two AI agent approach, and a fine-tuning approach.
  arXiv  Detail & Related papers  (2025-12-18T13:37:58Z)
• A Dataset for Distilling Knowledge Priors from Literature for Therapeutic Design [16.22074029228085]
  We introduce Medex, a dataset of priors for design problems extracted from literature describing compounds used in lab settings.<n>Medex consists of 32.3 million pairs of natural language facts, and appropriate entity representations.
  arXiv  Detail & Related papers  (2025-08-14T17:59:37Z)
• Zero-Shot Document-Level Biomedical Relation Extraction via Scenario-based Prompt Design in Two-Stage with LLM [6.26004554105527]
  We propose a novel approach to achieve the same results from unannotated full documents using general large language models (LLMs) with lower hardware and labor costs.<n>Our approach combines two major stages: named entity recognition (NER) and relation extraction (RE)<n>To enhance the effectiveness of prompt, we propose a five-part template structure and a scenario-based prompt design principles.
  arXiv  Detail & Related papers  (2025-05-02T07:33:20Z)
• Knowledge-augmented Pre-trained Language Models for Biomedical Relation Extraction [3.13957359732631]
  Several studies report improved performance when incorporating additional context information while fine-tuning pre-trained language models (PLMs) for automatic relationship extraction (RE)<n>Our study addresses this research gap by evaluating PLMs enhanced with contextual information on five datasets spanning four relation scenarios within a consistent evaluation framework.<n>Although inclusion of context information yield only minor overall improvements, an ablation study reveals substantial benefits for smaller PLMs when such external data was included during fine-tuning.
  arXiv  Detail & Related papers  (2025-05-01T19:16:18Z)
• SampleLLM: Optimizing Tabular Data Synthesis in Recommendations [46.689486044254544]
  Tabular data synthesis is crucial in machine learning, yet existing general methods are highly data-dependent and often fall short in recommender systems.<n>This limitation arises from their difficulty in capturing complex distributions and understanding feature relationships from sparse and limited data.<n>We propose a novel two-stage framework named SampleLLM to improve the quality of LLM-based data synthesis for recommendation tasks.
  arXiv  Detail & Related papers  (2025-01-27T15:12:27Z)
• ClinicRealm: Re-evaluating Large Language Models with Conventional Machine Learning for Non-Generative Clinical Prediction Tasks [37.544994716002016]
  Large Language Models (LLMs) are increasingly deployed in medicine.<n>However, their utility in non-generative clinical prediction remains under-evaluated.<n>Our ClinicRealm study addresses this by benchmarking 15 GPT-style LLMs, 5 BERT-style models, and 11 traditional methods.
  arXiv  Detail & Related papers  (2024-07-26T06:09:10Z)
• PathLDM: Text conditioned Latent Diffusion Model for Histopathology [62.970593674481414]
  We introduce PathLDM, the first text-conditioned Latent Diffusion Model tailored for generating high-quality histopathology images. Our approach fuses image and textual data to enhance the generation process. We achieved a SoTA FID score of 7.64 for text-to-image generation on the TCGA-BRCA dataset, significantly outperforming the closest text-conditioned competitor with FID 30.1.
  arXiv  Detail & Related papers  (2023-09-01T22:08:32Z)
• BioREx: Improving Biomedical Relation Extraction by Leveraging Heterogeneous Datasets [7.7587371896752595]
  Biomedical relation extraction (RE) is a central task in biomedical natural language processing (NLP) research. We present a novel framework for systematically addressing the data heterogeneity of individual datasets and combining them into a large dataset. Our evaluation shows that BioREx achieves significantly higher performance than the benchmark system trained on the individual dataset.
  arXiv  Detail & Related papers  (2023-06-19T22:48:18Z)
• Interpretable Medical Diagnostics with Structured Data Extraction by Large Language Models [59.89454513692417]
  Tabular data is often hidden in text, particularly in medical diagnostic reports. We propose a novel, simple, and effective methodology for extracting structured tabular data from textual medical reports, called TEMED-LLM. We demonstrate that our approach significantly outperforms state-of-the-art text classification models in medical diagnostics.
  arXiv  Detail & Related papers  (2023-06-08T09:12:28Z)
• Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function [106.69643619725652]
  We develop a training strategy that allows even a simple BiLSTM model, when trained with cross-entropy loss, to achieve competitive results. We report state-of-the-art results for text classification task on several benchmark datasets.
  arXiv  Detail & Related papers  (2020-09-08T21:55:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.