When Does Synthetic Patent Data Help? Volume-Fidelity Trade-offs in Low-Resource Multi-Label Classification

• URL: http://arxiv.org/abs/2605.24296v2
• Date: Tue, 26 May 2026 03:53:29 GMT
• Title: When Does Synthetic Patent Data Help? Volume-Fidelity Trade-offs in Low-Resource Multi-Label Classification
• Abstract summary: It is shown that the claimed improvements in micro F1 for BERT-for-Patents mainly reflect a volume effect.<n>A corpus that allows for improvement in classification performance up to +0.58 in raw micro F1 may adversely affect a Jaccard-overlap retrieval proxy.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The issues that must be considered regarding the utilization of synthetic data generated through LLMs for multilabel patent classification include (i) when the use of such data may help and (ii) why. Indeed, the former part appropriately adjusts for the possibility of improving results by an increase in sample size. The current experiment involves six open-source LLMs (from 3.8B to 12B parameters) for four real-data regimes in classification of 64 WIPO labels of assistive technologies. Both full-synthesis generation, conditioned on the label set, and paraphrasing methods are applied, with each used in combination with three classifier categories. It is shown that the claimed improvements in micro F1 for BERT-for-Patents from 0.120 to 0.702 mainly reflect a volume effect; indeed, replication with replacement in 165 examples produces 0.678. Thus, the improvement over the control is +0.024, while compared to the best baseline (focal loss reweighting) is +0.219. The second crucial point to consider here is that of evolving fidelity scores as the data generation regime varies. For low real-data regimes, the volume effect dominates and the correlation coefficient between maximum mean discrepancy (MMD) and classification performance equals r = +0.95. As more real data is used, the correlation becomes inverted and reaches r = -0.73 at the 1:10 regime (Fisher z = +6.47, p < 0.001, 95% CI on Delta r [ +0.96, +1.00 ]). In terms of a fixed budget allocation, combining real data (about 20-30%) with synthetic (70-80%) outperforms both purely synthetic and purely real strategies. Moreover, a corpus that allows for improvement in classification performance up to +0.58 in raw micro F1 may adversely affect a Jaccard-overlap retrieval proxy. Prompt-family variations for other genres may provide some explanation of the phenomenon, but using the standard-patent filter still decreases nDCG@10 by 26%.

Related papers

• Improving O-RADS Risk Stratification from Ultrasound Reports: A Comparative Evaluation of Hybrid versus End-to-End LLM Reasoning Strategies [17.85382282659117]
  We compared the performance of large language models (LLMs) and reasoning strategies for automated OvarianAd Reportingal and Data Fable System (ORADS) classification.<n>The feature-based hybrid architecture demonstrated almost the best performance, achieving an accuracy of 99.2% (387 of 390) and agreement with the reference standard.
  arXiv  Detail & Related papers  (2026-08-24T10:04:17Z)
• Synthetic Network Packet Generation through Statistical Learning and Genetic Algorithms [6.42094170804706]
  This paper proposes and compares two constraint-enforcing approaches for synthetic IoT network packet generation.<n> Evaluation on the complete ACI IoT 2023 dataset demonstrates that both methods achieve PASS status across all categories.
  arXiv  Detail & Related papers  (2026-06-18T18:52:51Z)
• Benchmarking Patent Embeddings: A Multi-Task Evaluation of 22 Models Across Retrieval, Classification, and Clustering [0.0]
  The study uses 113,148 assistive-technology patents, 46,069 citation-graph retrieval queries, and the public DAPFAM dataset for external validation.<n>Our framework covers citation-based retrieval, hybrid sparse-dense fusion, multi-label classification over five datasets, unsupervised clustering, six text-section views, domain-adaptive fine-tuning of four models, jurisdiction analysis, and proprietary DWPI (Derwent World Patents Index, Clarivate) expert-written content.
  arXiv  Detail & Related papers  (2026-05-22T23:51:13Z)
• Provable Sparse Inversion and Token Relabel Enhanced One-shot Federated Learning with ViTs [77.42033827176806]
  One-Shot Federated Learning, where a central server learns a global model in a single communication round, has emerged as a promising paradigm.<n>We propose a novel Federated Model Inversion and Token Relabel framework, which trains the global model by fully exploiting all patches of synthetic images.
  arXiv  Detail & Related papers  (2026-05-11T15:49:08Z)
• Reducing Maintenance Burden in Behaviour-Driven Development: A Paraphrase-Robust Duplicate-Step Detector with a 1.1M-Step Open Benchmark [1.9537983097153042]
  Behaviour-Driven Development suites accumulate step-duplicate duplication with documented maintenance cost.<n>We release the largest cross-organisational BDD step corpus to date.
  arXiv  Detail & Related papers  (2026-04-22T11:44:05Z)
• Putting HUMANS first: Efficient LAM Evaluation with Human Preference Alignment [53.72927532626824]
  We show that subsets of just 50 examples (0.3% of data) can achieve over 0.93 Pearson correlation with full benchmark scores.<n>To better predict preferences, we trained regression models on these selected subsets, achieving 0.98 correlation.<n>This demonstrates that in regression modeling, well-curated subsets outpredict the full benchmark, showing quality over quantity.
  arXiv  Detail & Related papers  (2026-04-20T00:57:31Z)
• Do We Still Need Humans in the Loop? Comparing Human and LLM Annotation in Active Learning for Hostility Detection [68.37351671559675]
  Active learning can annotate thousands of instances from a short prompt at negligible cost.<n>This raises two questions for active learning (AL): can LLM labels replace human labels within the AL loop, and does AL remain necessary when entire corpora can be labelled at once?<n>We investigate both questions on a new dataset of 277,902 German political TikTok comments.
  arXiv  Detail & Related papers  (2026-04-15T14:10:58Z)
• Do Generative Metrics Predict YOLO Performance? An Evaluation Across Models, Augmentation Ratios, and Dataset Complexity [43.338311770275745]
  We present a controlled evaluation of synthetic augmentation for YOLOv11 across three single-class detection regimes.<n>We benchmark six GAN-, diffusion-, and hybrid-based generators over augmentation ratios from 10% to 150% of the real training split.<n>For each dataset-generator-augmentation configuration, we compute pre-training dataset metrics under a matched-size bootstrap protocol.
  arXiv  Detail & Related papers  (2026-02-20T03:02:36Z)
• Enhancing Diversity and Feasibility: Joint Population Synthesis from Multi-source Data Using Generative Models [4.73459038844245]
  This study proposes a novel method to simultaneously integrate and synthesize multi-source datasets using a Wasserstein Generative Adversarial Network (WGAN) with gradient penalty.<n>Results show that the proposed joint approach outperforms the sequential baseline, with recall increasing by 7% and precision by 15%.<n>Since synthetic populations serve as a key input for agent-based models (ABM), this multi-source generative approach has the potential to significantly enhance the accuracy and reliability of ABM.
  arXiv  Detail & Related papers  (2026-02-17T00:02:30Z)
• Identifying Evidence-Based Nudges in Biomedical Literature with Large Language Models [2.2015514798912412]
  We present a scalable, AI-powered system that identifies and extracts evidence-based behavioral nudges from unstructured biomedical literature.<n>Nudges are subtle, non-coercive interventions that influence behavior without limiting choice, showing strong impact on health outcomes like medication adherence.
  arXiv  Detail & Related papers  (2026-02-10T22:36:07Z)
• Synthetic Data for Veterinary EHR De-identification: Benefits, Limits, and Safety Trade-offs Under Fixed Compute [0.0]
  This study evaluates whether large language model (LLM)-generated synthetic narratives improve de-identification safety.<n>We generated 10,382 synthetic notes using a privacy-preserving "template-only" regime.<n>We conclude that synthetic augmentation is effective for expanding exposure but is complementary, not substitutive, for safety-critical veterinary de-identification.
  arXiv  Detail & Related papers  (2026-01-13T19:35:25Z)
• Causal Data Augmentation for Robust Fine-Tuning of Tabular Foundation Models [45.21399037022976]
  CausalMixFT is a method that enhances fine-tuning robustness and downstream performance.<n>It generates structurally consistent synthetic samples using Structural Causal Models (SCMs) fitted on the target dataset.<n> evaluated across 33 classification datasets from TabArena and over 2300 fine-tuning runs.
  arXiv  Detail & Related papers  (2026-01-07T17:16:39Z)
• RESTRAIN: From Spurious Votes to Signals -- Self-Driven RL with Self-Penalization [52.01526898310723]
  We introduce RESTRAIN, a self-penalizing RL framework that converts the absence of gold labels into a useful learning signal.<n>Instead of overcommitting to spurious majority votes, RESTRAIN exploits signals from the model's entire answer distribution.<n>On challenging reasoning benchmarks, RESTRAIN delivers large gains using only unlabeled data.
  arXiv  Detail & Related papers  (2025-10-02T16:24:01Z)
• Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout [62.73150122809138]
  Federated Learning (FL) is a promising distributed machine learning approach that enables collaborative training of a global model using multiple edge devices.<n>We propose the FedDHAD FL framework, which comes with two novel methods: Dynamic Heterogeneous model aggregation (FedDH) and Adaptive Dropout (FedAD)<n>The combination of these two methods makes FedDHAD significantly outperform state-of-the-art solutions in terms of accuracy (up to 6.7% higher), efficiency (up to 2.02 times faster), and cost (up to 15.0% smaller)
  arXiv  Detail & Related papers  (2025-07-14T16:19:00Z)
• Revisiting Sparsity Constraint Under High-Rank Property in Partial Multi-Label Learning [48.528799044535155]
  Partial Multi-Label Learning (PML) extends the multi-label learning paradigm to scenarios where each sample is associated with a candidate label set.<n>Existing PML methods rely on two assumptions: sparsity of the noise label matrix and low-rankness of the ground-truth label matrix.<n>We propose a novel method Schirn, which introduces a sparsity constraint on the noise label matrix while enforcing a high-rank property on the predicted label matrix.
  arXiv  Detail & Related papers  (2025-05-27T09:25:50Z)
• Scalable Unit Harmonization in Medical Informatics via Bayesian-Optimized Retrieval and Transformer-Based Re-ranking [0.0]
  We develop a scalable methodology for harmonizing inconsistent units in large-scale clinical datasets.<n>We implement a multi-stage pipeline: filtering, identification, harmonization proposal generation, automated re-ranking, and manual validation.<n>The system achieved 83.39% precision at rank 1 and 94.66% recall at rank 5.
  arXiv  Detail & Related papers  (2025-05-01T19:09:15Z)
• Group-Adaptive Threshold Optimization for Robust AI-Generated Text Detection [58.419940585826744]
  We introduce FairOPT, an algorithm for group-specific threshold optimization for probabilistic AI-text detectors.<n>We partitioned data into subgroups based on attributes (e.g., text length and writing style) and implemented FairOPT to learn decision thresholds for each group to reduce discrepancy.<n>Our framework paves the way for more robust classification in AI-generated content detection via post-processing.
  arXiv  Detail & Related papers  (2025-02-06T21:58:48Z)
• PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learning [54.409634256153154]
  In Reinforcement Learning (RL), enhancing sample efficiency is crucial. In principle, off-policy RL algorithms can improve sample efficiency by allowing multiple updates per environment interaction. Our study investigates the underlying causes of this phenomenon by dividing plasticity into two aspects.
  arXiv  Detail & Related papers  (2023-06-19T06:14:51Z)
• HFedMS: Heterogeneous Federated Learning with Memorable Data Semantics in Industrial Metaverse [49.1501082763252]
  This paper presents HFEDMS for incorporating practical FL into the emerging Industrial Metaverse. It reduces data heterogeneity through dynamic grouping and training mode conversion. Then, it compensates for the forgotten knowledge by fusing compressed historical data semantics. Experiments have been conducted on the streamed non-i.i.d. FEMNIST dataset using 368 simulated devices.
  arXiv  Detail & Related papers  (2022-11-07T04:33:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.