A Hybrid Architecture with Efficient Fine Tuning for Abstractive Patent Document Summarization

• URL: http://arxiv.org/abs/2503.10354v1
• Date: Thu, 13 Mar 2025 13:30:54 GMT
• Title: A Hybrid Architecture with Efficient Fine Tuning for Abstractive Patent Document Summarization
• Authors: Nevidu Jayatilleke, Ruvan Weerasinghe,
• Abstract summary: This study proposes a system for efficiently creating abstractive summaries of patent records.<n>The procedure involves leveraging the LexRank graph-based algorithm to retrieve the important sentences from input parent texts.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic patent summarization approaches that help in the patent analysis and comprehension procedure are in high demand due to the colossal growth of innovations. The development of natural language processing (NLP), text mining, and deep learning has notably amplified the efficacy of text summarization models for abundant types of documents. Summarizing patent text remains a pertinent challenge due to the labyrinthine writing style of these documents, which includes technical and legal intricacies. Additionally, these patent document contents are considerably lengthier than archetypal documents, which intricates the process of extracting pertinent information for summarization. Embodying extractive and abstractive text summarization methodologies into a hybrid framework, this study proposes a system for efficiently creating abstractive summaries of patent records. The procedure involves leveraging the LexRank graph-based algorithm to retrieve the important sentences from input parent texts, then utilizing a Bidirectional Auto-Regressive Transformer (BART) model that has been fine-tuned using Low-Ranking Adaptation (LoRA) for producing text summaries. This is accompanied by methodical testing and evaluation strategies. Furthermore, the author employed certain meta-learning techniques to achieve Domain Generalization (DG) of the abstractive component across multiple patent fields.

Related papers

• Advancements in Natural Language Processing for Automatic Text Summarization [0.0]
  Authors explored existing hybrid techniques that have employed both extractive and abstractive methodologies.<n>Process of summarizing textual information continues to be significantly constrained by the intricate writing styles of a variety of texts.
  arXiv  Detail & Related papers  (2025-02-27T05:17:36Z)
• Context-Aware Hierarchical Merging for Long Document Summarization [56.96619074316232]
  We propose different approaches to enrich hierarchical merging with context from the source document.<n> Experimental results on datasets representing legal and narrative domains show that contextual augmentation consistently outperforms zero-shot and hierarchical merging baselines.
  arXiv  Detail & Related papers  (2025-02-03T01:14:31Z)
• Pap2Pat: Benchmarking Outline-Guided Long-Text Patent Generation with Patent-Paper Pairs [13.242188189150987]
  We build PAP2PAT, an open benchmark for patent drafting consisting of 1.8k patent-paper pairs describing the same inventions.<n>Our evaluation using PAP2PAT and a human case study show that LLMs can effectively leverage information from the paper, but still struggle to provide the necessary level of detail.
  arXiv  Detail & Related papers  (2024-10-09T15:52:48Z)
• BERT-VBD: Vietnamese Multi-Document Summarization Framework [2.2526595080231857]
  An emerging and promising strategy involves a synergistic fusion of extractive and abstractive summarization methods. This paper presents a novel Vietnamese MDS framework leveraging a two-component pipeline architecture. The proposed framework attains ROUGE-2 scores of 39.6% on the VN-MDS dataset and outperforming the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-09-18T16:56:06Z)
• Natural Language Processing in Patents: A Survey [0.0]
  Patents, encapsulating crucial technical and legal information, present a rich domain for natural language processing (NLP) applications. As NLP technologies evolve, large language models (LLMs) have demonstrated outstanding capabilities in general text processing and generation tasks. This paper aims to equip NLP researchers with the essential knowledge to navigate this complex domain efficiently.
  arXiv  Detail & Related papers  (2024-03-06T23:17:16Z)
• TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision [61.186488081379]
  We propose TextFormer, a query-based end-to-end text spotter with Transformer architecture. TextFormer builds upon an image encoder and a text decoder to learn a joint semantic understanding for multi-task modeling. It allows for mutual training and optimization of classification, segmentation, and recognition branches, resulting in deeper feature sharing.
  arXiv  Detail & Related papers  (2023-06-06T03:37:41Z)
• Active Learning for Abstractive Text Summarization [50.79416783266641]
  We propose the first effective query strategy for Active Learning in abstractive text summarization. We show that using our strategy in AL annotation helps to improve the model performance in terms of ROUGE and consistency scores.
  arXiv  Detail & Related papers  (2023-01-09T10:33:14Z)
• The Harvard USPTO Patent Dataset: A Large-Scale, Well-Structured, and Multi-Purpose Corpus of Patent Applications [8.110699646062384]
  We introduce the Harvard USPTO Patent dataset (HUPD) With more than 4.5 million patent documents, HUPD is two to three times larger than comparable corpora. By providing each application's metadata along with all of its text fields, the dataset enables researchers to perform new sets of NLP tasks.
  arXiv  Detail & Related papers  (2022-07-08T17:57:15Z)
• GERE: Generative Evidence Retrieval for Fact Verification [57.78768817972026]
  We propose GERE, the first system that retrieves evidences in a generative fashion. The experimental results on the FEVER dataset show that GERE achieves significant improvements over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-04-12T03:49:35Z)
• Patent Sentiment Analysis to Highlight Patent Paragraphs [0.0]
  Given a patent document, identifying distinct semantic annotations is an interesting research aspect. In the process of manual patent analysis, to attain better readability, recognising the semantic information by marking paragraphs is in practice. This work assist patent practitioners in highlighting semantic information automatically and aid to create a sustainable and efficient patent analysis using the aptitude of Machine Learning.
  arXiv  Detail & Related papers  (2021-11-06T13:28:29Z)
• Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation [93.87095877617968]
  We propose Constrained Abstractive Summarization (CAS), a general setup that preserves the factual consistency of abstractive summarization. We adopt lexically constrained decoding, a technique generally applicable to autoregressive generative models, to fulfill CAS. We observe up to 13.8 ROUGE-2 gains when only one manual constraint is used in interactive summarization.
  arXiv  Detail & Related papers  (2020-10-24T00:27:44Z)
• Multi-Fact Correction in Abstractive Text Summarization [98.27031108197944]
  Span-Fact is a suite of two factual correction models that leverages knowledge learned from question answering models to make corrections in system-generated summaries via span selection. Our models employ single or multi-masking strategies to either iteratively or auto-regressively replace entities in order to ensure semantic consistency w.r.t. the source text. Experiments show that our models significantly boost the factual consistency of system-generated summaries without sacrificing summary quality in terms of both automatic metrics and human evaluation.
  arXiv  Detail & Related papers  (2020-10-06T02:51:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.