PatentGPT: A Large Language Model for Patent Drafting Using Knowledge-based Fine-tuning Method
- URL: http://arxiv.org/abs/2409.00092v1
- Date: Mon, 26 Aug 2024 12:00:29 GMT
- Title: PatentGPT: A Large Language Model for Patent Drafting Using Knowledge-based Fine-tuning Method
- Authors: Runtao Ren, Jian Ma,
- Abstract summary: Existing large language models (LLMs) often fall short in this IP creation domain due to their lack of specialized knowledge and context-awareness.
We propose a groundbreaking framework for Knowledge Fine-Tuning (KFT) of LLMs, designed to endow AI with the ability to autonomously mine, understand, and apply domain-specific knowledge.
Our model, PatentGPT, has demonstrated outstanding performance, scoring up to approximately 400% higher in patent related benchmark tests compared to state-of-the-art models.
- Score: 1.4496326701907591
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: As humanity stands on the brink of a new era of technological innovation, the ability to rapidly transform creative ideas into protected intellectual property (IP) is more crucial than ever. However, the conventional processes for patent drafting are fraught with challenges, demanding a nuanced understanding of advanced field knowledge and technical concepts. Existing large language models (LLMs), while powerful, often fall short in this IP creation domain due to their lack of specialized knowledge and context-awareness necessary for generating technically accurate patent documents. To bridge this critical gap, we propose a groundbreaking framework for Knowledge Fine-Tuning (KFT) of LLMs, designed to endow AI with the ability to autonomously mine, understand, and apply domain-specific knowledge. Our model, PatentGPT leverages a unique combination of knowledge graph-based pre-training, domain-specific supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF). Through extensive evaluation, PatentGPT has demonstrated outstanding performance, scoring up to approximately 400% higher in patent related benchmark tests compared to state-of-the-art models. By KFT method the model's capability to not only assist but also augment human creativity and innovation, our approach sets a new standard for AI-driven intellectual property generation, paving the way for more efficient and effective invention processes.
Related papers
- Patent Novelty Assessment Accelerating Innovation and Patent Prosecution [0.873811641236639]
This report introduces a ground-breaking Patent Novelty Assessment and Claim Generation System.
Our system provides college students and researchers with an intuitive platform to navigate and grasp the intricacies of patent claims.
Unlike conventional analysis systems, our initiative harnesses a proprietary Chinese API to ensure unparalleled precision and relevance.
arXiv Detail & Related papers (2025-01-12T22:25:46Z) - EvoPat: A Multi-LLM-based Patents Summarization and Analysis Agent [0.0]
EvoPat is a multi-LLM-based patent agent designed to assist users in analyzing patents through Retrieval-Augmented Generation (RAG) and advanced search strategies.
We demonstrate that EvoPat outperforms GPT-4 in tasks such as patent summarization, comparative analysis, and technical evaluation.
arXiv Detail & Related papers (2024-12-24T02:21:09Z) - Early screening of potential breakthrough technologies with enhanced interpretability: A patent-specific hierarchical attention network model [4.779196219827507]
We propose an interpretable machine learning approach to predicting future citation counts from patent texts.
A case study of 35,376 pharmaceutical patents demonstrates the effectiveness of our approach.
It is expected that the proposed approach will enhance expert-machine collaboration in identifying breakthrough technologies.
arXiv Detail & Related papers (2024-07-24T02:17:10Z) - Natural Language Processing in Patents: A Survey [0.0]
Patents, encapsulating crucial technical and legal information, present a rich domain for natural language processing (NLP) applications.
As NLP technologies evolve, large language models (LLMs) have demonstrated outstanding capabilities in general text processing and generation tasks.
This paper aims to equip NLP researchers with the essential knowledge to navigate this complex domain efficiently.
arXiv Detail & Related papers (2024-03-06T23:17:16Z) - Federated Learning-Empowered AI-Generated Content in Wireless Networks [58.48381827268331]
Federated learning (FL) can be leveraged to improve learning efficiency and achieve privacy protection for AIGC.
We present FL-based techniques for empowering AIGC, and aim to enable users to generate diverse, personalized, and high-quality content.
arXiv Detail & Related papers (2023-07-14T04:13:11Z) - UNTER: A Unified Knowledge Interface for Enhancing Pre-trained Language
Models [100.4659557650775]
We propose a UNified knowledge inTERface, UNTER, to provide a unified perspective to exploit both structured knowledge and unstructured knowledge.
With both forms of knowledge injected, UNTER gains continuous improvements on a series of knowledge-driven NLP tasks.
arXiv Detail & Related papers (2023-05-02T17:33:28Z) - Multi label classification of Artificial Intelligence related patents
using Modified D2SBERT and Sentence Attention mechanism [0.0]
We present a method for classifying artificial intelligence-related patents published by the USPTO using natural language processing technique and deep learning methodology.
Our experiment result is highest performance compared to other deep learning methods.
arXiv Detail & Related papers (2023-03-03T12:27:24Z) - Great Truths are Always Simple: A Rather Simple Knowledge Encoder for
Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models [89.98762327725112]
Commonsense reasoning in natural language is a desired ability of artificial intelligent systems.
For solving complex commonsense reasoning tasks, a typical solution is to enhance pre-trained language models(PTMs) with a knowledge-aware graph neural network(GNN) encoder.
Despite the effectiveness, these approaches are built on heavy architectures, and can't clearly explain how external knowledge resources improve the reasoning capacity of PTMs.
arXiv Detail & Related papers (2022-05-04T01:27:36Z) - Counterfactual Explanations as Interventions in Latent Space [62.997667081978825]
Counterfactual explanations aim to provide to end users a set of features that need to be changed in order to achieve a desired outcome.
Current approaches rarely take into account the feasibility of actions needed to achieve the proposed explanations.
We present Counterfactual Explanations as Interventions in Latent Space (CEILS), a methodology to generate counterfactual explanations.
arXiv Detail & Related papers (2021-06-14T20:48:48Z) - Pre-Trained Models: Past, Present and Future [126.21572378910746]
Large-scale pre-trained models (PTMs) have recently achieved great success and become a milestone in the field of artificial intelligence (AI)
By storing knowledge into huge parameters and fine-tuning on specific tasks, the rich knowledge implicitly encoded in huge parameters can benefit a variety of downstream tasks.
It is now the consensus of the AI community to adopt PTMs as backbone for downstream tasks rather than learning models from scratch.
arXiv Detail & Related papers (2021-06-14T02:40:32Z) - An interdisciplinary conceptual study of Artificial Intelligence (AI)
for helping benefit-risk assessment practices: Towards a comprehensive
qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence.
The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
arXiv Detail & Related papers (2021-05-07T12:01:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.