Early screening of potential breakthrough technologies with enhanced interpretability: A patent-specific hierarchical attention network model

• URL: http://arxiv.org/abs/2407.16939v1
• Date: Wed, 24 Jul 2024 02:17:10 GMT
• Title: Early screening of potential breakthrough technologies with enhanced interpretability: A patent-specific hierarchical attention network model
• Authors: Jaewoong Choi, Janghyeok Yoon, Changyong Lee,
• Abstract summary: We propose an interpretable machine learning approach to predicting future citation counts from patent texts. A case study of 35,376 pharmaceutical patents demonstrates the effectiveness of our approach. It is expected that the proposed approach will enhance expert-machine collaboration in identifying breakthrough technologies.
• Score: 4.779196219827507
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the usefulness of machine learning approaches for the early screening of potential breakthrough technologies, their practicality is often hindered by opaque models. To address this, we propose an interpretable machine learning approach to predicting future citation counts from patent texts using a patent-specific hierarchical attention network (PatentHAN) model. Central to this approach are (1) a patent-specific pre-trained language model, capturing the meanings of technical words in patent claims, (2) a hierarchical network structure, enabling detailed analysis at the claim level, and (3) a claim-wise self-attention mechanism, revealing pivotal claims during the screening process. A case study of 35,376 pharmaceutical patents demonstrates the effectiveness of our approach in early screening of potential breakthrough technologies while ensuring interpretability. Furthermore, we conduct additional analyses using different language models and claim types to examine the robustness of the approach. It is expected that the proposed approach will enhance expert-machine collaboration in identifying breakthrough technologies, providing new insight derived from text mining into technological value.

Related papers

• Cutting Through the Confusion and Hype: Understanding the True Potential of Generative AI [0.0]
  This paper explores the nuanced landscape of generative AI (genAI) It focuses on neural network-based models like Large Language Models (LLMs)
  arXiv  Detail & Related papers  (2024-10-22T02:18:44Z)
• PatentGPT: A Large Language Model for Patent Drafting Using Knowledge-based Fine-tuning Method [1.4496326701907591]
  Existing large language models (LLMs) often fall short in this IP creation domain due to their lack of specialized knowledge and context-awareness. We propose a groundbreaking framework for Knowledge Fine-Tuning (KFT) of LLMs, designed to endow AI with the ability to autonomously mine, understand, and apply domain-specific knowledge. Our model, PatentGPT, has demonstrated outstanding performance, scoring up to approximately 400% higher in patent related benchmark tests compared to state-of-the-art models.
  arXiv  Detail & Related papers  (2024-08-26T12:00:29Z)
• InstructPatentGPT: Training patent language models to follow instructions with human feedback [0.9790236766474201]
  This research aims to increase the likelihood for a language model to generate patent claims that have a higher chance of being granted. To showcase the controllability of the language model, the system learns from granted patents and pre-grant applications with different rewards.
  arXiv  Detail & Related papers  (2024-05-25T11:48:50Z)
• Unveiling Black-boxes: Explainable Deep Learning Models for Patent Classification [48.5140223214582]
  State-of-the-art methods for multi-label patent classification rely on deep opaque neural networks (DNNs) We propose a novel deep explainable patent classification framework by introducing layer-wise relevance propagation (LRP) Considering the relevance score, we then generate explanations by visualizing relevant words for the predicted patent class.
  arXiv  Detail & Related papers  (2023-10-31T14:11:37Z)
• Predictable Artificial Intelligence [77.1127726638209]
  This paper introduces the ideas and challenges of Predictable AI. It explores the ways in which we can anticipate key validity indicators of present and future AI ecosystems. We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI ecosystems.
  arXiv  Detail & Related papers  (2023-10-09T21:36:21Z)
• AutoPrognosis 2.0: Democratizing Diagnostic and Prognostic Modeling in Healthcare with Automated Machine Learning [72.2614468437919]
  We present a machine learning framework, AutoPrognosis 2.0, to develop diagnostic and prognostic models. We provide an illustrative application where we construct a prognostic risk score for diabetes using the UK Biobank. Our risk score has been implemented as a web-based decision support tool and can be publicly accessed by patients and clinicians worldwide.
  arXiv  Detail & Related papers  (2022-10-21T16:31:46Z)
• Patent Sentiment Analysis to Highlight Patent Paragraphs [0.0]
  Given a patent document, identifying distinct semantic annotations is an interesting research aspect. In the process of manual patent analysis, to attain better readability, recognising the semantic information by marking paragraphs is in practice. This work assist patent practitioners in highlighting semantic information automatically and aid to create a sustainable and efficient patent analysis using the aptitude of Machine Learning.
  arXiv  Detail & Related papers  (2021-11-06T13:28:29Z)
• Counterfactual Explanations as Interventions in Latent Space [62.997667081978825]
  Counterfactual explanations aim to provide to end users a set of features that need to be changed in order to achieve a desired outcome. Current approaches rarely take into account the feasibility of actions needed to achieve the proposed explanations. We present Counterfactual Explanations as Interventions in Latent Space (CEILS), a methodology to generate counterfactual explanations.
  arXiv  Detail & Related papers  (2021-06-14T20:48:48Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)
• A Diagnostic Study of Explainability Techniques for Text Classification [52.879658637466605]
  We develop a list of diagnostic properties for evaluating existing explainability techniques. We compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones.
  arXiv  Detail & Related papers  (2020-09-25T12:01:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.