Anticipating Innovation Using Large Language Models

• URL: http://arxiv.org/abs/2605.04875v1
• Date: Wed, 06 May 2026 13:11:01 GMT
• Title: Anticipating Innovation Using Large Language Models
• Abstract summary: We show that forthcoming combinations leave an early trace in the collective language of patents, with predictive signals detectable decades in advance.<n>We show that signal is not attributable to any single inventor, but emerges as a collective shift in how technologies are described across thousands of patents.<n>We introduce TechToken, a transformer-based model that treats technologies as words in its vocabulary, learning the language of technologies by embedding these codes during fine-tuning.
• Score: 0.7340017786387767
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Forecasting innovation, intended as the emergence of new technological combinations, is a fundamental challenge for science and policy. We show that forthcoming combinations leave an early trace in the collective language of patents, with predictive signals detectable even decades in advance. We show that signal is not attributable to any single inventor, but emerges as a collective shift in how technologies are described across thousands of patents. To this end, we introduce TechToken, a transformer-based model that treats technologies, classified by International Patent Classification codes, as words in its vocabulary, learning the language of technologies by embedding these codes during fine-tuning. We define context similarity between code embeddings as a measure of linguistic convergence and show that it accurately predicts first technological combinations. TechToken also improves general representation quality, outperforming state-of-the-art models across different patent-related tasks.

Related papers

• Accelerating scientific discovery with the common task framework [48.92654976046941]
  Machine learning (ML) and artificial intelligence (AI) algorithms are transforming the characterization and control of dynamic systems in the engineering, physical, and biological sciences.<n>These emerging modeling paradigms require comparative metrics to evaluate a diverse set of scientific objectives.<n>We introduce a common task framework (CTF) for science and engineering, which features a growing collection of challenge data sets.
  arXiv  Detail & Related papers  (2025-11-06T02:53:07Z)
• Monitoring Transformative Technological Convergence Through LLM-Extracted Semantic Entity Triple Graphs [35.70283902821063]
  We propose a novel, data-driven pipeline to monitor the emergence of transformative technologies.<n>Our approach leverages advances in Large Language Models (LLMs) to extract semantic triples from unstructured text.<n>The pipeline includes multi-stage filtering, domain-specific clustering, and a temporal trend analysis of topic co-occurence.
  arXiv  Detail & Related papers  (2025-10-29T10:41:03Z)
• DiTTO-LLM: Framework for Discovering Topic-based Technology Opportunities via Large Language Model [5.923472343317637]
  This paper proposes a framework based on the temporal relationships between technologies to identify emerging technology opportunities.<n>The framework was evaluated using an artificial intelligence patent dataset provided by the United States Patent and Trademark Office.
  arXiv  Detail & Related papers  (2025-09-10T05:47:25Z)
• A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models [71.66119575697458]
  parallel text generation techniques aimed at breaking the token-by-token generation bottleneck and improving inference efficiency.<n>We categorize existing approaches into AR-based and Non-AR-based paradigms, and provide a detailed examination of the core techniques within each category.<n>We highlight recent advancements, identify open challenges, and outline promising directions for future research in parallel text generation.
  arXiv  Detail & Related papers  (2025-08-12T07:56:04Z)
• Technology Mapping with Large Language Models [1.1900482352079937]
  STARS (Semantic Technology and Retrieval System) is a novel framework that harnesses Large Language Models (LLMs) and Sentence-BERT.<n>It pinpoints relevant technologies within unstructured content, build comprehensive company profiles, and rank each firm's technologies according to their operational importance.<n> Experimental results show that STARS markedly boosts retrieval accuracy, offering a versatile and high-performance solution for cross-industry technology mapping.
  arXiv  Detail & Related papers  (2025-01-25T08:18:15Z)
• Large Language Model for Patent Concept Generation [2.4368308736427697]
  Existing large language models (LLMs) often fall short in the innovative concept generation due to a lack of specialized knowledge.<n>We propose a novel knowledge finetuning (KFT) framework to endow LLM-based AI with the ability to autonomously mine, understand, and apply domain-specific knowledge.<n>Our proposed PatentGPT integrates knowledge injection pre-training (KPT), domain-specific supervised finetuning (SFT), and reinforcement learning from human feedback.
  arXiv  Detail & Related papers  (2024-08-26T12:00:29Z)
• Early screening of potential breakthrough technologies with enhanced interpretability: A patent-specific hierarchical attention network model [4.779196219827507]
  We propose an interpretable machine learning approach to predicting future citation counts from patent texts. A case study of 35,376 pharmaceutical patents demonstrates the effectiveness of our approach. It is expected that the proposed approach will enhance expert-machine collaboration in identifying breakthrough technologies.
  arXiv  Detail & Related papers  (2024-07-24T02:17:10Z)
• Measuring Technological Convergence in Encryption Technologies with Proximity Indices: A Text Mining and Bibliometric Analysis using OpenAlex [46.3643544723237]
  This study identifies technological convergence among emerging technologies in cybersecurity. The proposed method integrates text mining and bibliometric analyses to formulate and predict technological proximity indices. Our case study findings highlight a significant convergence between blockchain and public-key cryptography, evidenced by the increasing proximity indices.
  arXiv  Detail & Related papers  (2024-03-03T20:03:03Z)
• Language Models: A Guide for the Perplexed [51.88841610098437]
  This tutorial aims to help narrow the gap between those who study language models and those who are intrigued and want to learn more. We offer a scientific viewpoint that focuses on questions amenable to study through experimentation. We situate language models as they are today in the context of the research that led to their development.
  arXiv  Detail & Related papers  (2023-11-29T01:19:02Z)
• Unveiling Black-boxes: Explainable Deep Learning Models for Patent Classification [48.5140223214582]
  State-of-the-art methods for multi-label patent classification rely on deep opaque neural networks (DNNs) We propose a novel deep explainable patent classification framework by introducing layer-wise relevance propagation (LRP) Considering the relevance score, we then generate explanations by visualizing relevant words for the predicted patent class.
  arXiv  Detail & Related papers  (2023-10-31T14:11:37Z)
• Adaptive Taxonomy Learning and Historical Patterns Modelling for Patent Classification [26.85734804493925]
  We propose an integrated framework that comprehensively considers the information on patents for patent classification. We first present an IPC codes correlations learning module to derive their semantic representations. Finally, we combine the contextual information of patent texts that contains the semantics of IPC codes, and assignees' sequential preferences to make predictions.
  arXiv  Detail & Related papers  (2023-08-10T07:02:24Z)
• Deep Technology Tracing for High-tech Companies [67.86308971806322]
  We develop a novel data-driven solution, i.e., Deep Technology Forecasting (DTF) framework, to automatically find the most possible technology directions customized to each high-tech company. DTF consists of three components: Potential Competitor Recognition (PCR), Collaborative Technology Recognition (CTR), and Deep Technology Tracing (DTT) neural network.
  arXiv  Detail & Related papers  (2020-01-02T07:44:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.