FAME: Forecasting Academic Impact via Continuous-Time Manifold Evolution

• URL: http://arxiv.org/abs/2605.07208v1
• Date: Fri, 08 May 2026 03:57:34 GMT
• Title: FAME: Forecasting Academic Impact via Continuous-Time Manifold Evolution
• Abstract summary: Large Language Models (LLMs) are increasingly used to brainstorm and evaluate research ideas.<n>We address this challenge by using the impact forecasting of human-authored manuscripts as a verifiable proxy task.
• Score: 9.700601627690466
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) are increasingly used to brainstorm and evaluate research ideas, yet assessing such judgments is fundamentally difficult because the true impact of a new idea may take years to emerge. We address this challenge by using the impact forecasting of human-authored manuscripts as a verifiable proxy task. In a prospective forecasting study, we find that frontier LLMs fail to reliably distinguish high-impact papers from ordinary publications, suggesting that static text-based judging is insufficient for scientific evaluation. To address this limitation, we propose $\textbf{FAME}$ ($\underline{\text{F}}$orecasting $\underline{\text{A}}$cademic Impact via Continuous-Time $\underline{\text{M}}$anifold $\underline{\text{E}}$volution), a spatiotemporal framework for modeling the dynamic trajectories of scientific topics. FAME projects papers into a dynamic latent space informed by textual features and a verified knowledge-flow graph, learning geometric constraints that align impactful manuscripts with the forward momentum of their fields. Experiments on 3,200 arXiv papers across three fast-evolving subfields show that FAME consistently and substantially outperforms state-of-the-art LLM evaluators in prospective multidimensional impact forecasting. Furthermore, integrating FAME's dynamic geometric signals into LLMs significantly improves their forecasting performance. These results support manuscript impact forecasting as a useful, measurable proxy benchmark and position FAME as a strong, trajectory-aware foundation for automated scientific evaluation.

Related papers

• AgentIdeaBench: Benchmarking Scientific Ideation in the Agent Era [55.37558486468993]
  We introduce AgentIdeaBench, a benchmark that evaluates scientific ideation under two matched settings, static observation and active exploration.<n>We report matched Static-Active evaluations for 33 LLMs across 40 densely scored subfields spanning five disciplines.<n>Active exploration reveals considerably more capability headroom, and that headroom is unevenly distributed across models.
  arXiv  Detail & Related papers  (2026-09-07T15:19:13Z)
• Large Discovery Models: Empirically-grounded Model-Based Open-Ended Search [32.21307731426909]
  Generative models such as large language models (LLMs) provide expressive priors over vast, structured, and open-ended hypothesis spaces.<n>We introduce the Large Discovery Model (LDM), an empirically grounded recurrent architecture that couples a generative model with a Bayesian non-parametric reward surrogate model.<n>LDM achieves a $2.4times$ greater reduction in validation BPB, an $18.2%$ relative decrease in binding energy, and more than $60%$ relative gains in molecular multi-objective performance.
  arXiv  Detail & Related papers  (2026-08-16T10:27:06Z)
• Learning to Predict Future-Aligned Research Proposals with Language Models [59.79457676644722]
  We build a time-consistent dataset of 17,771 papers from targets and their pre-cutoff citations.<n>To train models, we build a time-consistent dataset of 17,771 papers from targets and their pre-cutoff citations.<n>Across Llama-3.1 and Qwen2.5 models, future-aligned tuning improves future alignment over unaligned baselines.
  arXiv  Detail & Related papers  (2026-03-28T05:41:15Z)
• Evaluating Large Language Models in Scientific Discovery [91.732562776782]
  Large language models (LLMs) are increasingly applied to scientific research, yet prevailing science benchmarks probe decontextualized knowledge.<n>We introduce a scenario-grounded benchmark that evaluates LLMs across biology, chemistry, materials, and physics.<n>The framework assesses models at two levels: (i) question-level accuracy on scenario-tied items and (ii) project-level performance.
  arXiv  Detail & Related papers  (2025-12-17T16:20:03Z)
• Does GenAI Rewrite How We Write? An Empirical Study on Two-Million Preprints [15.070885964897734]
  Generative large language models (LLMs) introduce a further potential disruption by altering how manuscripts are written.<n>This paper addresses the gap through a large-scale analysis of more than 2.1 million preprints spanning 2016--2025 (115 months) across four major repositories.<n>Our findings reveal that LLMs have accelerated submission and revision cycles, modestly increased linguistic complexity, and disproportionately expanded AI-related topics.
  arXiv  Detail & Related papers  (2025-10-18T01:37:40Z)
• When can isotropy help adapt LLMs' next word prediction to numerical domains? [53.98633183204453]
  It is shown that the isotropic property of LLM embeddings in contextual embedding space preserves the underlying structure of representations.<n> Experiments show that different characteristics of numerical data and model architectures have different impacts on isotropy.
  arXiv  Detail & Related papers  (2025-05-22T05:10:34Z)
• ForeCite: Adapting Pre-Trained Language Models to Predict Future Citation Rates of Academic Papers [0.0]
  We present ForeCite, a framework to predict the future citation rates of academic papers.<n>ForeCite achieves a test correlation of $rho = 0.826$ on a curated dataset of 900K+ biomedical papers published between 2000 and 2024.<n>These results establish a new state-of-the-art in forecasting the long-term influence of academic research and lay the groundwork for the automated, high-fidelity evaluation of scientific contributions.
  arXiv  Detail & Related papers  (2025-05-13T20:10:00Z)
• Optimizing Research Portfolio For Semantic Impact [55.2480439325792]
  Citation metrics are widely used to assess academic impact but suffer from social biases.<n>We introduce rXiv Semantic Impact (XSI), a novel framework that predicts research impact.<n>XSI tracks the evolution of research concepts in the academic knowledge graph.
  arXiv  Detail & Related papers  (2025-02-19T17:44:13Z)
• From Words to Worth: Newborn Article Impact Prediction with LLM [69.41680520058418]
  This paper introduces a promising approach, leveraging the capabilities of LLMs to predict the future impact of newborn articles.<n>The proposed method employs LLM to discern the shared semantic features of highly impactful papers from a large collection of title-abstract pairs.<n>The quantitative results, with an MAE of 0.216 and an NDCG@20 of 0.901, demonstrate that the proposed approach achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-08-07T17:52:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.