HybridQuestion: Human-AI Collaboration for Identifying High-Impact Research Questions
- URL: http://arxiv.org/abs/2602.03849v1
- Date: Thu, 18 Dec 2025 15:10:38 GMT
- Title: HybridQuestion: Human-AI Collaboration for Identifying High-Impact Research Questions
- Authors: Keyu Zhao, Fengli Xu, Yong Li, Tie-Yan Liu,
- Abstract summary: "AI Scientist" paradigm is transforming scientific research by automating key stages of the research process.<n>Key question remains unclear: can AI scientists identify meaningful research questions?<n>We propose a human-AI hybrid solution that integrates scalable data processing capabilities of AI with the value judgment of human experts.
- Score: 48.1029746371619
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The "AI Scientist" paradigm is transforming scientific research by automating key stages of the research process, from idea generation to scholarly writing. This shift is expected to accelerate discovery and expand the scope of scientific inquiry. However, a key question remains unclear: can AI scientists identify meaningful research questions? While Large Language Models (LLMs) have been applied successfully to task-specific ideation, their potential to conduct strategic, long-term assessments of past breakthroughs and future questions remains largely unexplored. To address this gap, we explore a human-AI hybrid solution that integrates the scalable data processing capabilities of AI with the value judgment of human experts. Our methodology is structured in three phases. The first phase, AI-Accelerated Information Gathering, leverages AI's advantage in processing vast amounts of literature to generate a hybrid information base. The second phase, Candidate Question Proposing, utilizes this synthesized data to prompt an ensemble of six diverse LLMs to propose an initial candidate pool, filtered via a cross-model voting mechanism. The third phase, Hybrid Question Selection, refines this pool through a multi-stage filtering process that progressively increases human oversight. To validate this system, we conducted an experiment aiming to identify the Top 10 Scientific Breakthroughs of 2025 and the Top 10 Scientific Questions for 2026 across five major disciplines. Our analysis reveals that while AI agents demonstrate high alignment with human experts in recognizing established breakthroughs, they exhibit greater divergence in forecasting prospective questions, suggesting that human judgment remains crucial for evaluating subjective, forward-looking challenges.
Related papers
- ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning [118.46980291324148]
ATLAS is a large-scale, high-difficulty, and cross-disciplinary evaluation suite composed of approximately 800 original problems.<n>Its key features include: High Originality and Contamination Resistance, with all questions newly created or substantially adapted to prevent test data leakage.<n>Preliminary results on leading models demonstrate ATLAS's effectiveness in differentiating their advanced scientific reasoning capabilities.
arXiv Detail & Related papers (2025-11-18T11:13:06Z) - A Self-Evolving AI Agent System for Climate Science [59.08800209508371]
We introduce EarthLink, the first self-evolving AI agent system designed as an interactive "copilot" for Earth scientists.<n>Through natural language interaction, EarthLink automates the entire research workflow by integrating planning, code execution, data analysis, and physical reasoning.<n>It exhibits human-like cross-disciplinary analytical ability and proficiency comparable to a junior researcher in expert evaluations on core large-scale climate tasks.
arXiv Detail & Related papers (2025-07-23T08:29:25Z) - ResearcherBench: Evaluating Deep AI Research Systems on the Frontiers of Scientific Inquiry [22.615102398311432]
We introduce ResearcherBench, the first benchmark focused on evaluating the capabilities of deep AI research systems.<n>We compiled a dataset of 65 research questions expertly selected from real-world scientific scenarios.<n>OpenAI Deep Research and Gemini Deep Research significantly outperform other systems, with particular strength in open-ended consulting questions.
arXiv Detail & Related papers (2025-07-22T06:51:26Z) - AI4Research: A Survey of Artificial Intelligence for Scientific Research [55.5452803680643]
We present a comprehensive survey on AI for Research (AI4Research)<n>We first introduce a systematic taxonomy to classify five mainstream tasks in AI4Research.<n>We identify key research gaps and highlight promising future directions.
arXiv Detail & Related papers (2025-07-02T17:19:20Z) - Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions [0.0]
Agentic AI systems are capable of reasoning, planning, and autonomous decision-making.<n>They are transforming how scientists perform literature review, generate hypotheses, conduct experiments, and analyze results.
arXiv Detail & Related papers (2025-03-12T01:00:05Z) - Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation [58.064940977804596]
A plethora of new AI models and tools has been proposed, promising to empower researchers and academics worldwide to conduct their research more effectively and efficiently.<n>Ethical concerns regarding shortcomings of these tools and potential for misuse take a particularly prominent place in our discussion.
arXiv Detail & Related papers (2025-02-07T18:26:45Z) - Applications and Challenges of AI and Microscopy in Life Science Research: A Review [7.771558261139913]
This paper explores the intersection of AI and microscopy in life sciences, emphasizing their potential applications and associated challenges.<n>We provide a detailed review of how various biological systems can benefit from AI, highlighting the types of data and labeling requirements unique to this domain.<n>Specifically attention is given to microscopy data, exploring the specific AI techniques required to process and interpret this information.
arXiv Detail & Related papers (2025-01-22T08:32:36Z) - Can Artificial Intelligence Generate Quality Research Topics Reflecting Patient Concerns? [0.2801039649976666]
We propose an automated framework leveraging innovative natural language processing (NLP) and artificial intelligence (AI)<n>We analyzed 614,464 patient messages from 25,549 individuals with breast or skin cancer obtained from a large academic hospital.<n>We generated research topics to resolve the defined issues using a widely used AI.
arXiv Detail & Related papers (2024-11-15T20:24:38Z) - Can AI Serve as a Substitute for Human Subjects in Software Engineering
Research? [24.39463126056733]
This vision paper proposes a novel approach to qualitative data collection in software engineering research by harnessing the capabilities of artificial intelligence (AI)
We explore the potential of AI-generated synthetic text as an alternative source of qualitative data.
We discuss the prospective development of new foundation models aimed at emulating human behavior in observational studies and user evaluations.
arXiv Detail & Related papers (2023-11-18T14:05:52Z) - The Future of Fundamental Science Led by Generative Closed-Loop
Artificial Intelligence [67.70415658080121]
Recent advances in machine learning and AI are disrupting technological innovation, product development, and society as a whole.
AI has contributed less to fundamental science in part because large data sets of high-quality data for scientific practice and model discovery are more difficult to access.
Here we explore and investigate aspects of an AI-driven, automated, closed-loop approach to scientific discovery.
arXiv Detail & Related papers (2023-07-09T21:16:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.