Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering

• URL: http://arxiv.org/abs/2411.11504v1
• Date: Mon, 18 Nov 2024 12:04:52 GMT
• Title: Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering
• Authors: Xinyan Guan, Yanjiang Liu, Xinyu Lu, Boxi Cao, Ben He, Xianpei Han, Le Sun, Jie Lou, Bowen Yu, Yaojie Lu, Hongyu Lin,
• Abstract summary: Verifier engineering is a novel post-training paradigm specifically designed for the era of foundation models. We systematically categorize the verifier engineering process into three essential stages: search, verify, and feedback.
• Score: 51.31836988300326
• License:
• Abstract: The evolution of machine learning has increasingly prioritized the development of powerful models and more scalable supervision signals. However, the emergence of foundation models presents significant challenges in providing effective supervision signals necessary for further enhancing their capabilities. Consequently, there is an urgent need to explore novel supervision signals and technical approaches. In this paper, we propose verifier engineering, a novel post-training paradigm specifically designed for the era of foundation models. The core of verifier engineering involves leveraging a suite of automated verifiers to perform verification tasks and deliver meaningful feedback to foundation models. We systematically categorize the verifier engineering process into three essential stages: search, verify, and feedback, and provide a comprehensive review of state-of-the-art research developments within each stage. We believe that verifier engineering constitutes a fundamental pathway toward achieving Artificial General Intelligence.

Related papers

• Case for a unified surrogate modelling framework in the age of AI [0.0]
  We argue for the urgent need for a unified framework to guide the development and evaluation of surrogate models. This paper seeks to improve the reliability of surrogate modelling, foster cross-disciplinary knowledge transfer, and, as a result, accelerate scientific progress.
  arXiv  Detail & Related papers  (2025-02-10T18:31:15Z)
• Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems [92.89673285398521]
  o1-like reasoning systems have demonstrated remarkable capabilities in solving complex reasoning tasks. We introduce an imitate, explore, and self-improve'' framework to train the reasoning model. Our approach achieves competitive performance compared to industry-level reasoning systems.
  arXiv  Detail & Related papers  (2024-12-12T16:20:36Z)
• Vision Foundation Models in Remote Sensing: A Survey [6.036426846159163]
  Foundation models are large-scale, pre-trained AI models capable of performing a wide array of tasks with unprecedented accuracy and efficiency. This survey aims to serve as a resource for researchers and practitioners by providing a panorama of advances and promising pathways for continued development and application of foundation models in remote sensing.
  arXiv  Detail & Related papers  (2024-08-06T22:39:34Z)
• Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities [59.02391344178202]
  Vision foundation models (VFMs) serve as potent building blocks for a wide range of AI applications. The scarcity of comprehensive training data, the need for multi-sensor integration, and the diverse task-specific architectures pose significant obstacles to the development of VFMs. This paper delves into the critical challenge of forging VFMs tailored specifically for autonomous driving, while also outlining future directions.
  arXiv  Detail & Related papers  (2024-01-16T01:57:24Z)
• Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review [1.6006550105523192]
  Review explores the pivotal role of prompt engineering in unleashing the capabilities of Large Language Models (LLMs) Examines both foundational and advanced methodologies of prompt engineering, including techniques such as self-consistency, chain-of-thought, and generated knowledge. Review also reflects the essential role of prompt engineering in advancing AI capabilities, providing a structured framework for future research and application.
  arXiv  Detail & Related papers  (2023-10-23T09:15:18Z)
• Review of Large Vision Models and Visual Prompt Engineering [50.63394642549947]
  Review aims to summarize the methods employed in the computer vision domain for large vision models and visual prompt engineering. We present influential large models in the visual domain and a range of prompt engineering methods employed on these models.
  arXiv  Detail & Related papers  (2023-07-03T08:48:49Z)
• Challenges and Practices of Deep Learning Model Reengineering: A Case Study on Computer Vision [3.510650664260664]
  Many engineering organizations are reimplementing and extending deep neural networks from the research community. Deep learning model reengineering is challenging for reasons including under-documented reference models, changing requirements, and the cost of implementation and testing. Our study is focused on reengineering activities from a "process" view, and focuses on engineers specifically engaged in the reengineering process.
  arXiv  Detail & Related papers  (2023-03-13T21:23:43Z)
• Intrinsic Motivation in Model-based Reinforcement Learning: A Brief Review [77.34726150561087]
  This review considers the existing methods for determining intrinsic motivation based on the world model obtained by the agent. The proposed unified framework describes the architecture of agents using a world model and intrinsic motivation to improve learning.
  arXiv  Detail & Related papers  (2023-01-24T15:13:02Z)
• Data-Driven and SE-assisted AI Model Signal-Awareness Enhancement and Introspection [61.571331422347875]
  We propose a data-driven approach to enhance models' signal-awareness. We combine the SE concept of code complexity with the AI technique of curriculum learning. We achieve up to 4.8x improvement in model signal awareness.
  arXiv  Detail & Related papers  (2021-11-10T17:58:18Z)
• Roadmap on Signal Processing for Next Generation Measurement Systems [0.222020259427608]
  Recent advances in artificial intelligence and machine learning are shifting the research attention towards intelligent, data-driven, signal processing. This roadmap presents a critical overview of the state-of-the-art methods and applications aiming to highlight future challenges and research opportunities towards next generation measurement systems. It covers a broad spectrum of topics ranging from basic to industrial research, organized in concise thematic sections that reflect the trends and the impacts of current and future developments per research field.
  arXiv  Detail & Related papers  (2021-11-03T19:39:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.