Integration of cognitive tasks into artificial general intelligence test
for large models
- URL: http://arxiv.org/abs/2402.02547v2
- Date: Wed, 6 Mar 2024 02:46:40 GMT
- Title: Integration of cognitive tasks into artificial general intelligence test
for large models
- Authors: Youzhi Qu, Chen Wei, Penghui Du, Wenxin Che, Chi Zhang, Wanli Ouyang,
Yatao Bian, Feiyang Xu, Bin Hu, Kai Du, Haiyan Wu, Jia Liu, Quanying Liu
- Abstract summary: We advocate for a comprehensive framework of cognitive science-inspired artificial general intelligence (AGI) tests.
The cognitive science-inspired AGI tests encompass the full spectrum of intelligence facets, including crystallized intelligence, fluid intelligence, social intelligence, and embodied intelligence.
- Score: 54.72053150920186
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: During the evolution of large models, performance evaluation is necessarily
performed to assess their capabilities and ensure safety before practical
application. However, current model evaluations mainly rely on specific tasks
and datasets, lacking a united framework for assessing the multidimensional
intelligence of large models. In this perspective, we advocate for a
comprehensive framework of cognitive science-inspired artificial general
intelligence (AGI) tests, aimed at fulfilling the testing needs of large models
with enhanced capabilities. The cognitive science-inspired AGI tests encompass
the full spectrum of intelligence facets, including crystallized intelligence,
fluid intelligence, social intelligence, and embodied intelligence. To assess
the multidimensional intelligence of large models, the AGI tests consist of a
battery of well-designed cognitive tests adopted from human intelligence tests,
and then naturally encapsulates into an immersive virtual community. We propose
increasing the complexity of AGI testing tasks commensurate with advancements
in large models and emphasizing the necessity for the interpretation of test
results to avoid false negatives and false positives. We believe that cognitive
science-inspired AGI tests will effectively guide the targeted improvement of
large models in specific dimensions of intelligence and accelerate the
integration of large models into human society.
Related papers
- AI-Compass: A Comprehensive and Effective Multi-module Testing Tool for AI Systems [26.605694684145313]
In this study, we design and implement a testing tool, tool, to comprehensively and effectively evaluate AI systems.
The tool extensively assesses adversarial robustness, model interpretability, and performs neuron analysis.
Our research sheds light on a general solution for AI systems testing landscape.
arXiv Detail & Related papers (2024-11-09T11:15:17Z) - OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI [73.75520820608232]
We introduce OlympicArena, which includes 11,163 bilingual problems across both text-only and interleaved text-image modalities.
These challenges encompass a wide range of disciplines spanning seven fields and 62 international Olympic competitions, rigorously examined for data leakage.
Our evaluations reveal that even advanced models like GPT-4o only achieve a 39.97% overall accuracy, illustrating current AI limitations in complex reasoning and multimodal integration.
arXiv Detail & Related papers (2024-06-18T16:20:53Z) - Understanding and Evaluating Human Preferences for AI Generated Images with Instruction Tuning [58.41087653543607]
We first establish a novel Image Quality Assessment (IQA) database for AIGIs, termed AIGCIQA2023+.
This paper presents a MINT-IQA model to evaluate and explain human preferences for AIGIs from Multi-perspectives with INstruction Tuning.
arXiv Detail & Related papers (2024-05-12T17:45:11Z) - Large Multi-modality Model Assisted AI-Generated Image Quality Assessment [53.182136445844904]
We introduce a large Multi-modality model Assisted AI-Generated Image Quality Assessment (MA-AGIQA) model.
It uses semantically informed guidance to sense semantic information and extract semantic vectors through carefully designed text prompts.
It achieves state-of-the-art performance, and demonstrates its superior generalization capabilities on assessing the quality of AI-generated images.
arXiv Detail & Related papers (2024-04-27T02:40:36Z) - Position Paper: Agent AI Towards a Holistic Intelligence [53.35971598180146]
We emphasize developing Agent AI -- an embodied system that integrates large foundation models into agent actions.
In this paper, we propose a novel large action model to achieve embodied intelligent behavior, the Agent Foundation Model.
arXiv Detail & Related papers (2024-02-28T16:09:56Z) - Designing Novel Cognitive Diagnosis Models via Evolutionary
Multi-Objective Neural Architecture Search [13.9289351255891]
We propose to automatically design novel cognitive diagnosis models by evolutionary multi-objective neural architecture search (NAS)
Experiments on two real-world datasets demonstrate that the cognitive diagnosis models searched by the proposed approach exhibit significantly better performance than existing models and also hold as good interpretability as human-designed models.
arXiv Detail & Related papers (2023-07-10T09:09:26Z) - Brain in a Vat: On Missing Pieces Towards Artificial General
Intelligence in Large Language Models [83.63242931107638]
We propose four characteristics of generally intelligent agents.
We argue that active engagement with objects in the real world delivers more robust signals for forming conceptual representations.
We conclude by outlining promising future research directions in the field of artificial general intelligence.
arXiv Detail & Related papers (2023-07-07T13:58:16Z) - Beyond Interpretable Benchmarks: Contextual Learning through Cognitive
and Multimodal Perception [0.0]
This study contends that the Turing Test is misinterpreted as an attempt to anthropomorphize computer systems.
It emphasizes tacit learning as a cornerstone of general-purpose intelligence, despite its lack of overt interpretability.
arXiv Detail & Related papers (2022-12-04T08:30:04Z) - QKSA: Quantum Knowledge Seeking Agent [0.0]
We present the motivation and the core thesis towards the implementation of a Quantum Knowledge Seeking Agent (QKSA)
QKSA is a general reinforcement learning agent that can be used to model classical and quantum dynamics.
arXiv Detail & Related papers (2021-07-03T13:07:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.