Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions

• URL: http://arxiv.org/abs/2305.10435v2
• Date: Sun, 21 May 2023 10:12:02 GMT
• Title: Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions
• Authors: Gokul Yenduri, Ramalingam M, Chemmalar Selvi G, Supriya Y, Gautam Srivastava, Praveen Kumar Reddy Maddikunta, Deepti Raj G, Rutvij H Jhaveri, Prabadevi B, Weizheng Wang, Athanasios V. Vasilakos, and Thippa Reddy Gadekallu
• Abstract summary: The Generative Pre-trained Transformer (GPT) represents a notable breakthrough in the domain of natural language processing. GPT is based on the transformer architecture, a deep neural network designed for natural language processing tasks.
• Score: 11.959434388955787
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Generative Pre-trained Transformer (GPT) represents a notable breakthrough in the domain of natural language processing, which is propelling us toward the development of machines that can understand and communicate using language in a manner that closely resembles that of humans. GPT is based on the transformer architecture, a deep neural network designed for natural language processing tasks. Due to their impressive performance on natural language processing tasks and ability to effectively converse, GPT have gained significant popularity among researchers and industrial communities, making them one of the most widely used and effective models in natural language processing and related fields, which motivated to conduct this review. This review provides a detailed overview of the GPT, including its architecture, working process, training procedures, enabling technologies, and its impact on various applications. In this review, we also explored the potential challenges and limitations of a GPT. Furthermore, we discuss potential solutions and future directions. Overall, this paper aims to provide a comprehensive understanding of GPT, enabling technologies, their impact on various applications, emerging challenges, and potential solutions.

Related papers

• A Survey on Large Language Models from Concept to Implementation [4.219910716090213]
  Recent advancements in Large Language Models (LLMs) have broadened the scope of natural language processing (NLP) applications. This paper investigates the multifaceted applications of these models, with an emphasis on the GPT series. This exploration focuses on the transformative impact of artificial intelligence (AI) driven tools in revolutionizing traditional tasks like coding and problem-solving.
  arXiv  Detail & Related papers  (2024-03-27T19:35:41Z)
• From Generative AI to Generative Internet of Things: Fundamentals, Framework, and Outlooks [82.964958051535]
  Generative Artificial Intelligence (GAI) possesses the capabilities of generating realistic data and facilitating advanced decision-making. By integrating GAI into modern Internet of Things (IoT), Generative Internet of Things (GIoT) is emerging and holds immense potential to revolutionize various aspects of society.
  arXiv  Detail & Related papers  (2023-10-27T02:58:11Z)
• Recommender Systems in the Era of Large Language Models (LLMs) [62.0129013439038]
  Large Language Models (LLMs) have revolutionized the fields of Natural Language Processing (NLP) and Artificial Intelligence (AI) We conduct a comprehensive review of LLM-empowered recommender systems from various aspects including Pre-training, Fine-tuning, and Prompting.
  arXiv  Detail & Related papers  (2023-07-05T06:03:40Z)
• A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks [60.38369406877899]
  Transformer is a deep neural network that employs a self-attention mechanism to comprehend the contextual relationships within sequential data. transformer models excel in handling long dependencies between input sequence elements and enable parallel processing. Our survey encompasses the identification of the top five application domains for transformer-based models.
  arXiv  Detail & Related papers  (2023-06-11T23:13:51Z)
• Interactive Natural Language Processing [67.87925315773924]
  Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP. This paper offers a comprehensive survey of iNLP, starting by proposing a unified definition and framework of the concept.
  arXiv  Detail & Related papers  (2023-05-22T17:18:29Z)
• Prompt Engineering for Healthcare: Methodologies and Applications [93.63832575498844]
  This review will introduce the latest advances in prompt engineering in the field of natural language processing for the medical field. We will provide the development of prompt engineering and emphasize its significant contributions to healthcare natural language processing applications.
  arXiv  Detail & Related papers  (2023-04-28T08:03:42Z)
• ChatGPT: Applications, Opportunities, and Threats [0.0]
  ChatGPT is an artificial intelligence technology that is fine-tuned using supervised machine learning and reinforcement learning techniques. The system combines the power of pre-trained deep learning models with a programmability layer to provide a strong base for generating natural language conversations. Despite its exceptional ability to generate natural-sounding responses, the authors believe that ChatGPT does not possess the same level of understanding, empathy, and creativity as a human.
  arXiv  Detail & Related papers  (2023-04-14T16:25:03Z)
• ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning [70.57126720079971]
  Large language models (LLMs) have emerged as the most important breakthroughs in natural language processing (NLP) This paper evaluates ChatGPT on 7 different tasks, covering 37 diverse languages with high, medium, low, and extremely low resources. Compared to the performance of previous models, our extensive experimental results demonstrate a worse performance of ChatGPT for different NLP tasks and languages.
  arXiv  Detail & Related papers  (2023-04-12T05:08:52Z)
• Geotechnical Parrot Tales (GPT): Harnessing Large Language Models in geotechnical engineering [2.132096006921048]
  GPT models can generate plausible-sounding but false outputs, leading to hallucinations. By integrating GPT into geotechnical engineering, professionals can streamline their work and develop sustainable and resilient infrastructure systems.
  arXiv  Detail & Related papers  (2023-04-04T21:47:41Z)
• Unlocking the Potential of ChatGPT: A Comprehensive Exploration of its Applications, Advantages, Limitations, and Future Directions in Natural Language Processing [4.13365552362244]
  ChatGPT has been successfully applied in numerous areas, including chatbots, content generation, language translation, personalized recommendations, and even medical diagnosis and treatment. Its success in these applications can be attributed to its ability to generate human-like responses, understand natural language, and adapt to different contexts. This article provides a comprehensive overview of ChatGPT, its applications, advantages, and limitations.
  arXiv  Detail & Related papers  (2023-03-27T21:27:58Z)
• A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models [21.124096884958337]
  Controllable Text Generation (CTG) is emerging area in the field of natural language generation (NLG) We present a systematic critical review on the common tasks, main approaches, and evaluation methods in this area. We discuss the challenges that the field is facing, and put forward various promising future directions.
  arXiv  Detail & Related papers  (2022-01-14T08:32:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.