Online Training of Large Language Models: Learn while chatting
- URL: http://arxiv.org/abs/2403.04790v1
- Date: Mon, 4 Mar 2024 10:00:55 GMT
- Title: Online Training of Large Language Models: Learn while chatting
- Authors: Juhao Liang, Ziwei Wang, Zhuoheng Ma, Jianquan Li, Zhiyi Zhang,
Xiangbo Wu and Benyou Wang
- Abstract summary: This paper introduces a novel interaction paradigm-'Online Training using External Interactions'-that merges the benefits of persistent, real-time model updates with the flexibility for individual customization.
- Score: 23.995637621755083
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Large Language Models(LLMs) have dramatically revolutionized the field of
Natural Language Processing(NLP), offering remarkable capabilities that have
garnered widespread usage. However, existing interaction paradigms between LLMs
and users are constrained by either inflexibility, limitations in
customization, or a lack of persistent learning. This inflexibility is
particularly evident as users, especially those without programming skills,
have restricted avenues to enhance or personalize the model. Existing
frameworks further complicate the model training and deployment process due to
their computational inefficiencies and lack of user-friendly interfaces. To
overcome these challenges, this paper introduces a novel interaction
paradigm-'Online Training using External Interactions'-that merges the benefits
of persistent, real-time model updates with the flexibility for individual
customization through external interactions such as AI agents or online/offline
knowledge bases.
Related papers
- Modulating Language Model Experiences through Frictions [56.17593192325438]
Over-consumption of language model outputs risks propagating unchecked errors in the short-term and damaging human capabilities in the long-term.
We propose selective frictions for language model experiences, inspired by behavioral science interventions, to dampen misuse.
arXiv Detail & Related papers (2024-06-24T16:31:11Z) - LEGENT: Open Platform for Embodied Agents [60.71847900126832]
We introduce LEGENT, an open, scalable platform for developing embodied agents using Large Language Models (LLMs) and Large Multimodal Models (LMMs)
LEGENT offers a rich, interactive 3D environment with communicable and actionable agents, paired with a user-friendly interface.
In experiments, an embryonic vision-language-action model trained on LEGENT-generated data surpasses GPT-4V in embodied tasks.
arXiv Detail & Related papers (2024-04-28T16:50:12Z) - Scalable Language Model with Generalized Continual Learning [58.700439919096155]
The Joint Adaptive Re-ization (JARe) is integrated with Dynamic Task-related Knowledge Retrieval (DTKR) to enable adaptive adjustment of language models based on specific downstream tasks.
Our method demonstrates state-of-the-art performance on diverse backbones and benchmarks, achieving effective continual learning in both full-set and few-shot scenarios with minimal forgetting.
arXiv Detail & Related papers (2024-04-11T04:22:15Z) - Text-centric Alignment for Multi-Modality Learning [3.6961400222746748]
We propose the Text-centric Alignment for Multi-Modality Learning (TAMML) approach.
By leveraging the unique properties of text as a unified semantic space, TAMML demonstrates significant improvements in handling unseen, diverse, and unpredictable modality combinations.
This study contributes to the field by offering a flexible, effective solution for real-world applications where modality availability is dynamic and uncertain.
arXiv Detail & Related papers (2024-02-12T22:07:43Z) - PROMISE: A Framework for Developing Complex Conversational Interactions (Technical Report) [33.7054351451505]
We present PROMISE, a framework that facilitates the development of complex language-based interactions with information systems.
We show the benefits of PROMISE in the context of application scenarios within health information systems and demonstrate its ability to handle complex interactions.
arXiv Detail & Related papers (2023-12-06T18:59:11Z) - Prompt-to-OS (P2OS): Revolutionizing Operating Systems and
Human-Computer Interaction with Integrated AI Generative Models [10.892991111926573]
We present a paradigm for human-computer interaction that revolutionizes the traditional notion of an operating system.
Within this innovative framework, user requests issued to the machine are handled by an interconnected ecosystem of generative AI models.
This visionary concept raises significant challenges, including privacy, security, trustability, and the ethical use of generative models.
arXiv Detail & Related papers (2023-10-07T17:16:34Z) - In-context Interference in Chat-based Large Language Models [8.197259049834038]
Large language models (LLMs) have had a huge impact on society due to their impressive capabilities and vast knowledge of the world.
Various applications and tools have been created that allow users to interact with these models in a black-box scenario.
This paper shows how the model can suffer from interference between information that continually flows in the context, causing it to forget previously learned knowledge.
arXiv Detail & Related papers (2023-09-22T09:18:55Z) - When Large Language Models Meet Personalization: Perspectives of
Challenges and Opportunities [60.5609416496429]
The capability of large language models has been dramatically improved.
Such a major leap-forward in general AI capacity will change the pattern of how personalization is conducted.
By leveraging large language models as general-purpose interface, personalization systems may compile user requests into plans.
arXiv Detail & Related papers (2023-07-31T02:48:56Z) - Interactive Text Generation [75.23894005664533]
We introduce a new Interactive Text Generation task that allows training generation models interactively without the costs of involving real users.
We train our interactive models using Imitation Learning, and our experiments against competitive non-interactive generation models show that models trained interactively are superior to their non-interactive counterparts.
arXiv Detail & Related papers (2023-03-02T01:57:17Z) - Sparsity-aware neural user behavior modeling in online interaction
platforms [2.4036844268502766]
We develop generalizable neural representation learning frameworks for user behavior modeling.
Our problem settings span transductive and inductive learning scenarios.
We leverage different facets of information reflecting user behavior to enable personalized inference at scale.
arXiv Detail & Related papers (2022-02-28T00:27:11Z) - Learning Adaptive Language Interfaces through Decomposition [89.21937539950966]
We introduce a neural semantic parsing system that learns new high-level abstractions through decomposition.
Users interactively teach the system by breaking down high-level utterances describing novel behavior into low-level steps.
arXiv Detail & Related papers (2020-10-11T08:27:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.