xLAM: A Family of Large Action Models to Empower AI Agent Systems

• URL: http://arxiv.org/abs/2409.03215v1
• Date: Thu, 5 Sep 2024 03:22:22 GMT
• Title: xLAM: A Family of Large Action Models to Empower AI Agent Systems
• Authors: Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, Zhiwei Liu, Yihao Feng, Tulika Awalgaonkar, Rithesh Murthy, Eric Hu, Zeyuan Chen, Ran Xu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Silvio Savarese, Caiming Xiong,
• Abstract summary: We release xLAM, a series of large action models designed for AI agent tasks. xLAM consistently delivers exceptional performance across multiple agent ability benchmarks.
• Score: 111.5719694445345
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous agents powered by large language models (LLMs) have attracted significant research interest. However, the open-source community faces many challenges in developing specialized models for agent tasks, driven by the scarcity of high-quality agent datasets and the absence of standard protocols in this area. We introduce and publicly release xLAM, a series of large action models designed for AI agent tasks. The xLAM series includes five models with both dense and mixture-of-expert architectures, ranging from 1B to 8x22B parameters, trained using a scalable, flexible pipeline that unifies, augments, and synthesizes diverse datasets to enhance AI agents' generalizability and performance across varied environments. Our experimental results demonstrate that xLAM consistently delivers exceptional performance across multiple agent ability benchmarks, notably securing the 1st position on the Berkeley Function-Calling Leaderboard, outperforming GPT-4, Claude-3, and many other models in terms of tool use. By releasing the xLAM series, we aim to advance the performance of open-source LLMs for autonomous AI agents, potentially accelerating progress and democratizing access to high-performance models for agent tasks. Models are available at https://huggingface.co/collections/Salesforce/xlam-models-65f00e2a0a63bbcd1c2dade4

Related papers

• FuXi-$α$: Scaling Recommendation Model with Feature Interaction Enhanced Transformer [81.12174905444229]
  Recent advancements have shown that expanding sequential recommendation models to large-scale recommendation models can be an effective strategy. We propose a new model called FuXi-$alpha$ to address these issues. Our model outperforms existing models, with its performance continuously improving as the model size increases.
  arXiv  Detail & Related papers  (2025-02-05T09:46:54Z)
• Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling [128.24325909395188]
  We introduce InternVL 2.5, an advanced multimodal large language model (MLLM) series that builds upon InternVL 2.0. InternVL 2.5 exhibits competitive performance, rivaling leading commercial models such as GPT-4o and Claude-3.5-Sonnet. We hope this model contributes to the open-source community by setting new standards for developing and applying multimodal AI systems.
  arXiv  Detail & Related papers  (2024-12-06T18:57:08Z)
• MALT: Improving Reasoning with Multi-Agent LLM Training [64.13803241218886]
  We present a first step toward "Multi-agent LLM training" (MALT) on reasoning problems. Our approach employs a sequential multi-agent setup with heterogeneous LLMs assigned specialized roles. We evaluate our approach across MATH, GSM8k, and CQA, where MALT on Llama 3.1 8B models achieves relative improvements of 14.14%, 7.12%, and 9.40% respectively.
  arXiv  Detail & Related papers  (2024-12-02T19:30:36Z)
• xGen-MM (BLIP-3): A Family of Open Large Multimodal Models [157.44696790158784]
  This report introduces xGen-MM, a framework for developing Large Multimodal Models (LMMs) The framework comprises meticulously curated datasets, a training recipe, model architectures, and a resulting suite of LMMs. Our models undergo rigorous evaluation across a range of tasks, including both single and multi-image benchmarks.
  arXiv  Detail & Related papers  (2024-08-16T17:57:01Z)
• VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents [50.12414817737912]
  Large Multimodal Models (LMMs) have ushered in a new era in artificial intelligence, merging capabilities in both language and vision to form highly capable Visual Foundation Agents. Existing benchmarks fail to sufficiently challenge or showcase the full potential of LMMs in complex, real-world environments. VisualAgentBench (VAB) is a pioneering benchmark specifically designed to train and evaluate LMMs as visual foundation agents.
  arXiv  Detail & Related papers  (2024-08-12T17:44:17Z)
• Coalitions of Large Language Models Increase the Robustness of AI Agents [3.216132991084434]
  Large Language Models (LLMs) have fundamentally altered the way we interact with digital systems. LLMs are powerful and capable of demonstrating some emergent properties, but struggle to perform well at all sub-tasks carried out by an AI agent. We assess if a system comprising of a coalition of pretrained LLMs, each exhibiting specialised performance at individual sub-tasks, can match the performance of single model agents.
  arXiv  Detail & Related papers  (2024-08-02T16:37:44Z)
• LAMBDA: A Large Model Based Data Agent [7.240586338370509]
  We introduce LArge Model Based Data Agent (LAMBDA), a novel open-source, code-free multi-agent data analysis system. LAMBDA is designed to address data analysis challenges in complex data-driven applications. It has the potential to enhance data analysis paradigms by seamlessly integrating human and artificial intelligence.
  arXiv  Detail & Related papers  (2024-07-24T06:26:36Z)
• Arcee's MergeKit: A Toolkit for Merging Large Language Models [0.6374098147778188]
  MergeKit is a framework to efficiently merge models on any hardware. To date, thousands of models have been merged by the open-source community.
  arXiv  Detail & Related papers  (2024-03-20T02:38:01Z)
• ModelGPT: Unleashing LLM's Capabilities for Tailored Model Generation [35.160964210941955]
  We propose ModelGPT, a framework designed to determine and generate AI models tailored to the data or task descriptions provided by the user. Given user requirements, ModelGPT is able to provide tailored models at most 270x faster than the previous paradigms.
  arXiv  Detail & Related papers  (2024-02-18T11:24:34Z)
• TrainerAgent: Customizable and Efficient Model Training through LLM-Powered Multi-Agent System [14.019244136838017]
  TrainerAgent is a multi-agent framework including Task, Data, Model and Server agents. These agents analyze user-defined tasks, input data, and requirements (e.g., accuracy, speed), optimizing them from both data and model perspectives to obtain satisfactory models, and finally deploy these models as online service. This research presents a significant advancement in achieving desired models with increased efficiency and quality as compared to traditional model development.
  arXiv  Detail & Related papers  (2023-11-11T17:39:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.