AgentLogs: A Dataset for Opening the Black Box of GitHub's Cloud Agent

• URL: http://arxiv.org/abs/2608.29204v1
• Date: Sat, 29 Aug 2026 11:38:19 GMT
• Title: AgentLogs: A Dataset for Opening the Black Box of GitHub's Cloud Agent
• Abstract summary: AgentLogs is a large-scale dataset of agent activity on GitHub.<n>It comprises 307,416 agent tasks and 549,239 agent sessions in 35,810 of the 1,812,362 popular public repositories.
• Score: 0.8644909837301148
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative AI-based software engineering agents are becoming routine contributors to real-world software projects. On GitHub, developers can assign tasks to the Copilot cloud agent, which autonomously explores the repository, edits code, runs commands, and opens or reviews pull requests, producing a detailed log of every step along the way. While existing datasets capture outcomes of agent contributions, such as agent-authored pull requests, the process by which agents produce these contributions remains largely unexplored. To address this gap, we introduce AgentLogs, a large-scale dataset of agent activity on GitHub. AgentLogs comprises 307,416 agent tasks and 549,239 agent sessions in 35,810 of the 1,812,362 popular public repositories that we scanned, together with 64,255,174 session log entries that record each agent run step by step, including prompts, intermediate reasoning, tool calls (e.g., file edits, git operations, and GitHub interactions), and token usage. By exposing not only what agents contribute but also how they work, AgentLogs enables research on agent behavior, efficiency and cost, task formulation, failure modes, and human-agent collaboration in agentic software engineering.

Related papers

• Agent ATO: Visualizing Agent Interaction Timelines from Logs [1.5639957559514246]
  Agent ATO is a tool for visualizing AI coding agent interaction timelines from console logs.<n>In addition to an all-interaction timeline, Agent ATO provides filtered timelines that emphasize file discovery, file reading, file editing, and execution.
  arXiv  Detail & Related papers  (2026-09-08T06:22:44Z)
• An Exploratory Study of Agent Plans for Agentic AI Coding Tools in Open-Source Software [16.01774662182876]
  Repository-level configuration artifacts allow developers to provide guidance for agentic AI coding tools, such as Claude Code, Gemini, etc.<n>We present an exploratory study of Agent Plans in open-source software repositories, examining how plan files are preserved, which development activities they support, and what information they provide to guide agent execution.
  arXiv  Detail & Related papers  (2026-08-05T10:15:28Z)
• ToFu: A White-Box, Token-Efficient Agent Harness for Researchers [73.72929185946842]
  ToFu is an agentic harness for researchers that reads your files, edits files, runs commands, and integrates with your development tools.<n>As a research assistant, it supports practical research with superior token efficiency, lower cost, and multilingual capability.<n>As a research object, ToFu provides a white-box agentic harness that allows researchers to inspect, modify, and evaluate its orchestration logic, tool-use behavior, and harness design.
  arXiv  Detail & Related papers  (2026-07-13T11:26:06Z)
• Code as Agent Harness [107.31925305395957]
  In emerging agentic systems, code is no longer only a target output.<n>Code increasingly serves as an operational substrate for agent reasoning, acting, environment modeling, and execution-based verification.<n>This survey provides a unified roadmap toward executable, verifiable, and stateful AI agent systems.
  arXiv  Detail & Related papers  (2026-05-18T17:59:03Z)
• SWE-chat: Coding Agent Interactions From Real Users in the Wild [70.18158706281724]
  SWE-chat is the first large-scale dataset of real coding agent sessions collected from open-source developers in the wild.<n>The dataset currently contains 6,000 sessions, comprising more than 63,000 user prompts and 355,000 agent tool calls.
  arXiv  Detail & Related papers  (2026-04-22T17:08:19Z)
• AIDev: Studying AI Coding Agents on GitHub [9.1677959533705]
  We introduce AIDev, a large-scale dataset focused on agent-authored pull requests (Agentic-PRs) in real-world GitHub repositories.<n> AIDev aggregates 932,791 Agentic-PRs produced by five agents: OpenAI Codex, Devin, GitHub Copilot, Cursor, and Claude Code.<n>In addition, AIDev includes a curated subset of 33,596 Agentic-PRs from 2,807 repositories with over 100 stars, providing further information such as comments, reviews, commits, and related issues.
  arXiv  Detail & Related papers  (2026-02-09T20:45:58Z)
• AgentMesh: A Cooperative Multi-Agent Generative AI Framework for Software Development Automation [0.0]
  We propose a Python-based framework that uses multiple cooperating LLM-powered agents to automate software development tasks.<n>In AgentMesh, specialized agents - a Planner, Coder, Debugger, and Reviewer - work in concert to transform a high-level requirement into fully realized code.
  arXiv  Detail & Related papers  (2025-07-26T10:10:02Z)
• Cerebrum (AIOS SDK): A Platform for Agent Development, Deployment, Distribution, and Discovery [33.89476893368382]
  We present Cerebrum, an Agent SDK for AIOS that addresses the gap through three key components.<n>A comprehensive SDK featuring a modular four-layer architecture for agent development; (2) a community-driven Agent Hub for sharing and discovering agents; and (3) an interactive web interface for testing and evaluating agents.<n>Cerebrum advances the field by providing a unified framework that standardizes agent development while maintaining flexibility for researchers and developers to innovate and distribute their agents.
  arXiv  Detail & Related papers  (2025-03-14T14:29:17Z)
• CowPilot: A Framework for Autonomous and Human-Agent Collaborative Web Navigation [70.3224918173672]
  CowPilot is a framework supporting autonomous as well as human-agent collaborative web navigation.<n>It reduces the number of steps humans need to perform by allowing agents to propose next steps, while users are able to pause, reject, or take alternative actions.<n>CowPilot can serve as a useful tool for data collection and agent evaluation across websites.
  arXiv  Detail & Related papers  (2025-01-28T00:56:53Z)
• AgentGym: Evolving Large Language Model-based Agents across Diverse Environments [116.97648507802926]
  Large language models (LLMs) are considered a promising foundation to build such agents. We take the first step towards building generally-capable LLM-based agents with self-evolution ability. We propose AgentGym, a new framework featuring a variety of environments and tasks for broad, real-time, uni-format, and concurrent agent exploration.
  arXiv  Detail & Related papers  (2024-06-06T15:15:41Z)
• Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub [79.31134731122462]
  We introduce OpenAct benchmark to evaluate the open-domain task-solving capability, built on human expert consultation and repositories in GitHub.<n>We present OpenAgent, a novel LLM-based agent system that can tackle evolving queries in open domains through autonomously integrating specialized tools from GitHub.
  arXiv  Detail & Related papers  (2023-12-28T15:47:30Z)
• Agents: An Open-source Framework for Autonomous Language Agents [98.91085725608917]
  We consider language agents as a promising direction towards artificial general intelligence. We release Agents, an open-source library with the goal of opening up these advances to a wider non-specialist audience.
  arXiv  Detail & Related papers  (2023-09-14T17:18:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.