AgentBeats

AgentBeats is an open platform for Agentified Agent Assessment (AAA), focused on open, standardized, and reproducible evaluation for real-world agents. It provides a unified evaluation interface based on A2A and MCP, supports multi-agent assessments natively, and helps reduce test-production mismatch.

Authors: Xiaoyuan Liu, Jianhong Tu, Yuqi Chen, Siyuan Xie, Sihan Ren, Tianneng Shi, Gal Gantar, Evan Sandoval, Donghyun Lee, Daniel Miao, Peter J. Gilbert, Nick Hynes, Mauro Staver, Warren He, David Marn, Andrew Low, Xi Zhang, Elron Bandel, Michal Shmueli-Scheuer, Siva Reddy, Alexandre Drouin, Alexandre Lacoste, Ramayya Krishnan, Elham Tabassi, Yu Su, Victor Barres, Chenguang Wang, Wenbo Guo, Dawn Song.

🚀 Visit Website

Product site, updates, and community resources.

Open Website
📘 Read Paper

Official arXiv page for AAA and evaluation details.

Open arXiv
🎤 ICML Poster

ICML 2026 virtual poster entry and publication context.

Open Poster Page
🐞 Give Feedback

Report issues, suggest improvements, and discuss documentation updates.

Open GitHub Issues

Citation Metadata

@article{liu2026agentbeats,
  title={AgentBeats: Agentifying Agent Assessment for Openness, Standardization, and Reproducibility},
  author={Liu, Xiaoyuan and Tu, Jianhong and Chen, Yuqi and Xie, Siyuan and Ren, Sihan and Shi, Tianneng and Gantar, Gal and Sandoval, Evan and Lee, Donghyun and Miao, Daniel and others},
  journal={arXiv preprint arXiv:2606.13608},
  year={2026}
}

Competitions

🏆 AgentX-AgentBeats Competition

Open competition track for agent assessment and benchmarking.

Open Competition
🔐 Secure-Agent CTF

Security-focused challenge environment for robust agent behavior.

Enter CTF Portal

Blog Index

Start here if you are new to AgentBeats: read Blog 0 for the core AAA idea, then continue with implementation and integration posts. If you are preparing a submission, Blog 2 and Blog 3 are the fastest path to hands-on setup.

RDI Foundation