ARES: An open-source platform for robot data ingestion, annotation, and curation

ARES (Automatic Robot Evaluation System) is a open-source (Apache 2.0) platform for automatically ingesting, curating, and evaluating robot data using ML models to quickly and accurately understand policy performance, identify areas for improvement, and generate new robot datasets, all without setting up any heavy infrastructure. Researchers tend to chase point-solutions for specific tasks or paper implementations, but ARES is designed to be a generalized platform for long-term robot research that scales from a laptop to the cloud. ARES is built to be simple and scalable, with a special focus on ease of use. All computation and model inference can be run through local resources or cloud APIs via model providers like OpenAI, Anthropic, Gemini, Modal, Replicate, etc., requiring only a credit card for access - no complex cloud infrastructure or GPU setup needed. We make our data available on the Hugging Face Hub, which contains roughly 5000 rollouts from the Open X-Embodiment project. See the Data section for more details.

At a high level, ARES is composed of three main components:

- Ingestion: automatically transform raw robot data into a structured format with VLMs - Annotation: annotate the rollouts with pseduo-labels for downstream tasks - Curation and Modeling: understand data distributions and select data for training or evaluation

Who and what is ARES for?

ARES is a platform for understanding robot data, targeted at robot developers and researchers. Researchers and developers suffer from two main problems: building point-solutions for specific tasks or papers and transitioning from research-scripts to production-tools. ARES aims to solve both of these problems by providing a platform for robot data understanding, enabling rapid development of new robot systems.

You can use ARES to:

- Curate and annotate ground-truth teleoperation data - Evaluate the performance of robot models - Analyze batches of robot data to improve policies

I wrote a longer blog post about ARES and why robotics researchers need to adopt scalable platforms instead of script-based point-solutions here: https://a16z.com/ares-an-open-source-platform-for-robot-data...