Research Built to Be Used
We build benchmarks, datasets, and infrastructure to advance machine learning research.
Our Mission
We are a research organization building open benchmarks, datasets, and infrastructure. Our work explores machine learning, autonomous systems, and how computational tools can advance scientific research.
What We Build
Benchmarks
We build benchmarks to measure how AI agents perform on real research tasks.
Datasets
1.1M enriched papers. 129K research repositories. 778K code functions. The raw material for studying how AI systems interact with real scientific work.
Infrastructure
Runtimes for structured agent workloads. Orchestration topologies for recursive improvement loops. The scaffolding to run experiments at scale.
Research
ARIA Benchmark: How Much Machine Learning Do AI Models Actually Know?
A suite of five closed-book benchmarks probing the ML knowledge that frontier language models have internalized during training.
ArXiv Research Code Dataset: 129K Research Repositories
A collection of 4.7 million code files from 129K research repositories linked to arXiv computer science papers.
ArXivDLInstruct: 778K Research Code Functions for Instruction Tuning
A dataset of 778,152 functions extracted from arXiv-linked research code, each paired with instruction prompts, for training...
DeltaML-Bench: Evaluating Machine Learning Agents on Real-World Research Repositories
A 48-task benchmark for evaluating autonomous ML experimentation in real research repositories.
ML Research Benchmark: Can AI Agents Do Real ML Research?
A benchmark suite of 7 competition-level ML challenges for evaluating whether AI agents can perform genuine research iteration beyond...
About
Algorithmic Research Group builds tools and infrastructure for research. Benchmarks for evaluating autonomous agents. Datasets for studying machine learning systems. Runtimes for running agent workloads at scale.