01 / 08
Agents
Who is in the system?
arXiv:2607.22682
cs.AI · cs.LG · cs.MA
A Vocabulary for Multi-Agent Automated Research Systems
Eight coordinates for describing what changed and what was evaluated.
𝓜
=
⟨
The tuple
One coordinate per design choice.
Select a component to see what it controls and where its internal choices live.
Configuration explorer
Ten systems, the same vocabulary.
Each paper sets only a few coordinates. Select a system to trace its structure and see the choices it deliberately sets.
Structural view
AIRA₂
- Task
- Artifact
- Evaluator
- Budget
Coordinates this paper sets
Coordinate map
selected system
Contribution grid
Compare all ten at once.
A filled cell marks a coordinate the paper deliberately sets, not one shown to improve performance. Select any system or coordinate to inspect the underlying specification.
View
Swipe horizontally to compare every coordinate.
● coordinate the system sets
○ documented proxy-quality gap
generic, inherited from the harness
default in every system
Transcribed from Figure 4 and the case-study descriptions. The operation universe and external world state are omitted because every system has them and none sets them. Exploration is shaded because every system leaves it at the default temperature.
The consequence
“Taste” is two different failures.
Separating the generator from the evaluator turns a vague complaint into two problems with different fixes.
01
Generative taste
Does the system propose anything new?
This is the rate of novel trajectories before any score is observed. Change the backbone priors, initialization, exploration, or learned cross-run state.
02
Evaluative taste
Does the score reflect true quality?
Evaluative taste is the gap between proxy evaluator e and true quality q. More search can widen that gap. Strengthen evaluator integrity or change its mechanism before crediting a more inventive generator.
From the paper
The framework in four figures.
Rendered as vectors so the notation stays sharp at any scale.
Visual guide
A visual guide to every coordinate.
Eleven shareable plates: the tuple, taste, recursive self-improvement, and one for each component.
Abstract
A common language for autoresearch.
We introduce a vocabulary for automated research systems built from one or more agents to make their design choices easier to describe and compare. The vocabulary specifies who the agents are, which operations the harness exposes, who may invoke them, how agents communicate, what state is visible within and across runs, how the next action is chosen, how a run begins, and how outputs are scored. A trajectory records one run from the input task to the returned artifact. Because agents, operations, and initialization may be stochastic, repeated runs on the same task induce a distribution over trajectories rather than a single behavior.
The vocabulary assigns each structural design question, such as when agents should communicate, gain or lose a capability, or carry information across runs, to a distinct coordinate that can be varied on its own. It also makes the evaluator a component of the system, since reported gains depend on how closely the proxy score matches true quality. That separation also splits the vague complaint that these systems lack taste into two failures with different fixes. Generative taste is the rate at which a system proposes novel trajectories before any score is observed, and evaluative taste is the gap between the proxy score and the quality it should match. We instantiate the vocabulary on recent autoresearch systems to illustrate that it covers designs that differ widely in structure.
Citation
Use the vocabulary.
The framework is a specification language, not a benchmark. Use it to make system descriptions and ablations legible.
doi.org/10.48550/arXiv.2607.22682@misc{akhbari2026vocabulary,
title = {A Vocabulary for Multi-Agent
Automated Research Systems},
author = {Bardiya Akhbari},
year = {2026},
eprint = {2607.22682},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2607.22682}
}