The thing discovering your service is increasingly an AI agent, not a human. DomainArena tests whether agents can infer what service sits behind a domain — before you spend money on a name.
In pairwise tests, 87% of agents picked the first option regardless of which domain was shown. TLD effects (.com vs .dev vs .ai) were statistically insignificant. Order matters more than extension.
Llama 3.3, Mistral Small, and Qwen3 produced different rankings for the same domains. A domain cannot be called "agent-legible" based on one model. Cross-family replication is essential.
Some model families selected broken tools when they had enterprise-sounding descriptions. Agent discovery systems can be manipulated by presentation rather than capability.
Blind inference is a useful first filter, but execution testing is ground truth. AgentSearchBench (10K agents) confirms description similarity is weaker than execution-grounded performance.
Identical prompts at temperature zero produced materially different choices across time windows. One-shot domain ratings are scientifically weak. DomainArena replicates across windows.
The tested model never scores itself. Llama generates the inference, independent Mistral evaluates the match. This prevents self-reinforcing bias in comprehension testing.
The entire decision basis is frozen into a content-addressed SHA-256 receipt. Domain, intent, score, inference, prices, registration status, DNS verification — all hashed together. Proof that the decision was made with verified economic state, not gut feeling.
DomainArena attacks the decision before deployment: which available hostname best communicates the intended service to agents?