Autonomous agents expand testing; scaling requires limits
In May 2026, Prime Intellect described an experiment in which Codex and Claude Code worked autonomously on optimizing nanoGPT. According to the company, the agents carried out about 10,000 runs over roughly two weeks and beat the human reference used in that environment.
The result doesn't prove agents will solve any problem. It shows another possibility: an execution layer can test, compare and record alternatives for much longer than a person could follow manually.
Persistence helps when the problem is measurable
Prime Intellect also reports using roughly 14,000 hours of H200 GPUs in the experiment. In a company, the scale will be different, but the principle holds. A task that runs for hours can consume resources, produce results of little use or move down a path nobody reviewed.
Agents make more sense when there is a verifiable goal, testable alternatives, comparable results and a safe environment. Software tests, configuration comparisons and scenario analysis can benefit from persistence, as long as the task is bounded.
Autonomy doesn't replace framing the problem
Prime Intellect itself reports greater difficulty generating truly new ideas without prior human references. This separates two roles: AI can test, record, compare and repeat; people remain responsible for framing the problem, choosing what to measure and deciding whether the result has value outside the controlled environment.
An improved metric doesn't guarantee quality, safety or a positive effect for the customer. The greater the autonomy, the greater the need for limits on time, cost, attempts, permissions and human intervention.
The first step is a test with a defined stop
Before letting an agent work for hours, define the goal, the maximum cost, the allowed environment, the stopping criteria and the person who will validate the result. Record each run and establish what happens when the agent finds a promising result or strays from the expected behavior.
An initial test can have a fixed budget, a limit on attempts, restricted access and a review deadline. This design makes it possible to compare gain, consumption and quality before expanding autonomy.
The experiment needs a boundary
Define before the run the goal, the maximum budget, the accessible data and systems, the person responsible for review and the events that stop the agent. Separate exploratory results from actions that change real records or decisions.
Sources
Prime Intellect, 14 May 2026. https://www.primeintellect.ai/auto-nanogpt
Prime Intellect, 14 Aug 2026. https://www.primeintellect.ai/blog/measuring-autonomous-research
Prime Intellect. https://github.com/PrimeIntellect-ai/experiments-autonomous-speedrunning
Anthropic, 23 Mar 2026. https://www.anthropic.com/research/long-running-Claude
Anthropic, 26 Nov 2025. https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents