Fable 5 posted a 16.1% automation rate on the Remote Labor Index, nearly double Opus 4.8 at 8.3% and about 2.5 times GPT-5.5 at 6.3%, according to the latest results from the Center for AI Safety (CAIS). CAIS said in a new blog post that the frontier has improved by more than fourfold in less than eight months, compared with the top score of 2.5% when the benchmark first appeared.
All three newly tested models outperformed every system previously evaluated on the benchmark.
What the Remote Labor Index measures
The Remote Labor Index (RLI) was developed by CAIS and Scale AI. Its paper was published in October 2025, with 47 researchers involved. The benchmark includes 240 real freelance projects drawn from 358 verified freelancers on Upwork across 23 categories, including 3D modeling, CAD, architecture, graphic design, video animation, audio production, data analysis, and web applications. The total project value exceeds $144,000.
The core metric is automation rate: the share of projects where an agent’s deliverable is judged by human reviewers to meet at least a paying client’s acceptable standard. Each deliverable is compared one by one against a gold-standard version produced by a professional freelancer, using a direct test: would a reasonable client accept this work?
That makes RLI different from a conventional AI benchmark. Each task is a full commercial assignment, with a client brief, input files, and multi-format outputs spanning 72 file types. The median time for a human professional to complete a project is 11.5 hours, and the average is 28.9 hours.
How scores moved from 2.5% to 16.1%
When RLI launched in October 2025, Manus led the board with a 2.5% automation rate. Opus 4.6 paired with Claude Cowork later pushed the mark to 4.17%. In the latest round, three new models arrived with stronger agent frameworks and the scores jumped.
CAIS pointed to several factors behind Fable 5’s 16.1% result. One was the use of a worker-critic loop. A separate review agent checked the output from the perspective of a demanding client, opened files, captured screenshots, and compared the work against the brief item by item. If it found problems, it sent the job back to the execution agent for revision until the review passed or the budget ran out. CAIS said this setup let extra budget translate into better deliverable quality.
Budget also differed across models. Fable 5 had a per-project cap of $150 because of its higher token pricing, while other models were capped at $50. All agents were given a 24-hour limit, access to A100 GPUs, and computer-use tools.
Fable 5’s evaluation was interrupted by U.S. government export controls, so it completed 218 out of 240 projects. CAIS said the 22 unevaluated projects were evenly distributed across fields and difficulty bands. Even if all 22 were counted as failures, Fable 5 would still post a 14.6% automation rate, ahead of every other model.
CAIS says AI judges still fall short
CAIS also tested whether AI-based evaluation could replace expensive human review. Its answer was no. After automated judging was calibrated on older models and then applied to newer ones, it overstated GPT-5.5 by nearly 3x and Opus 4.8 by about 2.5x. The rank order stayed roughly right, but the absolute scores drifted far from reality.
CAIS said the reason is straightforward: judging is itself a hard agentic task. To evaluate a deliverable fairly, the reviewer has to open files in the right professional software, use the tools correctly, and make the same kind of call a paying client would make. That is also where current agents are weak.
The blog cited one example involving GPT-5.5 on a 3D modeling task. The model submitted a fabricated render, and the issue only became obvious after opening the 3D file and checking the actual geometry. In CAIS’s view, AI judges are running into the same capability ceiling as AI workers.
What 16.1% does and does not mean
CAIS said the “time horizon” hypothesis does not hold on RLI. The idea suggests that tasks taking humans longer are harder for AI, which appears to work in areas such as coding, but not across the broader mix of remote work covered here. Success rates did not decline as human completion time rose. Instead, performance showed a jagged frontier, suggesting that more than time complexity determines whether AI can complete a project.
Progress has been fast, but the absolute level is still low. CAIS showed three Fable 5 examples in its blog — jewelry 3D modeling, a 2D animated ad, and an architectural drawing — and said none met a professional delivery standard. In the ring design case, Fable 5 looked visibly better than older models, but the prong setting was still rough on close inspection.
That leaves 84% of real freelance projects outside AI’s current reach on this benchmark. CAIS argues that RLI matters because it is calibrated to economic value. It is not asking whether AI can solve a test problem. It is asking whether AI can earn money.
CAIS said the next points to watch are the supplemental results for Fable 5’s remaining 22 projects, along with future performances from Gemini 3.5 Pro, currently at 1.25%, and GPT-5.6.