AI agents have gotten very good at terminal-based software engineering benchmarks. A new benchmark built by scientists shows they still have a long way to go on real research work. On Terminal-Bench-Science 0.1, released Thursday, the best result came from Claude Opus 5 running in Claude Code, which resolved 30% of tasks.
The benchmark is led by researchers at Stanford University and built by the team behind Terminal-Bench, working with domain experts from institutions around the world. “Scientists, not model developers or data vendors, set the bar for scientific capability in AI,” the announcement says.
70 tasks from real labs
The first release has 70 tasks drawn from researchers’ own workflows across the life, physical, earth, mathematical and engineering sciences. They cover data analysis, statistical inference, simulation, optimization, theorem proving, image reconstruction, signal processing, sensor calibration and scientific machine learning. Agents are graded on concrete artifacts such as analyses, simulations, proofs, code and data products, using reproducible task-specific tests.
Getting in was hard. Researchers submitted 920 proposals through an open process on GitHub; 464 were approved for implementation and 386 pull requests opened, but only 70 tasks passed domain review, technical review and a final “bar raiser” check. The team says 376 contributors across 22 countries took part.
The results
Each model ran three trials per task. After Opus 5’s 30.0%, GPT-5.6 Sol with Codex reached 22.4% and Claude Fable 5 with Claude Code 21.4%. Claude Opus 4.8 scored 10.5%. GPT-5.6 Terra, Kimi K3 and Grok 4.6 each resolved under 10%, GLM 5.3 was the strongest open model at 8.1%, and GPT-5.6 Luna came last at 3.3%.
Cost varied widely. GPT-5.6 Sol matched Claude Fable 5’s performance for less than a third of the evaluation cost ($4.2k vs. $14.2k), while Opus 5’s top score cost $7.0k. By domain, Anthropic and OpenAI models held the top two spots everywhere except engineering, where Grok 4.6 tied GPT-5.6 Sol for second at 14.8% with lower cost and token use. In mathematics, Fable 5 (33.3%) and Sol (31.4%) beat Opus 5.
Every model scored more than 10 percentage points lower than on Terminal-Bench 3.0, a gap the team says is deliberate: tasks were calibrated against the newest frontier models during review. The goal, the announcement says, is agents that can take on technically demanding, time-consuming workflows so scientists can spend more time on hypotheses, interpretation and communication.
The benchmark is meant to be continuous. Future releases will add tasks and retire ones that agents saturate, and work on version 0.2 is underway, with an Oct. 5 deadline for contributed pull requests.
WebProNews is an iEntry Publication