To Measure AI Bio-Risk, Test the Experts

This is an Alice version of an essay published as an exclusive on AI Frontiers on November 9, 2026. The full original article is linked at the end.

--

Most AI-biosecurity research asks one question: can a general-purpose model help a novice, someone with little biology training, carry out dangerous work? It is an intuitive worry, and it is worth studying. But it may be aimed at the wrong people. The ones best placed to turn AI advice into real laboratory results are not novices. They are experienced biologists, who already have the hands, the judgment, and the perseverance to push an experiment through when it fails. The near-term risk could sit with the experts, and almost no one is measuring it there.

To see why experience matters so much, picture a lab experiment that keeps failing for months, and no one can say why. Every step was followed, yet the result was wrong. Often the cause is mundane, like a single ingredient that has quietly gone bad, and nothing in the written procedure points there, because a procedure assumes the ingredients work. What rescues the experiment is an experienced colleague who has seen the problem before and knows what it means.

That hard-won instinct is what separates a written procedure from a result that works. It is also what AI could hand to someone who already has that tacit knowledge, and unfortunately what most AI bio-risk tests have overlooked.

For the teams training frontier AI and LLMs, that gap matters, and it is one even the biggest labs, OpenAI, Anthropic, Google DeepMind, Meta, and Microsoft, have left largely untested. 

This piece walks through what today's evals actually measure, why an expert may gain more from AI than a novice, and the kind of testing that could finally close it.

What the current evidence actually measures

The number that matters is uplift: how much better someone performs with AI than without it. The studies so far have measured it only in pieces, and each piece looks at a different stage of research.

At the information stage, which is simply gathering what you need to know, OpenAI's 2024 study paired fifty biology PhDs and fifty students with either the internet or GPT-4, but could not confirm any benefit from the model. At the planning stage, writing up how you would actually do it, Anthropic's Claude 4 system card and its Claude Mythos trials found AI could produce better-written bioweapon plans, yet every plan still carried mistakes serious enough to make it fail in practice. On computer-based problems, the kind researchers call in silico because no physical lab is involved, Zhang et al. (2026) found AI gave novices more than four times the odds of answering correctly, enough to beat unaided experts on several tests, though the advantage disappeared once people were simply handed the relevant research paper.

Then there is the wet lab, the hands-on work with real cells, chemicals, and equipment, where biology actually happens. Here the evidence nearly runs out. The largest trial, Hong et al. (2025), put 153 novices through eight weeks of lab work, but found no clear difference between the people using AI and the people using only the internet, partly because so few succeeded at all. Smaller pilots at Los Alamos and Anthropic hit the same wall from the other side: the task was too easy, or the group too small to read anything into.

One pattern runs through all of it. Every hands-on lab trial tested novices. What a trained biologist gains from a frontier model at the bench has never been measured.

Why experience could widen the gap, not close it

It is tempting to assume AI helps beginners most, since they have the most to learn. The opposite may be true.

Experienced biologists are better equipped to use AI well. They can tell which suggestion is plausible and which is nonsense, catch the confident hallucination (a model stating something false as if it were fact: in one Los Alamos pretest, a model invented a piece of lab equipment that did not exist), and keep going through the failures that make most people give up. Novices often cannot tell good advice from bad and need visual demonstration to learn. In the Hong trial mentioned above, they rated YouTube as more useful than any frontier model. The smaller the gap a person has to close, the more a capable assistant is worth, and that points the advantage toward experts, not away from them.

The risk math follows. A 2025 GovAI analysis estimates that PhD biologists with hands-on lab experience, though a tiny share of possible bad actors, account for a disproportionate share of the danger, because they are far likelier to actually succeed in the lab: an assumed 20 in 100, against 1 in 100 for someone with a science bachelor's degree and 1 in 1,000 for everyone else. If AI meaningfully raises the capability of the much larger group with a degree and a year or two of lab experience, the picture shifts considerably.

The test that is missing

The fix is to measure expert uplift directly, under real conditions. That means hands-on lab trials built on safe biological tasks that still demand genuine problem-solving, with experienced scientists randomly split into a group that uses AI and a group that does not, the same model version throughout, and success judged by whether the experiment actually worked, not by how good the write-up reads.

These trials are hard, which is why they barely exist. They need labs, equipment, safety and ethics approval, and months of scientists' time, and by the time results are published the model being tested may already be out of date. But Hong and colleagues have proven that wet-lab trials can be run at scale. The gap is funding and intention, not possibility.

Why this matters for the people building the models

Every major AI lab's safety framework (OpenAI, Anthropic, Google DeepMind, Meta, Microsoft) now addresses biological misuse, and several already treat expert and novice capability as separate questions. What those frameworks still lack is evidence on what a frontier model does for a trained biologist's hands-on work. 

Measuring it would sharpen pre-deployment risk assessment in both directions: real uplift would justify stronger safeguards, and no uplift would spare useful research from limitations it does not need.

At Alice, we work with frontier labs to generate this evidence through expert-led evaluations and red-teaming that measure real-world model capabilities and help inform proportionate, evidence-based safeguards.

See how Alice Labs works with frontier models: labs.alice.io

Our approach to red teaming and evaluations: labs.alice.io/red-teaming

--

This piece is a condensed version of an essay by Sumnima Singh, GenAI Biosecurity Expert at Alice, published as an exclusive on AI Frontiers on November 9, 2026.

Sumnima is a physician-scientist with nearly two decades of experience in clinical medicine, immunology, and infectious diseases, and her work has appeared in Nature, Cell, and the Bulletin of the Atomic Scientists. Read the full essay on AI Frontiers.