Source to life · science

Reading gels, checking controls, counting colonies: everyday wet-lab work becomes AI's new exam

OpenAI built BioBasicsBench with Ligo, a biotech company focused on enzyme design: 100 tasks drawn entirely from raw experimental evidence in scientists' own experiments — gel and blot images, plate photographs, microscopy stacks, sequencing reads, plate-reader exports and bench videos, testing whether agents can read data, spot errors, and recognize when the evidence cannot support a conclusion. Seven models, 37 model-and-reasoning settings, five attempts per task per setting — 18,500 results in all: at the xhigh setting GPT-6 Astra earns an average of 66.7% of the available credit, against 43.8% for GPT-5.6 Sol. Astra catches a negative control labeled with the wrong antibiotic, recognizes a microscopy z-stack, and asks for better controls when a dual dye makes a signal ambiguous; it also lets a familiar assumption override the controls, discounts contradicting observations in the same field of view, and overcounts colonies by roughly 12%, inflating a transformation-efficiency estimate by the same proportion. OpenAI's own line: higher benchmark scores do not yet mean reliable judgments in the lab.

This account draws on OpenAI's announcement page (page metadata dated November 4, 2026; first seen in this site's collection on November 6, read in full on November 10). All scores, examples and boundary statements are OpenAI and Ligo's official caliber: this site ran no model, reproduced no task and touched no benchmark data.

In this articleContents4

中文

Why routine work deserves an exam

In early November, OpenAI released BioBasicsBench together with Ligo, a biotech company focused on enzyme design: 100 tasks that measure not scientific creativity but the things a wet lab repeats every day — interpreting raw experimental data, spotting errors, and recognizing the moment when the evidence at hand cannot support a conclusion. The page is blunt about the motive: everyday wet-lab work is full of failed experiments, ambiguous images and conflicting observations, while existing benchmarks are mostly built around charts and images selected for papers — missing exactly this ambiguity and the troubleshooting it requires. Misreading an image or overlooking a control problem has a real cost: scientists draw the wrong conclusion and spend time on experiments that cannot answer their question.

The exam covers five areas of routine work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. Tasks range from simple requests — reading the molecular weight of a DNA sample in a gel — to detailed assay analysis, with one thing in common: they are all routine work that wet-lab scientists should be able to complete reliably. The inputs deliberately preserve the forms of evidence scientists actually face: gel and blot images, plate photographs, microscopy stacks, sequencing reads, plate-reader exports, videos of bench work — a single task can mix several kinds; reference sheets such as ladder charts and plasmid maps appear as support. What is being tested is not recall but reading evidence the way one does at the bench.

The scoring is built to separate "partially right" from "right". Each task was written by a scientist from their own experimental data, paired with an expert-written grading rubric, and passed through three reviews — an internal review by a Ligo scientist who had not written it, an external review by a PhD-level scientist working in a wet lab, and a final internal review; reviewers checked the prompt, the evidence, the rubric, author notes and a graded model attempt. Agents return structured answers — conclusion, measurement or recommendation, with supporting reasoning where required — scored against the rubric's individual criteria, so an answer with a correct conclusion and a flawed analysis does not earn full credit. Most tasks test several skills at once: biological knowledge, interpretation of images and other data, error detection, protocol understanding.

The evaluation's scale: seven models — GPT-5.2, GPT-5.4, GPT-5.5, GPT-5.6 Sol, GPT-6.1 Sol, GPT-6 Luna and GPT-6 Astra — across 37 model-and-reasoning-setting combinations, five attempts per task per combination, 18,500 results in all. The metric is mean reward: the average task score as a percentage of available credit, with confidence intervals from resampling whole tasks and each task's five attempts kept together — variance within one task is not allowed to pose as stability across tasks.

Sources and further reading
  • OpenAI: Measuring how AI agents handle everyday wet-lab science (official announcement)Official release · page metadata dated November 4, 2026 · first collected November 6, read in full November 10Read in full. Key points: BioBasicsBench is built with Ligo, a biotech company focused on enzyme design; 100 tasks across five areas of routine wet-lab work — protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning; tasks range from reading the molecular weight of a DNA sample in a gel to detailed assay analysis, all routine work wet-lab scientists should complete reliably. Inputs preserve the evidence forms scientists actually face: gel and blot images, plate photographs, microscopy stacks, sequencing reads, plate-reader exports and videos of bench work; one task can mix several kinds, with reference sheets such as ladder charts and plasmid maps as support. Scientists wrote the tasks from their own experiments, paired with expert-written rubrics; each task passed an internal review by a Ligo scientist who had not written it, an external review by a PhD-level scientist working in a wet lab, and a final internal review — reviewers checked the prompt, the evidence, the rubric, author notes and a graded model attempt. Agents return structured answers (conclusion/measurement/recommendation, with supporting reasoning where required), scored per rubric criterion, distinguishing a correct conclusion from a partially correct analysis; most tasks test several skills together (biological knowledge, interpretation of images and other data, error detection, protocol understanding). Seven models evaluated: GPT-5.2, GPT-5.4, GPT-5.5, GPT-5.6 Sol, GPT-6.1 Sol, GPT-6 Luna and GPT-6 Astra; 37 model-and-reasoning settings, five attempts per task per setting, 18,500 results in all. The metric is mean reward — the average task score as a percentage of available credit — with 95% confidence intervals from resampling whole tasks, keeping each task's five attempts together. Scores given in the text: at xhigh, GPT-6 Astra 66.7% versus 43.8% for GPT-5.6 Sol; Astra improves substantially even at low reasoning effort, which OpenAI reads as stronger native multimodal capabilities possibly being key. Success examples: the dual-dye ambiguity (the SYTOX detector channel can also pick up FUN-1 green light; Sol classified cells as dead from brightness alone, Astra recognized the ambiguity and called for better controls); a negative-control plate whose rim label named the wrong antibiotic (LB Kan50) — Sol concluded from the absent colonies, Astra noticed the mislabel and rejected the plate as a reliable control; a microscopy series with cells drifting in and out of focus — Sol sought a biological explanation, Astra recognized a z-stack (images taken at different depths). Failure examples (all Astra): letting a familiar assumption override the controls (colony-PCR lane selection — Astra separates the two band groups correctly but assumes a larger PCR product means successful cloning and picks lanes 4, 6, 7, 8 where the reference's control comparison selects 5, 9, 10; lane 3 holds the original-plasmid PCR reference position, lane 2 carries both patterns because incomplete assembly leaves old template alongside new product, and colonies can carry either plasmid — the visual grouping is right while the experimental identities are reversed, which would purify the wrong plasmid downstream); discounting contradicting observations (vacuoles "generally darker", though some appear lighter in the same image — changing focus reverses the contrast; Astra inspects several regions and even notes the contrast varies with focus, yet the final answer keeps only the darker description); unreliable counting marks (close neighbours unresolved, visible colonies missed, empty agar marked; the net result is an overcount of about 12% on that plate, and with plated volume and dilution fixed the transformation-efficiency estimate inflates by the same proportion). Boundaries: higher benchmark scores do not yet mean reliable judgments in the lab; progress could let scientists spend less time troubleshooting and more time deciding which experiments to pursue; tracking these capabilities also serves understanding of biosafety and biosecurity implications.

中文

66.7 vs 43.8: the gap lives in seeing

The text gives one comparison, and it is the durable one: at the xhigh setting — which allows more reasoning per task — GPT-6 Astra earns an average of 66.7% of the available credit against 43.8% for GPT-5.6 Sol. The full chart on the page plots all seven models; the text does not enumerate their values. One directional statement is worth keeping too: Astra improves substantially over Sol even at low reasoning effort — OpenAI's reading is that stronger native multimodal capabilities may be the key to everyday wet-lab analysis, rather than more reasoning time.

Where exactly the gap lives, the page answers with three examples. First, reading ambiguity in a signal: in a dead-cell task, Sol classified cells as dead from a dye's signal — but that detector channel can also pick up light from a second dye (FUN-1 green), so brightness alone cannot justify "dead"; Astra recognized the ambiguity and called for better controls to separate the signals. Second, reading an error on a label: a DNA cloning experiment asked for conclusions drawn through controls — controls being essential; the negative-control plate had no colonies, and Sol concluded from that; Astra noticed the plate's rim label read LB Kan50 — the wrong antibiotic for the experiment — and rejected the plate as a reliable control (see the photo). Third, reading imaging context: a series of micrographs where cells drift in and out of view sent Sol looking for a biological explanation for the disappearance; Astra recognized a z-stack — a series of images taken at different depths — where cells entering and leaving focus are a property of the imaging, not a biological event.

Read together, the three examples share one trait: the gap is not "knowing more biology" but "seeing evidence the way a person at the bench does" — where the signal comes from, what the label says, how the image series was taken. None of that lives in the conclusion; it lives in the evidence's physical context. That is an editorial reading, not an official one; the official mechanism statement is a single sentence: stronger native multimodal capabilities may be the key.

A real task photo from the benchmark: a teal-gloved hand holds a culture plate, the handwritten rim label circled in yellow — "1B25 18F26D1 (LB Kan50)", an antibiotic that does not match what the experiment's control required.
The evidence that the negative control was "off" sits on the rim: the label reads LB Kan50, the wrong antibiotic for the experiment. GPT-5.6 Sol concluded from the absent colonies on the plate; GPT-6 Astra read the mislabel and rejected the plate as a reliable control. Image: task-example photo from OpenAI's announcement page. · Open full-size image
Sources and further reading
  • OpenAI: Measuring how AI agents handle everyday wet-lab science (official announcement)Official release · page metadata dated November 4, 2026 · first collected November 6, read in full November 10Read in full. Key points: BioBasicsBench is built with Ligo, a biotech company focused on enzyme design; 100 tasks across five areas of routine wet-lab work — protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning; tasks range from reading the molecular weight of a DNA sample in a gel to detailed assay analysis, all routine work wet-lab scientists should complete reliably. Inputs preserve the evidence forms scientists actually face: gel and blot images, plate photographs, microscopy stacks, sequencing reads, plate-reader exports and videos of bench work; one task can mix several kinds, with reference sheets such as ladder charts and plasmid maps as support. Scientists wrote the tasks from their own experiments, paired with expert-written rubrics; each task passed an internal review by a Ligo scientist who had not written it, an external review by a PhD-level scientist working in a wet lab, and a final internal review — reviewers checked the prompt, the evidence, the rubric, author notes and a graded model attempt. Agents return structured answers (conclusion/measurement/recommendation, with supporting reasoning where required), scored per rubric criterion, distinguishing a correct conclusion from a partially correct analysis; most tasks test several skills together (biological knowledge, interpretation of images and other data, error detection, protocol understanding). Seven models evaluated: GPT-5.2, GPT-5.4, GPT-5.5, GPT-5.6 Sol, GPT-6.1 Sol, GPT-6 Luna and GPT-6 Astra; 37 model-and-reasoning settings, five attempts per task per setting, 18,500 results in all. The metric is mean reward — the average task score as a percentage of available credit — with 95% confidence intervals from resampling whole tasks, keeping each task's five attempts together. Scores given in the text: at xhigh, GPT-6 Astra 66.7% versus 43.8% for GPT-5.6 Sol; Astra improves substantially even at low reasoning effort, which OpenAI reads as stronger native multimodal capabilities possibly being key. Success examples: the dual-dye ambiguity (the SYTOX detector channel can also pick up FUN-1 green light; Sol classified cells as dead from brightness alone, Astra recognized the ambiguity and called for better controls); a negative-control plate whose rim label named the wrong antibiotic (LB Kan50) — Sol concluded from the absent colonies, Astra noticed the mislabel and rejected the plate as a reliable control; a microscopy series with cells drifting in and out of focus — Sol sought a biological explanation, Astra recognized a z-stack (images taken at different depths). Failure examples (all Astra): letting a familiar assumption override the controls (colony-PCR lane selection — Astra separates the two band groups correctly but assumes a larger PCR product means successful cloning and picks lanes 4, 6, 7, 8 where the reference's control comparison selects 5, 9, 10; lane 3 holds the original-plasmid PCR reference position, lane 2 carries both patterns because incomplete assembly leaves old template alongside new product, and colonies can carry either plasmid — the visual grouping is right while the experimental identities are reversed, which would purify the wrong plasmid downstream); discounting contradicting observations (vacuoles "generally darker", though some appear lighter in the same image — changing focus reverses the contrast; Astra inspects several regions and even notes the contrast varies with focus, yet the final answer keeps only the darker description); unreliable counting marks (close neighbours unresolved, visible colonies missed, empty agar marked; the net result is an overcount of about 12% on that plate, and with plated volume and dilution fixed the transformation-efficiency estimate inflates by the same proportion). Boundaries: higher benchmark scores do not yet mean reliable judgments in the lab; progress could let scientists spend less time troubleshooting and more time deciding which experiments to pursue; tracking these capabilities also serves understanding of biosafety and biosecurity implications.

中文

Three ways it fails on the same exam

The page does not hide the failures in an appendix — it gives Astra's mistakes a section of their own, opening with the sentence most worth remembering: higher benchmark scores do not yet mean a model can make reliable judgments in the lab. The three failure examples each map onto a mistake humans make too.

The first: letting a familiar assumption override the controls. In the colony-PCR lane-selection task (see the gel), Astra's visual grouping is not wrong — it does separate the bands into two patterns; the error is assuming "a larger PCR product means the cloning worked", and under that premise it nominates lanes 4, 6, 7 and 8. The benchmark reference reads the controls instead: lane 3 holds only the original plasmid PCR, giving the old-plasmid reference position; lane 2 carries both patterns, because incomplete assembly leaves old template backbone alongside the new product; colonies can carry either the old or the new plasmid, and their bands should be compared against those reference patterns — the selection is lanes 5, 9 and 10. The two readings pick entirely different lanes: downstream purification by Astra's choice would yield the wrong plasmid. The vision was right; the experimental identities were reversed.

The second: seeing contradicting evidence and dropping it from the conclusion. In a vacuole-contrast task, Astra concluded that vacuoles — compartments inside cells — are generally darker than the surrounding cell, even though some appear lighter in the very same image. Both appearances matter: changing the microscope's focus reverses the contrast, so the darker examples establish no general rule. The page notes that Astra inspected several regions and even commented that contrast varies with focus — and then gave greater weight to the cells it judged better focused, keeping only the darker-vacuole description in the final answer. The contradiction was seen but never entered the conclusion. A correct intermediate observation failing to guide the final decision: of the three failures, this is the one closest to human confirmation bias.

The third: miscounting, and letting the error propagate. In the colony-counting task, Astra's counting marks do not reliably correspond to individual colonies: close neighbours go unresolved, visible colonies get missed, empty agar gets marked — a net overcount of about 12% on that plate relative to the benchmark reference. The page carries the consequence through: with plated volume and dilution held fixed, the estimate of how efficiently DNA was introduced into cells inflates by the same proportion — a small visual miscount becomes a systematic bias in an experimental parameter.

An annotated colony-PCR gel, lanes 1–10: lanes 4, 6, 7 and 8 numbered in pink align with the old-plasmid band position (Astra's selection), lanes 5, 9 and 10 in teal align with the new-product band (the benchmark reference's selection); lane 2 is boxed in teal (carries both band patterns) and lane 3 in pink (the original-plasmid PCR reference position).
Two readings of one gel: Astra's visual grouping is not wrong — it does separate the two band patterns; the error is assuming "a larger PCR product means the cloning worked", so it nominates lanes 4, 6, 7 and 8 where the reference's control comparison selects 5, 9 and 10. Downstream purification by Astra's choice would yield the wrong plasmid. Image: task example from OpenAI's announcement page (annotations are the official ones). · Open full-size image
Sources and further reading
  • OpenAI: Measuring how AI agents handle everyday wet-lab science (official announcement)Official release · page metadata dated November 4, 2026 · first collected November 6, read in full November 10Read in full. Key points: BioBasicsBench is built with Ligo, a biotech company focused on enzyme design; 100 tasks across five areas of routine wet-lab work — protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning; tasks range from reading the molecular weight of a DNA sample in a gel to detailed assay analysis, all routine work wet-lab scientists should complete reliably. Inputs preserve the evidence forms scientists actually face: gel and blot images, plate photographs, microscopy stacks, sequencing reads, plate-reader exports and videos of bench work; one task can mix several kinds, with reference sheets such as ladder charts and plasmid maps as support. Scientists wrote the tasks from their own experiments, paired with expert-written rubrics; each task passed an internal review by a Ligo scientist who had not written it, an external review by a PhD-level scientist working in a wet lab, and a final internal review — reviewers checked the prompt, the evidence, the rubric, author notes and a graded model attempt. Agents return structured answers (conclusion/measurement/recommendation, with supporting reasoning where required), scored per rubric criterion, distinguishing a correct conclusion from a partially correct analysis; most tasks test several skills together (biological knowledge, interpretation of images and other data, error detection, protocol understanding). Seven models evaluated: GPT-5.2, GPT-5.4, GPT-5.5, GPT-5.6 Sol, GPT-6.1 Sol, GPT-6 Luna and GPT-6 Astra; 37 model-and-reasoning settings, five attempts per task per setting, 18,500 results in all. The metric is mean reward — the average task score as a percentage of available credit — with 95% confidence intervals from resampling whole tasks, keeping each task's five attempts together. Scores given in the text: at xhigh, GPT-6 Astra 66.7% versus 43.8% for GPT-5.6 Sol; Astra improves substantially even at low reasoning effort, which OpenAI reads as stronger native multimodal capabilities possibly being key. Success examples: the dual-dye ambiguity (the SYTOX detector channel can also pick up FUN-1 green light; Sol classified cells as dead from brightness alone, Astra recognized the ambiguity and called for better controls); a negative-control plate whose rim label named the wrong antibiotic (LB Kan50) — Sol concluded from the absent colonies, Astra noticed the mislabel and rejected the plate as a reliable control; a microscopy series with cells drifting in and out of focus — Sol sought a biological explanation, Astra recognized a z-stack (images taken at different depths). Failure examples (all Astra): letting a familiar assumption override the controls (colony-PCR lane selection — Astra separates the two band groups correctly but assumes a larger PCR product means successful cloning and picks lanes 4, 6, 7, 8 where the reference's control comparison selects 5, 9, 10; lane 3 holds the original-plasmid PCR reference position, lane 2 carries both patterns because incomplete assembly leaves old template alongside new product, and colonies can carry either plasmid — the visual grouping is right while the experimental identities are reversed, which would purify the wrong plasmid downstream); discounting contradicting observations (vacuoles "generally darker", though some appear lighter in the same image — changing focus reverses the contrast; Astra inspects several regions and even notes the contrast varies with focus, yet the final answer keeps only the darker description); unreliable counting marks (close neighbours unresolved, visible colonies missed, empty agar marked; the net result is an overcount of about 12% on that plate, and with plated volume and dilution fixed the transformation-efficiency estimate inflates by the same proportion). Boundaries: higher benchmark scores do not yet mean reliable judgments in the lab; progress could let scientists spend less time troubleshooting and more time deciding which experiments to pursue; tracking these capabilities also serves understanding of biosafety and biosecurity implications.

中文

Boundaries, caliber, and the two lines it connects to

The caliber first: this is a vendor-built benchmark, vendor-run — OpenAI built it with Ligo, and the page's numbers have no independent third-party reading; the text gives only the Astra and Sol figures, and the full seven-model comparison lives in the page's chart. One hundred tasks bound the confidence intervals, and the whole-task resampling honestly preserves between-task variance. This site ran no model and reproduced no task; every score and example here is the official caliber.

The page's statement of purpose reads fully too: making everyday wet-lab work a target for improvement — interpreting raw data, checking controls, recognizing unsupported conclusions — so progress could let scientists spend less time troubleshooting and more time deciding which experiments to pursue next; the same page states that tracking these capabilities also helps understand their implications for biosafety and biosecurity. That second sentence is not a formality: an agent that reliably reads experimental evidence can also be pointed at experiments it should not run, and OpenAI writing this into the announcement itself means the line stays under official watch.

Two lines on this site connect. First, PSI, covered November 8: an AI research system produced a complete proof of a 2008 conjecture in three days — the front end of "AI does science"; BioBasicsBench is the other end — the routine a lab repeats daily: reading gels, checking controls, counting colonies. One end solves hard problems, the other does chores; the shared gate in both is whether the evidence gets read correctly. Second, the same GPT-6 Astra appears in the November 6 cybersecurity story: on RemoteExploitBench with Daybreak Blue it refused 100% of tasks — a measurement of refusal behavior, not capability. One model family, three exams: attack capability behind an approval gate, mathematical proof inside a research system, wet-lab chores on a public benchmark. For readers who run experiments, one actionable takeaway: the best score on this exam is currently 66.7% — so using it as an assistant today means its gel readings, control checks and counts all need human review; and it can already catch the error most easily missed through fatigue — a control plate labeled with the wrong antibiotic.

Sources and further reading
  • OpenAI: Measuring how AI agents handle everyday wet-lab science (official announcement)Official release · page metadata dated November 4, 2026 · first collected November 6, read in full November 10Read in full. Key points: BioBasicsBench is built with Ligo, a biotech company focused on enzyme design; 100 tasks across five areas of routine wet-lab work — protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning; tasks range from reading the molecular weight of a DNA sample in a gel to detailed assay analysis, all routine work wet-lab scientists should complete reliably. Inputs preserve the evidence forms scientists actually face: gel and blot images, plate photographs, microscopy stacks, sequencing reads, plate-reader exports and videos of bench work; one task can mix several kinds, with reference sheets such as ladder charts and plasmid maps as support. Scientists wrote the tasks from their own experiments, paired with expert-written rubrics; each task passed an internal review by a Ligo scientist who had not written it, an external review by a PhD-level scientist working in a wet lab, and a final internal review — reviewers checked the prompt, the evidence, the rubric, author notes and a graded model attempt. Agents return structured answers (conclusion/measurement/recommendation, with supporting reasoning where required), scored per rubric criterion, distinguishing a correct conclusion from a partially correct analysis; most tasks test several skills together (biological knowledge, interpretation of images and other data, error detection, protocol understanding). Seven models evaluated: GPT-5.2, GPT-5.4, GPT-5.5, GPT-5.6 Sol, GPT-6.1 Sol, GPT-6 Luna and GPT-6 Astra; 37 model-and-reasoning settings, five attempts per task per setting, 18,500 results in all. The metric is mean reward — the average task score as a percentage of available credit — with 95% confidence intervals from resampling whole tasks, keeping each task's five attempts together. Scores given in the text: at xhigh, GPT-6 Astra 66.7% versus 43.8% for GPT-5.6 Sol; Astra improves substantially even at low reasoning effort, which OpenAI reads as stronger native multimodal capabilities possibly being key. Success examples: the dual-dye ambiguity (the SYTOX detector channel can also pick up FUN-1 green light; Sol classified cells as dead from brightness alone, Astra recognized the ambiguity and called for better controls); a negative-control plate whose rim label named the wrong antibiotic (LB Kan50) — Sol concluded from the absent colonies, Astra noticed the mislabel and rejected the plate as a reliable control; a microscopy series with cells drifting in and out of focus — Sol sought a biological explanation, Astra recognized a z-stack (images taken at different depths). Failure examples (all Astra): letting a familiar assumption override the controls (colony-PCR lane selection — Astra separates the two band groups correctly but assumes a larger PCR product means successful cloning and picks lanes 4, 6, 7, 8 where the reference's control comparison selects 5, 9, 10; lane 3 holds the original-plasmid PCR reference position, lane 2 carries both patterns because incomplete assembly leaves old template alongside new product, and colonies can carry either plasmid — the visual grouping is right while the experimental identities are reversed, which would purify the wrong plasmid downstream); discounting contradicting observations (vacuoles "generally darker", though some appear lighter in the same image — changing focus reverses the contrast; Astra inspects several regions and even notes the contrast varies with focus, yet the final answer keeps only the darker description); unreliable counting marks (close neighbours unresolved, visible colonies missed, empty agar marked; the net result is an overcount of about 12% on that plate, and with plated volume and dilution fixed the transformation-efficiency estimate inflates by the same proportion). Boundaries: higher benchmark scores do not yet mean reliable judgments in the lab; progress could let scientists spend less time troubleshooting and more time deciding which experiments to pursue; tracking these capabilities also serves understanding of biosafety and biosecurity implications.