Nature Biotechnology analysis finds deep learning perturbation models can beat baselines

A paper published in Nature Biotechnology on 1 October 2026 reports that deep-learning-based genetic perturbation models can outperform uninformative baselines when they are judged with well-calibrated metrics ¹. The finding runs against recent benchmarks, which the paper says reported that such models fail to beat those baselines ¹.

What the analysis found

The authors introduce two tools: a positive control baseline and a framework for calibrating metrics ¹. They apply them across 14 datasets and 18 metrics ¹.

The central result concerns the measuring sticks rather than the models. The authors report that common benchmarking metrics, including mean squared error and Pearson Δ, are frequently miscalibrated, with reduced sensitivity to measure positive model performance ¹. On the paper's account, a metric of this kind is less able to register that a model is doing better than a baseline ¹.

Once the metrics are calibrated, the authors find that the deep learning models can outperform uninformative baselines ¹. The abstract does not name the models tested or the datasets used. It does not give the margin by which the models beat the baselines ¹.

What it means for the market

The paper matters most to R&D leaders who build, buy or fund genetic perturbation models for target discovery. Recent benchmark results had suggested that these models did no better than uninformative baselines ¹. This analysis argues that the choice of metric shaped that conclusion. It singles out mean squared error and Pearson Δ as common metrics that are frequently miscalibrated ¹.

The immediate change is to how model performance should be assessed. On the paper's account, a model judged only on metrics with reduced sensitivity to positive performance may look no better than a baseline ¹. Pharma companies weighing supplier claims, and internal teams defending their own models, now have a published basis for asking which metrics underpin a benchmark and whether those metrics are calibrated ¹.

Much of what buyers and investors would want to know is not in the source. The abstract gives no figures on the size of the performance gain and no comparison between named models ¹. It says nothing about cost, procurement, funding or regulation. It refers to genetic perturbation models only, so it does not establish anything about models that predict responses to chemical perturbations ¹. From this source alone, it is not possible to judge whether the gains are large enough to change a programme decision or a purchase.

The work is a benchmarking and metrics study. Its evidence comes from evaluating models across 14 datasets with the authors' own positive control baseline and calibration framework ¹. The abstract reports no evidence that model predictions led to validated targets or to experimental results in the laboratory ¹.

The claim is also bounded by its own wording. Under well-calibrated metrics, the authors find only that the models can outperform uninformative baselines, which leaves open how often and by how much ¹.