AI Engineer World's Fair 2026
From Tokens to Cells: Foundation Models for Single-Cell Biology - Akram Baharlouei, Altos Labs
Loading slides…
Read the talk
From Tokens to Cells: Foundation Models for Single-Cell Biology
Reprogramming a cell makes its state a modeling problem: what can we measure, what does sequencing miss, and can foundation models predict the full range of cellular responses?
From a talk by Akram Baharlouei
Before you start: Basic familiarity with machine-learning training and vector representations is helpful; no biology background is required.
What would it take to restore a cell’s youthful function?
What would it take to restore a cell’s youthful function without changing what kind of cell it is? That question motivates cellular rejuvenation and the mission Akram Baharlouei describes at Altos Labs: restoring cell health and resilience to address disease and disability. His perspective is that of a machine-learning engineer entering biology, with an emphasis on the engineering challenges between measuring cells and training useful foundation models.
The starting example is reprogramming. In 2006, Shinya Yamanaka discovered that overexpressing four transcription factors could push a differentiated cell toward an embryonic stem-cell-like state. In the talk’s skin-cell example, the intervention changes cell identity: a specialized cell becomes an induced pluripotent stem cell, or iPSC. The discovery, recognized with a Nobel Prize in 2012, opened possibilities for regenerative medicine, including producing cells needed to repair tissues or organs.
Partial reprogramming pursues a different endpoint: restoring youthful function while retaining cell identity. A rejuvenated skin cell should still be a skin cell. Baharlouei presents this as a therapeutic ambition, including the possibility of an mRNA medicine, rather than an established treatment. He tentatively points to an OSK-based human-testing milestone in 2026, roughly two decades after the original discovery; entering human testing is distinct from demonstrating rejuvenation in patients.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From cell models to the drug-development pipeline
Reprogramming is one reason to understand cellular state. A broader ambition is to build a unified model of a cell, then use it as a foundation for models of tissues, organs, and eventually humans. The Human Cell Atlas is an ongoing effort to map human cell types, not a completed census of every human cell. Virtual cells, virtual tissues, virtual humans, and digital twins name related ambitions at different biological scales: models that could help explain how organisms work and support treatment development.
The practical pressure comes from drug development. Baharlouei contrasts the rapid improvement associated with Moore’s Law with what he describes as declining drug-development productivity. Failure is routine: he offers a rough success estimate of five percent or less, without specifying a denominator or development stage. He describes a research-to-clinic pipeline that can take ten years and cost billions of dollars. These are broad estimates motivating the problem, not measurements of a defined cohort.
Faster discovery does not automatically make the whole pipeline fast. Research, preclinical work, and clinical trials impose successive constraints. Better protein design might save time in early research while leaving later stages largely unchanged. The attraction of virtual biological models is therefore wider than generating promising molecules: the goal is to improve decisions across the development pipeline, where downstream bottlenecks can otherwise absorb gains made upstream.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What can we measure inside a cell?
A cell contains much more information than any single measurement captures. Its genome supplies DNA sequence, which is broadly shared across cells; its transcriptome describes which genes are expressed and in what quantities at a particular time. For machine learning, single-cell RNA sequencing produces a familiar structure: a cell-by-gene matrix, with cells as rows, genes as columns, and measured expression quantities as entries. The talk places the gene dimension at roughly 20,000.
| Measurement | What it describes | Main distinction |
|---|---|---|
| Genome | DNA sequence | Broadly shared across cells |
| Transcriptome | Gene-expression profile | Varies with cellular state |
| Proteome | Proteins present in a cell | Functional machinery; difficult to measure at scale |
| Morphology | Cell shape and structure | Accessible through microscopy |
| Spatial information | Location within tissue | Preserves the cell’s surroundings |
Proteins carry out many cellular functions, making proteomics especially valuable, but Baharlouei describes its measurement as difficult and comparatively low throughput. Microscopy adds shape and structure, while spatial measurements help locate an individual cell within tissue. These modalities describe different aspects of the same biological system.
RNA-seq dominates much single-cell foundation-model training because it is comparatively easy to measure and scale, helped by PCR-based amplification. Baharlouei describes collections containing tens of millions of cells, says he has heard of a 500-million-cell collection, and mentions a billion-cell project. Those larger examples are unnamed; their scale should not be read as evidence that the samples are all independent or experimentally comparable. Even an enormous RNA-seq corpus remains a measurement of gene expression, so progress in other modalities is still necessary for a fuller model of cell biology.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A billion snapshots are still snapshots
Suppose a training collection did contain a billion RNA-seq samples. Would that be sufficient? The difficulty is that nominally identical cells need not produce identical measurements. Cells grow, change their molecular activity, and divide. Some activity happens in bursts. Sequencing captures one state during that process, not the trajectory that produced it: a snapshot from a biological movie. Increasing the number of snapshots does not by itself reveal the temporal history of each cell.
Biological variation is only part of the problem. Different laboratories and machines can produce different measurements, introducing technical variation alongside real differences in cellular state. A useful model must learn from both without simply treating every difference as biology. The data’s high dimensionality and many possible states prompt a brief quantum-computing aside, but Baharlouei presents that as speculation, with no demonstrated advantage. The immediate question is what current AI methods can extract from the measurements already available.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Cells as sentences, genes as tokens
scGPT and Geneformer bring transformer-based representation learning to single-cell data. The analogy is intuitive: treat a cell like a sentence and genes like tokens. Baharlouei explains the family through BERT-style masked prediction: hide part of a cell’s gene information and train a model to recover it from the remaining context. This is a shared conceptual introduction, not a claim that scGPT and Geneformer use identical encodings or training objectives.
For expression-value reconstruction, the cell-by-gene matrix makes the learning problem concrete. The following Python example hides two entries in a small expression vector and keeps their original values as targets. The mask is separate from the filled input, so a hidden measurement is distinguishable from an observed zero.
python
import numpy as np
genes = np.array(["GENE_A", "GENE_B", "GENE_C", "GENE_D"])
expression = np.array([12.0, 0.0, 7.0, 3.0])
masked = np.array([False, True, True, False])
model_input = expression.copy()
model_input[masked] = 0.0
target_genes = genes[masked]
target_values = expression[masked]
print(model_input) # [12. 0. 0. 3.]
print(target_genes) # ['GENE_B' 'GENE_C']
print(target_values) # [0. 7.]
The training signal asks the model to recover hidden information using relationships among genes. Bidirectional attention provides a way to use the surrounding gene context rather than predicting each entry independently.
The resulting cell representation can support cell-type prediction or perturbation-response modeling: predicting how a cell responds to an intervention. In the architecture pattern described here, the model compresses the input into a latent vector. A classification head uses that representation to assign a label; a decoder uses it to generate a predicted output. The important question is whether the representation retains the information needed by the downstream task.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Training expense does not guarantee useful representations
Compression can discard information that matters for prediction. Baharlouei offers this as an explanation for disappointing downstream results, rather than establishing it as the universal cause. He reports that simple linear models sometimes match or outperform expensive single-cell foundation models. That comparison makes the baseline essential: a complex representation is valuable only if it improves the task being evaluated.
The benchmarking work discussed next covers two settings: imaging together with RNA-seq, and perturbation-response modeling. Baharlouei describes comprehensive evaluations whose results echo the broader concern that costly training has not produced gains comparable to those seen in language and imaging. The claim is qualitative here; it does not establish a single ranking across every model, dataset, and biological task.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Predicting a distribution rather than its mean
Flow matching offers another way to formulate the prediction problem. Begin with samples of Gaussian noise and learn a transformation toward the distribution of cellular measurements. The goal is to reproduce a population’s range of responses, rather than reducing that population to a single representative prediction. Baharlouei introduces PRiMeFlow, available as an arXiv preprint, as a promising example of this approach.
The distinction matters when cells respond heterogeneously. A prediction near the average can fail to represent the different states actually present in a population. Baharlouei contrasts distribution matching with approaches organized around a compressed latent representation and reports favorable results for flow matching. Those results should be understood in the perturbation-response setting discussed here, not as universal superiority over transformer, autoregressive, or autoencoder architectures.
In the displayed comparison, Baharlouei identifies the green points as ground truth. He describes PRiMeFlow as following their distribution, while the autoencoder comparator concentrates predictions near the mean. The accompanying maximum mean discrepancy, or MMD, score addresses distributional agreement. The useful distinction is between reproducing an average response and reproducing the spread of observed responses; the talk supplies no numerical values here to quantify the difference.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Better measurements must accompany scale
The ambitions behind single-cell modeling—cellular rejuvenation, virtual biological systems, and a shorter drug-development cycle—depend on measurements that capture the relevant biology. RNA-seq is both useful and comparatively abundant, but more data and better measurements are separate requirements. Scaling the existing collection process alone does not ensure that the resulting corpus represents a living organism more faithfully.
Flow matching is the promising direction Baharlouei leaves open, especially for representing heterogeneous perturbation responses. The harder goal is to scale models that work on data they have not seen during training. That requires larger collections together with improved quality and measurement methods. He closes by pointing to scGeneScope and PerturBench, which he identifies as prior-year NeurIPS work, alongside PRiMeFlow: a progression from testing what current representations preserve to modeling the cellular variation those representations need to capture.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
A flow-matching model operating directly in gene-expression space, with perturbation-response benchmarks and distributional evaluation.
Code, data-access instructions and training workflows for a perturbationally paired imaging and transcriptomics benchmark.
Tools for evaluating cellular perturbation predictions with multiple aggregation methods and metrics.
The consortium's overview of its ongoing effort to map human cell types across tissues and the lifespan.
Further reading
A sponsor announcement describing an OSK-based investigational therapy and its planned Phase 1 evaluation for optic neuropathies.
Full transcript
Loading transcript…