Perspective

2026-04-21

What Are Foundation Models, and Why Should Biologists Care?

I wrote this as I explor the rapidly evolving landscape of foundation models in computational biology. The field is moving fast. What is state-of-the-art today may be a baseline tomorrow.

The Core Idea

Foundation models are machine learning models trained on large, diverse datasets using self-supervised objectives.

The goal is not to solve a specific task, but to learn a representation of the data.

In self-supervised learning, the model is trained to predict parts of the data from other parts. In doing so, it captures the statistical structure present in the measurements. What varies together, what is context-dependent, and what remains stable across samples.

If the training data is sufficiently broad, this representation becomes reusable. The same model can then be adapted to different downstream tasks with relatively little additional data.

This is the shift: from building task-specific models to learning something that can be reused across tasks.

Why “Foundation”?

The term is not just about scale. It is about role.

A foundation model is trained once, and then serves as a base layer for multiple downstream applications. Instead of fitting a new model for each task, you start from a shared representation and adapt it.

What distinguishes it from earlier self-supervised models is not just that it learns from unlabeled data, but that what it learns is general enough to transfer.

That generality, however, is bounded.

The model does not learn biology in a mechanistic sense. It learns the structure of the data generated by biological systems under specific measurement processes. Its usefulness depends on how well that structure aligns with the question being asked.

The name also carries a warning. Because downstream applications build on the same representation, any bias or blind spot in the pretraining data propagates. The foundation matters.

Why this matters for biological data

Biological data sits in an unusual regime.

It is high-dimensional, sparse, and context-dependent. The same gene can behave differently across cell types, tissues, or disease states. Measurements are shaped as much by the technology as by the underlying system. What we observe is not biology directly, but biology filtered through an assay.

This makes purely supervised approaches brittle. Labels are often limited or inconsistent, and models trained in one dataset do not reliably transfer to another.

This is where foundation models fit naturally.

By training across large and diverse datasets without relying on labels, they can learn patterns that persist across contexts-co-expression structure, spatial organization, and conditional dependencies. In principle, this allows them to separate the stable signal from dataset-specific variation.

But the same properties that make biology a good fit also make it difficult.

Heterogeneity is not just noise to be averaged out. Some of it is the signal of interest. A model that aggressively compresses variation may obscure meaningful differences, between cell states, disease subtypes, or environmental conditions.

More importantly, these models learn from measurements, not from the underlying system. If certain populations, conditions, or modalities are underrepresented, the learned representation will reflect those gaps. Generality is limited by what has been observed.

So the advantage is conditional.

Foundation models can be useful in biology because the data is large, unlabeled, and heterogeneous. But their behavior is tightly coupled to how that data was generated and what it contains.

When do you actually need a foundation model?

The decision comes down to scope.

If the goal is to solve a specific task on a specific dataset, a foundation model is often unnecessary. A well-specified supervised model will usually suffice.

Foundation models become useful when the problem extends beyond a single dataset.

This happens when the data is heterogeneous, when labels are limited, or when you expect the same representation to support multiple downstream tasks. In these settings, learning a shared representation is more valuable than optimizing for one task at a time.

The limitations follow from the same logic.

If the data is small or homogeneous, there is little to generalize over. If the task is narrow, the added flexibility is not used. And if the core problem is integration, dataset alignment, or correction of measurement differences, a foundation model is an indirect solution.

Foundation models are not a default upgrade. They are useful when the question is broader than the dataset and unnecessary when it is not.

What this means in practice

The practical shift is subtle but important.

Instead of building a new model for each task, you can start from a representation learned on broader data and adapt it to your specific question.

This reduces the dependence on large labeled datasets, but does not eliminate it. The quality of the outcome still depends on how well the pretrained representation aligns with the problem.

In that sense, the bottleneck shifts.

Less effort goes into building models from scratch, and more into formulating the right question and understanding the limits of the data.

2026-04-07

Heterogeneity Was Always There. Now It’s Unavoidable.

Henrik Zetterberg recently described heterogeneity as the emerging “mantra” of neurodegenerative disease research [Link to AlzForum article].

The question is not whether this is true.
The question is: why now?

One reason is straightforward: failure.

Despite decades of work, nearly all Alzheimer’s clinical trials around 98% have failed [Kim et al.,(2022), J. Alz. Dis., Cummings et al. (2014) Alz. Res. & Therapy, Andersen et al. (2017) Lancet]. These failures are not random. They reflect a mismatch between how we have modeled the disease and how the disease behaves.

Most therapeutic strategies have been built on a simplifying assumption: that Alzheimer’s disease can be understood as a relatively uniform process, often organized around a dominant pathway. But patient responses have consistently resisted that assumption. Individuals with similar clinical or pathological profiles diverge in progression, treatment response, and outcomes.

A homogeneous model of disease has produced heterogeneous results.

At the same time, the empirical picture has been fragmenting.

At the level of mechanism, there is no longer a single dominant explanation. Amyloid, tau, immune processes, metabolism, mitochondrial dysfunction, vascular changes. Each has evidence behind it. Rather than converging on one pathway, the field has expanded into multiple plausible ones.

This fragmentation is also visible in therapeutic development. The current drug pipeline targets a wide range of biological systems, extending well beyond amyloid. In practice, the field has already moved toward a multifactorial view of disease. Even if that shift has not always been explicitly articulated.

What has changed more recently is not just the presence of variability, but our ability to observe it systematically.

Large longitudinal cohorts, multimodal data (imaging, fluid biomarkers, omics), and cross-cohort analyses have made it possible to track disease trajectories with greater resolution. What emerges from these datasets is not random variation, but recurring patterns: different rates of progression, distinct combinations of pathologies, and varying responses to intervention.

As Zetterberg notes, combining biomarkers with careful phenotyping reveals that individuals with similar levels of pathology can follow markedly different trajectories.

The idea that Alzheimer’s disease is multifactorial is not new.
What is new is that it has become unavoidable.

Heterogeneity is no longer an anomaly to explain away. It is a pattern that persists: across cohorts, across modalities, and across levels of analysis.

This is not just a problem for Alzheimer’s disease. It reflects a broader challenge in how we model complex biological systems.

The question now is not whether heterogeneity exists. But how to represent it.

In my own work, I have repeatedly encountered this from a different direction: variation that does not behave like noise, but reflects underlying biological structure.

Understanding these structured perturbations, across molecular, cellular, and system levels, is the focus of my research.

More on that soon.

2026-03-31

Interviewing With an AI And Reviewing Applications

Recently, I interviewed with an AI screening system.

It was, frankly, odd.

There were moments where I paused to think and the system interpreted that pause as the end of my answer. I couldn’t clarify ambiguities in the questions. One of the prompts felt strangely phrased, almost as if the system had partially hallucinated it. There was no conversational repair. No recalibration. Just progression.

It felt procedural.

And yet.

I have also been on the other side of the table.

I have sat with colleagues sifting through CVs and cover letters. We read carefully. We give feedback even to poorly aligned applications. We try to be fair. And after the nth application, cognitive fatigue sets in. Attention narrows. Consistency drifts.

Scale changes everything.

The Candidate Experience: Where It Breaks

From the applicant’s perspective, AI screening exposes clear limitations:

Fragile turn-taking: Reflective pauses get treated as completed answers.

No clarification loop: Ambiguity cannot be resolved in real time.

Variable question quality: Dynamically generated prompts can be inconsistent or poorly grounded.

The interaction is linear rather than collaborative. You respond within a predefined frame, rather than co-constructing meaning.

And Yet, What It Changes

Despite that, AI screening alters parts of the process in ways that are difficult to ignore.

It shifts evaluation beyond the CV: Many candidates are filtered out before any interaction, often due to formatting, missing keywords, or differences in convention. An AI interview, at least in principle, evaluates how you articulate your skills, not just how you encode them in a document.

It redistributes early-stage bias: Human screeners bring preferences for communication style, familiarity, and immediacy of rapport. A structured AI interaction removes some of that variability. It does not remove bias, but it changes where it enters.

It enforces consistency in a way humans often cannot: Standardized questions have long been an ideal. In practice, human interviews drift in phrasing, tone, emphasis, and attention. AI systems do not drift.

Paradoxically, interviews that aim to be standardized at scale can feel more mechanical when conducted by humans than by machines.

The Other Side of the Table

From the committee perspective:

• Hundreds of applications

• Limited time

• Repeated evaluation tasks

Manual review is not just effortful, it is uneven.

AI systems compress, structure, and standardize.
They do not decide but they shape what is seen, and how.

The Tension

Something real is gained.

Candidates can move past the constraints of document filtering.
Evaluation becomes more consistent at scale.
Some forms of interpersonal bias recede.

And something else is lost.

The ability to pause and think without penalty.
The ability to clarify ambiguity.
The ability to shape the interaction as it unfolds.

Efficiency accumulates on one side.
Constraint accumulates on the other.

Holding Both

Interviewing with AI felt constrained in ways that were difficult to ignore.

Reviewing hundreds of applications without it is equally difficult to ignore.

Both are true.

And neither resolves the other.