The revolutionary Nature paper that wasn't

A CRISPR study claimed to upend the established science on genetic mutations. Did it?

Yeast cells under a scanning electron microscope. Not olives; not grapes either

Imagine a tourist comes up to you in the streets of your hometown and says:

Your weather is so very dull!

You probably wouldn’t be too bothered. Indeed, this is a common occurence in Scotland, where I’m from. What if they said the following?

Your weather is so very drab!

That’s okay too - all I’ve done is replace one of the words with a synonym, and it overall has the exact same effect. No big deal. But what about this one?

Your mother is so very dull!

Ah. I’ve changed one word to something that very much isn’t a synonym, and you suddenly have justification to be offended - and maybe even give the tourist a Will Smith-style smack in the mouth.

This is my—okay, rather tenuous—analogy to a particular kind of mutation in the genetic code. A new paper in Nature (the best scientific journal in the world, ladies and gentlemen!) has just shaken up some of the basic knowledge we had about those mutations. Or has it?

Filthy degenerate

We’ve known for a long time that the majority of genetic mutations—changes in the genetic code from one generation to the next—don’t have an effect on the organism’s evolutionary fitness. This idea was first put forward in the 1960’s by the biologist Motoo Kimura, and he later codified it in his “Neutral Theory” of evolution: most mutations are neutral with respect to whether the next generation will itself survive to reproduce. That’s completely consistent with the fact that many other mutations have a negative or a positive effect - say, producing a genetic disease that reduces fitness, or contributing to feature like camouflage that helps the organism survive.

One way that mutations can be neutral is when they’re synonymous - just like the first two sentences above. The genetic code is all about producing amino acids, which go on to make up proteins, which go on to affect on how an organism is built or behaves. Stretches of three “letters” of genetic code—three nucleotides—produce a particular amino acid. And sometimes, those three-letter stretches can be different, but still ultimately produce the same amino acid.

You can see how it works on the code wheel below. Follow each three-letter code sequence from the centre outwards: if the particular three-letter sequence in your genetic code starts with a A, and it’s followed by a G, and then another G (AGG), you’ll get the amino acid arginine. If the code is AGC, you’ll instead get the amino acid serine. But you can see that there are other ways to get to the same place: both AGG and AGA will get you arginine (they’re synonymous). Similarly, both AGC and AGU will get you serine. This kind of duplication makes the genetic sequence—and I don’t mean this in a pejorative sense—a “degenerate” code.

By the way, the reason the letters are G,U,A,C rather than the more familiar G,T,A,C are because this is RNA code rather than DNA code: in RNA the T (thymine) is replaced with a U (uracil), but they’re very similar. It’s just that (messenger) RNA is what’s directing the production of proteins, so it’s what’s on these wheel diagrams.

Since one of the more common kinds of mutations just changes one single letter in the genetic code, you can see from following some of the sequences on the wheel how a lot of mutations would be synonymous.

Even the scientists who are most interested in how the non-neutral mutations are selected are on board with the neutral theory. Here’s Richard Dawkins in his book The Greatest Show on Earth:

When the neutral theory of molecular evolution was first proposed… it was controversial. Some version of it is now widely accepted and… I am going to accept it in this book. Since I have a reputation as an arch-‘adaptationist’ (allegedly obsessed with natural selection as the major or even only driving force of evolution) you can have some confidence that if even I support the neutral theory it is unlikely that many other biologists will oppose it!

Those biologists support (some version of) the neutral theory because there’s lots of evidence for it. It’s not just about synonymous mutations, but since those are our main interest here, I should say that there’s a lot of evidence that they have a broadly neutral effect. For example:

• A variety of mutations can be involved in causing autism and other developmental disorders - large-scale studies find that synonymous mutations aren’t particularly prominent among these mutations;

• Mutations that cause cancer can include synonymous mutations, but far more frequently include other types of mutation;

• Synonymous mutations show little-to-no impact on the fitness of viruses.

But even though all of the above is the conventional wisdom, biology is complicated, and there are definitely an increasing number of papers providing evidence that synonymous mutations might sometimes be under selection, and some could have a “non-negligible” impact on traits. Since this can’t be via the standard amino-acid-making mechanisms, it might be through something less well-understood, like changes in gene expression, or in protein folding, or in biases towards using one three-letter sequence compared to others (like I say, biology is complicated).

But even those who argue that synonymous mutations are more important than we’d predict from the neutral theory generally haven’t argued that they might have a strong effect, or one that’s comparable to non-synonymous mutations. Until now.

Out of control(s)

“Synonymous mutations in representative yeast genes are mostly strongly non-neutral”. That’s the very straightforward title of a June 2022 paper in Nature that caused waves in the world of genetics. It seemed to be shaking up the whole of the well-accepted neutral theory of evolution that I described above: just to be clear, it isn’t saying “well, synonymous mutations are probably mostly neutral but through some complex mechanism they might have a non-negligible effect”. It’s saying that, to repeat that title, they are mostly, strongly, non-neutral. It’s a very big claim.

The study was hailed for “challenging dogma” with its “potential[ly] game-changing” results that are a “shake-up” for the field ( “WOW”, “Whaaat?”). It got its own accompanying “News & Views” article in Nature, which noted that:

Revising our expectations about synonymous mutations should expand our view of the genetic underpinnings of human health.

Wow indeed. And not only that, but the study—by researchers at University of Michigan—could have a big impact on how we think about natural history:

…the number of synonymous differences between species is used as a “molecular clock” to estimate how long ago speciation events took place - but these estimates would be inaccurate if selection actually constrains a lot of synonymous variation, effectively slowing the clock.

All this from a study of yeast! We had better work out what they did. First, the specific species used was standard brewer’s yeast (scientific name Saccharomyces cerevisiae). Using CRISPR-Cas9 gene editing, they created a whole host of new types of yeast, with sets of nucleotides deleted and then replaced with almost all the synonymous and nonsynonymous mutations that were possible in that stretch of their genome.

Then, they let them loose in a test tube, and compared the fitness of the mutant yeasts—that is, their survival over time—to the original, “wild-type” strain. That is, the wild-type was the control yeast (bear this in mind because it becomes important later).

After two or three days, they checked to see the relative proportions of mutant vs. wild-type yeast that were left. You’d expect that the nonsynonymous mutations, where the amino acid is actually changing, to have a much worse effect on the yeast’s fitness. But surprisingly, this was only the case for a few of the mutations. For the vast majority, synonymous and nonsynonymous mutations were comparably bad (resulting in something like a 1.5% reduction in survival on average compared to the wild-type). You can also see how similar the numbers are if you compare how many mutations were positive or negative:

Among synonymous mutations, 75.9% are significantly deleterious, whereas 1.3% are significantly beneficial. The corresponding values are 75.8% and 1.6% for nonsynonymous mutations.

Okay, so this is big news: you can see how this upends all the established science about mostly-neutral synonymous mutations that I discussed above, and how it goes way beyond the previous indications that maybe, sometimes, synonymous mutations have an effect.

But. The problem with using CRISPR is that it can have so-called “off-target” effects: mutations in other genes that you didn’t intend to produce. If any of these inadvertent mutations changed the survival rate of the yeast, the Nature authors might’ve confused these effects for effects of the synonymous mutations. This can happen when deleting the old nucleotides and inserting the new ones. The authors didn’t check for this by doing a full genome sequence of the new yeasts they’d created, so we can’t know how many off-target effects were present. As critics put it in a letter, currently posted as a preprint on bioRxiv:

…each genetically engineered deletion strain represents a unique genetic background potentially carrying unascertained mutations that may affect strain fitness.

Ouch. So the Nature paper might’ve varied more than just the one thing the authors wanted to study - if so, this would be a classic experimental blunder.

And it’s worse: there are good reasons to think the “wild-type” control was inappropriate. For instance, it might have been a particularly fast-growing yeast, since it had been selected by scientists as one that grows reliably quickly in their experiments. This would make it an even worse comparator for the mutant strains.

A better way to do this would be to create a different control along with each mutant in the exact same process, so that any “off-target” effects happened to both, making them more comparable. Comparing every mutant to a single control just isn’t enough.

Put simply, the critics are arguing that the Nature paper didn’t have a proper control group, rendering its claims invalid. This would be the same problem you see in, say, clinical “trials” that have no control group, or psychology intervention studies where the control group ends up doing something entirely different from the treatment group, producing lots of differences and leaving you with no idea whether it was the intervention or something else that had an effect. It’s harder to spot in this complex Nature CRISPR paper - but it’s the very same issue: the authors didn’t properly ask “compared to what?”.

I should also note that there are some things in the study that don’t sit right, given the authors’ interpretation. For instance, if you look at one of the figures in the paper (Figure 2b, which I probably shouldn’t reproduce here due to copyright), you can see there is, as the authors say:

a higher density of nonsynonymous mutants in the fitness range 0.91–0.97 but a higher density of synonymous mutants in the range 0.97–0.99.

That means the nonsynonymous variants are producing more negative effects on average (giving lower fitness values) - which is closer to what you’d expect given all the previous evidence. But this more subtle distinction is obscured if you just split mutations into percentages of “beneficial” versus “deleterious” effects, as the authors did in the statement I quoted above.

If the critical letter is accepted as a published commentary in Nature (I assume that it’s been submitted as such), the original authors will get a chance to respond. It’s hard to predict what they’ll say: maybe they’ll downplay the importance of off-target effects in an organism like yeast that has a very small genome (the yeast genome is about 12 megabases, compared to, say, the human one, which is about 3 gigabases). I don’t really see this debate being resolved until someone does a much more conservative, much more tightly-controlled, experiment. It’s hard to see why that kind of experiment wasn’t done in the first place.

It’s only Nature-al

The (very thorough) critics went even further. Remember the claim in the Nature News & Views article, that this new research might affect how we understand human disease? The critics did a whole analysis of that, too, using the massive UK Biobank sample. In a second critical preprint, they showed that synonymous mutations are far less likely to be linked to a huge range of human traits and diseases (nearly 5,000 of them) than mutations that actually alter the genetic sequence.

This isn’t what you’d predict if synonymous mutations—by whatever poorly-understood mechanism—are having just as big an effect on diseases as non-synonymous ones. And it feeds in to a more general argument made by the critics: how plausible are these findings, really? It would be totally wrong to say that because findings go against a lot of previous research this makes them incorrect - think of all the scientific progress we’d have missed if we believed that! But, as the critics put it:

…when results appear to contradict decades of evidence from multiple fields, the default assumption should be that they arise from a hidden artifact or artifacts, and every effort should be made to rule these out before taking the results at face value.

Well, quite: extraordinary claims require extraordinary evidence, and all that - and the setup of this experiment wasn’t anywhere near extraordinary enough. Perhaps the very enormity of the claim being made here—everything we thought we knew about synonymous mutations is wrong!—helped the paper get published somewhere as big as Nature. Perhaps it also blindsided the reviewers enough that they didn’t ask the kinds of questions asked in the critical letters.

In the world of scientific journals, Nature is synonymous with flashy, breakthrough research. But this story is yet another reminder that when a piece of research looks super-exciting, you should usually be more sceptical, not less.

Acknowledgements: Thanks very much to Daniel McArthur (a co-author of one of the two critical letters) for pointing me towards this story, and to Sebastian Worms and Adam Hunt for comments on a draft.

Image credits: Microscope image: Getty. Genetic code wheel: Wikimedia Commons (public domain).

Update 22 August 2022: Changed “bytes” to bases (d’oh!). Changed “Petri dish” to “test tube” (the CRISPR part was in a Petri dish but the actual competition between strains was in a test tube).