Science writing update, August 2023 edition

Why I was right about the LK-99 "room-temperature superconductor" - plus all the usual updates and collected links

Hello everyone - it’s time for your monthly update on what I’ve been writing about science! In this one, I’ve gone into a bit of detail on why I was so sceptical about LK-99, the magical “room-temperature superconductor”, plus some more writing on Alzheimer’s disease, the scientific publication system, and more.

And as always, I’ve added a collection of interesting science links at the end: you can click here to skip right to that.

Also, I’ll quickly give another mention to The Studies Show - it’s my new science podcast with fellow science writer Tom Chivers, where we take a controversial scientific topic and talk about it for an hour each week. It’s already critically-acclaimed!

So far you can hear episodes on Ozempic, breastfeeding, aspartame, psychedelics, and vaping, and there’s much more to come. We’d love it if you subscribed on The Studies Show’s own Substack - or on whichever app you normally use to listen to podcasts.

Now: on with the science writing.

LK-99: My part in its downfall

Discovering a material that’s a superconductor at room temperature would be a major technological advance: if we could conduct electricity with zero resistance and without having to cool the conducting material to near absolute zero, we’d get advances ranging from a vastly more efficient energy grid to viable quantum computers.

The other week a group of Korean scientists posted two preprints to arXiv claiming they’d invented just such a material, called LK-99. Twitter went mental.

I mean, really mental: threads with thousands and thousands of retweets and millions of views, with endless replies from people positively rapturous about this incredible discovery.

I didn’t believe a word of it. The same day that the hype began to build on social media, I had an article out calling it “probably nonsense”. I didn’t have any particular special insight, and of course I’m not a physicist. But this was the very basic stuff I was thinking:

• The main preprint was very badly presented. A lot of people became enraged when I posted a tweet saying the study was “not serious” on account of its presentation. How dare I criticise scientists whose first language isn’t English for not writing perfect English prose?! Well, first off, that’s the soft bigotry of low expectations: thousands of scientists with first languages other than English write papers with perfect English grammar every day. And second, the grammar was just a small part of it. The scrappy-looking figures and overblown pronouncements did not look like the work of sober, well-organised scientists but instead of people who were rushed, naive, or maybe even a bit delusional: not things you’d associate with careful, meticulous, world-changing work.

  • In their angry tweets, a lot of respondents seemed almost to be contorting themselves into the position that more poorly-presented papers are actually more likely to be true! Come on, man. Bad presentation might not be strongly predictive of bad research—it’s just a red flag—but in my experience reading and reviewing hundreds of papers, it’s certainly a correlation.

• Actual experts were highly sceptical. For my article I contacted some top UK materials physicists. They thought the preprint was missing crucial details. Others were quoted elsewhere saying something similar. If you’d really made a world-changing discovery, I feel like you’d be clued-in enough to include those details, since you’d know your paper would be subject to intense scrutiny. These authors did not seem particularly clued-in.

• Your priors should be very low. There are several holy grails in physics: cold fusion is maybe the most famous, but room-temperature superconductivity is another. Claims of its discovery are made every couple of years, and none of them have panned out. LK-99 just happens to have gotten astronomically more attention due to all the social-media hyping. So it’s not just that you should’ve been sceptical because rare things happen rarely - it’s because this specific thing has been claimed over and over again in the past, to no avail.

• This field has had prominent cases of bad research. Literally the day before the LK-99 news started spreading, a now-infamous researcher who’d claimed to have discovered room-temperature superconductors had had a(nother) paper retracted for scientific misconduct.

None of these things, by themselves or even together, mean that LK-99 isn’t a room-temperature superconductor. But they should’ve made you incredibly sceptical, and should’ve absolutely ruled out any excitement until we had solid replication data.

Regardless, I saw otherwise-sensible people saying that they were 99.9% sure LK-99 was a room-temperature superconductor, offering 10:1 odds that it would successfully replicate, or that it was a discovery on a par with the Green Revolution that saved more than a billion lives.

It really was a wild few days, showcasing extraordinary popular delusions and the madness of crowds. At the same time, amateur physics tweeters racked up the likes and retweets by dramatically hyping up every new development in posts full of false information (“insanely bullish for humanity”!). People created and shared likely-hoax videos of pieces of something floating above magnets, as superconductors sometimes can. Most bizarrely, an anonymous account with a cartoon avatar posted long, fictionalised stories of the discovery of LK-99 which were seen by millions but only sometimes had the word “fiction” written at the top.

Anyway, now we have some replication data. Several studies have come back finding that LK-99 is not, in fact, a room-temperature superconductor. Authoritative sources are saying it’s time to give up. Even the daft tweeters who misled millions with their OTT initial reports are slinking away. I do hope we hear much less from them in future.

Sure, maybe tomorrow we’ll see something that brings the story roaring back. But I rather doubt it. Hate to say I told you so.

Burying the dead - or the scientific publication system?

The LK-99 articles were preprints, and of course many people are concerned that a lack of peer-review allows false claims to circulate.

The journal eLife now has a pretty unique publication process where it publishes as preprints all the articles that it decides to send out for peer-review - and then publishes the reviews, too. A couple of weeks ago they published multiple papers about Homo naledi, the ancient hominins who, apparently, buried their dead and made deliberate cave etchings. But the claims went way beyond the data. Was it bad that eLife published these papers? Or good that the process was so transparent?

I wrote an article (£) that attempted to weave together the H. naledi tale with the story of eLife’s attempts to reform the scientific publication system.

Alzhypemer’s

There are now three drugs approved for patients with Alzheimers (aducanumab, lecanemab, and donanemab. Try saying those with a mouthful of boiled sweets. Actually, if you just say them normally, you sound like you have a mouthful of boiled sweets).

None of them have particularly stunning results. But every time a new study comes out, we’re told—by the media and scientists themselves—that it’s an enormous breakthrough, the beginning of the end for Alzheimer’s… and so on.

I tried to puncture this bubble with an article that pointed to some serious confusion about how to measure effect sizes in these kinds of trials, as well as references in the trial writeup that don’t support the authors’ case (I went into more detail on that here, and Jesse Singal wrote a much longer article on it here).

You don’t need a weatherman

Remember back in the day, when social psychology research was full of very silly studies about how certain unconscious influences (“primes”, like words, phrases, or images we happen to hear or see) had a massive influence on our behaviour?

A remnant of this kind of thinking still exists, and it’s “the wind speed changes your vote”! An article recently appeared claiming that the Brexit and Scottish Independence referendums, among other votes, were influenced by how windy the polling day was (the idea is that not that it changes turnout, but that it influences how “safe” people feel, and thus whether they vote for the “safe option”). You might be thinking “hmm, that doesn't sound very plausible” - and I’d agree with you. I critiqued the study here (£).

Things I didn’t write but that you might like anyway

Here are some interesting science links from the last month or so that you might’ve missed:

• There was a collective sharp intake of breath in cancer biology research this month: a devastating critique appeared pointing to some fatal flaws in a Nature study that claimed to be able to predict cancer types from the microbial DNA in tumours. It’s really brutal stuff.

• 176 new attempts to replicate various psychology studies. Only 49% were successful. And to think some people don’t believe there’s a replication crisis!

  • And ecology continues to be the field that’s maybe second only to psychology in facing up to its own big problems.

• According to a new analysis, scientific papers are getting less hedge-y over time: they contain fewer words like “might” and “probably” that signal uncertainty. It’s consistent with the idea that pressure to publish nudges scientists towards more overblown claims. Or, maybe it is. Probably.

• Here’s an astonishing thread about growth mindset, and a seemingly deeply inept attempt by pro-growth mindset researchers to defend themselves against criticism. Reading the list of errors in their rebuttal had me cringing half to death - the rebuttal should probably be corrected or retracted.

• How many clinical trials are either “problematic” or entirely fake? The answer is higher than you might think, and this article interviews some of the sleuths who are spotting the bad ones.

• A useful summary of where errors creep in at all points of the scientific research and publication process.

• The latest update in the Francesca Gino case: she’s now suing not just Harvard, but also the academics who wrote the (extremely careful, well-evidenced) Data Colada blog posts alleging that there was fraud in her research. This is really shameful, cynical stuff, and it’ll contribute to a chilling effect where people are even more afraid to point out anomalous-looking research by powerful, well-known, big-deal scientific figures.

  • Oh, and this tweet about the case from Keith Humphreys contains a pun so painful that it should be considered illegal under the Malicious Communications Act.

Image credit: Getty