A recent meta-analysis looked like good news for the effectiveness of "nudge" theory. Does a new set of rebuttal letters throw the whole idea into doubt?
I'm sorry to bother you with this as I am sure you must be thinking about it, but what do you think of the recent review which claimed that serotonin has no relationship to depression? Would love to read your take on it.
V kind! Recommended to take a look at the Christopher Snowdon article I linked to near the start - he has the interesting position of being a hardcore libertarian but also being relatively sympathetic to nudges, or at least some of them.
One major assumption here that is fundamentally wrong in the analysis of publication bias, is the requirement that all the reported effect "clouds" must be symmetrical even *throughout the zero effect*. Let me demonstrate with an example. You design a study to see whether say.. injecting bleach can cure Covid-19. You gather 100 people, inject 50 with bleach, 50 with saline. Hmm, weird, you replicate the study. Ten times, hundred times. You can plot funnels, clouds, but even if by some miracle you see a dot on the right, expecting any kind of symmetry is detached from reality.
So, yes, the variability in experimental design plays a role, but that's not all. You need to account for the fact that these experiments have been designed with a specific goal to achieve positive effects. And that may be a faulty mindset in science generally, but it is the fundamental nature of what nudges are about! In some cases, observing a negative effect size is actually absurd or even impossible. If you observe how many people sign up for organ donation form that's innocuously bundled to a driver's license application - would you expect that someone flips the table, storms off, and writes a last will saying: "When I die, dissolve my body in a vat of acid." ?
And even though thorough statistical interpretations are very much needed, sometimes the world is too complex for us to reduce into numbers.
Yes, this is fair! When I talk about publication bias tests I usually do add a disclaimer like this (see my breastfeeding post for example). But for some reason I didn't think to do so this time. Basically it's just circumstantial evidence - and circumstantial evidence can be very wrong!
i think a better term than "nudge" would be lying, manipulation, coercion, control etc. "nudge" is a propaganda term used to minimize the reality of how evil and sociopathic "nudging" (manipulating) people really is.
This is such a strange attitude to me. Every time a choice is placed in front of you, whether by a company or an individual, they have to decide how to present it to you. "Nudging" is just about choosing some presentations ("choice architectures") which encourage preferred outcomes. Because there is no default option for the presentation of choices, they have to make some such decision, so the real choice you make is between being nudged or making no choices at all.
Can this be taken too far? Yes. But there is no clear line between benign favorable presentation of preferred choices and acute manipulation.
Anticipated response, let me know if it's a straw man: "It's fine to talk about decision architectures in theory, but the truth is that everyone who's excited about nudges are extractive, psychopathic elites and the 'nudging' discourse just serves to justify their continued tightening of the vise around our throats>"
Grow up and accept that not everyone is going to take care of you like mama did when you were a toddler. I, personally, also worry about extractive elite culture, but that's not a substantive critique of the theory of nudging and it's not like we can't use more organ donors.
"Grow up and accept that not everyone is going to take care of you like mama did when you were a toddler."
Yes exactly. I'll make my own decisions without some elitist manipulating what's in front of me. "Nudging" is literally manipulation on a mass scale. You are manipulating people to achieve YOUR "preferred outcome". People who engage in this behavior are disgusting. Not only that, but the choices provided to people can be further manipulated to seem benign when in reality they are not.
It is just like propaganda. When there is an agenda behind perceived "choices" it is manipulation.
As all presentations have agendae simply by being stated, I think this is a false dichotomy. A stop sign presents you with a choice with a clear preference: "The State of NH insists you stop here."; an invitation to lunch presents you with a choice, but it has a manipulation, if you want to be sticky about it, behind it: "I would like to go to lunch with you;" For organ donation, the statement is "The State of NH thinks this is a good thing and would like to encourage it.
I don't think there is the least hope of eliminating it, as it is just normal human interaction. We simply have to be aware of them as they go by and weight them accordingly. If you are this determined to be offended I don't know that any of us can nudge you.
Can I guess that you have not brought up any children?
There's something I don't understand about the treatment here. Aren't all nudges apples and oranges, even closely related ones of the same type?
If one study finds with high certainty that a wasp picture in urinals have a positive effect and another study finds with high certainty that a butterfly picture in urinals have a negative effect (of the same size), is the relevant meta-conclusion that urinal-picture-nudges have no effect on average? Wouldn't the relevant conclusion for real-world practical applications be that nudges can be effective, but you have to test the effectiveness of the given nudge because you can't presume to know what it'll be in advance?
How does it make any kind of sense to look at an "average effect" of studies that study different nudges (even of the same type) and conclude that merely because the average is zero or very small - or would be if not for publication bias - this means we have no evidence for effectiveness of nudges in that domain overall? Just because a randomly designed nudge will have little to no effect on average doesn't mean nudges can't be effectively employed with the proper methodology for designing and testing them. Testing just has to be a part of the deployment process itself, rather than something left for academics to do later. or do I misunderstand something here?
The problem with having a nudge department is that, no matter what the situation, they're never going to say 'let's just be honest with the public and lay out the facts before them.'
As a nudger, your job security depends on persuading your bosses that the general public are a bunch of suicidal buffoons. Which can't be good for society.
You are British and the "nudge" guys are American yeah? I saw you say "pension scheme" and I knew you had to be British because in American English "scheme" is incredibly negative in connotation whereas "nudge" is about as neutral as you can get for what it is describing. We'd usually call them "pension plans" rather than "pension schemes". Planners are good and schemers are bad.
Many thanks Stuart for this excellent article. It would make a great addition to the nudge theory articles series I've been publishing at https://realkm.com/nudge-theory/
I wonder if I could obtain your permission for republication? The article would be republished verbatim, and I'm very happy to add backlinks and attribution as required.
This may be overly optimistic, but perhaps we can take encouragement that the open data provided alongside the original meta-analysis must surely be a factor which facilitated the speed of the corrections and commentary?
I'm sorry to bother you with this as I am sure you must be thinking about it, but what do you think of the recent review which claimed that serotonin has no relationship to depression? Would love to read your take on it.
Definitely something I want to look into!
(I sincerely hope) there’s a special place in hell for government bureaucrats and nudge-ists who think “changing the default” is a benign “nudge”.
Making something opt-out rather than opt-in is not some brilliant science-based choice architecture hack.
It’s simply coercing users by taking away their freedom to choose.
Compare “changing the default” to other judges and it’s clear that this is the most coercive technique (yet).
It is therefore no surprise that this is mentioned as one of the more effective methods.
Sorry for the rant but I had to get this off my chest.
Thank you for this essay/post (sorry, no idea what substack posts should be called), I really enjoy your writing
V kind! Recommended to take a look at the Christopher Snowdon article I linked to near the start - he has the interesting position of being a hardcore libertarian but also being relatively sympathetic to nudges, or at least some of them.
One major assumption here that is fundamentally wrong in the analysis of publication bias, is the requirement that all the reported effect "clouds" must be symmetrical even *throughout the zero effect*. Let me demonstrate with an example. You design a study to see whether say.. injecting bleach can cure Covid-19. You gather 100 people, inject 50 with bleach, 50 with saline. Hmm, weird, you replicate the study. Ten times, hundred times. You can plot funnels, clouds, but even if by some miracle you see a dot on the right, expecting any kind of symmetry is detached from reality.
So, yes, the variability in experimental design plays a role, but that's not all. You need to account for the fact that these experiments have been designed with a specific goal to achieve positive effects. And that may be a faulty mindset in science generally, but it is the fundamental nature of what nudges are about! In some cases, observing a negative effect size is actually absurd or even impossible. If you observe how many people sign up for organ donation form that's innocuously bundled to a driver's license application - would you expect that someone flips the table, storms off, and writes a last will saying: "When I die, dissolve my body in a vat of acid." ?
And even though thorough statistical interpretations are very much needed, sometimes the world is too complex for us to reduce into numbers.
Yes, this is fair! When I talk about publication bias tests I usually do add a disclaimer like this (see my breastfeeding post for example). But for some reason I didn't think to do so this time. Basically it's just circumstantial evidence - and circumstantial evidence can be very wrong!
i think a better term than "nudge" would be lying, manipulation, coercion, control etc. "nudge" is a propaganda term used to minimize the reality of how evil and sociopathic "nudging" (manipulating) people really is.
This is such a strange attitude to me. Every time a choice is placed in front of you, whether by a company or an individual, they have to decide how to present it to you. "Nudging" is just about choosing some presentations ("choice architectures") which encourage preferred outcomes. Because there is no default option for the presentation of choices, they have to make some such decision, so the real choice you make is between being nudged or making no choices at all.
Can this be taken too far? Yes. But there is no clear line between benign favorable presentation of preferred choices and acute manipulation.
Anticipated response, let me know if it's a straw man: "It's fine to talk about decision architectures in theory, but the truth is that everyone who's excited about nudges are extractive, psychopathic elites and the 'nudging' discourse just serves to justify their continued tightening of the vise around our throats>"
Grow up and accept that not everyone is going to take care of you like mama did when you were a toddler. I, personally, also worry about extractive elite culture, but that's not a substantive critique of the theory of nudging and it's not like we can't use more organ donors.
"Grow up and accept that not everyone is going to take care of you like mama did when you were a toddler."
Yes exactly. I'll make my own decisions without some elitist manipulating what's in front of me. "Nudging" is literally manipulation on a mass scale. You are manipulating people to achieve YOUR "preferred outcome". People who engage in this behavior are disgusting. Not only that, but the choices provided to people can be further manipulated to seem benign when in reality they are not.
It is just like propaganda. When there is an agenda behind perceived "choices" it is manipulation.
As all presentations have agendae simply by being stated, I think this is a false dichotomy. A stop sign presents you with a choice with a clear preference: "The State of NH insists you stop here."; an invitation to lunch presents you with a choice, but it has a manipulation, if you want to be sticky about it, behind it: "I would like to go to lunch with you;" For organ donation, the statement is "The State of NH thinks this is a good thing and would like to encourage it.
I don't think there is the least hope of eliminating it, as it is just normal human interaction. We simply have to be aware of them as they go by and weight them accordingly. If you are this determined to be offended I don't know that any of us can nudge you.
Can I guess that you have not brought up any children?
Says the person with a Stalin photo as his profile picture.
take a closer look genius.
This does seem to come up often. I wonder if we should train ourselves to think "Meta-analysis - Is it publication bias?"
There's something I don't understand about the treatment here. Aren't all nudges apples and oranges, even closely related ones of the same type?
If one study finds with high certainty that a wasp picture in urinals have a positive effect and another study finds with high certainty that a butterfly picture in urinals have a negative effect (of the same size), is the relevant meta-conclusion that urinal-picture-nudges have no effect on average? Wouldn't the relevant conclusion for real-world practical applications be that nudges can be effective, but you have to test the effectiveness of the given nudge because you can't presume to know what it'll be in advance?
How does it make any kind of sense to look at an "average effect" of studies that study different nudges (even of the same type) and conclude that merely because the average is zero or very small - or would be if not for publication bias - this means we have no evidence for effectiveness of nudges in that domain overall? Just because a randomly designed nudge will have little to no effect on average doesn't mean nudges can't be effectively employed with the proper methodology for designing and testing them. Testing just has to be a part of the deployment process itself, rather than something left for academics to do later. or do I misunderstand something here?
The problem with having a nudge department is that, no matter what the situation, they're never going to say 'let's just be honest with the public and lay out the facts before them.'
As a nudger, your job security depends on persuading your bosses that the general public are a bunch of suicidal buffoons. Which can't be good for society.
You are British and the "nudge" guys are American yeah? I saw you say "pension scheme" and I knew you had to be British because in American English "scheme" is incredibly negative in connotation whereas "nudge" is about as neutral as you can get for what it is describing. We'd usually call them "pension plans" rather than "pension schemes". Planners are good and schemers are bad.
Many thanks Stuart for this excellent article. It would make a great addition to the nudge theory articles series I've been publishing at https://realkm.com/nudge-theory/
I wonder if I could obtain your permission for republication? The article would be republished verbatim, and I'm very happy to add backlinks and attribution as required.
I look forward to hearing from you, and can be contacted through https://realkm.com/contact/
Many thanks,
Bruce Boyes.
This may be overly optimistic, but perhaps we can take encouragement that the open data provided alongside the original meta-analysis must surely be a factor which facilitated the speed of the corrections and commentary?