the New York Times blows it big time on brain imaging

The New York Times has a terrible, terrible Op-Ed piece today by Martin Lindstrom (who I’m not going to link to, because I don’t want to throw any more bones his way). If you believe Lindstrom, you don’t just like your iPhone a lot; you love it. Literally. And the reason you love it, shockingly, is your brain:

Earlier this year, I carried out an fMRI experiment to find out whether iPhones were really, truly addictive, no less so than alcohol, cocaine, shopping or video games. In conjunction with the San Diego-based firm MindSign Neuromarketing, I enlisted eight men and eight women between the ages of 18 and 25. Our 16 subjects were exposed separately to audio and to video of a ringing and vibrating iPhone.

But most striking of all was the flurry of activation in the insular cortex of the brain, which is associated with feelings of love and compassion. The subjects’ brains responded to the sound of their phones as they would respond to the presence or proximity of a girlfriend, boyfriend or family member.

In short, the subjects didn’t demonstrate the classic brain-based signs of addiction. Instead, they loved their iPhones.

There’s so much wrong with just these three short paragraphs (to say nothing of the rest of the article, which features plenty of other whoppers) that it’s hard to know where to begin. But let’s try. Take first the central premise–that an fMRI experiment could help determine whether iPhones are no less addictive than alcohol or cocaine. The tacit assumption here is that all the behavioral evidence you could muster–say, from people’s reports about how they use their iPhones, or clinicians’ observations about how iPhones affect their users–isn’t sufficient to make that determination; to “really, truly” know if something’s addictive, you need to look at what the brain is doing when people think about their iPhones. This idea is absurd inasmuch as addiction is defined on the basis of its behavioral consequences, not (right now, anyway) by the presence or absence of some biomarker. What makes someone an alcoholic is the fact that they’re dependent on alcohol, have trouble going without it, find that their alcohol use interferes with multiple aspects of their day-to-day life, and generally suffer functional impairment because of it–not the fact that their brain lights up when they look at pictures of Johnny Walker red. If someone couldn’t stop drinking–to the point where they lost their job, family, and friends–but their brain failed to display a putative biomarker for addiction, it would be strange indeed to say “well, you show all the signs, but I guess you’re not really addicted to alcohol after all.”

Now, there may come a day (and it will be a great one) when we have biomarkers sufficiently accurate that they can stand in for the much more tedious process of diagnosing someone’s addiction the conventional way. But that day is, to put it gently, a long way off. Right now, if you want to know if iPhones are addictive, the best way to do that is to, well, spend some time observing and interviewing iPhone users (and some quantitative analysis would be helpful).

Of course, it’s not clear what Lindstrom thinks an appropriate biomarker for addiction would be in any case. Presumably it would have something to do with the reward system; but what? Suppose Lindstrom had seen robust activation in the ventral striatum–a critical component of the brain’s reward system–when participants gazed upon the iPhone: what then? Would this have implied people are addicted to iPhones? But people also show striatal activity when gazing on food, money, beautiful faces, and any number of other stimuli. Does that mean the average person is addicted to all of the above? A marker of pleasure or reward, maybe (though even that’s not certain), but addiction? How could a single fMRI experiment with 16 subjects viewing pictures of iPhones confirm or disconfirm the presence of addiction? Lindstrom doesn’t say. I suppose he has good reason not to say: if he really did have access to an accurate fMRI-based biomarker for addiction, he’d be in a position to make millions (billions?) off the technology. To date, no one else has come close to identifying a clinically accurate fMRI biomarker for any kind of addiction (for more technical readers, I’m talking here about cross-validated methods that have both sensitivity and specificity comparable to traditional approaches when applied to new subjects–not individual studies that claim 90% with-sample classification accuracy based on simple regression models). So we should, to put it mildly, be very skeptical that Lindstrom’s study was ever in a position to do what he says it was designed to do.

We should also ask all sorts of salient and important questions about who the people are who are supposedly in love with their iPhones. Who’s the “You” in the “You Love Your iPhone” of the title? We don’t know, because we don’t know who the participants in Lindstrom’s sample, were, aside from the fact that they were eight men and eight women aged 18 to 25. But we’d like to know some other important things. For instance, were they selected for specific characteristics? Were they, say, already avid iPhone users? Did they report loving, or being addicted to their iPhones? If so, would it surprise us that people chosen for their close attachment to their iPhones also showed brain activity patterns typical of close attachment? (Which, incidentally, they actually don’t–but more on that below.) And if not, are we to believe that the average person pulled off the street–who probably has limited experience with iPhones–really responds to the sound of their phones “as they would respond to the presence or proximity of a girlfriend, boyfriend or family member”? Is the takeaway message of Lindstrom’s Op-Ed that iPhones are actually people, as far as our brains are concerned?

In fairness, space in the Times is limited, so maybe it’s not fair to demand this level of detail in the Op-Ed iteslf. But the bigger problem is that we have no way of evaluating Lindstrom’s claims, period, because (as far as I can tell), his study hasn’t been published or peer-reviewed anywhere. Presumably, it’s proprietary information that belongs to the neuromarketing firm in question. Which is to say, the NYT is basically giving Lindstrom license to talk freely about scientific-sounding findings that can’t actually be independently confirmed, disputed, or critiqued by members of the scientific community with expertise in the very methods Lindstrom is applying (expertise which, one might add, he himself lacks). For all we know, he could have made everything up. To be clear, I don’t really think he did make everything up–but surely, somewhere in the editorial process someone at the NYT should have stepped in and said, “hey, these are pretty strong scientific claims; is there any way we can make your results–on which your whole article hangs–available for other experts to examine?”

This brings us to what might be the biggest whopper of all, and the real driver of the article title: the claim that “most striking of all was the flurry of activation in the insular cortex of the brain, which is associated with feelings of love and compassion“. Russ Poldrack already tore this statement to shreds earlier this morning:

Insular cortex may well be associated with feelings of love and compassion, but this hardly proves that we are in love with our iPhones.  In Tal Yarkoni’s recent paper in Nature Methods, we found that the anterior insula was one of the most highly activated part of the brain, showing activation in nearly 1/3 of all imaging studies!  Further, the well-known studies of love by Helen Fisher and colleagues don’t even show activation in the insula related to love, but instead in classic reward system areas.  So far as I can tell, this particular reverse inference was simply fabricated from whole cloth.  I would have hoped that the NY Times would have learned its lesson from the last episode.

But you don’t have to take Russ’s word for it; if you surf for a few terms on our Neurosynth website, making sure to select “forward inference” under image type, you’ll notice that the insula shows up for almost everything. That’s not an accident; it’s because the insula (or at least the anterior part of the insula) plays a very broad role in goal-directed cognition. It really is activated when you’re doing almost anything that involves, say, following instructions an experimenter gave you, or attending to external stimuli, or mulling over something salient in the environment. You can see this pretty clearly in this modified figure from our Nature Methods paper (I’ve circled the right insula):

Proportion of studies reporting activation at each voxel

The insula is one of a few ‘hotspots’ where activation is reported very frequently in neuroimaging articles (the other major one being the dorsal medial frontal cortex). So, by definition, there can’t be all that much specificity to what the insula is doing, since it pops up so often. To put it differently, as Russ and others have repeatedly pointed out, the fact that a given region activates when people are in a particular psychological state (e.g., love) doesn’t give you license to conclude that that state is present just because you see activity in the region in question. If language, working memory, physical pain, anger, visual perception, motor sequencing, and memory retrieval all activate the insula, then knowing that the insula is active is of very little diagnostic value. That’s not to say that some psychological states might not be more strongly associated with insula activity (again, you can see this on Neurosynth if you switch the image type to ‘reverse inference’ and browse around); it’s just that, probabilistically speaking, the mere fact that the insula is active gives you very little basis for saying anything concrete about what people are experiencing.

In fact, to account for Lindstrom’s findings, you don’t have to appeal to love or addiction at all. There’s a much simpler way to explain why seeing or hearing an iPhone might elicit insula activation. For most people, the onset of visual or auditory stimulation is a salient event that causes redirection of attention to the stimulated channel. I’d be pretty surprised, actually, if you could present any picture or sound to participants in an fMRI scanner and not elicit robust insula activity. Orienting and sustaining attention to salient things seems to be a big part of what the anterior insula is doing (whether or not that’s ultimately its ‘core’ function). So the most appropriate conclusion to draw from the fact that viewing iPhone pictures produces increased insula activity is something vague like “people are paying more attention to iPhones”, or “iPhones are particularly salient and interesting objects to humans living in 2011.” Not something like “no, really, you love your iPhone!”

In sum, the NYT screwed up. Lindstrom appears to have a habit of making overblown claims about neuroimaging evidence, so it’s not surprising he would write this type of piece; but the NYT editorial staff is supposedly there to filter out precisely this kind of pseudoscientific advertorial. And they screwed up. It’s a particularly big screw-up given that (a) as of right now, Lindstrom’s Op-Ed is the single most emailed article on the NYT site, and (b) this incident almost perfectly recapitulates another NYT article 4 years ago in which some neuroscientists and neuromarketers wrote a grossly overblown Op-Ed claiming to be able to infer, in detail, people’s opinions about presidential candidates. That time, Russ Poldrack and a bunch of other big names in cognitive neuroscience wrote a concise rebuttal that appeared in the NYT (but unfortunately, isn’t linked to from the original Op-Ed, so anyone who stumbles across the original now has no way of knowing how ridiculous it is). One hopes the NYT follows up in similar fashion this time around. They certainly owe it to their readers–some of whom, if you believe Lindstrom, are now in danger of dumping their current partners for their iPhones.

h/t: Molly Crockett

44 thoughts on “the New York Times blows it big time on brain imaging”

  1. Terrific writeup/takedown. I can’t believe the NYT published this. What were they thinking? Maybe since it was in the Opinion section they figured it doesn’t have to be true.

    BTW, I like (a) that you tagged this ‘terrible’ and (b) that this isn’t the only item on your blog with that tag.

  2. What an awful blog post, Tal — talk about “burying it in paragraph 19”! You didn’t mention Lindstrom’s most compelling evidence for the accuracy of his claims until literally the last paragraph of your post:

    as of right now, Lindstrom’s Op-Ed is the single most emailed article on the NYT site

    There now, you see? Whatever other scientific mumbo jumbo you might throw at this piece, the decision of the Times’ editorial staff to accept Lindstrom’s claims as factual is clearly vindicated, don’t you think?

    I think you’ve committed two major logical fallacies here, Tal. The first, and less egregious, is your use of the Licit Minor, i.e. your insistence that fMRIs showing stimulation of the insular cortex in both people in love and in iPhone users is anything less than proof positive that iPhone use is identical to love.

    Far more embarrassing to you, though, (or at least it ought to be) is your employment of the Argument ad Factium, in which you suggest that scientific fact is somehow more relevant than how many e-mail forwards this article gets. Shame on you for such an undemocratic belief!

  3. As a neuroscientist and cognitive psychologist, I share your indignation over the abuse of physiological measures (brain activity) to draw overblown and unfounded conclusions abut personality traits and mental states. It seems scientists play this card often in order to attract non-scientific attention to themselves. It’s irresponsible. I agree wholeheartedly with your critique and would even further point out just how the insular cortex cortex – cited here as some biomarker for love – also known as a biomarker for disgust in other studies.

  4. Pingback: Quora
  5. Even if the imaging results could be interpreted as nothing other than love, if the stimuli are in the form of ringtones, who is to say it’s the object of the phone itself which elicits the reaction and not the ringing which signals imminent human contact, most likely with a loved one? Who hasn’t been waiting for a phone call from a loved one and had their heart skip a beat at the sound of the ringing phone? A ringtone on an iPhone signals the beginning of the least iPhoneish activity one performs on it; a telephone call. Every phone makes and receives phone calls, that’s not a reason to notice, let alone ‘Love’ an iPhone in particular.

  6. I fully agree and thank you for the analysis. I was however disappointed to see the whole thing flurred up an fMRI hate bandwagon again and couldn’t help but comment.

    http://www.sayginlab.org/blog/?p=135

    Sorry I don’t want to hijack your blog post. I rarely blog but it touched a nerve.

    I think bad reporting hurts all of us.

  7. Random thought:

    It’s an opinion piece rather than a serious academic study. Interesting, curious, but I doubt the Michelle Obama Clinic for the iPhone addicted will start anytime soon.

  8. I don’t see any comment relating to the fact that iPhone uses subliminal technologies (for example an alarm sound that to increase the compassion and/or need to protect and to love in the listener brain), or anything similar, but just the effects on the folks questioned.

    In the case of drugs, physicians already studied the chemical effects (and the dependencies those create on the humans), not just exposed some or another effect.

    Therefore, WHY is addictive, can you answer to that ? Oh, sorry, you’re not an expert, you’re just a person “sharing” with us. Personally, I would love to read some more informed article, this is why the journalists are not so welcome (they seem to know about everything and nothing, in the same time, which just spreads confusion around). This is the reason why readers use to comment – because the info was not whole, clear or at least enough for their sense of perception. No wonder that readers keep searching on the net some more informed opinion on same topic. And that happens in 95% from cases.

    Thank you.

  9. Great piece. There’s one claim that you might want to qualify, though. You and Russ say that the insula is “one of the most highly activated part[s] of the brain”, and that insula activation “shows up for almost everything.” That’s true in the context of the tasks that imaging researchers tend to study, but not necessarily in real life. If, hypothetically, there were more imaging studies on vision than on hearing (and this may be the case), this wouldn’t imply that visual cortex was more highly activated than auditory cortex IRL. Incidentally, this doesn’t take anything away from your argument about the NYT piece, which is right-on.

  10. Pingback: iPhone Love
  11. Personally, I would be surprised if the study ever took place.

    Isn’t the most parsimonious explanation that this Op-Ed was financed by the marketing hacks at a large corporation that would profit from the extraordinary publicity?

    It is, fundamentally, a *marketing* company, no?

    No publishing = No peer review = No evidence of data = No data fabrication violation = No retraction

  12. Thanks for all the comments!

    AP Saygin, my critique (and most of the others, I think) certainly isn’t meant to be a critique of fMRI; it’s a critique of the abuse of fMRI. I share the concerns you expressed in your blog post.

    D Reich, thanks for the link, wasn’t aware of that.

    Josh, agreed–we still need priors, and can’t just take the literature as published as an accurate representation of the true distribution of mental states. This is the main reason we used uniform priors in our Nature Methods paper. But in the context of this particular critique, I agree that it doesn’t make any difference, since what’s clear is that the insula doesn’t do love.

    neuromusic, I think the jury’s out on that, but I don’t have trouble believing some study was done. I’m just not sure it was done well (well, okay, I’m sure it wasn’t done well, or we wouldn’t be here).

  13. I know you don’t want to throw any clicks in his direction, but I visited Lindstrom’s website and enjoyed his hilarious “predictions” section, where he claims that in 2008 he proved “a correlation between brands and religion (and Apple and religion) in the world’s first fMRI study ever conducted.” Little known fact: Martin Lindstrom invented fMRI in 2008. So the question is, why would I care what you have to say, Tal Yarkoni?

  14. You can fake it all you like, Tim, but I know you hang on my every word. Well, maybe not every word. But surely some of them. Like when I say, “awesome Neuron paper, Tim”–I’m pretty sure you hang on those words, right?

    Speaking of your paper, I hope you credited Lindstrom in the introduction for developing fMRI. He probably also invented MVPA and the matching pennies task, so maybe you should send the guy a gift card or something.

  15. It’s true, Tal; I dote on you even if you didn’t invent fMRI in 2008. I do wish you had a “predictions” page on your site, though. And that you would use more Flash. Gobs and gobs of pointless Flash.

  16. Good news! After a recent brain study, it turns out that I’m not addicted to cocaine — I’m just in LOVE with it.

    Wait, what? It’s weird to interpret addictive behaviors as attachment based on flimsy brain scan evidence? Yeah, it’s weird.

  17. Thanks for debunking to the NYT piece. I’m a shopper marketer that got to hear Martin speak at an industry event several years ago. He was a very strong presenter, and based on his lecture I eagerly read (and believed) his new book Buyology largely based on the fMRI study he references in his subsequent coments. As a total layman, I bought his conclusions hook, line, and sinker. In fact, I enjoyed the book so much, I also read another fMRI based marketing book: The Buying Brain: Secrets for Selling to the Subconscious Mind, which I also believed and enjoyed. Now I’m looking at all the marketing texts leveraging brain imaging with a skeptical eye. Thanks for keeping me honest.

Leave a Reply to Tal YarkoniCancel reply