Archive for the ‘general silliness’ Category

babygate blues: a neuromarketing tale

Monday, July 26th, 2010

Cory Doctorow has a new short story (“Ghosts in my Head“) about the undesirable consequences of neuromarketing run amok up on the Subterranean Press website.  I liked the story, but thought the premise was pretty unrealistic (and, yes, I do know it’s called science fiction for a reason–I’m just sayin’). So as a counterpoint, here’s an alternative neuromarketing future that I personally find much more plausible.

Deborah Stojko didn’t care much for Pockter and Gramble’s corporate headquarters. The building smelled of disinfectant and organization; the halogen corridors all blended together into one giant dimly-lit maze. Stojko had been visiting P&G regularly for several years now; it was never a pleasant experience, but it couldn’t be avoided. Communicating with major stakeholders was a large part of her job as director of the International Consortium for Neuromarketing Research. And P&G was by far the largest stakeholder, contributing over 70% of the money that supported the consortium’s work.

For several years now, ICNR had been pumping out first-class scientific research on the neural mechanisms of economic decision-making. The Richelieu effect, Preinforcement Learning, the neurometric satisficing theorem… ICNR was behind any number of recent discoveries; its members were continually in the news. And all of it was made possible only through the generosity of the marketing and R&D wings of P&G.

The generosity, or the naivete? Stojko asked herself as she reached her destination and knocked softly on an office door. Somehow, the executives at Pockter and Gramble had managed to convince themselves that the survival of P&G rested on their ability to mine the deep secrets of the brain. For years now, they’d been throwing sums of money at cognitive neuroscientists that would make European royalty blush. That streak of good fortune, Stojko suspected, was now about to end. Recent events had rendered P&G’s massive investment in ICNR something of a political liability; she had the feeling this was the last time she’d be making the trip to P&G headquarters.

And not a moment too soon, she thought, as the door opened in front of her.

*    *    *

“How long has Pockter and Gramble been funding you, Deborah,” Bob Ramsey, Chief Executive Officer, asked, once Stojko was seated and they’d gotten the standard pleasantries out of the way.

Stojko did the arithmetic in her head. The International Neuromarketing Consortium had formed in 2013, following a massive infusion of P&G cash, so…

“Six years,” she said.

“Right. And do you know how much money Pockter and Gramble has given your consortium in those six years?”

“I’d put it somewhere between 251.8 and 251.9 million dollars.”

“Very clever. A quarter of a billion dollars. We’ve given you a quarter. Of a billion. Dollars.”

“Well, to be fair, that amount is spread out over 8 sites and 30 other investigators,” Stojko pointed out. “It’s not like you wrote me a check for 250 million. My institution only got about forty-five million.”

Ramsey didn’t say anything, but his expression bespoke a thinly-veiled irritation. He picked up a remote control on the desk and pushed a button. Behind Stojko, the wall turned translucent as the embedded display lit up.

“No doubt you’ll recognize this clip,” Ramsey said.

Stojko swiveled around to watch the giant screen. The camera faded in on a bright and comfortable-looking living room somewhere in America. Almost immediately, six or seven babies in diapers filed into the room and began dancing synchronously in a circle. After a few seconds of dancing, the babies started babbling an Eastern-sounding melody in a totally incomprehensible–and, Stojko suspected, nonexistent–language. And a few seconds after that, they started banging spoons on the tabletop in perfect unison, all the while still dancing and singing in tongues. The whole thing lasted exactly thirty seconds, and occupied a very narrow emotional niche between really adorable and utterly creepy.

Stojko did recognize the clip, of course; it was an ad for Dampers, a P&G-owned diaper brand. The consortium had selected the ad from over two dozen candidates that P&G had asked them to test. For reasons that remained unclear to Stojko–and to pretty much everyone else–singing, dancing, spoon-banging babies lit the brain up like a christmas tree.

Stojko had had her reservations about declaring a ‘winner’; she’d written several long emails to the P&G marketing brain trust explaining that, brain activation notwithstanding, there really wasn’t any evidence yet that this particular ad was going to help sell more diapers, and many more studies were needed before the consortium could confidently interpret its own results. But marketing wasn’t into the whole waiting thing, and the ad was on the air within three months of the consortium’s initial report.

As it turned out, it didn’t do so well.

“That ad bombed,” Ramsey said, wagging his finger in the general direction of the screen, “According to you people, it was supposed to push all of the brain’s buttons at once. You spent three million dollars of our money just on that one testing program. Two dozen ads to choose from, and the one you pick completely tanked. It was an epic failure. At this very moment, people in living rooms all over America are laughing at Pockter and Gramble because of that ad.”

“I’m sure it’s not that bad” said Stojko, smirking almost imperceptibly. She was well aware of the PR disaster P&G had on its hands, of course. But she couldn’t deny the warm feeling of schadenfreude that accompanied the knowledge that P&G was now paying many times over for disregarding just about every recommendation the consortium had made in its 480-page report. She was pretty sure the suits had never made it past the fifth or sixth page.

“It is that bad,” Ramsey shot back. “We blew half of our network budget for the year on this ad. Our initial focus groups were already pretty positive, and then we received your report saying things like–and I quote–”of all the ads tested, number seventeen elicited the largest response in brain areas associated with reward.” So we figured it was a sure thing, and started airing the ad in all the major markets. And then, out of nowhere, we get this massive backlash. Thousands of angry emails from people complaining that the ad was trite and we were shamefully “exploiting babies”. People saying they would never buy Dampers diapers again; that the CEO–that’s me, mind you–should resign; that someone should “just torch Pockter and Gramble headquarters”. And those were just the serious complaints. There were also the people who apparently thought the whole thing was just a big joke that gave them an opening to do their own thing. We had forty YouTube videos a day uploaded by people spoofing the ad. There was one clip of six guys in giraffe suits singing and doing our baby dance. Sixteen million hits.”

“All publicity is good publicity, right?”

“No. Not even close.”

Stojko chuckled just loudly enough for Ramsey to hear.

“Is this funny to you?” Ramsey asked. “We give you a quarter of a billion dollars for commercials designed to push the brain’s reward buttons, and we get grown men in giraffe suits?”

“Well, let me put it this way, Bob. If your goal was really to make commercials that light up the brain’s reward circuitry, you wouldn’t have needed to do any serious research in the first place; you could have just run 30-second clips of semi-nude women making out with each other, or couples giggling and cuddling in bed. That’d cover most of the bases. You’d have all the reward-related activation you could want. But how many deodorant sticks do you think commercials like that would sell?”

Ramsey stared at Stojko blankly.

“Porn, flashing lights, pictures of hundred-dollar bills, a basket of shiny fresh fruit… lots of things activate the brain’s reward centers,” Stojko continued. “What makes you think a commercial that tangentially elicits reward-related activation is going to make people buy any more of a product?”

“Well, can’t you tell that?”

“Can we?” asked Stojko rhetorically. “I don’t know. Can you tell that? You guys probably have labs full of people trying to figure out whether the fact that people tell you they like a commercial means they’re going to buy more of the product featured in that commercial. And what’s the answer?”

“I don’t know that myself,” Ramsey replied abruptly. “It’s not my job to know that. I can have marketing come up here and tell you the answer if you like.”

Stojko shook her head.

“Doesn’t matter. I mean, it can only go one of two ways. If marketing doesn’t know what makes a commercial good or bad, you can’t really expect us to tell you what it is about the brain that makes people buy things. We don’t track how well your products sell after different ads go into circulation; how the hell would we know which commercials have the largest impact on sales? I can tell you which commercials activate the nucleus accumbens more than others, but so what? How am I supposed to know if nucleus accumbens activation is a good predictor of actual purchases without actually knowing anything about real-world purchases?”

Ramsey had nothing to say to that; he stared down at his shoes.

“So clearly, that’s not going to help us,” Stojko continued. “But suppose instead we pretend that the people in your marketing department are smart cookies, and they do know what it is about commercials that makes people buy your products. Well, in that case, what the hell would you need us? If you’ve figured out that people are more likely to buy your anti-dandruff shampoo after watching ads they rate ‘extremely interesting’, what is peering into the brain going to tell you?”

“Well, I guess you could use brain imaging to figure out what it is that people find extremely interesting, right?”

“Sure, Bob, we could do that. And you know how we’d do that? By asking people which commercials they found interesting, and then correlating their verbal responses with what their brains were doing while they watched those commercials. And you know what that means? It means we can never do any better than your people can do with your focus groups and spreadsheets. Because basically, we’re stuck trying to predict the same variables that you guys are using to predict people’s buying behavior. We’re just one step further removed.”

Ramsey listened quietly, but anger visibly colored his face as Stojko spoke.

“This is the kind of thing that might have been good to bring up, oh, say, five years ago,” he said.

“Oh, believe me, we did bring it up,” Stojko smiled bitterly. “Or at least, we tried to.”

She tapped a few keys on her holoboard.

“Here’s an email dated June 18th, 2014: “Dear Mr. Chauahan–I believe that’s your VP of marketing, right?–senior members of the consortium continue to express their frustration at Pockter and Gramble’s failure to provide us with the sales data we requested. As we indicated in our letter dated April 21st, it is not possible for us to properly evaluate the efficacy of our program without the use of real-world performance metrics. We understand your concerns about sharing private data with outside contractors; however…”

Stojko shot Ramsey a pointed look.

“I’ll spare you the rest; it goes on like that for three pages. See, we’ve been asking for the data we need for six years now–pretty much since we started. And every time we ask, you throw more money at us and tell us to go back to work, that you’re not going to share your numbers with us because they’re confidential and we shouldn’t need that information anyway.”

She tapped a few more keys.

“Here’s another similar one. September 30th: Dear Mr. Chauahan, the consortium is at a loss to understand…”

“Enough!” yelled Ramsey, slamming his fist down on the desk. “I get the point! We’ve spent a quarter of a billion buying you new toys to play with, and all the while you’ve been playing us for idiots. Well, you know what–enjoy your toys while they last, because we’re going to have Legal look at our options for recovering that money first thing Monday morning. Those fancy new scanners of yours are going away.”

He wheeled his chair away from Stojko and sat there fuming. Stojko took it as a sign the meeting was over; she shrugged and got up to leave.

The falling out was unfortunate, she thought as she walked down the long sterile corridor towards the elevator. But it had been a long time coming, and after the whole Babygate episode (as the scientists at ICNR had started calling it), no one at ICNR would be surprised to hear that P&G was pulling the plug.

Nor would most of them mind terribly much. Stojko had always planned for a six or seven-year run, and had stopped hiring people on short-term contracts a couple of years ago. There would be no massive lay-offs, no collective plunge into obscurity for the many researchers invested in the project. The data was already collected, and she and her colleagues would be kept busy analyzing and publishing the results for years to come.

As for Ramsey’s legal threats, Stojko wasn’t the least bit worried. Universities had lawyers too, and there wasn’t a judge in the country who’d award P&G a single nickel for breach of contract; not after reading the long series of emails from the consortium that already explained in excruciating detail exactly why P&G was never going to recoup its financial investment unless it fundamentally changed the way it did things. Which, of course, hadn’t happened–and probably never would.

Stojko left Pockter and Gramble headquarters with a clear conscience. At the end of the day, she thought as she walked to her car, all you could do was represent yourself honestly to the other party and let the chips fall where they may. And that was what she’d done. She’d told P&G all along exactly how the consortium was going to spend the money they received; the service agreements she signed were very clearly delineated in legalese that several lawyers on the institutional payroll had contributed to and pored over. Stojko and her colleagues had worked hard to ensure that no one at P&G was laboring under false pretenses about the likely outcome of ICNR’s work. As she’d once put it to a mid-level P&G executive over dinner, neuromarketing research was great for science, and (in her estimation) utterly useless for advertising. But if the suits were willing to pay for it, she was willing to do the research. That, after all, was her job; it was what she’d be doing with her time anyway, ICNR or no ICNR.

No, she thought, turning the key in the ignition. She’d been right to take the industry money; ICNR had conducted itself impeccably over the past six years. If someone insisted on filling your cup up with change even after you very carefully explained to them that you were only going to buy beer with it, who could blame you for paying a visit to the bar once panhandling hours were over?

elsewhere on the net, vacation edition

Tuesday, July 13th, 2010

I’m hanging out in Boston for a few days, so blogging will probably be sporadic or nonexistent. Which is to say, you probably won’t notice any difference.

The last post on the Dunning-Kruger effect somehow managed to rack up 10,000 hits in 48 hours; but that was last week. Today I looked at my stats again, and the blog is back to a more normal 300 hits, so I feel like it’s safe to blog again. Here are some neat (and totally unrelated) links from the past week:

  • OKCupid has another one of those nifty posts showing off all the cool things they can learn from their gigantic userbase (who else gets to say things like “this analysis includes 1.51 million users’ data”???). Apparently, tall people (claim to) have more sex, attractive photos are more likely to be out of date, and most people who claim to be bisexual aren’t really bisexual.
  • After a few months off, my department-mate Chris Chatham is posting furiously again over at Developing Intelligence, with a series of excellent posts reviewing recent work on cognitive control and the perils of fMRI research. I’m not really sure what Chris spent his blogging break doing, but given the frequency with which he’s been posting lately, my suspicion is that he spent it secretly writing blog posts.
  • Mark Liberman points out a fundamental inconsistency in the way we view attributions of authorship: we get appropriately angry at academics who pass someone else’s work off as their own, but think it’s just fine for politicians to pay speechwriters to write for them. It’s an interesting question, and leads to an intimately related, and even more important question–namely, will anyone get mad at me if I pay someone else to write a blog post for me about someone else’s blog post discussing people getting angry at people paying or not paying other people to write material for other people that they do or don’t own the copyright on?
  • I like oohing and aahing over large datasets, and the Guardian’s Data Blog provides a nice interface to some of the most ooh- and aah-able datasets out there. [via R-Chart]
  • Ed Yong has a characteristically excellent write-up about recent work on the magnetic vision of birds. Yong also does link dump posts better than anyone else, so you should probably stop reading this one right now and read his instead.
  • You’ve probably heard about this already, but some time last week, the brain trust at ScienceBlogs made the amazingly clever decision to throw away their integrity by selling PepsiCo its very own “science” blog. Predictably, a lot of the bloggers weren’t happy with the decision, and many have now moved onto greener pastures; Carl Zimmer’s keeping score. Personally, I don’t have anything intelligent to add to everything that’s already been said; I’m literally dumbfounded.
  • Andrew Gelman takes apart an obnoxious letter from pollster John Zogby to Nate Silver of fivethirtyeight.com. I guess now we know that Zogby didn’t get where he is by not being an ass to other people.
  • Vaughan Bell of Mind Hacks points out that neuroplasticity isn’t a new concept, and was discussed seriously in the literature as far back as the 1800s. Apparently our collective views about the malleability of mind are not, themselves, very plastic.
  • NPR ran a three-part story by Barbara Bradley Hagerty on the emerging and somewhat uneasy relationship between neuroscience and the law. The articles are pretty good, but much better, in my opinion, was the Talk of the Nation episode that featured Hagerty as a guest alongside Joshua Greene, Kent Kiehl, and Stephen Morse–people who’ve all contributed in various ways to the emerging discipline of NeuroLaw. It’s a really interesting set of interviews and discussions. For what it’s worth, I think I agree with just about everything Greene has to say about these issues–except that he says things much more eloquently than I think them.
  • Okay, this one’s totally frivolous, but does anyone want to buy me one of these things? I don’t even like dried food; I just think it would be fun to stick random things in there and watch them come out pale, dried husks of their former selves. Is it morbid to enjoy watching the life slowly being sucked out of apples and mushrooms?

this year, i backed new zealand to go all the way

Tuesday, July 6th, 2010

Jerry Coyne ponders whether the best football/soccer team generally wins the World Cup. The answer is clearly no: any sporting event where games are settled on the basis of rare events (e.g., only one or two goals per match), and teams only play each other once to determine a winner, is going to be at the mercy of Lady Luck a good deal of the time. If we really wanted the best team to come out on top reliably, we’d probably need teams to play multiple games at every stage of the Cup, which isn’t very practical. Coyne discusses an (old) paper demonstrating that the occurrence of goals during World Cup matches is well fit by a poisson distribution, allowing one to calculate the probability of various unjust outcomes taking place (which turn out to be surprisingly high).

The curious thing, I think, is that it’s not really clear that sporting fans really do want the best team to come out on top. We don’t want outcomes to be determined by a coin toss, of course; it would kind of suck if, say, New Zealand had as much chance of lifting the cup as Brazil did. But it would also be pretty boring if it were a foregone conclusion that Brazil was going to win it all every time around. We want events to make sense, but we don’t want them to be too predictable. I suppose you could tell an interesting prediction error story about this kind of thing–e.g., that maximally engaging stimuli may be ones that seem to occur systematically yet defy easy explanation–but it’s probably more fun to sit around and curse at the television set as the Netherlands make short work of the Samba Kings (I don’t know if anyone actually uses that nickname; I just picked it off Wikipedia to make it look like I know what I’m talking about). Go Oranje!

and the runner up is…

Tuesday, June 22nd, 2010

This one’s a bit of a head-scratcher. Thomson-Reuters just released its 2009 Journal Citation Report–essentially a comprehensive ranking of scientific journals by their impact factor (IF). The odd part, as reported by Bob Grant in The Scientist, is that the journal with the second-highest IF is Acta Crystallographica – Section A–ahead of heavyweights like the New England Journal of Medicine. For perspective, the same journal had an IF of 2.051 in 2008. The reason for the jump?

A single article published in a 2008 issue of the journal seems to be responsible for the meteoric rise in the Acta Crystallographica – Section A‘s impact factor. “A short history of SHELX,” by University of Göttingen crystallographer George Sheldrick, which reviewed the development of the computer system SHELX, has been cited more than 6,600 times, according to ISI. This paper includes a sentence that essentially instructs readers to cite the paper they’re reading — “This paper could serve as a general literature citation when one or more of the open-source SHELX programs (and the Bruker AXS version SHELXTL) are employed in the course of a crystal-structure determination.” (Note: This may be a good way to boost your citations.)

Setting aside the good career advice (and yes, I’ve made a mental note to include the phrase “this paper could serve as a general literature citation…” in my next paper), it’s perplexing that Thomson-Reuters didn’t downweight Acta Crystallographica‘s IF considerably given the obvious outlier. There’s no question they would have noticed that the second-ranked journal was only there in virtue of one article, so I’m curious what the thought process was. Perhaps the deliberation went something like this:

Thomson-Reuters statistician A: We need to take it out! We can’t have a journal with an impact factor of 2 last year beat out the NEJM!

Thomson-Reuters statistician B: But if we take it out, it’ll look like we tampered with the IF!

TRS-A: But we already tamper with the IF! No one knows how we come up with these numbers! Sometimes we can’t even replicate our own results ourselves! And anyway, it’s really not a big deal if we just leave the article in; scientists know better than to think Acta Crystallographica is the second most influential science journal on the planet. They’ll figure it out.

TRS-B: But that’s like asking them to just disregard our numbers! If you’re supposed to ignore the impact factor in cases where it contradicts your perception of journal quality, what’s the point of having an impact factor at all?

TRS-A: Beats me.

So okay, I’m sure it didn’t go down quite like that. But it’s still pretty weird.
And now, having bitched about how arbitrary the IF is, I’m going to go off and spend the next 15 minutes perusing the psychology and neuroscience journal rankings…

elsewhere on the net

Friday, June 4th, 2010

Some neat links from the past few weeks:

  • You Are No So Smart: A celebration of self-delusion. An excellent blog by journalist David McCraney that deconstructs common myths about the way the mind works.
  • NPR has a great story by Jon Hamilton about the famous saga of Einstein’s brain and what it’s helped teach us about brain function. [via Carl Zimmer]
  • The Neuroskeptic has a characteristically excellent 1,000 word explanation of how fMRI works.
  • David Rock has an interesting post on some recent work from Baumeister’s group purportedly showing that it’s good to believe in free will (whether or not it exists). My own feeling about this is that Baumeister’s not really studying people’s philosophical views about free will, but rather a construct closely related to self-efficacy and locus of control. But it’s certainly an interesting line of research.
  • The Prodigal Academic is a great new blog about all things academic. I’ve found it particularly interesting since several of the posts so far have been about job searches and job-seeking–something I’ll be experiencing my fill of over the next few months.
  • Prof-like Substance has a great 5-part series (1, 2, 3, 4, 5) on how blogging helps him as an academic. My own (much less eloquent) thoughts on that are here.
  • Cameron Neylon makes a nice case for the development of social webs for data mining.
  • Speaking of data mining, Michael Driscoll of Dataspora has an interesting pair of posts extolling the virtues of Big Data.
  • And just to balance things out, there’s this article in the New York Times by John Allen Paulos that offers some cautionary words about the challenges of using empirical data to support policy decisions.
  • On a totally science-less note, some nifty drawings (or is that photos?) by Ben Heine (via Crooked Brains):

the perils of digging too deep

Wednesday, June 2nd, 2010

Another in a series of posts supposedly at the intersection of fiction and research methods, but mostly just an excuse to write ridiculous stories and pretend they have some sort of moral.


Dr. Rickles the postdoc looked a bit startled when I walked into his office. He was eating a cheese sandwich and watching a chimp on a motorbike on his laptop screen.

“YouTube again?” I asked.

“Yes,” he said. “It’s lunch.”

“It’s 2:30 pm,” I said, pointing to my watch.

“Still my lunch hours.”

Lunch hours for Rickles were anywhere from 11 am to 4 pm. It depended on exactly when you walked in on him doing something he wasn’t supposed to; that was the event that marked the onset of Lunch.

“Fair enough,” I said. “I just stopped by to see how things were going.”

“Oh, quite well.” said Rickles. “Things are going well. I just found a video of a chimp and a squirrel riding a motorbike together. They aren’t even wearing helmets! I’ll send you the link.”

“Please don’t. I don’t like squirrels. But I meant with work. How’s the data looking.”

He shot me a pained look, like I’d just caught him stealing video game money from his grandmother.

“The data are TERRIBLE,” he said in all capital letters.

I wasn’t terribly surprised at the revelation; I’d handed Rickles the dataset only three days prior, taking care not to  tell him it was the dataset from hell. Rickles was the fourth or fifth person in the line of succession; the data had been handed down from postdoc to graduate student to postdoc for several years now. Everyone in the lab wanted to take a crack at it when they first heard about it, and no one in the lab wanted anything to do with it once they’d taken a peek. I’d given it to Rickles in part to teach him a lesson; he’d been in the lab for several weeks now and somehow still seemed happy and self-assured.

“Haven’t found anything interesting yet?” I asked. “I thought maybe if you ran the Flimflan test on the A-trax, you might get an effect. Or maybe if you jimmied the cryptos on the Borgatron…”

“No, no,” Rickles interrupted, waved me off. “The problem isn’t that there’s nothing interesting in the data; it’s that there’s too MUCH stuff. There are too MANY results. The story is too COMPLEX.”

That didn’t compute for me, so I just stared at him blankly. No one ever found COMPLEX effects in my lab. We usually stopped once we found SIMPLE effects.

Rickles was unimpressed.

“You follow what I’m saying, Guy? There are TOO-MANY-EFFECTS. There’s too much going on in the data.”

“I don’t see how that’s possible,” I said. “Keith, Maria, and Lakshmi each spent weeks on this data and found nothing.”

“That,” said Rickles, “is because Keith, Maria, and Lakshmi never thought to apply the Epistocene Zulu transform to the data.”

The Epistocene Zulu transform! It made perfect sense when you thought about it; so why hadn’t I ever thought about it? Who was Rickles cribbing analysis notes from?

“Pull up the data,” I said excitedly. “I want to see what you’re talking about.”

“Alright, alright. Lunch hours are over now anyway.”

He grudgingly clicked on the little X on his browser. Then he pulled up a spreadsheet that must have had a million columns in it. I don’t know where they’d all come from; it had only had sixteen thousand or so when I’d had the hard drives delivered to his office.

“Here,” said Rickles, showing me the output of the Pear-sampled Tea test. “There’s the A-trax, and there’s its Nuffton index, and there’s the Zimming Range. Look at that effect. It’s bigger than the zifflon correlation Yehudah’s group reported in Nature last year.”

“Impressive,” I said, trying to look calm and collected. But in my head, I was already trying to figure out how I’d ask the department chair for a raise once this finding was published. Each point on that Zimming Range is worth at least $500, I thought.

“Are there any secondary analyses we could publish alongside that,” I asked.

“Oh, I don’t think you want to publish that,” Rickles laughed.

“Why the hell not? It could be big! You just said yourself it was a giant effect!”

“Oh sure. It’s a big effect. But I don’t believe it for one second.”

“Why not? What’s not to like? This finding make’s Yehudah’s paper look like a corn dog!”

I recognized, in the course of uttering those words, that they did not constitute the finest simile ever produced.

“Well, there are two massive outliers, for one. If you eliminate them, the effect is much smaller. And if you take into consideration the Gupta skew because the data were collected with the old reverberator, there’s nothing left at all.”

“Okay, fine,” I muttered. “Is there anything else in the data?”

“Sure, tons of things. Like, for example, there’s a statistically significant gamma reduction.”

“A gamma reduction? Are you sure? Or do you mean beta,” I asked.

“Definitely gamma,” said Rickles. “There’s nothing in the betas, deltas, or thetas. I checked.”

“Okay. That sounds potentially interesting and publishable. But I bet you’re going to tell me why we shouldn’t believe that result, either, right?”

“Well,” said Rickles, looking a bit self-conscious, “it’s just that it’s a pretty fine-grained analysis; you’re not really leaving a lot of observations when you slice it up that thin. And the weird thing about the gamma reduction is that it is essentially tantamount to accepting a null effect; this was Jayaraman’s point in that article in Statistica Splenda last month.”

“Sure, the Gerryman article, right. I read that. Forget the gamma reduction. What else?”

“There are quite a few schweizels,” Rickles offered, twisting the cap off a beer that had appeared out of the minibar under his desk.

I looked at him suspiciously. I suspected it was a trap; Rickels knew how much I loved Schweizel units. But I still couldn’t resist. I had to know.

“How many schweizels are there,” I asked, my hand clutching at the back of a nearby chair to help keep me steady.

“Fourteen,” Rickles said matter-of-factedly.

“Fourteen!” I gasped. “That’s a lot of schweizels!”

“It’s not bad,” said Rickles. “But the problem is, if you look at the B-trax, they also have a lot of schweizels. Seventeen of them, actually.”

“Seventeen schweizels!” I exclaimed. “That’s impossible! How can there be so many Schweizel units in one dataset!”

“I’m not sure. But… I can tell you that if you normalize the variables based on the Smith-Gill ratio, the effect goes away completely.”

There it was; the sound of the other shoe dropping. My heart gave a little cough–not unlike the sound your car engine makes in the morning when it’s cold and it wants you to stop provoking it and go back to bed. It was aggravating, but I understood what Rickles was saying. You couldn’t really say much about the Zimming Range unless your schweizel count was properly weighted. Still, I didn’t want to just give up on the schweizels entirely. I’d spent too much of my career delicately massaging schweizels to give up without one last tug.

“Maybe we can just say that the A-trax/Nuffton relationship is non-linear?” I suggested.

“Non-linear?” Rickles snorted. “Only if by non-linear you mean non-real! If it doesn’t survive Smith-Gill, it’s not worth reporting!”

I grudgingly conceded the point.

“What about the zifflons? Have you looked at them at all? It wouldn’t be so novel given Yehudah’s work, but we might still be able to get it into some place like Acta Ziffletica if there was an effect…”

“Tried it. There isn’t really any A-trax influence on zifflons. Or a B-trax effect, for that matter. There is a modest effect if you generate the Mish component for all the trax combined and look only at that. But that’s a lot of trax, and we’re not correcting for multiple Mishing, so I don’t really trust it…”

I saw that point too, and was now nearing despondency. Rickles had shot down all my best ideas one after the other. I wondered how I’d convince the department chair to let me keep my job.

Then it came to me in a near-blinding flash of insight. Near blinding, because I smashed my forehead on the overhead chandelier jumping out of my chair. An inch lower, and I’d have lost both eyes.

“We need to get that chandelier replaced,” I said, clutching my head in my hands. “It has no business hanging around in an office like this.”

“We need to get it replaced,” Rickles agreed. “I’ll do it tomorrow during my lunch hours.”

I knew that meant the chandelier would be there forever–or at least as long as Rickles inhabited the office.

“Have you tried counting the Dunams,” I suggested, rubbing my forehead delicately and getting back to my brilliant idea.

“No,” he said, leaning forward in his chair slightly. “I didn’t count Dunams.”

Ah-hah! I thought to myself. Not so smart are we now! The old boy’s still got some tricks up his sleeve.

“I think you should count the Dunams,” I offered sagely. “That always works for me. I do believe it might shed some light on this problem.”

“Well…” said Rickles, shaking his head slightly, “maaaaaybe. But Li published a paper in Psykometrika last year showing that Dunam counting is just a special case of Klein’s occidental protrusion method. And Klein’s method is more robust to violations of normality. So I used that. But I don’t really know how to interpret the results, because the residual is negative.”

I really had no idea either. I’d never come across a negative Dunam residual, and I’d never even heard of occidental protrusion. As far as I was concerned, it sounded like a made-up method.

“Okay,” I said, sinking back into my chair, ready to give up. “You’re right. This data… I don’t know. I don’t know what it means.”

I should have expected it, really; it was, after all, the dataset from hell. I was pretty sure my old RA had taken a quick jaunt through purgatory every morning before settling into the bench to run some experiments.

“I told you so,” said Rickles, putting his feet up on the desk and handing me a beer I didn’t ask for. “But don’t worry about it too much. I’m sure we’ll figure it out eventually. We probably just haven’t picked the right transformation yet. There’s Nordstrom, El-Kabir, inverse Zulu…”

He turned to his laptop and double-clicked an icon on the desktop that said “YouTube”.

“…or maybe you can just give the data to your new graduate student when she starts in a couple of weeks,” he said as an afterthought.

In the background, a video of a chimp and a puppy driving a Jeep started playing on a discolored laptop screen.

I mulled it over. Should I give the data to Josephine? Well, why not? She couldn’t really do any worse with it, and it would be a good way to break her will quickly.

“That’s not a bad idea, Rickles,” I said. “In fact, I think it might be the best idea you’ve had all week. Boy, that chimp is a really aggressive driver. Don’t drive angry, chimp! You’ll have an accid–ouch, that can’t be good.”

The

perils of digging too deep

Dr. Rickles the postdoc looked a bit startled when I walked into his office. He was eating a cheese sandwich and watching a chimp on a motorbike on his laptop screen.
“YouTube again?” I asked.
“Yes,” he said. “It’s lunch.”
“It’s 2:30 pm,” I said, pointing to my watch.
“Still my lunch hours.”
Lunch hours for Rickles were anywhere from 11 am to 4 pm. It depended on exactly when you walked in on him doing something he wasn’t supposed to; that was the event that marked the onset of Lunch.
“Fair enough,” I said. “I just stopped by to see how things were going.”
“Oh, quite well.” said Rickles. “Things are going well. I just found a video of a chimp and a squirrel riding a motorbike together. They aren’t even wearing helmets! I’ll send you the link.”
“Please don’t. I don’t like squirrels. But I meant with work. How’s the data looking.”
He shot me a pained look, like I’d just caught him stealing video game money from his grandmother.
“The data are TERRIBLE,” he said in all capital letters.
I wasn’t terribly surprised at that revelation; I’d handed Rickles the dataset only three days prior, taking care not to  tell him it was the dataset from hell. Rickles was the fourth or fifth person in the line of succession; the data had been handed down from postdoc to graduate student to postdoc for several years now. Everyone in the lab wanted to take a crack at it when they first heard about it, and no one in the lab wanted anything to do with it once they’d taken a peek. I’d given it to Rickles in part to teach him a lesson; he’d been in the lab for several weeks now and somehow still seemed happy and self-assured.
“Haven’t found anything interesting yet?” I asked. “I thought maybe if you ran the Flimflan test on the A-trax, you might get an effect. Or maybe if you jimmied the cryptos on the Borgatron…”
“No, no,” Rickles interrupted, waved me off. “The problem isn’t that there’s nothing interesting in the data; it’s that there’s too MUCH stuff. There are too MANY results. The story is too COMPLEX.”
That didn’t compute for me, so I just stared at him blankly. No one ever found COMPLEX effects in my lab. We usually stopped once we found SIMPLE effects.
Rickles was unimpressed.
“You follow what I’m saying, Guy? There are TOO-MANY-EFFECTS. There’s too much going on in the data.”
“I don’t see how that’s possible,” I said. “Keith, Maria, and Lakshmi each spent weeks on this data and found *nothing*.”
“That,” said Rickles, “is because Keith, Maria, and Lakshmi never thought to apply the Epistocene Zulu transform to the data.”
The Epistocene Zulu transform! It made perfect sense when you thought about it; so why hadn’t I ever thought about it? Who was Rickles cribbing analysis notes from?
“Pull up the data,” I said excitedly. “I want to see what you’re talking about.”
“Alright, alright. Lunch hours are over now anyway.”
He grudgingly clicked on the little X on his browser. Then he pulled up a spreadsheet that must have had a million columns in it. I don’t know where they’d all come from; it had only had sixteen thousand or so when I’d had the hard drives delivered to his office.
“Here,” said Rickles, showing me the output of the Pear-sampled Tea test. “There’s the A-trax, and there’s its Nuffton index, and there’s the Zimming Range. Look at that effect. It’s bigger than the zifflon correlation Yehudah’s group reported in Nature last year.”
“Impressive,” I said, trying to look calm and collected. But in my head, I was already trying to figure out how I’d ask the department chair for a raise once this finding was published. *Each point on that Zimming Range is worth at least $500*, I thought.
“Are there any secondary analyses we could publish alongside that,” I asked.
“Oh, I don’t think you want to publish *that*,” Rickles laughed.
“Why the hell not? It could be big! You just said yourself it was a giant effect!”
“Oh *sure*. It’s a big effect. But I don’t believe it for one second.”
“Why not? What’s not to like? This finding make’s Yehudah’s paper look like a corn dog!”
I recognized, in the course of uttering those words, that they did not constitute the finest simile ever.
“Well, there are two massive outliers, for one. If you eliminate them, the effect is much smaller. And if you take into consideration the Gupta skew because the data were collected with the old reverberator, there’s nothing left at all.”
“Okay, fine,” I muttered. “Is there anything else in the data?”
“Sure, tons of things. Like, for example, there’s a statistically significant Gamma reduction.”
“A gamma reduction? Are you sure? Or do you mean Beta,” I asked.
“Definitely gamma,” said Rickles. “There’s nothing in the betas, deltas, or thetas. I looked.”
“Okay. That sounds potentially interesting and publishable. But I bet you’re going to tell me why we shouldn’t believe that result, either, right?”
“Well,” said Rickles, looking a bit self-conscious, “it’s just that it’s a pretty fine-grained analysis; you’re not really leaving a lot of observations when you slice it up that thin. And the weird thing about the gamma reduction is that it is essentially tantamount to accepting a null effect; this was Jayaraman’s point in that article in *Statistica Splenda* last month.”
“Sure, the Gerryman article, right. Okay. Forget the gamma reduction. What else?”
“There are quite a few Schweizels,” Rickles offered, twisting the cap off a beer that had appeared out of the minibar under his desk.
I looked at him suspiciously. I suspected it was a trap; Rickels knew how much I loved Schweizel units. But I still couldn’t resist. I had to know.
“How many Schweizels are there,” I asked, my hand clutching at the back of a nearby chair to help me stay upright.
“Fourteen,” Rickles said matter-of-factedly.
“Fourteen!” I gasped. “That’s a lot of Schweizels!”
“It’s not bad,” said Rickles. “But the problem is, if you look at the B-trax, they also have a lot of Schweizels. Seventeen of them, actually.”
“Seventeen Schweizels!” I exclaimed. “That’s impossible! How can there be so many Schweizel units in one dataset!”
“I’m not sure. But… I can tell you that if you normalize the variables based on the Smith-Gill ratio, the effect goes away completely.”
There it was; the sound of the other shoe dropping. My heart gave a little cough–not unlike the sound your car engine makes in the morning when it’s cold and it wants you to go back to bed and stop stressing it out. It was aggravating, but I understood what Rickles was saying. You couldn’t really say much about the Zimming Range unless your Schweizel count was properly weighted. Still, I didn’t want to just give up on the Schweizels entirely.
“Maybe we can just say that the A-trax/Nuffton relationship is non-linear,” I proposed.
“Non-linear?” Rickles snorted. “Only if by non-linear you mean non-real! If it doesn’t survive Smith-Gill, it’s not worth reporting!”
I grudgingly conceded the point.
“What about the zifflons? Have you looked at them at all? It wouldn’t be so novel given Yehudah’s work, but we might still be able to get it into some place like *Acta Ziffletica* if there was an effect…”
“Tried it. There isn’t really any A-trax influence on zifflons. Or a B-trax effect, for that matter. There *is* a modest effect if you generate the Mish component for all the trax combined and look only at that. But that’s a lot of trax, and we’re not correcting for multiple Mishing, so I don’t really trust it…”
I saw that point too, and was now nearing despondency. Rickles had shot down all my best ideas one after the other. What else was left?
Then it came to me in a near-blinding flash of insight. *Near* blinding, because I smashed my forehead on the overhead chandelier jumping out of my chair. An inch lower, and I’d have lost both eyes.
“We need to get that chandelier replaced,” I said, clutching my head in my hands. “It has no business hanging around in an office like this.”
“We need to get it replaced,” Rickles agreed. “I’ll do it tomorrow during my lunch hours.”
I knew that meant the chandelier would be there forever–or at least as long as Rickles inhabited the office.
“Have you tried counting the Dunams,” I suggested, rubbing my forehead delicately and getting back to my brilliant idea.
“No,” he said, leaning forward in his chair slightly. “I didn’t count Dunams.”
Ah-hah! I thought to myself. Not so smart are we now! The old boy’s still got some tricks up his sleeve.
“I think you should count the Dunams,” I offered sagely. “That always works for me. I do believe it might shed some light on this problem.”
“Well…” said Rickles, shaking his head slightly, “maaaaaybe. But Li published a paper in Psychometrika last year showing that Dunam counting is just a special case of Klein’s occidental protrusion method. And Klein’s method is more robust to violations of normality. So I used that. But I don’t really know how to interpret the results, because the residual is *negative*.”
I really had no idea either. I’d never come across a negative Dunam residual, and I’d never even heard of occidental protrusion. As far as I was concerned, it sounded like a made-up method.
“Okay,” I said, sinking back into my chair, ready to give up. “You’re right. This data… I don’t know. I don’t know what it means.” I should have expected it, really; it was, after all, the dataset from hell. I was pretty sure my old RA had collected it after taking a quick jaunt through purgatory every morning.
“I told you so,” said Rickles, putting his feet up on the desk and handing me a beer I didn’t ask for. “But don’t worry about it too much. I’m sure we’ll figure it out eventually. We probably just haven’t picked the right transformation yet.”
He turned to his laptop and double-clicked an icon on the desktop that said “YouTube”.
“Maybe you can give the data to your new graduate student when she starts in a couple of weeks,” he said as an afterthought.
In the background, a video of a chimp and a puppy driving a Jeep started playing on a discolored laptop screen.
I mulled it over. Should I give the data to Josephine? Well, why not? She couldn’t really do any *worse* with it, and it *would* be a good way to break her will in a hurry.
“That’s not a bad idea, Rickles,” I said. “In fact, I think it might be the best idea you’ve had all week. Boy, that chimp is a really aggressive driver. Don’t drive angry, chimp! You’ll have an accid–ouch, that can’t be good.”

in brief…

Tuesday, May 4th, 2010

Some neat stuff from the past week or so:

  • If you’ve ever wondered how to go about getting a commentary on an article published in a peer-reviewed journal, wonder no longer… you can’t. Or rather, you can, but it may not be worth your trouble. Rick Trebino explains. [new to me via A.C. Thomas, though apparently this one's been around for a while.]
  • The data-driven life: A great article in the NYT magazine discusses the growing number of people who’re quantitatively recording the details of every aspect of their lives, from mood to glucose levels to movement patterns. I dabbled with this a few years ago, recording my mood, diet, and exercise levels for about 6 months. I’m not sure how much I learned that was actually useful, but if nothing else, it’s a fun exercise to play aroundwith a giant matrix of correlations that are all about YOU.
  • Cameron Neylon has an excellent post up defending the viability (and superiority) of the author-pays model of publication.
  • In typical fashion, Carl Zimmer has a wonderful blog up post explaining why tapeworms in Madagascar tell us something important about human evolution.
  • The World Bank, as you might expect, has accumulated a lot of economic data. For years, they’ve been selling it at a premium, but as of 2010 the World Development Indicators are completely free to access. via [via Flowing Data]
  • Every tried Jew’s Ear Juice? No? In China, you can–but not for long, if the government has its way. The NYT reports on efforts to eradicate Chinglish in public. Money quote:

“The purpose of signage is to be useful, not to be amusing,” said Zhao Huimin, the former Chinese ambassador to the United States who, as director general of the capital’s Foreign Affairs Office, has been leading the fight for linguistic standardization and sobriety.

CNS wrap-up

Thursday, April 22nd, 2010

I’m back from CNS in Montreal (actually, I’m not quite back; I’m in Ottawa for a few days–but close enough). Some thoughts about the experience, in no particular order, and with very little sense:

  • A huge number of registered attendees (basically, everyone from Europe who didn’t leave for Montreal early) couldn’t make it to the meeting because of that evil, evil Icelandic volcano. As a result, large swaths of posterboard were left blank–or would have been left blank, if not for the clever “Holy Smokes! So-and-so can’t be here…” notes taped to them. So that was really too bad; aside from the fact that the Europeans missed out on the meeting, which kind of sucks, there was a fair amount of chaos during the slide and symposium sessions as speakers were randomly shuffled around. I guess it’s a testament to the organizers that the conference went off relatively smoothly despite the loss of a large chunk of the attendance.
  • The symposium I chaired went well, as far as I can tell. Which is to say, no one streaked naked through the hall, no one went grossly over time, the audience hall was full, and the three talks I got to watch from the audience were all great. I think my talk went well too, but it’s harder to say. In theory, you should be able to tell how these things go based on the ratio of positive to negative feedback you get. But since people generally won’t tell you if they thought your talk sucked, you’re usually stuck trying to determine whether people are giving you well-I-didn’t-really-like-it-but-I-don’t-want-you-to-feel-bad compliments, or I-really-liked-it-and-I’m-not-even-lying-to-your-face compliments. In any case, good or bad reception, I think the topic is a really important one, and I’m glad the symposium was well attended.
  • I love Montreal. As far as I’m concerned they could have CNS in Montreal every year and I wouldn’t complain. Well, maybe I’d complain a little. But only about unimportant things like the interior decoration of the hotel lobby.
  • Speaking of which, I liked the Hilton Bonaventure and all, but the place did remind me a lot of a 70s porn set. All it’s missing are some giant ferns in the lobby and a table lined with cocaine next to the elevators. (You can probably tell that my knowledge of 70s porn is based entirely on watching two-thirds of Boogie Nights once). Also, what the hell is on floors 2 through 12 of Place Bonaventure? And how can a hotel have nearly 400 rooms, all on the same (13th) floor!?
  • That Vietnamese place we had lunch at on Tuesday, which apparently just opened up, isn’t going to last long. When someone asks you for “brown rice”, they don’t mean “white rice with some red food dye stirred in”.
  • Apparently, Mike X. Cohen is not only the most productive man in cognitive neuroscience, but also a master of the neuroimaging haiku (admittedly, a niche specialty).
  • Sushi and baklava at a conference reception? Yes please!
  • The MDRS party on Monday night was a lot of fun, though the downstairs room at the bar was double-booked. I’m sure the 20-odd people at salsa dancing night were a bit surprised, and probably not entirely appreciative, when 100 or so drunken neuroscientists collectively stumbled downstairs for a free drink, hung out for fifteen minutes, then disappeared upstairs again. Other than that–and the $8 beers–a good time was had.
  • Turns out that assortment of vegetables that Afghans call an Afghan salad is exactly what Turks call a Turkish salad and Israelis call an Israeli salad. I guess I’m not surprised that everyone in that part of the world uses the same four or five ingredients in their salad, but let’s not all rush to take credit for what is basically some cucumber, tomato, and parsley in a bowl. That aside, dinner was awesome. And I wish there were more cities full of restaurants that let you bring your own wine.

  • The talks and posters were great this year. ALL OF THEM. If I had to pick favorites, I guess I really liked the symposium on perceptual decision-making, and several of the posters in the reward/motivation session on Sunday or Monday afternoon. But really, ALL OF THEM WERE GREAT. So let’s all give ourselves giant gold medals with pictures of brains on them. And then… let’s melt down those medals, sell the gold, and buy some scanners with the money.

the grand canada tour, 2010 edition

Monday, April 12th, 2010

Blogging will be slow(er than normal) for the next couple of weeks. On Wednesday I’m off on a long-awaited Grand Tour of Canada, 2010 edition. The official purpose of the trip is the CNS meeting in Montreal, but seeing as I’m from Canada and most of my family is in Toronto and Ottawa, I’ll be tacking on a few days of R&R at either end of the trip, so I’ll be gone for 10 days. By R&R I mean that I’ll be spending most of my time in Toronto at cheap all-you-can-eat sushi restaurants, and most of my time in Ottawa sleeping in till noon in my mom’s basement.  So really, I guess my plan for the next two weeks is to turn seventeen again.

While I’m in Ottawa, I’ll also be giving a talk at Carleton University. I’d like to lump this under the “invited talks” section of my vita–you know, just to make myself seem slightly more important (being invited somewhere means people actually want to hear you say stuff!)–but I’m not sure it counts as “invited” if you invite yourself to give a talk somewhere else. Which is basically what happened; I did my undergraduate degree at Carleton, so when I emailed my honors thesis advisor to ask if I could give a talk when I was in town, he probably felt compelled to humor me, much as I know he’d secretly like to say no (sorry John!). At any rate, the talk will be closely based on this paper on the relation between personality and word use among bloggers. Amazingly enough, it turns out you can learn something (but not that much) about people from what they write on their blogs. It’s not the most exciting conclusion in the world, but I think there are some interesting results hidden away in there somewhere. If you happen to come across any of them, let me know.

my new favorite blog

Tuesday, April 6th, 2010

…teaches you How To Write Badly Well. For instance, if you want to write badly well, you must Refuse to leave the present tense:

I sit at my desk and remember how, years ago, I wonder what my life will be like when I am fifty, which I am now. I’m imagining that I’m living in a big house, I remember as I sit in my one-bedroom apartment. Now I pour myself a drink and cast my mind back to a time when I’m full of hope and passion which is never to be extinguished, as it is now.

‘What am I doing?’ I mutter to myself, taking a sip of my drink. In my memory, I’m seven years old, sitting in the highest branches of a tree which is being planted a hundred years before I am born. Now, though, the tree is long dead. I’m chopping it down at the age of twenty and thinking about when it is supporting my weight at the age of seven. I look at my watch.

‘Late,’ I mutter to myself. It is eight; the retrospective is just starting, half an hour ago.