[citation needed]

There’s a long tradition in the academic blogosphere (and the offlinesphere too, I presume) of complaining that academia is a pyramid scheme. In a strict sense, I guess you could liken academia to a pyramid scheme, inasmuch as there are fewer open positions at each ascending level, and supply generally exceeds demand. But as The Prodigal Academic points out in a post today, this phenomenon is hardly exclusive to academia:

I guess I don’t really see much difference between academic job hunting, and job hunting in general. Starting out with undergrad admissions, there are many more qualified people for desirable positions than available slots. Who gets those slots is a matter of hard work (to get qualified) and luck (to be one of the qualified people who is “chosen”). So how is the TT any different from grad school admissions (in ANY prestige program), law firm partnership, company CEO, professional artist/athlete/performer, attending physician, investment banking, etc? The pool of qualified applicants is many times larger than the number of slots, and there are desirable perks to success (money/prestige/fame/security/intellectual freedom) making the supply of those willing to try for the goal pretty much infinite.

Maybe I have rose colored glasses on because I have always been lucky enough to find a position in research, but there are no guarantees in life. When I was interviewing in industry, I saw many really interesting jobs available to science PhD holders that were not in research. If I hadn’t gone to National Lab, I would have been happy to take on one of those instead. Sure, my life would be different, but it wouldn’t make my PhD a waste of time or a failed opportunity.

For the most part, I agree with this sentiment. I love doing research, and can’t imagine ever voluntarily leaving academia. But If I do end up having to leave–meaning, if I can’t find a faculty position when I go on the job market in the next year or two–I don’t think it’ll be the end of the world. I see job ads in industry all the time that looks really interesting, and on some level, I think I’d find almost any job that involves creative analysis of very large datasets (which there are plenty of these days!) pretty gratifying. And no matter what happens, I don’t think I’d ever view the time I’ve spent on my PhD and postdoc training as a waste of time, for the simple reason that I’ve really enjoyed most of it (there are, of course, the nasty bits, like writing the Nth chapter of a dissertation–but those are transient, fortunately). So in that sense, I think all the talk about academia being a pyramid scheme is kind of silly.

That said, there is one sticking point to the standard pyramid scheme argument I do agree with, which is that, when you’re starting out as a graduate student, no one really goes out of their way to tell you what the odds of getting a tenure-track faculty position actually are (and they’re not good). The problem being that most of the professors that prospective graduate students have interacted with, either as undergraduates, or in the context of applying to grad school, are precisely those lucky souls who’ve managed to secure faculty positions. So the difficulty of obtaining the same type of position isn’t always very salient to them.

I’m not saying faculty members lie outright to prospective graduate students, of course; I don’t doubt that if you asked most faculty point blank “what proportion of students in your department have managed to find tenure-track positions,” they’d give you an honest answer. But when you’re 22 or 23 years old (and yes, I recognize some graduate students are much older, but this is the mode) and you’re thinking of a career in research, it doesn’t always occur to you to ask that question. And naturally, departments that are trying to recruit your services are unlikely to begin their pitch by saying, “in the past 10 years, only about 12% of our graduates have gone on to tenure-track faculty positions”. So in that sense, I don’t think new graduate students are always aware of just how difficult it is to obtain an independent research position, statistically speaking. That’s not a problem for the (many) graduate students who don’t really have any intention of going into academia anyway, but I do think a large part of the disillusionment graduate students often experience is about the realization that you can bust your ass for five or six years working sixty hours a week, and still have no guarantee of finding a research job when you’re done. And that could be avoided to some extent by making a concerted effort to inform students up front of the odds they face if they’re planning on going down that path. So long as that information is made readily available, I don’t really see a problem.

Having said that, I’m now going to blatantly contradict myself (so what if I do? I am large! I contain multitudes!). You could, I think, reasonably argue that this type of deception isn’t really a problem, and that it’s actually necessary. For one thing, the white lies cut both ways. It isn’t just faculty who conveniently forget to mention that relatively few students will successfully obtain tenure-track positions; many graduate students nod and smile when asked if they’re planning a career in research, despite having no intention of continuing down that path past the PhD. I’ve occasionally heard faculty members complain that they need to do a better job filtering out those applicants who really truly are interested in a career in research, because they’re losing a lot of students to industry at the tail end. But I think this kind of magical mind-reading filter is a pipe dream, for precisely the reasons outlined above: if faculty aren’t willing to begin their recruitment speeches by saying “most of you probably won’t get research positions even if you want them,” they shouldn’t really complain when most students don’t come right out and say “actually, I just want a PhD because I think it’ll be something interesting to do for a few years and then I’ll be able to find a decent job with better hours later”.

The reality is that the whole enterprise may actually require subtle misdirection about people’s intentions. If every student applying to grad school knew exactly what the odds of getting a research position were, I imagine many fewer people who were serious about research would bother applying; you’d then get predominantly people who don’t really want to do research anyway. And if you could magically weed out the students who don’t want to do research, then (a) there probably wouldn’t be enough highly qualified students left to keep research programs afloat, and/or (b) there would be even more candidates applying for research positions, making things even harder for those students who do want careers in research. There’s probably no magical allocation of resources that optimizes everyone’s needs simultaneously; it could be that we’re more or less at a stable equilibrium point built on little white lies.

tl;dr : I don’t think academia is really a pyramid scheme; more like a giant cesspool of little white lies and subtle misinformation that indirectly serves most people’s interests. So, basically, it’s kind of like most other domains of life that involve interactions between many groups of people.

That’s a terribly constructed title, I know, but bear with me. A couple of weeks ago I blogged about a courtroom case in Tennessee where the defense was trying to introduce fMRI to the courtroom as a way of proving the defendant’s innocence (his brain, apparently, showed no signs of guilt). The judge’s verdict is now in, and…. fMRI is out. In United States v. Lorne Semrau, Judge Pham recommended that the government’s motion to exclude fMRI scans from consideration be granted. That’s the outcome I think most respectable cognitive neuroscientists were hoping for; as many people associated with the case or interviewed about it have noted (and as the judge recognized), there just isn’t a shred of evidence to suggest that fMRI has any utility as a lie detector in real-world situations.

The judge’s decision, which you can download in PDF form here (hat-tip: Thomas Nadelhoffer), is really quite elegant, and worth reading (or at least skimming through). He even manages some subtle snark in places. For instance (my italics):

Regarding the existence and maintenance of standards, Dr. Laken testified as to the protocols and controlling standards that he uses for his own exams. Because the use of fMRI-based lie detection is still in its early stages of development, standards controlling the real-life application have not yet been established. Without such standards, a court cannot adequately evaluate the reliability of a particular lie detection examination. Cordoba, 194 F.3d at 1061. Assuming, arguendo, that the standards testified to by Dr. Laken could satisfy Daubert, it appears that Dr. Laken violated his own protocols when he re-scanned Dr. Semrau on the AIMS tests SIQs, after Dr. Semrau was found “deceptive” on the first AIMS tests scan. None of the studies cited by Dr. Laken involved the subject taking a second exam after being found to have been deceptive on the first exam. His decision to conduct a third test begs the question whether a fourth scan would have revealed Dr. Semrau to be deceptive again.

The absence of real-life error rates, lack of controlling standards in the industry for real-life exams, and Dr. Laken’s apparent deviation from his own protocols are negative factors in the analysis of whether fMRI-based lie detection is scientifically valid. See Bonds, 12 F.3d at 560.

The reference here is to the fact that Laken and his company scanned Semrau (the defendant) on three separate occasions. The first two scans were planned ahead of time, but the third apparently wasn’t:

From the first scan, which included SIQs relating to defrauding the government, the results showed that Dr. Semrau was “not deceptive.” However, from the second scan, which included SIQs relating to AIMS tests, the results showed that Dr. Semrau was “being deceptive.” According to Dr. Laken, “testing indicates that a positive test result in a person purporting to tell the truth is accurate only 6% of the time.” Dr. Laken also believed that the second scan may have been affected by Dr. Semrau’s fatigue. Based on his findings on the second test, Dr. Laken suggested that Dr. Semrau be administered another fMRI test on the AIMS tests topic, but this time with shorter questions and conducted later in the day to reduce the effects of fatigue. … The third scan was conducted on January 12, 2010 at around 7:00 p.m., and according to Dr. Laken, Dr. Semrau tolerated it well and did not express any fatigue. Dr. Laken reviewed this data on January 18, 2010, and concluded that Dr. Semrau was not deceptive. He further stated that based on his prior studies, “a finding such as this is 100% accurate in determining truthfulness from a truthful person.”

I may very well be misunderstanding something here (and so might the judge), but if the positive predictive value of the test is only 6%, I’m guessing that the probability that the test is seriously miscalibrated is somewhat higher than 6%. Especially since the base rate for lying among people who are accused of committing serious fraud is probably reasonably high (this matters, because when base rates are very low, low positive predictive values are not unexpected). But then, no one really knows how to calibrate these tests properly, because the data you’d need to do that simply don’t exist. Serious validation of fMRI as a tool for lie detection would require assembling a large set of brain scans from defendants accused of various crimes (real crimes, not simulated ones) and using that data to predict whether those defendants were ultimately found guilty or not. There really isn’t any substitute for doing a serious study of that sort, but as far as I know, no one’s done it yet. Fortunately, the few judges who’ve had to rule on the courtroom use of fMRI seem to recognize that.