[citation needed]

I like the PLoS journals, including PLoS ONE, a lot. But it drives me a little bit crazy that the list of PLoS ONE subjects includes things like Non-Clinical Health, Nutrition, and Science Policy, while perfectly respectable subjects like Psychology, Economics, and Political Science are nowhere to be found (note: I’m not saying there’s anything wrong with Nutrition, just that there’s also nothing wrong with Psychology).

I can sort of understand the rationale; PLoS ONE is supposed to be a science journal, and I imagine the editors feel that if they opened up the door to the aforementioned categories, some of the submissions they’d start receiving would have tenuous or nonexistent relationships to anything that you could call science. But in practice, PLoS ONE already does take articles in all of those subjects–and many others. And what then happens, no doubt, is that the editorial board has epic battles over which of the 40-odd existing subjects is going to become the proud beneficiary of a completely unrelated article.

I imagine it goes down something like this:

Editor A: Look, “Patriarchal principles of pop music in a post-Jacksonian era” is clearly an Epidemiology article. It’s going under Public Health and Epidemiology.

Editor B: Don’t be a fool. There isn’t a single word in the paper about health or disease. You’d know that if you’d bothered to read it. It obviously belongs under Mental Health.

Editor A: Absolutely not. Infectious Diseases, Pediatrics and Child Health, or Anesthesiology and Pain Management. Pick one. Final offer.

Editor B: No. But I’ll tell you what. Send it back to the authors, ask them to add a section on the influence of barbiturates and opiates on modern composition, and then we’ll stick it under Pharmacology.

Editor A: Deal.

Lest you think I’m making shit up exaggerating, witness exhibit A: a paper published today by Araújo et al entitled “Tactical Voting in Plurality Elections”. To be fair, I don’t know anything about tactics, voting, plurality, or elections, so I can’t tell you if the paper is any good or not. It looks interesting, but I don’t understand much more than the abstract.

What I can tell you though with something approaching certainty is that the paper has absolutely nothing to do with Neuroscience–which is one of the categories it’s filed under (the other is Physics, which it also seems to bear no relation to, save for the fact that the authors are physicists). It doesn’t mention the words ‘brain’, ‘neuro-’, ‘neural’, or ‘neuron’ anywhere in the text, which is pretty much a necessary condition for a neuroscience article in my book. The only conceivable link I can think of is that it’s a paper about voting, and voting is done by people, and people have brains. But that’s not very compelling. Really, it should go under Political Science, or Economics, or Applied Statistics, or even a catch-all category like Social Sciences. Except that none of those exist.

Pretty please, PLoS ONE, can we get a Social Sciences section?

This one’s a bit of a head-scratcher. Thomson-Reuters just released its 2009 Journal Citation Report–essentially a comprehensive ranking of scientific journals by their impact factor (IF). The odd part, as reported by Bob Grant in The Scientist, is that the journal with the second-highest IF is Acta Crystallographica – Section A–ahead of heavyweights like the New England Journal of Medicine. For perspective, the same journal had an IF of 2.051 in 2008. The reason for the jump?

A single article published in a 2008 issue of the journal seems to be responsible for the meteoric rise in the Acta Crystallographica – Section A‘s impact factor. “A short history of SHELX,” by University of Göttingen crystallographer George Sheldrick, which reviewed the development of the computer system SHELX, has been cited more than 6,600 times, according to ISI. This paper includes a sentence that essentially instructs readers to cite the paper they’re reading — “This paper could serve as a general literature citation when one or more of the open-source SHELX programs (and the Bruker AXS version SHELXTL) are employed in the course of a crystal-structure determination.” (Note: This may be a good way to boost your citations.)

Setting aside the good career advice (and yes, I’ve made a mental note to include the phrase “this paper could serve as a general literature citation…” in my next paper), it’s perplexing that Thomson-Reuters didn’t downweight Acta Crystallographica‘s IF considerably given the obvious outlier. There’s no question they would have noticed that the second-ranked journal was only there in virtue of one article, so I’m curious what the thought process was. Perhaps the deliberation went something like this:

Thomson-Reuters statistician A: We need to take it out! We can’t have a journal with an impact factor of 2 last year beat out the NEJM!

Thomson-Reuters statistician B: But if we take it out, it’ll look like we tampered with the IF!

TRS-A: But we already tamper with the IF! No one knows how we come up with these numbers! Sometimes we can’t even replicate our own results ourselves! And anyway, it’s really not a big deal if we just leave the article in; scientists know better than to think Acta Crystallographica is the second most influential science journal on the planet. They’ll figure it out.

TRS-B: But that’s like asking them to just disregard our numbers! If you’re supposed to ignore the impact factor in cases where it contradicts your perception of journal quality, what’s the point of having an impact factor at all?

TRS-A: Beats me.

Like many (most?) scientists, I’ve often wondered what a world without Elsevier would look like. Not just Elsevier, mind you; really, the entire current structure of academic publishing, which revolves around a gatekeeper model where decisions about what gets published where are concentrated in the hands of a very few people (typically, an editor and two or three reviewers). The way scientists publish papers really hasn’t kept up with the pace of technology; the tools we have these days allow us, in theory, to build systems that support the immediate and open publication of scientific findings, which could then be publicly reviewed, collaboratively filtered, and quantitatively evaluated using all sorts of metrics that just aren’t available in a closed system.

One particularly compelling vision is articulated by Niko Kriegeskorte, who presents a beautiful schematic of one potential approach to the future of academic publishing. I’m a big fan of Niko’s work (see e.g., this, this, or this)–almost everything he publishes is great, and his articles consistently feature absolutely stunning figures–and these ideas are no exception. The central motif, which I’m wholly sympathetic to, is to eliminate gatekeepers and promote open review and evaluation. Instead of a secret cabal small group of other researchers (and potential competitors) making behind-the-scenes decisions about whether to accept or reject your paper, you’d publish your findings online in a centralized repository as soon as you felt it was ready for prime time. At that point, the broader community of researchers would set about evaluating, rating, and commenting on your work. Crucially, all of the reviews would also be made public (either in signed or anonymous form), so that other researchers could evaluate not only the work itself, but also the responses to it. Reviews would therefore count as a form of publication, and one can then imagine all sorts of sophisticated metrics that could take into account not only the reception of one’s publications, but also the quality and nature of the reviews themselves, the quality of one’s own ratings of others’ work, and so on. Schematically, it looks like this:

Anyway, that’s just a cursory overview; Niko’s clearly put a lot of thought into developing a publishing architecture that overcomes the drawbacks of the current system while providing researchers with an incentive to participate (the sociological obstacles are arguably greater than the technical ones in this case). Well, at least in theory. Admittedly, it’s always easier to design a complex system on paper than to actually build it and make it work. But you have to start somewhere, and this seems like a pretty good place.