Glenn Ellison, from the MIT economics department, has a paper documenting and discussing the fact that “over the past decade there has been a decline in the fraction of papers in top economics journals written by economists from the highest-ranked economics departments”. Glenn argues that this trend is due to fact that “the necessity of going through the peer-review process has lessened for high status authors: in the old days peer-reviewed journals were by far the most effective means of reaching readers, whereas with the growth of the Internet high-status authors can now post papers online and exploit their reputation to attract readers.” He elaborates:

Journals serve two roles: they disseminate papers and provide quality certification. The Internet aids dissemination in many ways: papers are posted to authors’ websites and working paper archives; e-mail is used to inform potential readers about papers; Internet search tools help readers find papers; journals have been made more accessible; and so on. Many of these are substitutes for traditional journal dissemination, so the primary role of journals may increasingly be to provide quality certification. A shift in the role of journals could lead authors from top schools to withdraw from publishing in many journals for two reasons. First, highly-regarded and highly-visible authors will be able to make their work widely known (and widely read) without publishing it in journals. Such authors receive diminished dissemination benefits from journal publication. Second, highly-regarded authors may traditionally have been publishing in journals mostly for the dissemination benefits.

I believe that these observation are not just relevant to economics but may apply to our field as well. Established researchers have diminished incentive to publish in peer-reviewed journals, since their output will be noticed and cited wherever it appears. Of course, they have always had the venue of invited contributions to edited volumes (which are all too often only open to well-established or at least well-connected researchers). But with the internet and repositories like the Semantics Archive and LingBuzz, it has become even easier to make new research widely available. And if you’re a big cheese, everyone will read your new article on the archive.

Alan Prince, in a comment on phonoloblog discussing the announcement of our journal, wrote:

“Question. What is the marginal utility of ‘rigorous peer-review’? Specifically, is the difference in quality between article and a serious draft generally large enough to be worth the investment of time, energy, and stress on the part of all concerned? The alternative to a journal is simply an open archive, with matters of quality & improvement left up to the twin pressures of self-respect and the marketplace of ideas. How well does this work in comparison with the reviewing system now in place? This sounds like the kind of thing that social scientists would study (rigorously) — but, rigor aside, it might be interesting to gather some opinions.

Comment. In promotion cases, the publication list is a proxy for impact on the field (itself a proxy for value). With wholesale electronification and effective search engines, impact can be more directly assessed via citation indexing. It strikes me as possible that official use of the humanist hallmark of ‘publication by a major press’ [or other reputable entity] will dwindle as this becomes apparent.”

So, what is the role of peer-review nowadays? Why are we starting a new peer-reviewed journal?

• Publication in high quality, peer-reviewed journals is still the one of the main stamps of approval for research output for institutional recognition of a researcher’s status, as in tenure and promotion cases.
• In an increasingly larger field, such publications are also one of the primary ways in which new and promising researchers come to the attention of the field, building their reputation that way. (This is not to deny that professional networking is essential to getting noticed as well).
• These two reasons apply mainly to young, up-and-coming researchers. But we believe that the established researcher will also benefit from peer-review. The problem has been that the very real benefits of peer-review (in-depth comments from competent experts, placement in the spotlight, etc.) are often swamped by the pains associated with it, specifically the long time between the first circulated draft and final publication. We hope that with our efficiencies, we can remove many of the obstacles that keep established researchers from submitting their work to peer-review.

The importance of peer-review is often used by traditional publishers as an argument that open-access journals are a threat to the time-tested system of science. But that is a spurious argument. The role of traditional publishing houses in peer-review is (almost) entirely clerical — a role that is now made obsolete by modern journal management software. All the real work (reviewing, editing, revising, etc.) has always been done by scientists themselves. We’re just taking the last logical step.

The only factor that is seriously impacted in an interesting way by the advent of journals such as ours is that there is no technical limit on the number of articles that can be published (L&P, JoS, NLS all have a limited number of pages they can fill per year). We believe, however, that there will still be a significant limit on what gets published in journals such as ours: the quality of the submitted articles. Just because we can publish a practically unlimited number of pages doesn’t mean that we should and will. The success of our journal will depend on the high quality of what we publish and we will not compromise in that regard.

Two final remarks:

• As should be obvious, we are setting S&P up quite conservatively as a traditional peer-reviewed journal. This does not mean that we are not open to some amount experimentation, exploring some kind of open peer review (submitted articles are open for comments from everyone at the same time as they are being reviewed by designated members of the Editorial Board) and other ideas. But we will wait until after the journal has been established as a serious venue for work in semantics and pragmatics, before we consider such innovations. And then we will move deliberately and in consultation with our Advisory Board and the research community as a whole.
• It is important to distinguish the narrow sense of peer-review that we’ve been discussing here and the wide sense in which all scientific writing is peer-reviewed because other scientists read it, criticize it, utilize it in their own work, etc. In that wider sense, there is no crisis of peer-review at all. The question is whether the narrow sense of peer-review can and should still play its traditional role of highlighting work that should receive serious attention. We believe so.

PS. In the aftermath of Glenn’s paper, the topic has been discussed in a number of places:

• Inside Higher Ed
• Adventures in Ethics and Science
• Lemmings