Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)
EUGENE GARFIELD: Refereeing and Peer Review. Part 2. The Research on Refereeing and Alternatives in the Present System
17 GARFIELD: REFEREEING AND PEER REVIEW, PART 1 to 25 percent of the papers submitted to them. Biology journals rejected about 30 percent of the papers they received. Journals in experimental and physiological psychology had a rejection rate of over 50 percent, while sociology journals were over 80 percent and history journals hovered at 90 percent. Stephen Lock, editor, British Medical Journal (BMJ), made an observation that has also been noted by others who have read the study. He wrote that "the more humanistically oriented the journal, the higher the rate of [rejection]; the more experimentally and observationally oriented, with an emphasis on rigour of observation and analysis, the lower the rate of rejection." 1 5 (p. 17) Zuckerman and Merton also reported that the editorial staffs attitude concerning its own errors in judgment constitutes an often-overlooked factor influencing acceptance rates. 1 2 Although editors and referees want to avoid errors in judgment altogether, they recognize that they cannot be infallible; thus, since they must make mistakes, they tend to have preferences for the kind of mistakes they are willing to risk. The staffs of some journals —notably those prestigious journals with high rejection rates —seem more willing to reject "unorthodox" manuscripts that the wider community of scholars might eventually consider important, rather than to run the risk of publishing a substandard work. The staffs of low-rejection journals, on the other hand, apparently prefer to publish the occasional work that doesn't measure up, rather than reject a paper that later turns out to be significant. 1 2 The Research A research front consists of a group of current papers that, together, cite one or more of a cluster of older, core publications. Since I referred earlier 1 to the paucity of empirical research on refereeing and peer review and the abundance of anecdote and opinion on the subject, one may wonder how a research front of any size might be generated on this subject. But even a large anecdotal literature, through repeated citations of previous anecdotal literature, as well as reputable studies, can form a pseudo-research front. Only a careful analysis of the core and citing literature can determine the nature and extent of the research front —even when very useful core review papers can be found. Since the literature on peer review and refereeing is vast, at the end of Part 2 of this essay I have added a selected bibliography of publications not mentioned in the text. The 1983 ISI® research front entitled "Objectivity of reviewers in peer review" (#83-8291) consists of but 2 core papers and 12 citing papers. One core paper is the highly controversial 1982 study by Douglas P. Peters, University of North Dakota, Grand Forks, and Stephen J. Ceci, Cornell University, Ithaca, New York. 1 6 The other core paper is a 1982 editorial by Lock, entitled "Peer review weighed in the balance." 1 7 In it Lock discusses the conclusions drawn by Peters and Ceci and details some of the flaws in their study. In spite of these problems, however. Lock believes that Peters and Ceci have underscored some shortcomings within the system. Most of the recommendations Lock makes for improving refereeing —particularly doubleblind review —are discussed in detail below. Peters and Ceci This controversial study involved the resubmission of 12 psychology articles —published by authors from prestigious and highly productive departments —to the journals that originally published them. 1 8 Peters and Ceci became interested in doing the study after reading about an informal experiment conducted by Los Angeles free-lance writer Chuck Ross. 1 9 He reports having submitted the untitled, untyped manuscript of Polish-bom US literary author Jerzy Kosinski's novel Steps 2 0 under a pseudonym to publishers and literary agents to see if "unknown" authors re-
