Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)
EUGENE GARFIELD: Refereeing and Peer Review. Part 2. The Research on Refereeing and Alternatives in the Present System
19 GARFIELD: REFEREEING AND PEER REVIEW, PART 1 Lee Brazis, University of Cincinnati, Ohio, mailed abstracts of an empirical study on astrology to 282 members of the American Psychological Association. 28 They were asked to rate the design of the paper. Half were sent an abstract that reflected a conclusion confirming commonly held scientific attitudes toward astrology; the other half received an identical abstract, except that it included a conclusion that ran counter to scientific beliefs. The former was rated by most referees as better designed and having more significance for future research. The latter, which contradicted common wisdom, was rated as flawed. When Zuckerman and Merton examined the selection of articles for the Physical Review, they found that papers by physicists of great repute affiliated with prestigious institutions were more likely to be exempted entirely from the refereeing process. Their papers were accepted and published more quickly than papers by lesser known physicists. 1 2 And in a large-scale study of papers submitted to physics journals, Gordon reported a strong bias in referees from major universities toward papers by authors who were also from large, well-known universities. 2 9 Lock, however, found no evidence of referee bias in a study of 1,558 manuscripts submitted to BMJ between January and August 1979. The study was published in his book A Difficult Balance: Editorial Peer Review in Medicine. 15 Of the 246 external referees who were sent manuscripts by BMJ, 143 held academic positions, while the rest had nonacademic affiliations; yet the proportion of papers recommended for acceptance did not differ from one group to the other. 1 5 (p. 56-71) Moreover, regardless of the affiliations of both referee and author, Lock said that referees judged manuscripts "to an equal standard." 15 (p. 61) Suggestions for Improvement A few years ago, Norton D. Zinder, Rockefeller University, New York City, sent me the text of a talk he gave to the Society of Editors in 1969, when he was an associate editor of Virology. 30 Tongue partially in cheek, Zinder asked, "What would be so terrible if there were no refereeing of scientific papers?... As we now operate, with [the] restriction of publication by reviewing, the number of publications becomes a thing in itself.... If we were to cease refereeing papers,...there'd be little bar [to publication, and] quality might reassert its role, since there'd be less pressure to have long lists of publications." 3 0 The Perloffs write that the "caveat emptor approach [of having no refereeing system at all] might be viewed as a nod to the free market of ideas. Let millions of flowers bloom." 2 6 Some may feel that the continued growth of the literature may lend support to these views. However, others, including myself, believe that a few non-refereed publications can exist only because the refereed journals set the standards for all the others. I believe that most scientists would agree that if something is indeed shown to be wrong with refereeing, an attempt should be made to repair the system, rather than to abandon it. Unfortunately, with little or no solid, systematic evidence of refereeing's deficiencies, most suggestions for improvement are as conjectural as the ills they are meant to cure. Among the most discussed options —one that is already prevalent among sociology journals —is that of double-blind refereeing, also called reciprocal anonymity, in which neither the authors nor referees are aware of the others' identities. There is precedent for author anonymity: David A. Kronick, professor, medical bibliography. University of Texas Health Science Center at San Antonio, notes that "maintaining the anonymity of the author was a standard practice in the prize essay competitions (a sort of early form of sponsored research) of eighteenth-century scientific societies, which had elaborate devices to maintain the anonymity of contributors." 3 1 The rationale behind double-blind refereeing, as was pointed out in an ap-
