Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)
EUGENE GARFIELD: Refereeing and Peer Review. Part 1. Opinion and Conjecture on the Effectiveness of Refereeing
11 GARFIELD: REFEREEING AND PEER REVIEW, PART 1 nice ring of fairness to it.... However,... when a group of colleagues is permitted to have [their] comments taken as some kind of gospel, [they] are no longer peers but quite definitely superiors insofar as power and influence go." 4 3 It is in answer to just this kind of criticism, Harnad reports, that BBS is conducting an internal, statistical study of, among other things, the relationships among anonymity, referees' ratings of manuscripts, and authors' ratings of the usefulness of referee reports. 2 4 Another criticism of the system is of the "Newcomb variety." I have often referred to the career of Simon Newcomb, who proved conclusively —just months before the Wright Brothers took off from the sands of Kitty Hawk —that a flying machine was impossible. 4 4. 45 Sometimes this type of rejection is the result of referees who are hostile to innovative ideas or to those that clash with their own. 4 1 We don't know how often thoughtful, conscientious scientists —in good faith and in keeping with currently accepted theory —rendered an opinion concerning the implausibility of a given idea or theory, only to see that theory become the basis of a dramatic paradigm shift. Still, referees and journal editors should not consider such rejection experience as sufficient reason for extending some kind of "publication carte blanche" to would-be authors who want to prove, for example, that perpetualmotion machines are possible. I continue to be in favor of refereeing that prevents the publication of intellectual atrocities, including papers with inadequate documentation. For those articles straddling the border between science and speculation, there exist publications such as Speculations in Science and Technology, which was started specifically as a forum for the publication of ideas lacking support "in established theoretical and experimental work," according to an article by founder William M. Honig, senior lecturer in the physical sciences and engineering, Western Australian Institute of Technology, Perth, in the Sciences. 4 6 Refereeing and Garfield's Uncertainty Principle It is easy to "prove" on the basis of anecdotal evidence that the refereeing system doesn't work. From the hundreds of published Citation Classics® commentaries —such as those written by Oscar Buneman, Stanford University, California, 4 7 and Hans Lineweaver, US Department of Agriculture, Washington, DC 4 8 —or in correspondence with their authors, we know that dozens of significant papers have been rejected by some journals for various reasons. Some of these reasons might be described as "N-I-H," that is, "not invented here." Nevertheless, much scientific quackery is exposed by careful, insightful, constructive refereeing, and this far outweighs the ideas that have allegedly been suppressed because of referees who would not give them a chance to see the light of day. A scientist's appreciation of the collaborative, communal goal of refereeing —protecting science and the public from errors and inferior work —varies according to a host of factors, including the scientist's age, status, and temperament. Famous, tenured, or established researchers may be better able to weather the occasional rejection notice than scientists just starting their careers and trying to make their mark. No other activity is as fundamental to democratic scholarship as refereeing. From all this, I concluded that there is an Uncertainty Principle of Refereeing: The more we have of it, the less we like it —but the less we have of it, the more we miss it. We sometimes trivialize what we take for granted. Refereeing has been around for so long that it's easy to forget that it wasn't always there. The present stage of
