Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)
EUGENE GARFIELD: Refereeing and Peer Review. Part 1. Opinion and Conjecture on the Effectiveness of Refereeing
8 GARFIELD: REFEREEING AND PEER REVIEW, PART 1 ly few controlled studies that have been done, many suffer from such severe methodological shortcomings that their conclusions are questionable. More will be said about research on refereeing in Part 2. Refereeing and other forms of peer review have been discussed at length, especially in the four decades since World War II, but discussion alone does not constitute science or scholarship. Since we are all affected by peer review, it is not surprising that so many of us have opinions on the subject. Yet the literature representing controlled studies of peer review is either pitifully small or disgracefully absent, while the body of anecdote and opinion is quite large. We carefully distinguish here between studies, experiments, experience, and opinions. In researching this essay, we also found that most published opinion on refereeing is negative. But we suspect that this is due, ironically, to the widespread acceptance of and satisfaction with the current system of peer review: most scientists simply do not feel that refereeing needs defending, so positive opinions are relatively scarce. It should also be kept in mind that these opinions on refereeing are themselves unrefereed. Furthermore, the existence and ranking of hundreds of refereed journals is concrete evidence that they are the preferred medium of publication. Fbws in the System? In a note published in the New England Journal of Medicine (NEJM ), John C. Bailar III and Kay Patterson, Harvard School of Public Health, Boston, Massachusetts, speculate that current opinion on refereeing seems divided among one or more of four paradigms. 2 7 Based on their own informal observations, the authors assert that many scientists seem to perceive the process as a sieve, sifting the wheat from the chaff. Many also liken the process to a smithy, in which "papers are pounded into new and better shapes between the hammer of peer review and the anvil of editorial standards." Some seem to view it as a switch, reflecting the widespread belief that a persistent author can eventually publish a manuscript somewhere (although refereeing may determine exactly where). Finally, some scholars seem to consider refereeing a capricious and essentially unpredictable process —a "shot in the dark." 2 7 Stephen Lock, editor, British Medical Journal, feels that refereeing "favours unadventurous nibblings at the margin of truth rather than quantum leaps." 28 An example supporting his opinion is the reception given the early demonstration, via radioimmunoassay, of insulin-binding antibody by the late Solomon A . Berson and Rosalyn S. Yalow, Veterans Administration, New York. This work was fundamental to the development of the radioimmunoassay into a "powerful tool for determination of virtually any substance of biologic interest," according to Yalow. 2 9 Although Yalow would share the 197-7 Nobel Prize with Roger Guillemin, Salk Institute, San Diego, and Andrew Schally, Veterans Administration Hospital, New Orleans, the initial research concerning radioiodine-labeled insulin was rejected both by Science and, at first, by the Journal of Clinical Investigation (JCI) as erroneous. 2 9 Nevertheless, when the paper was revised to meet the objections of reviewers, it was published in the JCI. 3 0 A comparatively recent poll of the authors of manuscripts rejected by the JCI, conducted by editor Jean D. Wilson, Department of Internal Medicine, University of Texas Health Science Center at Dallas, found that 85 percent of the rejected papers were subsequently published elsewhere. And Wilson also re-
