Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)

MARTIN RUDERFER: The Fallacy of Peer Review: Judgement without Science and a Case History

MARTIN RUDERFER: The Fallacy of Peer Review: Judgement without Science and a Case History Speculations in Science and Technology, 3 (1980) 533-562 Peer review, the process of judging the contributions that make up the archives of science, is not now justifiable as a scientific endeavour. Yet this process, via the archives of science, is a key factor in determining man's ability to cope with the growing global problems of the population explosion stimulated by past scientific progress. There exists an urgency to improve peer review in order to guarantee the technological growth rate vital for long-term survival. However, a science of peer review has thus far been precluded by the secrecy imposed on the primary raw data — review histories. To begin to rectify this, a case history of an erroneous rejection is presented in detail. The rejected paper, which claimed to correct a published dispute involving atomic timekeeping, was published in SST in 1979 along with a follow-up paper confirming and extending it. The case history leads to the hypothesis that the probability of rejection increases with the degree of innovation in a publishable work. This is validated by the follow-up paper which shows the rejected paper to require a paradigm shift to correct a widespread misinterpretation of rotating clock behaviour now erroneously attributed to special relativity. This results in a simple unification of rotating clock behaviour in atomic timekeeping, the Sagnac effect and the Hafele-Keating experiment. The ability of this case history to clearly delineate origins of human error in review processes demonstrates the need for publication of many more and the desirability of stressing erroneous rejection in peer review at least as much as the traditional emphasis on erroneous acceptance. This one case also supports the urgency required for improving the accuracy and speed of peer review and recommends a number of specific means for accomplishing this. 1. INTRODUCTION The spectacular growth of modem science following the invention of printing attests to the supremacy of the printed word in man's pursuit of knowledge. The archives of science nurtured the rapid rise of technology in the last few centuries. Demands on reports for research journals — the roots of these archives — are among the most exacting in all the fields of literature: extensive preliminary research, precise explication, maximum objectivity and rigour, absence of bias and error, novel or utilitarian content, all presented with optimum economical clarity. It is consequently not surprising that the decision of what to accept or reject has always been difficult. For the last three centuries the principal method of judging potential contributions has been peer review, commonly known as the referee system, wherein acceptance or rejection is decided by an editor based on reports of