Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)

MARTIN RUDERFER: The Fallacy of Peer Review: Judgement without Science and a Case History

RUDERFER: THE FALLACY OF PEER REVIEW 197 Appendix R 16 March 1977 The Editor, Science. Dear Dr Abelson, I cannot dispute that your nine reviewers are of "known competence" provided we understand the term to be restricted to what is commonly known. There is no justification for assuming that any man's competence encompasses the unknown. Otherwise every "expert" may be expected to solve any problem that represents a good jump beyond the state of the art which, judging from the plethora of unsolved problems, has not yet come to pass. My report concerns a little explored area — one-way light propagation — for which there is a paucity of experts of known competence. This applies to the nine referees: although one stated the report was excellent, one offered criticism which I answered and the others would not or could not comprehend my contribution. The last attempt was a travesty. It is precisely in such cases — the revolutionary quantum jump — that significant progress often results as opposed to the smaller evolutionary advances contained in most reports. Both types are essential to scientific progress. However the review system is primarily suited for the latter; it fails miserably for the former and history bears this out. This is my whole point. The inadequate review of my paper attests to a glaring deficiency in the referee system. A thousand more like rejections would still not justify your decision. Something must be done for those few cases that probe uncharted territory. Man has too many urgent unsolved problems and too little time to continue to indiscriminately squander and squash creative efforts. This is not a matter for authors or referees to resolve — it is clearly a matter of editorial policy and practice; it represents a timely challenge of lasting potential benefit if it can be done. I can only repeat my offer — as an editorial experiment I am willing to participate in a test for the concept of closed-loop review for the type of dispute represented by my paper. Whether or not it results in acceptance, the experience is bound to be of future utility to you and other editors if properly done. Sincerely, Martin Ruderfer.