Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)

MARTIN RUDERFER: The Fallacy of Peer Review: Judgement without Science and a Case History

179 RUDER

I ER: T

IIK I'A

I.I ACY O

l PEER REVIEW Besides confirming the rejected paper and clarifying clock behaviour, the follow-up paper indicates how applications may directly proliferate from innovation and hence that these are delayed by improper rejection. This case history thereby links the probability of self-destruction of science from a deficient growth rate directly to the mental inertia inherent in the minds of men as summarised by the Innovation Theorem. 7. TIME FACTORS IN THE REVIEW PROCESS In view of the increasing need for urgency in solving today's technological problems, opportunities for decreasing unnecessary time delays are of interest. Accordingly, the time factors in this review process are summarised in Table 2 (at the end of this paper). The greatest delays were incurred by the reviews themselves which accounted for 75.5 percent of the total time to reach a final decision. The time for a single review response averaged to 109 days, which includes the two-way transit time between editor and referees. Since the review cycle represents the greatest opportunity for time reduction, it would be useful to know the distribution of "dead time", i.e. time delays not affecting review quality, due to editorial office, referees and transit. Such data are intrinsically determinable by the journals and demand systematic analysis. The major cause of the 433 days required to reach a decision is the large number of reviews which, in turn, was aggravated by the low review precision. This case history thereby suggests that an increase in review precision presages shorter review times and reduced load on the journals and research community with its attendant financial, psychological and social benefits. The total time for a review may be small compared to the delay in final publication caused by a rejection. Due to the nature and period of the subject review, there were no plans to repeat such an exasperating experience. Publication of the two papers would have been indefinitely delayed were it not for the circumstances associated with the inception of SST (p.386) and insistence of the editor of SST on further corroboration. The high delay per response by reviewers suggests a general disregard for urgency in present review practice. Due to the sequential proliferation of discoveries, which depend on prior discoveries dependent on still prior discoveries, etc., the development of science is exponentially stretched out by unnecessary delays. That a marked reduction in total publication time is feasible has been demonstrated by the Journal of Clinical Psychiatry'

3

2> and in physics by Azbel<

9

) in his comparison of JETP Letters and Physical Review Letters. He attributes the greater speed of JETP to editorial requirements for greater review precision and shorter response time. In reply, the editors of PRL state'

1

0 ' that publication delay time "can be reduced substantially only by increasing our costs — and our page charges — significantly and we choose not to do so" (italics added). If this direct affirmation of a low priority for urgency is merely a matter of cost to the journals, why is it not also evaluated with respect to the inordinate cost of slower technological growth'

1

5' to society? Can we put a price on increasing the probability of civilization's survival and the attendant improvement in the quality of life? In this light the solution is self-evident: since society is the chief beneficiary it should provide the