Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)
RUSTUM ROY: Alternatives to Review by Peers: A Contribution to the Theory of Scientific Choice
143 ROY: ALTERNATIVES TO REVIEW BY PEERS of peers. The entire operation, moreover, further exacerbated the differences between the successful and the unsuccessful applicants, since the unsuccessful universities wasted their efforts, while those with the more successful applicants gained even more. The failure of the system of review by peers is not that it fails to provide financial support for most good scientists. No system could fail to do that! The unnecessary waste of limited resources of scientific talent is the single most telling failure of the peer-review system. There is no single system of review by peers. There is an infinity of systems in which the influence of the scientific "peer" varies from almost nothing more than the passive lending of a name for legitimation, to almost complete control. I will consider only four major categories of review by peers: reviews submitted by peers, plus an assessment by an assembled panel, plus a site visit; reviews submitted by post, plus an assembled panel; standardised postal reviews, usually with quantified ratings or preferences within prescribed categories; and postal advice or comments without ratings. Only the most sanguine advocate unfamiliar with the literature would claim that there is any basis for expecting a correlation between a scientist's ability to present an essay and the actual future production of the "best science". 5 The weak links in a "theoretical" sense are that we have no definition of what constitutes the "best science". With the total confusion between the terms "basic" and "applied" and over the value of relevance, and the very major psychosocial differences in perceptions and values, between —let us say —civil engineers and theoretical physicists, the entire system of review by peers is one of reinforcements of the idiosyncracies or the ruling paradigms of any group which is constituted and supported as a unit. We have no definition of a peer. According to current practice —strongly departing from the judicial model —a peer is defined as one who works in the same narrow subspeciality of scientific research. For example, proposals in microwave plasma synthesis exclude those working in radio-frequency plasmas, or those working in chemical vapour deposition synthesis. The working definition of peers in the present peer-review system certainly means as narrow a group of specialists as can be found to match the subject defined in the proposal. But is not a better definition of a peer, a person of equal "rank" and "experience" in science, drawn not only from the narrow speciality, but explicitly including the neighbouring fields? Dr Weinberg's insight that the best science would be that which affected a wide group of fields indicates that peers be explicitly defined to include some from neighbouring fields. Not a single agency of the United States government does this. In the present system, the simplest precautions against conflict of interest 5 Peters, D. P. and Ceci, S. J., "Peer Review Practice of Psychological Journals", The Behavioral and Brain Sciences, V, 2 (1982), pp. 187-255; Harnad. Stephen, "Peer Commentary on Peer Review"; followed by 56 comments, ibid., pp. 185-186, 196-225.
