Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)
IAN I. MITROFF and DARYL E. CHUBIN: Peer Review at the NSF: A Dialectical Policy Analysis
108 MITROFF & CHUBIN: PEER REVIEW AT THE NSF very institution of science, precluding systematic investigation of mechanisms by which the institution's autonomy, self-governance, and 'uneasy partnership' with government is maintained. 3 As sanguinity and tenacity have flagged, a more defensive posture has emerged, resulting in empirical investigation of contemporary peer review. The results of such investigation occasion this paper, the purpose of which is fourfold : 1. to outline the nature of the debate: that is, to present systematically the position of the contending parties; 2. to review critically some of the evidence, particularly that emanating from two studies of peer review at the National Science Foundation, which bears upon the debate; 3. to raise issues not addressed in recent studies, but which bear fundamentally on the debate; and 4. to propose a strategy for future studies that will clarify old and new issues and hasten collection of appropriate data for informing, if not resolving, the debate — a debate which centres on peer review as an evaluative mechanism in the execution of science policy. Before proceeding, two caveats must be sounded, lest we be misconstrued. Firstly, notwithstanding our commentary, we are neither anti-science not anti-peer review. Indeed, we regard peer review in principle as the best available system; this does not mean that the system in practice cannot be improved. Likewise, and one would think with greater ease, studies of peer review can be improved, not merely in terms of measurement and modes of analysis, but in approach. If scientists refuse to be reflexive, sceptical, and probing of their own institution — its organization and management — then can they really decry congressional 'incursions' into their policies and practices? Part of the responsibility of autonomy and self-governance is self-scrutiny. This is a reasonable expectation, yet too few dissenting voices within the scientific community tend to be heard." The second caveat is closely related to our first, but pertains to the scope of this paper. We regard peer review as a kind of science advice involving select members of the community, all of whom act — to a greater or lesser degree — as gatekeepers. These gatekeepers help to regulate the flows of information and fiscal resources through the community by directing, impeding, and expediting flows based upon judgments of quality and merit, allegiances and biases and, probably, on sheer caprice as well. 5 The point is that
