Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)

IAN I. MITROFF and DARYL E. CHUBIN: Peer Review at the NSF: A Dialectical Policy Analysis

114 MITROFF & CHUBIN: PEER REVIEW AT THE NSF thoughtful members felt that going to public markup sessions was an exercise in sheer idiocy. They feared that the majority and minority would not be able to compromise, and that we could never settle anything or report a bill out of committee. We changed this procedure and I think that everyone agreed that the system has been vastly improved. You have described the [peer] review system as one based on mores. Mores can change. They can yield to the pressures of changing time and conditions ... In the past, the lifestyle based on these mores [that is, secrecy] promoted a degree of integrity, decency, and internal fair dealing ... An awful lot of problems would be solved if we changed our system from one based on confidentiality to one based on openness and fair treatment. I don't think that we are now aware of all the problems connected with an open system, but I am willing to bet that many of 19 these problems would not materialize. Although these quotations reveal the intensity of the differences between the two sides in the debate, they underestimate the significant propositions on which both sides agree. We have detected at least five points of consensus, namely: (1) 'No method superior to peer review has been found forjudging the scientific competence of proposers. Scientific peers are better able than others to judge the design of proposed work, the importance of proposed work to the scientific field, and the past performance of the proposer. Appropriate peer review procedures generally lead to the support of proposals in a high quality range. Using peer review procedures [NSF] has successfully fostered significant advances in basic science over the past 25 years.'

2

0 (2) 'Witnesses agreed overwhelmingly that some form of peer review should continue to be used to assist in the allocation of Federal funds for scientific research. Not a single witness suggested that peer review be abandoned, although several witnesses proposed changes in the decision-making processes of [NSF] — some minor and some major in their potential effects.'

2

1 (3) 'While many witnesses avowed that peer review results in the support of high-quality research, some of which is truly innovative, there was not much confidence expressed that peer review consistently leads to the support of innovative research if it challenges the mainstream of scientific thought or if it seems unlikely to succeed. Arguments and the weight of opinion to the contrary were rather persuasive.'

2

2 (4) 'The Subcommittee had ample opportunity during the hearings to explore whether Congressional review of individual [NSF] grants should be required in addition to Foundation approval before Foundation's action becomes final . . . Opinion was over-