Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)
IAN I. MITROFF and DARYL E. CHUBIN: Peer Review at the NSF: A Dialectical Policy Analysis
127 MITROFF & CHUBIN: PEER REVIEW AT THE NSF neglected or refused to test critically their theories, but that their actual conduct of science is more complicated than that portrayed in conventional accounts. The relevance of these findings to the present discussion is this: Can we expect scientists' views on the general operation of science as an intellectual and social system not to influence their views regarding peer review? Were the moon scientists' views exceptional (they too, constitute an élite sample) or more the rule than previously thought? These are fittings topics for future studies, and in our opinion, vital for further assessment of the peer review system. What we now know better about the peer review debate from the Hensler and Cole studies is summarized in Table 3. In our judgment, these data may be necessary, but are insufficient, to settle the principal issues. Although they enhance our understanding in dialectical terms, the data underscore the serious gaps that exist in our knowledge — gaps which must narrow if the debate is ever to approach closure. Conclusions and Recommendations The principal conclusion of this review is that the current data are inconclusive to resolve the debate represented in Tatole 1; nevertheless, some data do exist to support contentions on each side of the debate. Because issues such as those inherent in peer review expose tensions in the workings of science as a social system, they call forth deep divisions of value. Such issues, therefore, may not be amenable to treatment (and hence, to resolution) via conventional methods. The debate instead calls for treatment of the issues from more than one theoretical point of view. Philosophers of science have long recognized that scientific data can neither be collected in the first place, nor analyzed in the second, apart from some prior theoretical point of view. 6 9 That is, one does not collect data without having presupposed some hypothesis, theory, or model, no matter how implicit, unconscious, or informal it may be. We would assert that at least three models undergird the peer debate; (1) the Accumulative Advantage Model; (2) the Political Model; and (3) the Merit Model. The Accumulative Advantage Model derives from the 'Matthew effect', as explicated by Merton: 7 0 one who has developed a good reputation based on past
