Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)
IAN I. MITROFF and DARYL E. CHUBIN: Peer Review at the NSF: A Dialectical Policy Analysis
136 MITROFF & CHUBIN: PEER REVIEW AT THE NSF will neither gain ready access to the literature nor approval of proposals to pursue their research programme. Mainstream thought, in short, can sustain only moderate innovation. The issue, in the context of peer review, is whether the agencies which administer the system and its resources are the guardians of the mainstream or a refuge for innovators. Surely they are a little of both; hence, the issue of bias and evidence are elusive at best. For related discussion, see below, plus T. S. Kuhn, 'Second Thoughts on Paradigms', in F. Suppe (ed.), The Structure of Scientific Theories (Urbana, III.: University of Illinois Press, 1974), 459-82; M. J. Mulkay, The Social Process of Innovation (London: Macmillan, 1972); H. M. Collins, 'The TEA Set: Tacit Knowledge and Scientific Networks', Science Studies, Vol. 4 (1974), 165-86; D. E. Chubin, 'The Conceptualization of Scientific Specialties', Sociological Quarterly, Vol. 17 (Autumn 1976), 448-76, esp. 459-70. 32. While Hensler does not specify these 20 institutions, the following ten US universities have been identified as receiving more than a third of all federal expenditures in universities, producing about a third of all the doctorates, and providing 37 percent of the members of federal review panels in the 1960s: California, Caltech, Chicago, Columbia, Cornell, Harvard, Illinois, MIT, Michigan and Minnesota. See W. Hirsch, Scientists in American Society (New York: Random House, 1968), 106. 33. These findings accord with previous perceptions, as Hensler (op. cit. note 25, 50) observes that 'reviewers in general, and applicants who have also served as reviewers, are significantly less likely to perceive bias in the process than other applicants .... Applicants who have not been successful in obtaining NSF grants recently or in the past are most likely to think that process is biased.' 34. Hensler, op. cit. note 25, 84. 35. Ibid., 1. 36. Ibid. 37. It could well be the case that those who have either experienced the most 'bias' (no matter how it is defined) or who attribute bias to the system may be those who have either (a) 'dropped out' of the system prior to the sampling period or (b) never 'dropped in' in the first place. Without sampling this group, such conjectures simply cannot be evaluated; yet they cannot be dismissed out of hand. In this respect, Hensler can be criticized for a uniform lack of conjecture; she apparently feels no compulsion to explain why her survey generated the responses it did. 38. This quotation appears in an earlier version of Hensler's report (op. cit. note 25), dated September 1976, on p. 80. 39. G. Gordon and E. V. Morse, 'Creative Potential and Organizational Structure', Academy of Management Journal, Vol. 12 (1969), 37-49; I. I. Mitroff, The Subjective Side of Science, A Philosophical Inquiry into the Psychology of the Apollo Moon Scientists (New York: Elsevier, 1974); E. V. Morse and G. Gordon, 'Cognitive Skills: A Determinant of Scientists' Local-Cosmopolitan Orientation', Academy of Management Journal, Vol. 17 (1974), 709-23. 40. A prior question (underlying all of these) concerns the distribution of innovativeness in the community at large, and whether the purposive sampling of scientists for the role of peer reviewer proportionately captures this characteristic. Given the profile of reviewers developed in the works cited in note 6, one would think so. Indeed, one would think that innovators are overrepresented among reviewers; likewise, one would hope that innovators are overrepresented among the recipients of research funds. (As far as we can tell, there are no questions in the
