Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)

IAN I. MITROFF and DARYL E. CHUBIN: Peer Review at the NSF: A Dialectical Policy Analysis

133 MITROFF & CHUBIN: PEER REVIEW AT THE NSF fulfill the promise of a dialectical policy analysis; only then will the debate prescribe changes of policy into practice. • NOTES A presentation based on an early version of this paper was made at the Second Annual Meeting of the Society for Social Studies of Science, held in Boston, Massachusetts, and Harvard University, 14-16 October 1977. The incisive comments of P. Thomas Carroll on that version were most helpful in rethinking and rewriting. t. See T. Gustafson, 'The Controversy Over Peer Review', Science, Vol. 190 (2 December 1975), 1060-66; D. Shapley, 'House Votes Veto Power on All NSF Research Orants', ibid., Vol. 188 (25 April 1975), 338-41; Shapley, 'NSF Violations of Personnel Code Alleged', ibid. (31 May 1975), 915; J. Walsh, 'NSF and Its Critics in Congress: New Pressures on Peer Review', ibid. (6 June 1975), 999-1001; Walsh, 'NSF House Appropriations Panel Gives Warning Tug on Purse Strings', ibid., Vol. 189 (4 July 1975), 26-28; Walsh, "NSF Peer Review Hearings: House Panel Starts with Critics', ibid. (8 August 1975), 435-37; Walsh, 'Peer Review: NSF Faces Changes, the Question is How Extensive', ibid., Vol. 190(17 October 1975), 253-56. 2. S. MacLane, 'Peer Review and the Structure of Science,' Science, Vol. 190 (14 November 1975), 617. 3. G. M. Lyons, The Uneasy Partnership: Social Science and the Federal Government in the Twentieth Century (New York: Russell Sage, 1969). See also J. Haberer, Politics and the Community of Science (New York: Van Nostrand Reinhold, 1969). 4. Our mention of this caveat stems from two sets of experiences: firstly, in response to the first public draft of this paper, where colleagues rhetorically asked, 'You're not against peer review, are you?'; and secondly, our inability to convince NSF programme managers that despite the recent studies of their system, the 'crucial experiment' had yet to be performed. We claimed that the present system had been legitimated, but not accurately appraised because (1) many of the right questions had not been asked, or if they had, the answers were not carefully weighed (for example, could not be quantified), or (2) the crucial data were not in the public domain, but with special consent could be made available for analysis. Consequently, the study whose findings could be used to alter policy and strengthen the operation of the system never materialized. Our interpretation may smack of sour grapes, but we think the opportunity for internal self-scrutiny was lost (although congressional critics were pacified — for the moment). We do not think our argument fell on deaf ears; rather, no one was willing to handle a hot potato which had temporarily cooled. 5. For a review, see H. Zuckerman and R. K. Merton, 'Age, Aging, and Age Structure in Science', in M. W. Riley, M. Johnson and A. Foner (eds), A Theory of Age Stratification, Vol. 3, Aging and Society (New York: Russell Sage, 1972), 292-356.