Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)

EUGENE GARFIELD: Refereeing and Peer Review. Part 3. How the Peer Review of Research-Grant Proposals Works and What Scientists Say About It

27 GARFIELD: REFEREEING AND PEER REVIEW, PART 1 maneuvered.

1

6 The practice may also divert scarce resources from research projects that the scientific community considers to be of higher priority. As Atkinson and Blanpied write, "At issue is not whether meritorious research will be carried out in facilities obtained through pork-barrel tactics. Rather, [such] tactics violate the understanding that available resources are to be allocated in the best overall interests of science—and the public—rather than in the interests of individual claimants."

9 Roy, however, pointedly wonders who will define what the "best overall interests of science" are.

1

7 How Peer Review Works The principal agencies that support basic scientific research in the US are the NSF, the NIH, the Veterans Administration (VA), the Department of Defense, the DOE, the Department of Agriculture, and the National Aeronautics and Space Administration (NASA),

9 although hinds are also provided through such organizations as the American Chemical Society, which administers the Petroleum Research Fund.

1

8 Various congressional committees and the US Office of Management and Budget determine the amount of money each government agency has available to disburse.

9 In general, how that money is spent depends largely on peer review: area experts judge proposals on their scientific and technical merit and make recommendations accordingly. But each agency or organization charged with dispensing funds for scholarship operates with a somewhat different set of procedures. The NIH, which accounts for most US basic research-grant funding in terms of total dollars per year, makes use of a two-tiered system called "dual review."

1

9 (p. 41) At the first level, a panel of experts in a given field, called the Initial Review Group (IRG), evaluates a research application for its scientific merit. The IRG also comments on the applicant's performance on any previous grants and recommends a funding priority for the application, as well as the amount and duration of the grant.

1

8 At the second level, the respective National Advisory Council/Board of each bureau, institute, or division of the NIH reviews the recommendations made by the IRG and makes its own, final judgment concerning the application's relevance to the NIH's various programs and priorities.

1

9 (p. 42-3) At the NSF, applications in chemistry, physics, and mathematics are mailed to 3 to 10 independent experts selected by a program officer.

I

8'

2

0 (p. 7) These experts, referred to as mail reviewers, individually evaluate applications for their scientific quality through written comments and boxes checked off on a multiple-choice form.

20 (p. 7) Applications in the earth, biological, and social and behavioral sciences are usually reviewed by a combination of mail reviewers and panel reviewers.

1

8'

2

0 (p. 7) Panels consist of scientists selected by program directors; the size of panels varies from section to section, but each meets in Washington, DC, three times a year to evaluate proposals.

2

0 (p. 8) Like the NIH's IRG, NSF reviewers report on the applicant's track record, as well as on the relevance of the work and on the capability of the applicant's institution to provide technical support. Based on these comments, the program officer makes a recommendation to higher-ranking officials, who in tum make the final decision.

181

9 (p. 22-4) 2

0 (p. 3-11) One significant difference between the NIH and the NSF procedures is that NSF officers have considerable discretion to modify or even disregard peer-review recommendations, whereas at the NIH, all recommendations by the IRGs are followed very closely.

1

0 The procedures of other agencies and organizations are, for the most part, variations on either the NSF or the NIH models. For instance, the VA, like the NIH, conducts an initial review of a grant application through a discipline-based review board that makes a recommendation to VA administrators; they constitute a second level of review.

19 (p. 25-6) Unlike the NSF or the NIH, however, the VA attempts to provide some funding for all approved proposals. When NASA receives unsolicited research proposals, it operates in much the same fashion as the NSF, with ad hoc reviewers who make recommendations based on scientific merit. Review procedures at the ONR are also sim-