Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)

EUGENE GARFIELD: Refereeing and Peer Review. Part 3. How the Peer Review of Research-Grant Proposals Works and What Scientists Say About It

29 GARFIELD: REFEREEING AND PEER REVIEW, PART 1 search. The money from these agencies is fiumeied into grants by the German Research Society (DFG), a scholarly society that operates beyond the boundaries of formal government. For scientists affiliated with German universities, which are stateowned, DFG grants supplement a certain minimum level of funding. DFG support goes out mainly in the form of small, individual project grants that run from one to three years. Grant applications are evaluated by peers who are elected to their positions by the entire scientific community.

2

1 In summary, the three countries briefly discussed here, as well as others, provide a relatively stable level of operating support to their universities, as well as to a parallel basic-research system separate from the university system.

2

1 And although many governments provide some funds on a competitive, peer-reviewed basis to scientists working both within and outside of the university systems, such support is relatively small compared with the baseline funding. Since research support in other countries is not limited to individual projects for short periods of time, foreign scientists, unlike their US counterparts, do not have to cope with the distractions of securing grant money and the disruptions suffered when grants are reduced or discontinued.

2

1 Atkinson and Blanpied claim that systems outside the US are less effective in encouraging competition among the most innovative ideas, and that other nations' faculty members, who are virtually or even literally employees of their respective governments, "cannot claim the same degree of autonomy they can in the United States."

9 Roy says that "not one study has ever been supported to test this" claim

1

7 or to compare the review systems of various government agencies with other methods of allocating funds.

2

2 Criticisms of Peer Review With research projects, jobs, and even careers at stake each time the review process renders a verdict, it is not surprising that the effectiveness and fairness of the system are matters of great concern—especially since the mid-1970s, when government support began declining.

1

8 Making matters worse, the NIH and the NSF, the major grant-giving agencies, do not have the money to fund every application they approve. In a remark reported in Science, NIH director James B. Wyngaarden said that the issue of distinguishing between "shades of excellence" was among those that most concerned scientists. The distinctions between one excellent proposal and another are often so fine that judgments concerning relative quality cannot be rendered on an objective basis, leaving those whose top-rated proposals are rejected "angry and frustrated."

2

3 Many scientists also feel it is inappropriate to rank disparate proposals that have littlejn common with one another.

5 And Roy goes even further, charging that "there is no theoretical or empirical justification to support the contention that 'good' research can be predicted on the basis of the 'evaluation of proposed ideas contained in an essay."

8 Another, almost universal, concern about fieer review

—found even among reviewers and agency administrators—is the time, effort, and money it takes to complete the paperwork involved in applying for and evaluating proposals. To ensure approval of a grant application, physiologist Daniel H. Osmond, University of Toronto, Canada, writes, "many have sacrificed 1-3 months of productive research.... The entire year is dominated by thoughts of preparation, and of the tragic consequences of refusal

.... We must do quick experiments, write them up fast, and publish, publish, publish.... Innovative, time-consuming work must be done on the side with unbudgeted dollars to diminish the risk of rejection by an overcautious grants committee when the work eventually surfaces."

2

4 Rosalyn S. Yalow, VA, New York, and the 1977 Nobel laureate in physiology or medicine, adds that grant proposals are "inherently dishonest," since "few established investigators whose contributions are highly original and imaginative can spell out... detailed plans for a three- or a five-year period."

2

3 If the investigator can do so, she continues, then ' 'he does not expect to make a discovery; in fact, that mind-set can keep him from recognizing a discovery."

2

6 And Osmond adds that the constant pressure of applying for grant renewals can cause seien-