Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)
EUGENE GARFIELD: Refereeing and Peer Review. Part 2. The Research on Refereeing and Alternatives in the Present System
21 GARFIELD: REFEREEING AND PEER REVIEW, PART 1 paying referees would encourage them to perform their task more thoroughly and impartially. 2 6 Although they do not say how much referees should receive, they suggest that such fees could come from "authors' institutions, their research funding, or their personal resources." 2 6 They present no empirical evidence supporting their argument, but the notion of paying reviewers, like other ideas reported in this essay, could form the basis of an interesting study. In this case, the questions might be, "Do paid referees perform better than unpaid ones?" and "How much money does it take before a significant effect is noticed?" Conclusion It is difficult to draw substantive conclusions about how well the refereeing process functions. But Lock makes an interesting observation: the validating of experimental results and theoretical conclusions is ultimately not through the refereeing process but through the broader evaluation that articles receive over time at the hands of a larger, informed scientific community. 1 5 (p. 128) Of course, refereeing does not always detect fraud, plagiarism, errors, and muddy thinking. Still, it is probably impossible for most journals to switch to a system of in-house evaluation: despite its faults, real or imagined, refereeing is probably the most efficient and effective method for distinguishing the promising from the meretricious —at least, until it is proven otherwise. In assessing refereeing's supposed flaws, one of the key issues seems to be delays in publication. Much of the accumulated anxiety about refereeing in many fields seems traceable to the tedious process that is often made out of what should be a straightforward decision. At the heart of many delays are referees who allow manuscripts, to gather dust on their desks without informing editors that they cannot complete a review in a timely fashion. As I see it, at the root of many of the alleged deficiencies of peer review are the attitudes of the scientific community itself. Were quality valued over quantity, and spurious "productivity" deplored instead of rewarded with tenure and promotions or research grants, then the incentive to publish shoddy or half-finished research would diminish. This might reduce the burden placed upon editors and reviewers because of the publish-or-perish syndrome. Unfortunately, we have not yet emerged from the stage of regarding the sheer number of publications as significant, 3 9 but there is a growing tendency to limit the number of papers to be listed on nominations for awards, grants, and so on. 4 0 And in fact, one of the often-stated goals of citation analysis is to encourage quality, high-impact work, rather than publication for the sake of pure output. Of the myriad comments about refereeing, it is difficult to find one brief, allencompassing statement that says it all. But John Ziman, Imperial College of Science and Technology, London, UK, and editor. Science Progress, has come close. In a commentary on Peters and Ceci, he wrote: Informed discourse on the primary communication system of science takes for granted the basic utility and reliability of the peer-review process, at least up to some modest practical level of human competence. The height of this level should not be exaggerated: It is not an indicator of permanent scientific worth. Acceptance for publication by a reputable journal implies no more than that the work is superficially sound, mildly interesting, and moderately original. The opinion that it should at least be taken into consideration by other scientists is only a preliminary assessment, likely to be contradicted and entirely superseded in the light of further study. Nevertheless, this weak and uneven standard of quality appears real enough to the authors, editors, and reviewers who tussle endlessly to establish and maintain it. Specific accusations of prejudice, inquiries concerning systematic bias, and demands for institutional reform have all been addressed to imperfection of performance around and about this hypothetical benchmark. 4 1
