Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)

DOMENIC V. CLCCHETTI: The Reliability of Peer Review for Manuscript and Grant Submissions: A Cross-Disciplinary Investigation

103 CICHETTI: THE RELIABII

.ITY OF PEER REVIEW Garcia, C., Rosenfield, N. S., Markowitz, R. K., Seashore, J. H., Touloukian, R. J. Őr Cicchetti, D. V. (1987) Appendicitis in children: Accuracy of the barium enema. American Journal of Diseases of Children 141:130912. [rDVCJ Gardner, M. J.. Snee, M. P.. Hall, A J., Powell, C. A., Dowries, S. őr Terrell, J. D. (1990) Results of case-control study of leukaemia and lymphoma among young people near Sellafield nuclear plant in West Cumbria. British Medical Journal 300 423-29. [SPL] Garfield, E. (1972) Citation analysis as a tool in journal evaluation. Sciencc 178:471-79. [RFB] Cáriunkéi, J. M.. Ulshen. R. H . Hamrick, H. J őr Lawson, E E. (1990) Problems identified by secondary review of accepted manuscripts. Journal of the American Medical Association 263:1369-71. [rDVC. SPL] Gamer. W. R. (1962) Uncertainty and structure as psychological concepts. Wiley. (DL) Camer, W. R. őr McGill, W. J. (1956) The relation between information and variance analyses. Psychometrika 21:219-28. [arDVC], JBC Garvey, W. D., Lin, N. őr Nelson, C. E. (1970) Some comparisons of communication activities in the physical and social sciences. In: Communication among scientists and engineers, ed. C. E. Nelson őt D. K. Pollock. Health. [SC] (1979) Communication in the physical and social sciences. In: Communication: The essence of science, ed. W. D. Garvey. Pergamon Press. [aDVC] Gholson, B. őr Barker, B. (1985) Kuhn, Lakatos, and Laudan: Applications in the history of physics and psychology. American Psychologist 40:75569. [aDVC] Ciere, R. N. (1988). Explaining science: A cognitive approach. University of Chicago Press. [MEG] Gillett, R. (1985) Nominal scale response agreement and rater uncertainty. British Journal of Mathematical and Statistical Psychology 38:5866. [rDVC] Gilmore, J. B. (1979) Illusory reliability in journal reviewing. Canadian Psychological Review 20:157-58. [arDVC, JBG] Glenn, N. D. (1976) The journal article review process. Some proposals for change. American Sociologist 11:179-85. [aDVC] Goodman, L. A. & Kniskal, W. H. (1954) Measures of association for cross classifications. Journal of the American Statistical Association 49:73264. [aDVC] Goodrich, D. W. (1945) An analysis of manuscripts received by the editors of the American Sociological Review from May 1, 1944, to September 1, 1945. American Sociological Review 10:716-25. [aDVC] Goodstein, L. D. (1982) When will the editors start to edit? Behavioral and Brain Sciences 5:212-13. [LJS] Goodstein, L. D. őc Brazis, K. L. (1970) Credibility of psychologists: An empirical study. Psychological Reports 27:835-38. [aDVC, JSA] Gordon. M. D. (1977) Evaluating the evaluators. New Scientist 73:34243. [aDVC] (1978) A study of the evaluation of papers by primary journals in the U.K. University of Leicester. [LLH] Gorman, M. E. (1986) How the possibility of error affccts falsification on a task that models scientific problem-solving. British Journal of Psychology 77:65-79. [MEG] (1989) Error, falsification and scientific inference: An experimental investigation. Quarterly Journal of Experimental Psychology, 41A, 385412. [MEC] Gorman, Michael E. őt Gorman, Margaret E. (1984) A comparison of disconfirmatory, confirmatory and a control strategy on Wason's 2, 4, 6 task. Quarterly Journal of Experimental Psychology 12:129-40. [MEC] Gottfredson, S. D. (1978) Evaluating psychology research reports: Dimensions, reliability, and correlates of quality judgments. American Psychologist 33:920-34. [aDVC, RFB, JBC] Green, D. M., Luce, R. D. őr Duncan. J. E. (1977) Variability and sequential effects in magnitude production and estimation of auditory intensity. Perception 0 Psychophysics 22:450-56. [DL] Green, D. M., Luce, R. D. őr Smith, A. F. (1980) Individual magnitude estimates for various distributions of signal intensity. Perception Lr Psychophysics 27:483-88. [DL] Crecnwald, A. G. (1975) Consequences of prejudice against the null hypothesis. Psychological Bulletin 82:1-20. [aDVC, PHS] (1976) An editorial. Journal of Personality and Social Psychology 33:17. [aDVC] Creenwald, A. C., Pratkanis, A. R., Leippe, M. R. őr Baumgardner, M. H. (1986) Under what conditions does theory obstruct research progress? Psychological Review 93:216-29. [aDVC] Cross, S. T. (1986) The kappa coefficient of agreement for multiple observers when the number of subjects is small. Biometrics 42:883-93. [rDVC] Grove, W. M . Andreasen, N C . McDonald-Scott P . Keller. M B ér Shapiro, R. W. (1981) Reliability studies of psychiatric diagnosis Theory and practice Archives of General Psychiatry 38.408-13 |rDVC] Guilford, J P (1954) Psychometric methods, 2nd ed McCraw-Hill (RR Culliksen, H. O (1950) Theory of mental tests Wiley [DL. LJSi Guyatt. G. H . Townsend, M őr Berman. L (1987) A comparison of Liker* and visual analogue scales for measuring change in function Journal of Chronic Diseases 40:1129-33 [rDVC] Hall, J. A. (1979) Author review of reviewers. American Psychologist 34 798. [aDVC] Hargens, L. L. (1988) Scholarly consensus and journal rejection rates American Sociological Review 53:139-51. [aDVC, SC] (1990) Variation in journal peer-review systems Possible causes and consequences. Journal of the American Medical Association 263 134852. [arDVC. LLH] Hargens, L. L. őc Herting, J. R (1990a) A new approach to referees' assessments of manuscripts Social Science Research 191-16 [arDVC, LLH] (1990b) Neglected considerations in the analysis of agreement among journal referees. Scientometrics 19:91-106 [aDVC. LLH] Hamad, S. (1979) Creative disagreement The Sciences 19:18-20 [aDVC] ed. (1983) Peer commentary on peer review: A case study in scientific quality control. Cambridge University Press (reprinted from Behavioral and Brain Sciences, vol. 5). [aDVC] (1985) Rational disagreement in peer review. Science, Technology O Human Values 10(3):55-62. [aDVC, LJS) (1986) Policing the paper chase. Nature 322:24-25. [aDVC, JBC] Hartog, P.. Rhodes. E. C., and Burt, C. (1936) The marks of examiners. Macmillan. [DL] Heavens, R. H. Jr. őt Cicchetti, D. V. (1978) A computer program for calculating rater agreement and bias statistics using contingency table input. Proceedings of the American Statistical Association (Statistical Computing Section) 21.366-70. [aDVC] Hendrick, C. (1976) Editorial comment. Personality and Social Psychology Bulletin 2:27-08. [aDVC] (1977) Editorial comment. Personality and Social Psychology Bulletin 3:12. [aDVC] Hensler, D. (1976) Perceptions of the National Science Foundation peerreview process: A report on a survey of NSF reviewers and applicants. NSF publication 77-33. [aDVC] Heskin, K. (1984) The Milwaukee Project: A cautionary comment. American Psychologist 39:1316-17. [aDVC] Holt, V. E. (1985) Research briefings: Peer-review appeals system established American Psychological Association (APA) Monitor 16:18. [aDVC] Horrobin, D. F. (1990) The philosophical basis of peer review and the suppression of innovation. Journal of the American Medical Association 263:1438-41. [JSA] Howe, M. J. A. (1982) Peer reviewing: Improve or be rejected. Behavioral and Brain Sciences 5:218-19. [aDVC] Hubbard, R. őt Armstrong, J. S. (1990) Replication and the development of marketing science. Marketing Department working paper. The Wharton School, University of Pennsylvania. [JSA] Hughes, H. M. (1976) Letter to the editor. American Sociologist 1117879. [aDVC] Hull, D. L. (1988) Science as a process. University of Chicago Press. [LLH] Hunt. E. (1971) Psychological publications. American Psychologist 26:311. [aDVC] Hunt, K. (1975) Do we really need more replications? Psychological Reports 36:587-93. [aDVC] Ingelfinger, F. J. (1974) Peer review in biomedical publication. American Journal of Medicine 56:686-92. [aDVC] (1975) Charity and peer review in publication. New England Journal of Medicine 293:1371-72. [aDVC] Ison, J. R. (1985) The granting system and healthy research. Science 230:376. [aDVC] Iyengar, S. őr Creenhouse, J. B. (1988) Selection model and the file drawer hypothesis. Statistical Science 33:109-35. |PHS) Jesteadt, W., Luce, R. D. őt Green, D. M. (1977) Sequential effects in judgment of loudness. Journal of Experimental Psychology: Human Perception and Performance 3:92-104. [DL] Jesteadt, W., Wier, C. C. őr Green, D. M. (1977) Intensity discrimination as a function of frequency and sensation level. Journal of the Acoustical Society of America 61:169-77. | DL] Jonckheere, A. R. (1970) Techniques for ordered contingency tables. In: Proceedings of the NUFFIC International Summer Session in Science, Het Oude Hof, ed. J. B. Riemcrsma őr H. C. van der Meer. The Hague. [aDVC]