Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)

DOMENIC V. CLCCHETTI: The Reliability of Peer Review for Manuscript and Grant Submissions: A Cross-Disciplinary Investigation

104 CICHETTI: THE RELIABII

.ITY OF PEER REVIEW Jones, R. (1974) Rights, wrongs, and referees. Neu: Scientist 61:75859 laDVC] Kahneman, D.. Slovic, P. Ac Tversky. A., eds. (1982) Judgment under uncertainty: Heuristics and biases. Cambridge University Press. [HLR] Kamin. L. J. (1981) The intelligence controversy, ed. H. J. Eysenck. Wiley IPHSJ Kazdin, A. E. (1982) Single-case research designs: Methods for clinical and applied settings. Oxford University Press. [aDVC] Kerr, S.. Tolliver, J. At Petree, D. (1977) Manuscript characteristics which influence acceptance for management and social science journals. Academy of Management Journal 20:132-41. [aDVC] Klayman, J. & Ha. Y.-W. (1987) Confirmation, disconfirmation and information in hypothesis testing. Psychological Review 94:21128. |MEC) Koran, L. M. (1975a) The reliability of clinical methods, data, and judgments. New England Journal of Medicine 293:642-46. |rDVC] (1975b) The reliability of clinical methods, data, and judgments. New England Journal of Medicine 293:695-701. [rDVC] Koshland, D. E. Jr. (1985) Peer review of peer review. Science 228.1387. [aDVC] Kraemer, H. C. (1980) Extension of the kappa coefficient. Biometrics 36:20716. (rDVC) (1982) Estimating false alarms and missed events from interobserver agreement: Comment on Kaye. Psychological Bulletin 92:74954. [rDVC] (1988) Assessment of 2x2 associations: Generalization or signal-detection methodology. The American Statistician 42:37-49. [rDVC] Kraus, C. A. (1950) The present state of academic research. Chemical and Engineering News 28:3203-04. [aDVC] Krippendorff, K. (1970) Bivariate agreement coefficients for reliability of data. In: Sociological methodology, ed. E. C. Borgatta. Jossey-Bass. [aDVC] Krystal, J., Ciller, E. Ac Cicchetti. D. V. (1986) Assessment of alexithymia in post-traumatic stress disorder and psychosomatic illness: Introduction of a reliable measure. Psychosomatic Medicine 48:84-94. [rDVC] Kuhn, T. (1962) The structure of scientific revolutions. University of Chicago Press. [aDVC, LDN] Lakatos, I. (1972) Falsification and the methodology of scientific research programmes. In: Criticism and the growth of knowledge, ed. I. Lakatos Ac A. Musgrave. Cambridge University Press. [aDVC] Laming, D. (1984) The relativity of "absolute" judgments. British Journal of Mathematical and Statistical Psychology 37:152-83. [DL] (1990) The reliability of a certain university examination compared with the precision of absolute judgments. Quarterly Journal of Experimental Psychology 42A:239-54 [DL] (in press) Reconciling Fechner and Stevens? Behavioral and Brain Sciences. (DL) Landis, J. R. Ac Koch, C. C. (1977) The measurement of observer agreement for categorical data. Biometrics 33:1599-74. [rDVC] Latane, B., Williams, K. Ac Harkins, S. (1979) Many hands make light work. The causes and consequences of social loafing. Journal of Personality and Social Psychology 37:822-32. [AMC] Laudan, L. (1984) Science and values: The aims of science and their role in scientific debate. University of California Press. [aDVC] Lawlis, G. F. At Lu, E. (1972) Judgment of counseling process: Reliability. agreement, and error. Psychological Bulletin 78:17-20. [aDVC] Lazarus, D. (1982) Intcrreferee agreement and acceptance rates in physics. Behavioral and Brain Sciences 5:219. [aDVC] LeLewis. D. At Burke, C. J. (1949) The use and misuse of the chi square test. Psychological Bulletin 46:433-89. [rDVC] Leach, C. (1979) Introduction to statistics: A nonparametric approach for the social sciences. John Wiley. [aDVC] Lindsey, D. (1977) Participation and influence in publication review proceedings. American Psychologist 32:379-86. [RFB] (1978) The scientific publication system in social science. JosseyBass. [aDVC, RFB] (1988) Assessing precision in the manuscript review process: A little better than a dice roll. Scientometrics 14:75-82. [LLH] Lock, (1985) A difficult balance: Editorial peer review in medicine. 1SI Pre«s. [aDVC, JF) Lodahl, J. B. (1970) Paradigm development as a source of consensus in scientific fields (unpublished master's thesis). [aDVC] Lord, F. N. At Novick, M. R. (1968) Statistical theories of mental test scores. Addison-Wesley. [LLH] Luce, R. D. (1989) A history of psychology in autobiography, vol. 8, ed. C. Lindzey. Stanford University Press. [PHS] Luce. R. D. At Creen, D. M. (1978) Two tests of a neural attention hypothesis for auditory psychophysics. Perception O Psychophysics 23:363-71. [DL] Luce. R. D., Nosofsky, R. M., Creen, D. M. Ac Smith, A. F. (1982) The bow and sequential effects in absolute identification. Perception 6 Psychophysics 32:397-408. |DL] Machol, R. (1981) Letter to the editor. The Sciences 21:xxi. [aDVC] Mäher. B. A. (1978) A reader's, writer's, and reviewer's guide to assessing research reports in clinical psychology. Journal of Consulting and Clinical Psychology 46:835-38. [aDVC] Mahoney, M. J. (1976) Scientist as subject: The psychological imperative. Ballinger [LLH] (1977) Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy Research 1:161-75. |aDVC, LDN. PHS, SPL, JSAJ (1978) Publish and perish. Human Behavior 7:38-41. [aDVC] (1982) Publication, politics, and scientific progress. Behavioral and Brain Sciences 5:220-21. [AMC] (1985) Open exchange and epistemic progress. American Psychologist 40:29-39. [ADVC. JBC, RFB. JF) (1987) Scientific publication and knowledge politics. Journal of Social Behavior and Personality 2:165-76. [RFB] (1990) Bias, controversy, and abuse in the study of the scientific publication system. Science, Technology, 0 Human Values 15:50-55. [MJM] Mahoney, M. J., Kazdin, A. E. Ar Kenigsberg, M. (1978) Getting published. Cognitive Therapy and Research 2:69-70. [aDVC] Margulis, L. (1977) Letter to the editor: Peer review attacked. The Sciences 17:5, 31. |aDVC] Marsh. H. W. Ac Ball, S. (1981) Interjudgmental reliability of reviews for the Journal of Educational Psychology. Journal of Educational Psychology 73:872-80. [aDVC], HWM (1989) The peer review process used to evaluate manuscripts submitted to academic journals: Interjudgmental reliability. Journal of Experimental Education 57:151-69. [HWM] McCarthy. P., Sharpé, M. R., Spiesel, S. Z., Dolan, T. F., Forsyth, B. W„ De Witt. T. G., Fink, H. D., Baron. M. A. Ac Cicchetti, D. V. (1982) Observation scales to identify serious illness in febrile children. Pediatrics 70:802-09. [rDVC] McCarthy, P. L., Sznajderman, S. D., Lustman-Findling. K., Baron, M. A., Fink, H. D., Czarkowski, N., Bauchner, H.. Forsyth, B. C. At Cicchetti, D. V. (1990) Mothers' clinical judgment: A randomized trial of the acute illness observation scales. Journal of Pediatrics 116:200-06. [rDVC] McCartney, J. L. (1978) Making sense of reviewers' comments. Paper presented to the Southern Sociological Association Meetings, New Orleans, LA. [aDVC] McNcmar, Q. (1947) Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12:15357. [rDVC] (1955) Psychological statistics. Wiley. [CSW] McNutt, R. A.. Evans, A. T.. Fletcher, R. H. At Fletcher, S. W. (1990) The effects of blinding on the quality of peer review. Journal of the American Medical Association 263:137-76. [rDVC, JSA, SPL] Merton, R. K. (1973) The sociology of science: Theoretical and empirical investigations. University of Chicago Press. [aDVC, DLE] Meyer, G. S. (1979) Academic labor and the development of scicncc (unpublished doctoral dissertation). State University of New York at Stony Brook. [SC] Mezzich, J. E., Kraemer, H. C., Worthington, D. R. L. Ac Coffman, G. A. (1981) Assessment of agreement among several raters formulating multiple diagnoses. Journal of Psychiatric Research 16:29-39, [rDVC] Mitroff, 1. I. At Chubin, D. E. (1979) Peer review at the NSF: A dialectical policy and analysis. Social Studies of Science 9:199-232. Sage [aDVC] Mulkay, M. (1977) Sociology of the scientific research community. In: Science, technology, and society, ed. I. Spiegel-Rosing Ac D. de Sofia Price. Sage. [aDVC] Mulkay, M. At William, A. T. (1971) A sociological study of a physics department. British Journal of Sociology 22:68-80. [SC] Murphy. R. J. L. (1978) Reliability of marking in eight GCE examinations. British Journal of Educational Psychology 48:196-200. [DL] (1982) A further report of investigations into the reliability of marking of CCE examinations. British Journal of Educational Psychology 52:5863. [DL] National Research Council (1988) The behavioral and social sciences: Achievements and opportunities. National Academy Press. [MJM] Nelson, L., Satz, P., Mitnishina, M., Van Corp, W., Cicchetti, D., Lewis, R. Ac Van Lancker, D. (1989) Development and validation of the Neuropsychology Behavior and Affect Profile. Psychological Assessment: A Journal of Consulting and Clinical Psychology 1:266-72. [rDVC] Newman, H., Freeman, F. Ac Hoizinger, K. (1937) Twins. A study of heredity and environment. University of Chicago Press. [PHS]