Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)

DOMENIC V. CLCCHETTI: The Reliability of Peer Review for Manuscript and Grant Submissions: A Cross-Disciplinary Investigation

105 CICHETTI: THE RELIABII

.ITY OF PEER REVIEW Newman, S. H. (1966) Improving the evaluation of submitted manuscripts. American Psychologist 21:980-81. (aDVC) Nisbett, R., Ót Ross, L. (1980) Human inference: Strategies and shortcomings in human judgments. Prentice-Hall. [HLR] Nisbett, R. E. öt Wilson, T. D. (1977) The halo effect: Evidence for unconscious alteration of judgments. Journal of Personality and Social Psychology 35:250-56. |RFB) Noble, J. H. (1974) Peer review: Quality control of applied social research. Science 185:916-21. (aDVC) Nunnally, J. C. (1978) Psychometric theory, 2nd ed. McGraw-Hill. [aDVC] Orlinsky D. ót Howard, K. (1978) The relation of process to outcome in psychotherapy. In: Handbook of psychotherapy and behavior change, ed. S. Garfield it A. Bergin. John Wiley ót Sons. (LDN) Orr, R. H. it Kassab, J. (1965) Peer group judgments on scientific merit: Editorial refereeing. Paper presented to the Congress of the International Federation of Documentation, Washington, D.C. (aDVC) Over, R. (1982) What is the source of bias in peer review? Behavioral and Brain Sciences 5:229-30. (aDVC) Oxman, A. D., Cuyatt, G. H., Singer. J.. Goldsmith, C. H., Hutchison, B. C., Milner, R. A. it Streiner, D. L. (1991) Agreement among reviewers of review articles. Journal of Clinical Epidemiology 44:9198. [rDVC] Patterson, E. H. (1969) Evaluation of manuscripts submitted for publication. American Psychologist 24:73. (aDVC) Patterson, K. it Bailar, J. C. Ill (1985) A review of journal peer review. In: Selectivity in information systems: Survival of the fittest, ed. K. S. Warren. Praeger Scientific. (aDVC) Peters, C. (1976) Multiple submissions: Why not? American Sociologist 11:165-79. (aDVC) Peters, D. P. it Ceci, S. J. (1982) Peer-review practices of psychological journals: The fate of published articles submitted again. Behavioral and Brain Sciences 5:187-255. [aDVC, AMC, DLE) Pfeffer, J., Leong, A. it Strehl, K. (1977) Paradigm development and particularism: Journal publication in three scientific disciplines. Social Forces 55:938-51. [aDVC] Physical Review it Physical Review Letters (1987) Annual report 1986. (rDVC) Pollack, I. (1952) The information of elementary auditory displays. Journal of the Acoustical Society of America 24:745-49. [DL] (1953) The information of elementary auditory displays. II Journal of the Acoustical Society of America 25:765-69. (DL) Price, D. de Sollt (1963) Little science, big science. Columbia University Press. (aDVC) Reid, L. N., Soley, L. C. it Wimmer, R. D. (1981) Replications in advertising research: 1977, 1978, 1979. Journal of Advertising 10:313. (aDVC) Relman, A. S. (1978) Are journals really quality filters? Rockefeller Foundation working papers (conference, May 22-23). Coping with the biomedical literature explosion: A qualitative approach. [aDVC] Remington, M., Tyrer, P. J., Newson-Smith, J. ót Cicchetti, D. V. (1979) Comparative reliability of categorical and analogue rating scales in the assessment of psychiatric symptomatology. Psychological Medicine 9:76570. (aDVC) Rennie, D. (1986) Guarding the guardians: A conference on editorial peer review. Journal of the American Medical Association 256:239192. (MJM) Roberts, W. A. (1976) Failure to replicate visual discrimination learning with a 1-min delay of reward. Learning and Motivation, 7, 313-25. [TRZ] Robertson, P. (1976) Towards open refereeing. New Scientist 71:410. (aDVC) Robinson, W. S. (1957) The statistical measurement of agreement. American Sociological Review 22:17-25. (aDVC) Rodman, H. (1970) The moral responsibility of journal editors and referees. American Sociologist 5:351-57. |RC] Roediger, H. L. Ill (1987) The role of journal editors in the scientific process. In: Scientific excellence: Origins and assessment, ed. D. N. Jackson and J. P. Rushton. Sage. (LLH, HLR] Rogot, E. 6r Goldberg, I. D. (1966) A proposed index for measuring agreement in test-retest studies. Journal of Chronic Diseases 19:9911006. I rDVC] Romanczyk, R. C., Kent. R. N., Diament, C. ót O'Leary, K. D. (1973) Measuring the reliability of observational data: A reactive process. Journal of Applied Analysis 6:175-84. [JDC] Rosenfield, N. S., Ablow, R. C., Markowitz, R. I., DiPietro, M., Seashore, J. H., Touloukian, R. J. Ót Cicchetti, D. V. (1984) Hirschsprung Disease: Accuracy of the barium enema examination. Radiology 150:393400. [rDVC] Rosenthal, R. (1979) The "file drawer problem" and tolerance for null results. Psychological Bulletin 86:638-41. [PHS] (1984) Meta-analytic procedures for social research. Sage. | RH] (1987) Judgment studies: Design, analysis, and meta-analysis. Cambridge University Press. [RR] Rosenthal, R. Ót Rosnow, R. L. (1984) Essentials of behavioral research Methods and data analysis. McGraw-Hill. [RR] (1985) Contrast analysis: Focused comparisons in the analysis of vanance. Cambridge University Press. [RR] Rosenthal, R. öt Rubin. D. B. (1978) Interpersonal expectancy effects: The first 345 studies. Behavioral and Brain Sciences 3:377-415. [PHS] (1982) A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology 74:166-69. [RR] Rourke, B. P. ót Costa, L. (1979) Editorial policy II. Journal of Clinical Neuropsychology 1:93-96. [aDVC] Rowney, J. A. ót Zenisek, T. J. (1980) Manuscript characteristics influencing reviewers' decisions. Canadian Psychology 21:17-21. |aDVC] Roy, R. (1985) Funding science: The real defects of peer review and an alternative to it. Science, Technology, and Human Values 10:7381. [aDVC] Rubin, D. B. (1982) Rejection, rebuttal, revision: Some flexible features of peer review. Behavioral and Brain Sciences 2:236-37. [PHS] Scarr, S. (1982) Anosmic peer review: A rose by another name is evidently not a rose. Behavioral and Brain Sciences 5:237-38. [aDVC] Scarr, S. ót Weber, B. L. R. (1978) The reliability of reviews for the American Psychologist. American Psychologist 33:935. [aDVC, LJS] Schönemann, P. H. (1971) The minimum average correlation between equivalent sets of uncorrected factors. Psychometrika 36:21-30. [PHS] (1989) New questions about old heritability estimates. Bulletin of the Psychonomic Society 27:175-78. [PHS] " Schönemann, P. H. ót Wang, M. M. (1972) Some new results on factor indeterminancy. Psychometrika 37:61-91. [PHS] Scott, W. A. (1974) Interreferee agreement on some characteristics of manuscripts submitted to the Journal of Personality and Social Psychology. American Psychologist 29:698-702. [aDVC, CAK] Sharp, D. W. (1990) What can and should be done to reduce publication bias? Journal of the American Medical Association 263:1390-91. [SPL] Shrout, P. E., Spitzer, R. L. ót Fleiss, J. L. (1987) Quantification of agreement in psychiatric diagnosis revisited. Archives of General Psychiatry 44:172-77. |rDVC] Smart, R. (1964) The importance of negative results in psychological research. Canadian Psychologist 5a.225-32. [aDVC] Smigel, E. O. öt Ross, H. L. (1970) Factors in the editorial decision. American Sociologist 5:19-21. |aDVC] Smith, K. (1977) Letter to the editor: Peer review defended. The Sciences 17:5. [aDVC] Snedecor, C. W. öt Cochran, W. C. (1967) Statistical methods, 6th ed. Iowa State University Press. [RR] (1980) Statistical methods, 7th ed. Iowa State University Press. [RR] Solomon, D. L. (1989) Editorial communication. Biometrics, June 21. [PHS] Sommer, R. öt Sommer, B. A. (1984) Reply from Sommer öt Sommer. American Psychologist 39:1318-19. [aDVC] Soper. H. V., Cicchetti, D. V., Satz, P., Light. R. öt Orsini, D. L. (1988) Null hypothesis disrespect in neuropsychology: Dangers of alpha and beta errors. Journal of Clinical and Experimental Neuropsychology 10:25570. [aDVC] Sparrow, S. S.. Balla. D. A. ót Cicchetti, D. V. (1984a) The Vineland Adaptive Behavior Scales: A revision of the Vineland Social Maturity Scale by E. A. Doll. I. Survey form. American Guidance Service (rDVC) (1984b) The Vineland Adaptive Behavior Scales: A revision of the Vineland Social Maturity Scale by E. A. Doll. 11. Expanded form. American Cuidance Service. [rDVC] (1985) The Vineland Adaptive Behavior Scales: A revision of the Vineland Social Maturity Scale by E. A. Doll. III. Classroom edition. American Cuidance Service. [rDVC] Spearman. K. (1927) The abÜities of man. MacMillan [PHS] Spitzer, R. L. öt Fleiss. J. L. (1974) A reanalysis of the reliability of psychiatric diagnosis. British Journal of Psychiatry 125:341-47. [aDVC] Steiger, J. J. öt Schönemann, P. H. (1976) A history of factor indeterminancy. In: Theory construction and data analysis in the behavioral sciences, ed. S. Shye. Jossey-Bass. [PHS] Steinberg, M., Rounsaville, B. öt Cicchetti, D. V. (1990) Interview for DSMIII-R dissociative disorders: Preliminary report on a new diagnostic instrument. American Journal of Psychiatry 147:76-82. [rDVC] Sterling, T. D. (1959) Publication decisions and their possible effects on inferences drawn from tests of significance - or vice versa. Journal of the American Statistical Association 54:30-34. [aDVC] Stevens, J. C. öt Tulving, E. (1957) Estimations of loudness by a group of untrained observers. Atnerican Journal of Psychology 70:600-05. |DL]