Braun Tibor, Schubert András (szerk.): Szakértői bírálat (peer review) a tudományos kutatásban : Válogatott tanulmányok a téma szakirodalmából (A MTAK Informatikai És Tudományelemzési Sorozata 7., 1993)
ANGELO S. DENISI, W. ALAN RANDOLPH and ALLYN G. BLENCOE: Potential Problems with Peer Ratings
167 D E NISI & AL.: POTENTIAL PROBLEMS WI TH PEER RATINGS recognized. Perhaps foremost among these is the short time perspective studied. It is suspected that performance following negative peer ratings would deteriorate over time, but there are several other possibilities. Negative ratings conceivably could motivate a worker to try even harder in order to improve the ratings received; thus performance might actually improve over time. It also is possible that, over time, group members might become accustomed to poor peer ratings and ignore them completely. A different limitation of this study arising from the short time perspective is that in existing work groups, cohesiveness and satisfaction generally have had time to develop more fully than in this study, and workers might be less susceptible to changes following negative peer ratings. One could argue, however, that learning that close and trusted peers have given negative ratings might have an even greater debilitating effect on the group. In any event, given more time together, the apparent deterioration in group-member relations might well lead to a decline in actual performance. There is a clear need for further research with a more realistic time horizon. Other limitations to the external validity of the present study stem from the manipulation of the peer feedback. Negative peer ratings were operationalized here as a mean rating of about 2.5 on a 7-point scale, but rarely would one peer rate another so poorly. Thus, it is possible that the relatively strong effects found for negative peer feedback were due to the extreme nature of the feedback received. Nonetheless, such a reaction would indicate that subjects did believe that such low ratings were possible. Of course, in existing groups, negative ratings probably are defined more by group norms than by scale points, and what seems to be a much more positive rating could actually be viewed as quite severe by group members. In general, even given these limitations, the results of the present study suggest the need for further research on peer ratings and peer evaluations in general. This research needs to go beyond demonstrating significant relationships between peer evaluations and some criterion measures. This study, performed in a controlled setting, found that knowledge of how one's peers have rated a person had a definite impact on group behavior (especially when those ratings were negative). This finding cannot be ignored simply because the setting was somewhat artificial. Instead, field research needs to be conducted utilizing designs that allow assessment of effects over longer periods of time so as to understand how group interactions and patterns of behavior might be affected by feedback from peers. References Adams, J. S. Injustice in social exchange. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 2). New York: Academic Press, 1965, 267-300. Bales, R. F. Interaction process analysis: A method for the study of small groups. Cambridge, Mass.: Addison-Wesley, 1950. Bernardin, H. J. The effect of reciprocal leniency on the relationship between consideration scores from the LBDQ and performance ratings. Proceedings of the 40th Annual Meetings of the Academy of Management, Detroit, 1980, 131-136.
