|Journal of Engineering Education, Vol. 94, No. 3, pp. 319-326, July 2005.
|Effects of behavioral anchors on peer evaluation reliability
|M.W. Ohland, R.A. Layton, M.L. Loughry and A.G. Yuhasz
This paper presents comparisons of three peer evaluation instruments tested among students in undergraduate engineering
classes: a single-item instrument without behavioral anchors, a ten-item instrument, and a single-item behaviorally
anchored instrument. Studies using the instruments in undergraduate engineering classes over four years show that the
use of behavioral anchors significantly improves the inter-rater reliability of the single-item instrument.
The inter-rater reliability (based on four raters) of the behaviorally anchored instrument was 0.78, which was not
significantly higher than that of the ten-item instrument (0.74), but it was substantially more parsimonious.
The results of this study add to what is known about evaluating studentsí performance in teams. This is critical since
the ability to function in multidisciplinary teams is a required outcome of engineering graduates.
| ©2005 Journal of Engineering Education.
|Return to list of publications