Testing the Sametest-effect in a BSc-level Business Communication Course Examination

Abstract

Using secondary data, we empirically examine two biasing effects that may arise in the written evaluation of large groups of students. Suppose the students take the examination in consecutive groups, and we wish to avoid the distortion caused by tests of different difficulty. In that case, we can decide to use the same examination questions. However, the danger of the "same test effect" arises, according to which the group writing later can perform better if it receives information from the examinees in the previous round. Using the same examination tests cannot be recommended if that effect is significant. Another related potential phenomenon is the "revealed sameness effect". Accordingly, if the examinees are aware of the repetition of the questions, it significantly increases the scores of the following group. We tested these phenomena using the data of a three-round written examination. A previously published analysis of a larger sample found that the "same test effect" can be expected if the students decide in which round they take the examination. Since it was possible to freely register for the examination rounds for the assessment analyzed in this study, we assume that the "same test effect" will be significant. Based on the literature, we also expected that the "revealed sameness effect" would occur in the third round. The performed linear regression analysis (N=77) only found some weak evidence for the 'revealed sameness effect' but not for the 'same test effect'.

Keywords

• same test effect
• higher education
• business communication
• educational assessment
• assessment bias