Assessing Information and Communication Technology Competence of Students: Approaches, Tools, Validity and Reliability of Results

Abstract

The study describes the Information and Communication Literacy Test (the ICL Test), an instrument to measure information and communication technology competence in middle school students (grades 7–9). An overview of the existing instruments proves that there are no other ICT competence assessment instruments applicable to Russian students today. The assessment was constructed using an innovative systematic approach (evidence-centered assessment design, ECD). The ICL Test results were calculated using Bayesian networks, which are more effective than the Item Response Theory in this case. The ICL Test revealed a high level of construct, content, convergent, divergent and face validity as well as good reliability. The instrument conforms to both Russian and international educational standards according to Webb’s criteria for alignment. The ICL Test results were used to assess ICT competence and identify the factors that influence its development in regions of Russia, Armenia and Belarus. The conclusion is that the ICL Test can be applied to both Russian and foreign education systems.

Keywords

• tests
• validity
• information and communication technology competence
• information and communication literacy test
• competence
• competence assessment
• cross-disciplinary educational outcomes