Validity evidence based on internal structure of a pilot scientific literacy assessment instrument

Abstract

This study presents the validity evidence based on internal structure gathered from the application of a pilot instrument to assess the scientific literacy of students at the end of the 3rd cycle of basic education. The instrument, composed of the subtests nature of science, impact of science and technology on society and content of science, was applied to 176 students in the 10th grade from eight Portuguese schools. Evidence was gathered using the Item Response Theory information function. The analysis revealed that the three subtests show evidence of validity based on internal structure that makes it possible to use the results for decision making. Nevertheless, it was found to be necessary to revise the items of the content of science subtest in order for them to better fit the characteristics of the sample.