Relating a concordance-based cloze test to the model of communicative language ability: a verbal protocol study

Abstract

Abstract The literature of empirical studies on the concordance-based cloze test (ConCloze) is far and few between. This is despite the fact that it has a potential for item writing without the aid of native speakers and for making use of corpus-based technology in task design. This article explores the test-taking processes and strategies of a ConCloze item variant. The aim is to investigate the substantive aspect of the construct validity for the item type and increase the generalizability of the findings in the universe of admissible observations. The sample consists of 14 non-native English users who are in higher education and engage with 3 test tasks each, totaling 42 verbal reports. The sampling method is purposive sampling, in which their first language profiles are maximized in terms of heterogeneity for increased power of generalization. It is found that Reading concordance lines and recognizing clue words inside is a major process, and Assessing item components and testing a meaningful compatibility of a word in context a secondary one. A model of communicative language ability is used to provide a context for task use in this study, where strategic competence is represented in ConCloze substantive validity.

Keywords

• Concordance-based cloze
• Item type
• Verbal protocol analysis
• Test-taking processes
• Substantive validity
• Communicative language ability