Orthographic neighborhood size volatility analysis in the context of writing assessment

Abstract

Writing assessment was conducted using the orthographic neighborhood size as a numerical measure of transforming literacy to numeracy. The neighborhood size was calculated using the free online CLEARPOND technology from Northwestern University. The transformed numerical information can be viewed as a time series where fluctuation analysis could be applied. The concept of volatility used in stock market analysis was employed to analyze the fluctuation of the orthographic neighborhood sizes in short essays assigned to students in an introductory astronomy course offered by an open admission community college. The results showed that writing containing scientific words with low neighborhood sizes exhibited high volatility, consistent with high-demand cognitive tasks students were asked to perform. A random shuffling of the words in a sentence generated word-list sequences where the students would be asked to exercise rules of grammar to select the correct sequence. The hypothesis that the reading of a high volatility in the neighborhood size histogram would be expected to correspond to a high-demand cognitive task was tested in this exploratory project. The CLEARPOND output contained a list of phonological neighborhood sizes and the association of orthographic volatility with phonological volatility is also studied with additional examples from presidential debate sentences and song lyrics. This proof of concept volatility project brings stock market analysis, based on econometric methodology, into writing and language studies that demand precision comparison. The volatility analysis results in comparison to lexical diversity analysis results are also discussed.

Keywords

• Volatility
• Orthographic neighborhood size
• Phonological neighborhood size