Automatic Exam Correction Framework (AECF) for the MCQs, Essays, and Equations Matching

Abstract

Automatic grading requires the adaption of the latest technologies. It has become essential especially when most of the courses became online courses (MOOCs). The objectives of the current work are (1) Reviewing the literature on the text semantic similarity and automatic exam correction systems, (2) Proposing an automatic exam correction framework (HMB-AECF) for MCQs, essays, and equations that is abstracted into five layers, (3) Suggesting equations similarity checker algorithm named “HMB-MMS-EMA”, (4) Presenting an expression matching dataset named “HMB-EMD-v1”, (5) Comparing the different approaches to convert textual data into numerical data (Word2Vec, FastText, Glove, and Universal Sentence Encoder (USE)) using three well-known Python packages (Gensim, SpaCy, and NLTK), and (6) Comparing the proposed equations similarity checker algorithm (HMB-MMS-EMA) with a Python package (SymPy) on the proposed dataset (HMB-EMD-v1). Eight experiments were performed on the Quora Questions Pairs and the UNT Computer Science Short Answer datasets. The best-achieved highest accuracy in the first four experiments was 77.95% without fine-tuning the pre-trained models by the USE. The best-achieved lowest root mean square error (RMSE) in the second four experiments was 1.09 without fine-tuning the used pre-trained models by the USE. The proposed equations similarity checker algorithm (HMB-MMS-EMA) reported 100% accuracy over the SymPy Python package which reported 71.33% only on “HMB-EMD-v1”.

Keywords

• Automatic exam correction
• document embedding
• expression trees
• MCQ matching
• word embedding