Multi-Criteria Evaluation and Benchmarking for Young Learners’ English Language Mobile Applications in Terms of LSRW Skills

Abstract

This study proposes an evaluation and benchmarking decision matrix (DM) on the basis of multi-criteria decision making (MCDM) for young learners' English mobile applications (E-apps) in terms of listening, speaking, reading and writing (LSRW) skills. Benchmarking E-apps for young learners is challenging due to (a) multiple criteria, (b) criteria importance and (c) data variation. The DM was constructed on the basis of the intersection amongst evaluation criteria in terms of LSRW and E-apps for young learners. The criteria were adopted from a preschool education curriculum standard. The DM data included six E-apps as alternatives and 17 skills as criteria. Thereafter, the six E-apps were evaluated by distributing a checklist form amongst six English learning experts. These apps were subsequently benchmarked by utilising MCDM methods, namely, best-worst method (BWM) and technique for order of preference by similarity to ideal solution (TOPSIS). BWM was used for criterion weighting, whereas TOPSIS was employed to benchmark and rank the apps. TOPSIS was utilised in two contexts, namely, individual and group. In the group context, internal and external aggregations are applied. Mean was computed to ensure that the E-apps undergo a systematic ranking for objective validation. This study provides scenarios and a benchmarking checklist to evaluate and compare the proposed work with six relative studies. Results indicated that (1) BWM is suitable for criteria weighting. (2) TOPSIS is suitable for benchmarking and ranking E-apps. Moreover, the internal and external TOPSIS group decision making exhibited similar findings, with the best app being `Montessori' and the worst app being `FunWithFlupe.' (3) For objective validation, remarkable differences were observed amongst the group scores, which indicate that the internal and external ranking results are identical. (4) In the evaluation, the proposed DM revealed advantages over the six relative studies by 40.00%, 53.33%, 40.00%, 46.67%, 46.67% and 46.67%.

Keywords

• Language learning app evaluation
• language learning app assessment
• language teaching/learning strategies