Quasi-experimental quality evaluation of educational-purposed user-generated contents under a stochastic multi-criteria environment

Abstract

As social platforms experience an influx of diverse content from users, the need to determine high-quality contributions becomes crucial, especially for educational purposes. This paper highlights the pivotal role of quality in assessing how educational-purposed user-generated content (UGC) shapes user experiences, fosters engagement, and establishes credibility. This study proposes a computational framework using a quasi-experimental evaluation through the sorting-based ELimination Et Choice TRanslating Reality, termed ELECTRE-SORT, with a dataset randomly generated from normally distributed user evaluations. Considering the diverse nature of contents, the method evaluates 16 educational-purposed UGC videos from different online media platforms (i.e. Facebook, YouTube, TikTok). These videos were categorized based on their concordance and discordance to three (3) main criteria: content quality, design quality, and technology quality. Employing the ELECTRE-SORT reveals that most UGC videos (i.e. 14 out of 16) fall into the “medium quality” category, possessing a considerable standard for the quality of educational purpose content. Their characteristics generally satisfy the quality attributes and can be used to guide the development of future relevant UGC videos. Finally, to demonstrate the robustness of the proposed approach, we presented a sensitivity analysis by designing different weight assignments to the quality attributes. Practical insights are outlined in this work.