數位學習實驗研究品質評估與現況分析：以行動學習為例 Quality Assessment and Situational Analysis of Experimental E-Learning Designs: A Case Study of Mobile Learning

Abstract

由於近年來行動裝置與教育應用軟體的快速發展，探討行動裝置輔助學習效果的實驗研 究大量增加。然而，目前尚缺乏以評估實驗研究品質為主題所進行的回顧性研究。因此，本 研究的目的即為廣泛蒐集 ERIC 和 SSCI 資料庫自 2003 年至 2013 年間，197 篇以行動裝置輔 助學習的實驗研究，並探討其在實驗設計嚴謹度、統計考驗之過程與結果的正確性，以及成 效評估工具的品質等議題。研究結果顯示，實驗設計以準實驗研究占最多數（61%），但有四 分之一的準實驗研究未考量實驗組與控制組之起點能力均等性問題。統計結果效度部分則發 現，半數以上研究未符合統計基本假設，七成研究所採用之樣本數偏低，使其統計考驗力低 落，效果值估計情形未臻精準。成效評估工具品質部分，則發現半數研究者未提供相關的測 驗信度與效度資訊。最後，本研究根據研究結果與討論，對未來研究者提出相關建議。 Because of the substantial development of mobile devices and educational software in recent years, the results of mobile learning–based interventions represent a popular research topic for investigation. Although experiment quality is the basis of empirical research, few studies have explored this issue. Thus, the present study investigated the shortfalls in existing experimental research designs related to mobile learning over the past decade and offers suggestions in this paper. The researchers collected data from all 197 experimental studies on mobile learning published in the Education Resources Information Center and Institute of Science Index from 2003 to 2013. The findings of the present study are described as follows: (1) Quasi-experimental designs represent the most frequently used design type (61%); however, among the quasi-experimental studies, 25% did not consider baseline equivalence. (2) Over half of the studies may not have met basic statistical assumptions, and approximately 70% used insufficient sample sizes, leading to low statistical power and imprecise effect size estimation. (3) Half of the studies did not provide information on test reliability and validity. Finally, this paper discusses the results and their implications for future research and practice.

Keywords

• 行動學習
• 研究品質
• 實驗設計
• experimental design
• mobile learning
• research quality