Evaluation of Evidence-Based Practices in Online learning.pdf

1.      What were the measures used in this study? *

This study is a study of other studies. While the "other studies" used various measures of student performance, etc., THIS STUDY first employed several measures (see next paragraph) to screen over 1000 empirical studies of online learning to determine which effects could be subjected to meta-analysis. 

The screening required that the studies under consideration contrast online and F2F learning, that they measured learning outcomes, that the research design used was rigorous, and that enough information was provided to calculate an effect size.  These were the initial measures used to screen the studies.  From this, 50 independent effects were identified and subjected to meta-analysis.

2.      How did the researchers define “better”? *

Learning outcomes for online students were compared with those of F2F students by taking the difference between the means for each group in any given study and dividing by the pooled standard deviation.  This was the measurement of "better" used by the researchers. It is referred to as "effect size." Blended learning "beat" F2F learning by even more than pure online learning did (i.e. it had a greater effect size).

3.      How did the researchers define “performance? *

Performance was defined in terms of the effect size and its statistical significance. Thus the two groups were compared (blended learning students vs. F2F students, or Online-only vs. F2F students), to determine the effect size (as described above) and the probability of such an effect size occurring purely by chance.

* It should be noted that ultimately, the "measures" used, and the definitions of "better" and "performance" were determined by the researchers in each individual study, moreso than by the researchers who conducted the meta-analysis of these studies.