• If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • Finally, you can manage your Google Docs, uploads, and email attachments (plus Dropbox and Slack files) in one convenient place. Claim a free account, and in less than 2 minutes, Dokkio (from the makers of PBworks) can automatically organize your content for you.


Meta-Analysis Assignment

Page history last edited by Jack Busby 9 years, 9 months ago

Evaluation of Evidence-Based Practices in Online Learning

A Meta-Analysis and Review of Online Learning Studies


Each of you please answer the following questions on the class wiki as indicated, although I want you to “talk/collaborate” with one another in the class as you answer these questions.


1.      What were the measures used in this study?

2.      How did the researchers define “better”?

3.      How did the researchers define “performance?


Note to Dr. McFadden,  At least 10 of the classmates held a conference call online (3/9/2011) to discuss this topic.  Some of us made comments below, and looked at other's answers when considering how to respond ourselves in an effort to "talk/collaborate."  The recording of the online conference call did not take so we could not post it here to share with others and you like we'd hoped. 


List your names in alphabetical order below (LastName, First Name).


Busby, Jack

Evans, Traci

Fields, Xan

Hardy, Anthony

Hill, Patricia

Jones, Joy

Mealer, Crystal

Miller, Tracy

Morgan, Pamela

Nichols, Adam

Outlaw, Veronica

Stazel, David

Trosclair, Kevin

Watson, Stephanie

Williams, Christopher


Analysis: meta analysis paper Evaluation of Evidence-Based Practices in Online learning.pdf



Comments (16)

jsfields@crimson.ua.edu said

at 5:06 pm on Feb 22, 2011

Good News: This is the same study that we looked at as part of our first collaborative effort in 570 and 566 this semester.

I thought I'd start a stream here in the comments for chatting about these questions. I'm open to other suggestions for collaborating, though.

Patricia said

at 8:59 pm on Feb 22, 2011

I knew that this information looked familiar - I couldn't put my finger on it at the time. Thanks for the refresher. Great idea and initiative to get the discussion started. I have more reading to do before I am READY to tackle the assignment. Very thought provoking comments though.

jsfields@crimson.ua.edu said

at 5:07 pm on Feb 22, 2011

About Question 1: MEASURES

jsfields@crimson.ua.edu said

at 5:13 pm on Feb 22, 2011

Page 17 of the document tells us they measured 50 independet effect sizes within 45 studies. Boy, that's a lot of measuring!

Getting a little deeper, though, we find that the study looked mainly at higher education and medical education classes that were either entirely online or blended. Strictly face-to-face was not a part of this meta-analysis (p. xii).

jsfields@crimson.ua.edu said

at 12:49 pm on Mar 9, 2011

Now that I've re-read it when I'm not drugged from being sick...it seems they did measure F2F also.

jsfields@crimson.ua.edu said

at 5:14 pm on Feb 22, 2011

About Question 2: "BETTER"

jsfields@crimson.ua.edu said

at 5:16 pm on Feb 22, 2011

I need to read through the report when I'm not on so much pain medication...but right now it seems that "better" simply refers to the effect sizes they found when measuring grades of students in online classes versus blended classes.

Patricia said

at 11:19 pm on Feb 25, 2011

Could this also refer to their use of "observed learning advantages" and "expansion of learning"

jsfields@crimson.ua.edu said

at 5:14 pm on Feb 22, 2011

About Question 3: "PERFORMANCE"

jsfields@crimson.ua.edu said

at 5:35 pm on Feb 22, 2011

p. 51 states that "all effects have been based on objective measures of learning" which leads me to believe the researchers were looking at grades of students in different online or blended classes to determine where higher grades showed enough of an effect size to say that one method was "better" or not.

jsfields@crimson.ua.edu said

at 5:37 pm on Feb 22, 2011

My comments below are meant to be a conversation starter, so I left a lot of my thoughts out of it. I look forward to chatting with you all about this.

Stephanie Watson said

at 10:57 pm on Feb 22, 2011

Thanks for getting us on track Xan. I'm devoting tomorrow to reading all this. Will give my thoughts during the day.

Kevin Trosclair said

at 12:20 pm on Mar 2, 2011

Reading studies like this, everything starts to blend together. Am I missing something when I say that doc's terms of "better" and "performance" are not directly addressed in the study but only a relative term when regarding the outcome of the studies, being either "not significant" or resulted in "stronger learner outcomes"?

Stephanie Watson said

at 5:57 pm on Mar 4, 2011

What Kevin Said.

Stephanie Watson said

at 8:24 pm on Mar 9, 2011

We had a great meeting tonight on Gotomeeting.com but unfortunately I did not record it right :( If someone remembers what we said maybe you can put up a note. I think what we basically came to the conclusion of on this is that the terms better and performance aren't really defined that clearly in this study. Although "better" seems to mean that based on the objective measurements (tests) that as long as the result was statistically better p>.05 then it was better. Performance means how they do on tests, and how the teachers teach their class???

Additions ?

jsfields@crimson.ua.edu said

at 9:26 pm on Mar 10, 2011

good summary of our call, Stephanie. Well done.

You don't have permission to comment on this page.