Grading MOOCs

Selena Larson took a 5 week MOOC on Coursera entitled Terrorism and Counterterrorism: Comparing Theory And Practice. She wrote an article on her experience in ReadWrite, March 08, 2014. She believes traditional university classrooms are doomed and is interested in the problems we face in finding a good substitute. One big problem is on the student side; on Coursera, the average student retention rate is only 4%. No more than 51 percent of students passed Udacity’s online math program offered at San Jose State. Apparently the average MOOC completion rate is only 6.8 percent.  

She reports what one might expect: The course was casual and unmotivated. Something was missing in the tension between the student and the subject. But she reports another interesting problem that was unexpected in several ways. One can take these courses with no investment for free--the way a course might be audited for no grade--or one can be graded. She chose to be graded (one pays a few dollars and gets a certificate). The course grade depends upon a few easy quizzes--quizzes that can be retaken if one does poorly--and a few short essays. The quizzes are graded by computer but the essays are graded anonymously by the student's peers. 

The graders--her peers--savaged her. She failed all the essays and failed the course.

There are some interesting questions here. Is anonymity bad? Do students have competitive animosity? Are they just unqualified? Is the author unqualified? But there was an interesting insight. Ms. Larson wrote an essay on the Oklahoma City Bombing and in it used a letter Tim McVeigh wrote. Sounds pretty reasonable. Then she reveals that one reviewer declared the source meaningless because it was from Fox News. That disreputable news source, said the reviewer, invalidated all her other sources. But McVeigh wrote the letter to Fox News. That's where the letter was. One can debate the relative lack of bias--Fox, NYT, Washington Times--but this sounds very close-minded and bigoted, especially when Fox was the very source of the letter. It gets stranger. Ms. Larson then says "Normally I would have agreed with the commentator."

While the average MOOC completion rate at 6.8%, and the six most-completed courses relied on automatic testing, not peer review grading.

I'll bet.