Skip to content

Lean Research Methodology

June 28, 2012


Dear Mr. Eric Reis, ImagineK12, and New Schools Venture Fund,

I am the director of a national network of visionary early-vangelist educators who love play testing beta ed. tech. We work with lean ed. tech. startups who use our feedback to develop awesome educational technologies.

As you may already know, the education market has some broken bits. I’m writing this letter because there’s one particular bit that I’m having trouble fixing. The problem was brought to my attention by Karen Cator, Director of the Office of Educational Technology, during her discussion of excellence and innovation in education at ISTE. In her talk, she identified a critical flaw in current frameworks used by entities like the Department of Education’s What Works Clearinghouse (WWC)-which sets standards for reviewing scientifically based research designed to determine if an intervention shows a positive effect on student learning.*

To be in accordance with the WWC standards, entrepreneurs must freeze the development of their product for the duration of the study. As Cator explained, its like the researchers are telling these entrepreneurs “Hold it still and don’t change it while we do a study against something else.”

This requirement flies in the face of lean startup methodology and becomes straight up absurd when one considers how many updates would typically occur during the 1-2 years it would take to complete WWC compliant research. These standards are fit for the educational technology that was made in the 1980s. Back in the day, aka 1986, The Learning Company created Reader Rabbit. Three years passed before they released Reader Rabbit 1.0 on floppy disk. Having won multiple awards back then, the first version for the ipad Reader Rabbit 1.0 ipad app. was updated to version 1.01 in less than five months. Today’s technology can’t “hold still” for 1-2 years and it shouldn’t have to.

The bottom line is that my students and I don’t have 1-2 years to wait for someone to figure out what works; nor do we want entrepreneurs and technologists to hold back from making important updates for that long.  I’m on a mission to give all teachers and students access to awesome technology and I’m not willing to watch us stand in our own way.

We’ve developed lean methods for developing a product, now we need to do the same for researching their efficacy. There’s incredible work being done in this field by genius researchers like Dr. Roland Fryer, Jr, the Robert M. Beren Professor of Economics at Harvard University, and faculty director at Harvard’s Ed Lab-an education research and development laboratory devoted to closing the achievement gap. Its going to take the most innovative researchers, teachers and entrepreneurs to raise the level of excellence and innovation in our nation’s education system. I want to thank you for the work you’ve already done towards this end. This summer I’m going to do everything I can to find a rapid and equally valid research methodology to demonstrate the positive effects  that innovations have on student learning.


Jennie Dougherty

@betaclassroom | English Teacher at Brockton High School | Director of Innovative Teacher Network at edUpgrade

*For more information see awesomely clarifying post by Lesson Cast and 3 part article on “what works” platforms by Center for American Progress. 

No comments yet

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: