Of That

Brandt Redd on Education, Technology, Energy, and Trust

16 September 2011

Tackling Bloom's 2 Sigma Problem

Recently I wrote about the tyranny of the bell curve. Benjamin Bloom was working on this problem back in the 1980s. As an experiment, he and some of his grad students combined Mastery Learning with 1:1 tutoring. They discovered that average students in the program performed two standard deviations (two sigmas) better than their peers receiving conventional instruction. Using on John Hattie's scales from Visible Learning I equate that to more than four times the rate of learning.

In a seminal paper on the subject, Bloom wrote that that 1:1 tutoring is "too costly for most societies to bear on a large scale" and reported on their efforts to find more scalable solutions. This has become known as Bloom's 2 Sigma Problem.

Like many others working on education technology, I believe that Bloom's 2 Sigma results can be achieved and even surpassed by appropriate use of computer technology. From a number of initiatives, we're getting results that confirm this belief. While approaches vary, they have common elements:

Mastery Learning: That's what Bloom called it. Other terms are Competency Based Pathways and Proficiency Based Learning. There are nuanced differences but the basic premise is that students don't advance until they have demonstrated competency in the current topic.

Asynchronous Learning: Students advance from topic to topic independently. To do mastery learning properly, this is a requirement. However, it doesn't mean that there aren't sync points. For example OLI Courses support students spending variable amounts of time (according to their skills and background) learning the basic material. This way they arrive in class equally prepared for the live debates that are so critical to teaching certain subjects. Some classes resync every Friday with those students who are ahead assisting those who are taking more time. Results from the Khan Academy and School of One are showing us that individual students aren't consistently fast or slow. The slow and fast students trade places from day to day or week to week and overall variability tends to balance out.

Emphasis on Principles more than Facts: A student who has command of the underlying principles of a subject can often derive the facts. And in today's world, memorizing facts is of diminishing importance. It's too easy to look them up.

Strategic Intervention: The teacher is more important than ever. After all, learning is fundamentally a human-to-human process. Deploying online curricula in such a way that supports independent work frees teachers to spend more time one-on-one with students. They are enabled to focus on things only teachers can do: diagnosing misunderstanding, demonstrating the value of the subject, motivating and rewarding achievement and developing a personal relationship with each student. Paradoxically, technology has potential humanize the classroom. In a very important TED talk, Salman Khan says that we should move from measuring the student to teacher ratio to measuring the "student to valuable human time with the teacher ratio." (Quote is at 14:30 but watch the whole thing.) Teacher Dashboards are an important mechanism for informing teachers about where they need to apply their skills.

Posts in this series:
Breaking the Tyranny of the Bell Curve
Tackling Bloom's Two Sigma Problem
The Personalized Learning Model


  1. When I was in first grade, I was in an experimental classroom in California. We had kindergarten through 3rd grade in one classroom with 2 teachers. We were all free to learn all the subjects at our own pace. Small groups would learn certain topics, and then we'd work with each other or alone to get our work done. The teachers would help us if we needed it. I easily worked through two grades (and to about 5th grade in reading, I think) with my friend. I think this matches your "proficiency based learning" model. It's made me wonder if the old idea of the school house with all the kids together wasn't so bad.

  2. I spent this summer putting together a sabbatical proposal (https://sites.google.com/site/sabbaticalucc) to to integrate Adaptive Learning (AL) and Computerized Essay Review (CER) software into developmental courses at Union County (NJ) College These two uses of computer technologies simulate the one-on-one tutoring that Benjamin Bloom found produces a 2 sigma difference (http://bit.ly/qvjvTI) in results. That is, one-on-one tutoring (emulated using AL and CER) gives results where the average tutored student does better than 98% of the students taught with current-traditional classroom techniques. AL and CER may be the ”silver bullets" needed to achieve drastically better learning and graduation rates, particularly for minorities and those who need remediation before starting college work.

    Instead of studying using the exact same text book as every other student, Computerized Adaptive Learning with its automated continuous formative assessment determines:

    customized content which is dynamically generated "instantaneously"
    based on what the student knows and doesn't
    how many lessons on each concept are needed for each individual student to achieve mastery
    the format of lessons that the student learns best with: video, text, interactive exercise or a game

    Computerized Adaptive Learning and computer essay review provides:

    an active, learner centered experience
    24/7 mobile technology solution favored by students
    performance data that is available immediately to students, faculty and administration

    The presentation I created (http://bit.ly/ALCER) explains why and how delegating a portion of content delivery and assessment to Adaptive Learning and Computerized Essay Review software benefits students, faculty and administration.

    I first learned of adaptive learning in 2002 from Alfred Bork of UC Irvine, where I believe the concept and early implementations were born. Faculty from UC Irvine went on to develop ALEKS, a widely used program of adaptive learning of math. As computers became faster and more capable they were finally used to personalize instruction. Carnegie Mellon University's Open Learning Initiative http://oli.web.cmu.edu/openlearning is based on Adaptive learning principle. Companies like Knewton company developed Adaptive Learning offerings for SAT and GMAT preparation. Faculty from CMU created Carnegie Learning, which offered adaptive learning courses to K-12. Apollo Group (Phoenix Univ) just bought Carnegie Learning.

    Several startup companies, have developed Adaptive Learning offerings. Dreambox and HeadSprout are in the K-12 arena. Knewton and Grocit have offered SAT and GMAT preparation for several years. Knewton’s programs are so successful they guarantee a 150 point increase in SAT scores. This year Arizona State University, SUNY and Penn state adopted the Knewton’s AL engine to deliver remedial/developmental instruction.

    Today, Adaptive Learning and Computerized Essay Review are being used by hundreds of thousands of students. Having community colleges to use the two computer teaching modes in their developmental courses is a game changer.

  3. Thanks, Maureen, for the success report! I've seen demonstrations of Knewton and OLI. And I know that several of these are in use at places like Rocketship Education. Many of my colleagues are involved in some of the other efforts you mention. Much of what we're working on at the Gates Foundation is intended to support interoperability among these and to make it easier to focus on one part without having to invent a whole learning system.

  4. There is a looming problem with this conversation as I see it, and it is one of definitions. Bloom's methodology included three modalities- conventional, mastery learning, and tutoring. In each case, CONTENT mastery seems to be in view, with mastery being the ability to apply the principles in a new context. The emerging discussion of competency-based learning certainly requires content mastery as a foundation, but moves beyond content and concept mastery to proficiency in be able to DO SOMETHING with the knowledge- in many cases, not just application in a new context of learned principles, but a skill or ability is in view. I can nowhere find just exactly what Bloom was measuring -was it knowledge, skills, or abilities, and how were these measured- was a product EVER produced as evidence of learning? What was the nature of the formative and summative assessment instruments? The literatures mentions the ability to apply principles within a new context or different problem situation as being one of the strategies; also mentioned is the use of heuristic problem solving. But do either of these assessment techniques approach the demonstration of performance -based competencies in areas that do involve the need to show competence in the performance of a real-world skill?