The Perverse Vocabulary of Feedback Loops

I come from a family of engineers. Naturally, we cannot talk about feedback loops without getting into control theory. Conveniently, engineering control theory can be adapted to education reasonably well as soon as you get over some vocabulary hurdles.

An open-loop control system.

Here is a basic control system. One example might be a cruise control for your car. In that case, the reference is the speed you want to go, e.g. 65 miles per hour. The controller translates that reference to an input value – the throttle position . Then the system (the engine, transmission, drive train, tires) produces an output – the actual speed of the car.

In control theory, this is an "open-loop" control system meaning that it has no feedback. The controller must have a very good mathematical model of the system and the system itself must be very precise to get a predictable output. Open loop systems are used when there's a large acceptable margin of error. A cooling fan, for example.

A closed-loop control system.

A car with an open-loop cruise control would slow down as it climbed hills and speed up when it descended. For more precise control, real cruise controls use a feedback loop. A sensor detects the output value (the speed of the car) and returns it to the controller. The controller take the difference between the feedback and the reference values and adjusts the input (throttle) accordingly.

Mathematically, taking the difference is subtraction so we call this "negative feedback." In contrast, "positive feedback" sends systems wildly out of control. A familiar example is pointing a microphone at a speaker. The terrific squeal that you hear is a "positive feedback loop."

So, when talking about control systems, "closed loop" is better than "open loop" and "negative feedback" is good while "positive feedback" is bad. The precise engineering vocabulary is the opposite of what we might use in casual conversation. Let's now apply this to education.

A personalized learning system.

With a few word substitutions we get a personalized learning system. For a moment, let's ignore the feedback loop. In an open-loop learning system a set of educational standards (such as the various state or common core standards) are translated into instruction in the form of textbooks, lectures and exercises. These activities are delivered to the student resulting in skills. Conventional schooling is much like this. In order to handle a class of a dozen or more students, all students perform the same activities. But, remember that open-loop systems require a very precise system to get predictable results. Students come to us with different personalities, talents, preferences and backgrounds. The result is a wide margin of error as illustrated by the spectrum of grades assigned at the end of of a course.

Good teachers do better than this. They find ways to adapt their teaching to the needs of individual students. Good students, being intelligent, can also use feedback to adapt their learning to match the instructional methods being used. Both kinds of adaption require feedback from assessment. And to do this well, the feedback should be compared with the standards for what is intended to be taught.

For feedback to be really effective, it must be frequent, fast and rich. Feedback must come often so that course corrections are made frequently. It must be fast, ideally immediate; otherwise it's too late to affect the learning process. And it should be rich. "Incorrect" is not nearly so meaningful as, "You misplaced the negative sign in step 3."

Frequent, fast and rich feedback depends on frequent, fast and rich assessments. In conventional schooling, assessment is expensive. It costs a lot of teacher time to compose, administer and score assessments. This is an excellent opportunity to apply technology. Computer supports can minimize the effort required to compose, administer and score assessments. And computers can tabulate the results into effective teacher and student dashboards. The result is a more personalized and effective learning experience. Or, as an engineer might say, "more predictable output with a smaller margin of error."

FETC Reminds Me How Hard it is to Change

Last week I attended the FETC conference. This picture from the show floor inadvertantly captures the de facto theme: "21st Century Technology that does not change the way you work.

The audience was composed of about 50% classroom teachers, 25% other school officials the the balance of technologists, vendors, district and state people and so forth. This an education technology conference and the teachers who attend are tech savvy and forward thinking. But the framework remains a conventional classroom and the attitude seems to be "Take school, add technology, mix well and serve."

In my presentation I argued for a different perspective. Digital abundance is nice but we've been adding technology to classrooms for decades with little impact on student learning. In order to the big improvements that society is demanding of schools, we have to change our education systems. Teachers will be more important than ever -- i'm under no illusions that computers can replace them. But just as the practice of business had to change to take advantage of new technology, so must the practice of education.

Resources for Today's Speech at FETC

I'm speaking today at the FETC conference. Here are links to some of the resources I reference in my talk:

• Jane McGonigal: Reality Is Broken http://www.janemcgonigal.com 
• Clayton Christensen – Disrupting Class http://www.innosightinstitute.org 
• Duncker’s Candle Problem http://anthonyogrady.wordpress.com/2010/05/02/carrots-and-sticks/ 
• Dan Pink: The Surprising Science of Motivation http://www.ted.com/talks/dan_pink_on_motivation.html 
• Learning Resource Metadata Initiative: http://lrmi.net
• Shared Learning Collaborative: http://slcedu.org

These previous posts on this blog discuss some of the same subjects:

Breaking the Tyranny of the Bell Curve
Tackling Bloom's 2 Sigma Problem
The Personalized Learning Model
The Learning Resource Metadata Initiative

Blackout: SOPA, PIPA

In solidarity with Wikipedia, ReddIt and others I'm posting a black message on my blog today in protest of SOPA and PIPA. For more information, Wikipedia has an excellent writeup.

Was Lou Gehrig’s ALS Caused by Tap Water?

My good friend (and sometimes mentor) Paul Cox has been researching ALS (Amyotrophic Lateral Sclerosis or Lou Gehrig's Disease). His theory, that ALS is caused by a non-protein amino acid called BMAA, was at first viewed with skepticism. But now his Ethnomedicine Foundation coordinates research at more than 21 universities worldwide. Collectively they've published numerous papers in peer-reviewed journals.

This new article offers an excellent introduction to his work:
Was Lou Gehrig’s ALS Caused by Tap Water?

Here's a quick summary: Cyanobacteria, also known as blue green algae, create BMAA which can contaminate water and food. Some plants, such as the Cycad Tree concentrate BMAA. The Flying Fox bat, which eats the nuts of the Cycad tree, concentrates it more. The Chamorro, natives of Guam, consider the flying fox to be a delicacy and they suffer from ALS and related diseases at a much higher rate than other populations.

The trail was difficult to follow and prove because BMAA gets incorporated into the body's proteins, possibly substituting for Serine, a protein amino acid. When incorporated into protein, BMAA is undetectable using normal methods. To find it, they had to break up proteins using enzymes and then test. It can also take many years from the time that a person is exposed to BMAA before symptoms appear. Cox theorizes that this is because BMAA can get bound up into proteins in the body's tissues and not delay symptoms until those tissues are broken down and replaced many years later. There was also skepticism about the substitution theory -- that BMAA might be used place of another amino acid. But evidence is building that substitution does occur.

BMAA is also being implicated in Parkinson's Disease and Alzheimer's Disease. Most exciting is the potential that this research may result in effective treatments for these terrible diseases. 

Quote: Peter Sandman on Climate Change Outrage

If you’re talking to a room full of people who hate the idea of the set of remedies you have proposed for climate change and instead of trying to reduce their outrage about the remedies, you’re busy trying to increase their outrage about climate change, you’re fighting the wrong fight.

(Peter Sandman, Quoted on Freakonomics)

Video Presentation - Changing the Rules to the Game of School

My presentation at the iNACOL VSS Conference two weeks ago was very well-received. The video version is below, it's just voice over slides but it flows pretty well.

VSS Presentation: Changing the Rules to the Game of School

Today I'm speaking at the iNACOL VSS conference. The following outline of my presentation is primarily a resource for attendees but others may find it valuable as well.

A New Perspective on Technology:

Changing the Rules to the Game of School

• Thesis
• The Game of School was designed around scarce resources but new technology offers abundance where scarcity once ruled.
• Digital Abundance
• Abundant Content
• LRMI
• Learning Maps
• Abundant Assessment
• Games
• Game of School
• Goal (from Disrupting Class)
• 1840: Preserve the Democracy
• 1890: Prepare Everyone for Vocations
• 1980: Keep America Competitive
• 2000: Eliminate Poverty
• Rules
• Failure Consequences
• Proxies for Achievement
• Unbundling
• Feedback
• Formative Assessment
• Control Theory
• Pedagogical Theory
• Voluntary Participation
• Duncker’s Candle Problem
• Motivation
• Autonomy
• Mastery
• Purpose
• Design of the Game
• Resources & References
• Jane McGonigal: Reality Is Broken
  http://www.janemcgonigal.com
• Clayton Christensen – Disrupting Class
  http://www.innosightinstitute.org
• Duncker’s Candle Problem
  http://anthonyogrady.wordpress.com/2010/05/02/carrots-and-sticks/
• Dan Pink: The Surprising Science of Motivation
  http://www.ted.com/talks/dan_pink_on_motivation.html
• Learning Resource Metadata Initiative:
  http://lrmi.net
• Watch for announcements on OfThat.com
  of Abundant Assessments, Learning Maps and more.

The Learning Resource Metadata Initiative

Try this: Browse to Google's Homepage and search for a recipe. Given the season, try "Pumpkin Pie." On the left the Recipe Search Bar automatically appears because Google sensed that a lot of the matches were recipes. Now you can narrow the list by selecting those that do have maple syrup but don't include amaretto since it's missing from your spice cabinet. There are also options to select for cooking time and calories.

Now, suppose teachers and students had the same kind of facilitation for their searches. This week I was experimenting with AdWords and discovered that 246,000 people searched for "right triangle." (many of them probably teachers) and 60,500 people searched for "triangle calculator" (most likely students). Wouldn't it be cool if such searches resulted in a Learning Search Bar that let you choose between videos, activities and lesson plans; or that let you target a particular age group or find resources for students with specific disabilities? Indeed, there were 880 searches for "math for the blind."

That's the idea behind the Learning Resource Metadata Initiative. In June, Google, Yahoo and Microsoft jointly announced Schema.org. This is a common metadata vocabulary for describing things like blog posts, audio recordings, organizations, places, news articles and things for sale. Then they encouraged industry-specific consortia to submit extensions to the vocabulary. So, we formed LRMI to represent the education industry.

It's an amazing group. Co-funded by the Hewlett and Gates foundations and co-sponsored by Creative Commons and the Association of Educational Publishers the team has representatives from major educational publishers as well as OER repositories. The technical working group involves a cross section of educators and metadata experts. We're making excellent progress and are on schedule to solicit public comments on a draft specification starting in December.

Before long, the search for quality learning materials on the web will become much easier.

Quote of the Day: Bill James on Trust

"We have a better society when we can trust one another. And wherever and whenever there’s an evaporation of systems based on trust I think there’s a loss to society. I also think that one evaporation of trust in society tends to feed another, and that we would have a better society if we could, rather than promoting fear and working to reduce the places where terrible things happen, if we could promote trust and work on building societies in which people are more trustworthy. I think we’re all better off in a million different ways if and when we can do that."

- Bill James, being interviewed by Freakonomics (quote appears at the very end of the podcast. Transcript here)

The Personalized Learning Model

The first two parts of this series discussed the Tyranny of the Bell Curve and a strategy for Tackling Bloom's Two Sigma Problem. In this third and last part I describe the Personalized Learning Model that many of us are using to guide investments in education technology.

The diagram to the right is similar to those used by other education technology organizations so it's not unique to the Gates Foundation. The key components in most any Adaptive Learning System or Instructional Improvement System are Student Data, Educational Content and Assessments. We use precise definitions of these:

• Learning Objectives are specific competencies to be learned in a particular subject domain. Most courses, both online and legacy media, start with a set of learning objectives. However, if data, content and assessment systems are to interoperate, a common set of objectives must be shared among them.
• Student Data is a collection of  evidence of what competencies or skills a student has achieved. On a scale of weak to strong evidence, it includes presence information (the student attended a class), activity information (the student viewed a particular video or performed a lab) and assessment results.
• Learning Content includes reading materials, textbooks, interactive activities, lesson plans, exercises and any other content that's intended to teach about a subject.
• Assessments are student activities that are instrumented in such a way that we can measure competence in knowledge or skills. You can think of multiple choice and true/false as activities that are deliberately simplified to make them easier to instrument. However, assessment technology is advancing in ways that make it possible to instrument more realistic activities.

The Feedback Loop describes the process of learning, from determining what a student doesn't know, to teaching the subject, to assessing competency. For the feedback loop to work effectively, it must cycle frequently supplying rich and accurate feedback to students and educators.

Most of our education technology investments involve some combination of improving the state of practice in these areas and improving interoperability among systems. Future posts in this blog will profile some of the most important initiatives we and others are working on.


Posts in this series:
Breaking the Tyranny of the Bell Curve
Tackling Bloom's Two Sigma Problem
The Personalized Learning Model

On Track for 50% of High School Courses Online by 2019

In the 2008 book, Disrupting Class, Clayton Christensen applied his theories of disruptive innovation to education. By that time, disruptive innovation had been studied well enough that Christensen and his colleagues could predict the adoption curve of such an innovation. It's an impressive feat -- telling us how soon something new is going to impact our lives.

They predicted that by 2014, 25% of high school courses would be taken online and that by 2019 fully half of them will be taught that way. When Christensen and his colleagues talk about online education, they include blended or hybrid formats in that bucket. This is important because the evidence shows that it's a blend of online materials and personal attention that results in superior learning outcomes.

In a recent Washington Post Column, Christensen and co-author Michael Horn offer an update citing emerging examples like Khan Academy in Los Altos and Rocketship Education. "In the year 2000, roughly 45,000 K-12 students took an online course. In 2010, roughly 4 million did." Then they reassert their prediction of 50% of high school courses online by 2019.

Three years into the prediction, we seem to be on track.

Steve Jobs: How to Live Before You Die

As a teenager I learned to program on an Apple ][. First BASIC and then Pascal and assembly language. I played computer games, hacked them and then wrote my own. I have fond memories of those times. But none of that, nor the careers that followed for me and countless others would have happened without Steve Jobs. There's hardly a person in the world whose life hasn't been impacted in some way by his vision and drive to see it realized.

I join many others in recommending the following speech he made at the 2005 Stanford University Commencement. Fittingly titled, "How to Live Before You Die":

May he rest in peace.

We Need an Energy Breakthrough

I haven't yet read The Quest by Daniel Yergin -- only this review. But it's nice to know that someone who has spent a career studying energy issues agrees with my conclusions. We need a breakthrough in energy technology. The environmental burden caused by fossil fuels is too great for us to rely on that source as we try to elevate the standard of living for the world's populations.

Tackling Bloom's 2 Sigma Problem

Recently I wrote about the tyranny of the bell curve. Benjamin Bloom was working on this problem back in the 1980s. As an experiment, he and some of his grad students combined Mastery Learning with 1:1 tutoring. They discovered that average students in the program performed two standard deviations (two sigmas) better than their peers receiving conventional instruction. Using on John Hattie's scales from Visible Learning I equate that to more than four times the rate of learning.

In a seminal paper on the subject, Bloom wrote that that 1:1 tutoring is "too costly for most societies to bear on a large scale" and reported on their efforts to find more scalable solutions. This has become known as Bloom's 2 Sigma Problem.

Like many others working on education technology, I believe that Bloom's 2 Sigma results can be achieved and even surpassed by appropriate use of computer technology. From a number of initiatives, we're getting results that confirm this belief. While approaches vary, they have common elements:


Mastery Learning: That's what Bloom called it. Other terms are Competency Based Pathways and Proficiency Based Learning. There are nuanced differences but the basic premise is that students don't advance until they have demonstrated competency in the current topic.

Asynchronous Learning: Students advance from topic to topic independently. To do mastery learning properly, this is a requirement. However, it doesn't mean that there aren't sync points. For example OLI Courses support students spending variable amounts of time (according to their skills and background) learning the basic material. This way they arrive in class equally prepared for the live debates that are so critical to teaching certain subjects. Some classes resync every Friday with those students who are ahead assisting those who are taking more time. Results from the Khan Academy and School of One are showing us that individual students aren't consistently fast or slow. It slow and fast students trade places from day to day or week to week and the overall variability tends to balance out.

Emphasis on Principles more than Facts: A student who has command of the underlying principles of a subject can often derive the facts. And in today's world, it's easy enough to look up facts that memorizing them is diminishing in importance.

Strategic Intervention: The teacher is more important than ever. After all, learning is fundamentally a human-to-human process. Deploying online curricula in such a way that supports independent work frees teachers to spend more time one-on-one with students. They are enabled to focus on things only teachers can do: diagnosing misunderstanding, demonstrating the value of the subject, motivating and rewarding achievement and developing a personal relationship with each student. Paradoxically, technology has potential humanize the classroom. In a very important TED talk, Salman Khan says that we should move from measuring the student to teacher ratio to measuring the "student to valuable human time with the teacher ratio." (Quote is at 14:30 but watch the whole thing.) Teacher Dashboards are an important mechanism for informing teachers about where they need to apply their skills.

Posts in this series:
Breaking the Tyranny of the Bell Curve
Tackling Bloom's Two Sigma Problem
The Personalized Learning Model

A Four Layer Framework for Data Standards

Recently I've been getting involved in a number of education data efforts. It's an alphabet soup of standards and specifications including CEDS, LRMI, SIF, PESC, Ed-Fi and more. As we've discussed these specs and how they fit together we developed a four-layer framework for how different data standards fit together. Our one-page outline of the framework has been used in ways we didn't foresee. I recently updated it with feedback from the CEDS team. Click here for the latest version. It's released under a CC0 license which pretty much means do what you want with it but don't blame me if something goes wrong. And see below for a graphic version.