inBloom - For My Concerned Friends

In my post about the Common Core State Standards I wrote about how concerned pundits have lumped together five related but independent efforts. Today I'm writing about inBloom which I'll contrast Statewide Longitudinal Data Systems – two more of those five.

InBloom is a service designed to help students achieve academic success through personalized learning. Those of us who helped develop the Shared Learning Collaborative (which was renamed inBloom in February) are convinced that personalizing the learning experience is the best way to improve student achievement. Whether personalization is being done by a teacher, an online learning system, or a synergistic combination of the two, it happens when information about what the student needs to learn intersects with information about available learning materials.

With that in mind, we set out to supply teachers and students with the data they need. That's what inBloom does. It taps into existing student data systems at schools, districts and states and makes that data available, in a secure way, to authorized teachers, students and parents. Simultaneously it indexes a library of teaching materials and makes them available to those same individuals.

A lot of work went into preserving student privacy. inBloom requires two things to happen before any student data can be retrieved. First, the application they are using must be authorized by the school district. Second, the individual using the application must be logged into inBloom and be authorized to access the requested data. This protection of student privacy is compliant with and goes beyond the requirements of FERPA and state data privacy laws.

So, who can access student data? Teachers can access data about students who are enrolled in their classes. Parents, if authorized by the school or district, can access their children's data. And students can access their own data. An application, such as a personalized learning system, can only access private student data if an authorized user is logged in to the app.

To match student achievement data against available learning resources, we need a common taxonomy of what it is that students need to learn. It's not sufficient to know that Johnny got an "A" on assignment number 5 but a "C" on assignment number 7. We need to know what learning objectives were represented by each of these assignments. That's why inBloom makes use of the Common Core State Standards. In the data, we can show that assignment 7 was on multi-digit multiplication. And, since it appears that Johnny needs some more practice, we can search the library for multiplication practice that's suitable to his age and preferences.

In a nutshell, inBoom supplies the student and content data needed for effective personalized learning.

Statewide Longitudinal Data Systems

For whatever reason, some people have confused inBoom with Statewide Longitudinal Data Systems (SLDS). The SLDS effort was launched more than a decade ago by the Bush Administration and funded by the Educational Technical Assistance Act of 2002. While a separate statute, it's related to the No Child Left Behind Act of 2001. The official SLDS website describes it this way:

Better decisions require better information. This principle lies at the heart of the Statewide Longitudinal Data Systems (SLDS) Grant Program. Through grants and a growing range of services and resources, the program has helped propel the successful design, development, implementation, and expansion of K12 and P-20W (early learning through the workforce) longitudinal data systems. These systems are intended to enhance the ability of States to efficiently and accurately manage, analyze, and use education data, including individual student records. The SLDSs should help states, districts, schools, educators, and other stakeholders to make data-informed decisions to improve student learning and outcomes; as well as to facilitate research to increase student achievement and close achievement gaps.

Under grants from the SLDS program, 47 states are developing longitudinal data systems that aspire to collect student data from preschool through college and even into workforce placement. Analysis of the data should help researchers understand the impact of different factors and programs on student achievement.

Before being analyzed to find trends, the data is either anonymized or aggregated in order to preserve the privacy of the students. However, the databases themselves necessarily contain personally identifiable information (PII). That's because the data comes from multiple sources: K-12 schools, colleges and workforce databases. In order to connect all of the data about an individual together, you need to be able to match up records and that requires the personal identity information about each individual.

This concentration of individual data spanning decades of educational experiences spooks a lot of people. Two factors help moderate those fears. First, according to federal regulation, data is not combined between states nor is it reported to the federal government. Only aggregate data (sums, averages and so forth) is reported to the federal government. Second, the Family Educational Rights and Privacy Act (FERPA) prohibits the release of any student information without permission from parent. Of course, that doesn't reassure everyone. The mere fact that such databases exist concerns many.

I have a different concern. I've previously written about Theories of Change for educational improvement. In this case, the theory is that over time the collected data will help government officials, education officials, teachers and curriculum developers make better decisions based on what really works. But if we're trying to figure out how a particular curriculum choice in elementary school affects a student's college prospects, it may take 10 years or more to have the data to measure that effect. My concern is that this effort will take a long time to make a difference.

~ ~ ~ ~ ~

inBloom and SLDS both collect student data. Both leverage CEDS definitions for the data fields they collect. But the purposes of the data sets and the people who have access to the data are entirely different. Of the two, I'm more optimistic that inBloom will achieve the impact on student learning that our country needs.

The Common Core State Standards - For My Concerned Friends

Even before their adoption in the Summer of 2010, the Common Core State Standards (CCSS) had their advocates and their critics. Recently, however, that criticism has made its way into the popular press. Knowing that I've worked on related projects at the Bill & Melinda Gates Foundation, friends and family have asked my opinion.

Several of the pundits have conflated five different projects as if they were all the Common Core. These are to some degree related but each has it's own sponsors and they are being managed and adopted separately. They are:

• The Common Core State Standards themselves.
• State or district curriculum.
• State Longitudinal Data Systems (SLDS).
• inBloom
• Race to the Top Assessment Consortia

In this post I'll address the Common Core and what distinguishes it from a curriculum. In a future post I'll write about inBloom and other data systems. And one more post will cover the assessment consortia.

The concept of state core standards gained prominence during the Bush Administration as part of the No Child Left Behind act. In a recent blog post I wrote about how they are part of the Standards and Accountability theory of education reform and how later and more promising theories also rely on quality standards.

The result of NCLB and related efforts is that each of the 50 states developed its own core standards. This has the vague advantage of more local influence but it has two significant disadvantages. First, there are differences between what students learn in different states. So colleges and universities don't have a consistent standard of preparation to expect from students. Second, developers of tests and curriculum spread their resources 50 different ways. The result is lower quality teaching materials and examinations.

Starting in 2008 a consortium of state representatives developed the Common Core State Standards for ELA/Literacy and Mathematics. They don't include Science, Social Studies, History or any other subject. However they do specify literacy standards for Science and Social Studies. In other words, they specify that reading should be a significant part of those subjects without specifying the actual titles or subjects to be read.

The standards are written in the form of "competencies" – that is, descriptions of things that students should be able to do. For example, standard CCSS.ELA-Literacy.RL.8.5 reads, "Compare and contrast the structure of two or more texts and analyze how the differing structure of each text contributes to its meaning and style." Standards like the common core describe what is to be taught while curriculum describes how it should be taught.

Here are some ways that distinction applies: The Common Core describes the difficulty of text to be read at each grade; curriculum gives a list of actual books and stories. The common core describes the kinds of problems a student should be able to solve; curriculum specifies the order concepts will be taught and includes exercises to be performed. The rivalry between Phonics and Whole Language is not resolved by the Common Core; that decision remains in the hands of district curriculum committees.

Critics of the core have missed an opportunity here. Since curriculum involves textbooks, lesson plans and teaching materials, it consists of thousands of pages, tens of hours of video and other media. It's also copyrighted. All of this makes reviewing a curriculum a daunting task – albeit an important one. Meanwhile, the standards are relatively short and accessible. They are released under an open license and you can read them online at http://corestandards.org. They total somewhere around 200 pages long including appendices so you can review them in an afternoon.

They are different from previous standards. The ELA/Literacy standards start with a 50/50 balance between literary and informational texts (fiction and non-fiction) in the lower grades and increase that to a 30/70 split when social studies and science reading are included in upper grades. Reading in the English classes remains a 50/50 split through all grades.

The focus in all texts, whether fiction or non-fiction, is on critical thinking and extracting arguments and meaning from the text itself. As a result, "response papers" where a student expresses their opinion or thoughts about a document are discouraged in favor of more analytical writing that identifies arguments, contrasts perspectives and uses evidence from the documents themselves.

Many English teachers have objected to the shift away from an emphasis on fictional reading and writing. Picking up on that, one pundit suggested that Huckleberry Finn would be eliminated in favor of the phone book. Of course, the phone book isn't what the Common Core means by "informational texts". A sample list can be found in Appendix B of the common core. Remember that actual reading lists are the domain of curriculum. That's why this is in an appendix; it's not normative to the standard. Examples of informational texts in that list include the founding documents of our country, Lincoln's "Gettysburg Address" and Ronald Reagan's “Address to Students at Moscow State University”. This isn't the phone book.

To get an idea of how these texts might be taught, I recommend this video from David Coleman. He was a coordinator and key author of the ELA standards. In this video he demonstrates how to teach the standards using Martin Luther King's "Letter from a Birmingham Jail" and Lincoln's "Gettysburg Address." In both cases he shows the brilliance of the authors and how it's not necessary to teach a lot of background because the authors include the needed information in the texts themselves.

Regarding the math standards, there are two important shifts from existing teaching practice. First is that they have reduced the total amount of information to be taught. The overall theme is narrower and deeper. For example they require fewer methods for solving quadratic equations (narrower), but they also introduce complex numbers and the possibility of an imaginary result to a quadratic (deeper).

The second change is that they teach mathematics at three levels: conceptual understanding, computational and procedural fluency. The overall goal is to help children become "numerate." That is, students should naturally apply mathematics to interpret things in their daily lives.

So, what are the objections? A common one is that this is a federal program to control what our students learn. They only control teaching if the standards are considered to be limits to what is taught. But they are really a floor, not a ceiling and most of the details remain left to the curriculum.

Other objections come from academics arguing for or against certain pedagogical theories that the rest of us aren't familiar with. But the common core aren't as opaque as all of that. As I wrote a couple of months ago, the English standards focus on a few basic skills applied to increasingly complex texts. The math standards cover the familiar topics of arithmetic, algebra, geometry and so forth.

The biggest issue is that change is difficult and frequently unpopular. The changes demanded by the common core aren't easy ones. They require changes to curriculum; they require new lesson plans; and they require teachers to approach subjects in new ways. Many people are excited by the possibilities but it's not surprising that some would prefer to preserve the status quo. Unfortunately, status quo isn't good enough.

The Common Core State Standards offer two important advantages over previous state core standards. First is simply that they are common. We hope that by concentrating their efforts on one standard instead of 45, developers of curriculum and examinations can do a better job than before. The second advantage is that the Common Core is a second-generation standard built on a foundation of the best state standards and informed by the experience of those who built the first generation.

Are they perfect? Not likely. But these new standards are better than previous ones and they will become a valuable tool in our personalized learning arsenal.

A Taxonomy of Education Standards

I've previously posted and updated A Four-Layer Framework for Data Standards. When working with education standards I've also used the following taxonomy that categorizes standards according to their purpose. For convenience, this taxonomy is also available in PDF form under a CC0 dedication.

Types of StandardsThere are three types of standards that are involved educational efforts: Academic Standards, Data Standards and Technology Standards.

Academic Standards include achievement standards like the Common Core State Standards (CCSS) plus curriculum and testing standards. Contemporary practice in the U.S. is to describe academic standards in the form of learning objectives – descriptions of skills that students can acquire or demonstrate. Historically it was more common to describe standards in syllabus form – as a list of subjects to be studied.

Encouraged by the No Child Left Behind Act, the 50 states have each defined core curriculum standards. More recently, the CCSS standards for Mathematics and ELA-Literacy have been adopted by 45 states. Using a similar process, the Next Generation Science Standards have been proposed for multi-state adoption. In higher education there is no such consistency. Some institutions have developed their own sets of standards but most leave the objectives up to the professor. A few industry organizations publish standard sets. These include the AAAS Benchmarks for Science Literacy[3] and the National Center for History in the Schools standards for History.

Data Standards define the data elements and structures used to store and exchange educational information. In the Four-Layer Framework data standards may include layers 1-3 (Data Dictionary, Data Model and Serialization).

For education, the three major domains of data standards are Student Data, Educator Data and Content Data. Important metrics like graduation rate, student financial aid repayment or college-going rate are derived from data sets but aren’t data in and of themselves.

Student Data includes traditional demographic information as well as a student record which includes academic achievements, assessment results, learning activities, attendance and so forth. Educator Data includes information about teachers and staff. It includes qualifying information like academic credentials, a portfolio of creative works and publications and data about teaching performance. Content Data, often called metadata, is information about learning materials including textbooks, assessments, multimedia and digital resources. Content data often indicates the alignment between learning resources and academic standards like the CCSS.

Technical Standards define how systems interoperate. Accordingly, they usually include the protocol layer of the Four-Layer Framework. A wide variety of standards may fit into this category but the majority of education-related technical standards involve Content Packaging Formats, Interoperability Protocols and Data Exchange Protocols.

Content Packaging Formats support the transport of learning content (e.g. text, video, graphics, etc.) and assessments between systems. Examples include IMS Common Cartridge and SCORM.

Interoperability Protocols support interoperability among learning systems. The most common use case is integration of learning tools (like simulations, games or assessments) into learning environments (like a learning management system). Key functions are to identify the user to the learning tool, ensure that they are authorized to access the content, transfer control to the tool, and collect data back. Common examples include OpenID, SAML, OAuth and IMS QTI. Data Exchange Protocols represent layer 4 in the Four Layer Framework for Data Standards. Thus, data exchange protocols are usually paired with a corresponding data standard. Frameworks for setting up data exchange protocols include ESB, SOAP and REST.

Progress Report: The Personalized Learning Model

A bit more than two years ago my colleagues and I at the Gates Foundation came up with the Personalized Learning Model. Eighteen months ago I introduced it on this blog. Two weeks ago, at SXSWedu, we celebrated the launch of inBloom which is a set of services that support the Personalized Learning Model.

The concept of personalized learning was not new or unique to us. Indeed, we chose it because the benefits have been well-proven. Our model was a way to describe how technological supports could be designed to facilitate personalized learning. As we've been working on this for a couple of years now, it's time for a progress report.

Learning Objectives
In 2010 a consortium of states, coordinated by the Council of Chief State School Officers (CCSSO) and the National Governor's Association (NGA), introduced the Common Core State Standards for English/Literacy and Mathematics. They were rapidly adopted by 45 U.S. states. Having common standards across states is, of course, convenient but these standards seek to be an improvement on the previous generation.

The Common Core State Standards were written by building on the best and highest state standards in existence in the U.S., examining the expectations of other high performing countries around the world, and careful study of the research and literature available on what students need to know and be able to do to be successful in college and careers. No state in the country was asked to lower their expectations for their students in adopting the Common Core. The standards are evidence-based, aligned with college and work expectations, include rigorous content and skills, and are informed by other top performing countries. They were developed in consultation with teachers and parents from across the country so they are also realistic and practical for the classroom. (From the CCSS FAQ.)

In August of 2012, the CCSSO and NGA released official identifiers and an XML representation of the Common Core thereby facilitating alignment of digital learning resource to the core standards. Driven by the need to measure and prove coverage of the standards, finer-grained identifiers are being assigned to individual learning objectives within the common core standards.

The Next Generation Science Standards are also under development with an expected release before the end of March. Following their release, state education boards will consider adoption.

Postsecondary education is taking a different approach. There's little formal agreement between colleges and universities on the learning objectives that compose common courses. However, college and university departments are defining the objectives for core curriculum and there is growth in the sharing of these objectives within university systems. Colleges are also considering use of the Common Core for developmental education courses.

Student Data
Common Education Data Standards (CEDS) is a project to create a common data dictionary and logical data model for education data. Applications that align to CEDS use the same definitions for data fields making data exchange easier and increasing fidelity.

The inBloom Data Store uses CEDS for its data model and ingests data in SIF and Ed-Fi data formats. It offers an API through which personalized learning applications can store and retrieve common student data. Security features preserve the privacy of data and ensure that only authorized people can access it.

Newer data stores align student activity and assessment data to standard learning objectives. The goal is derive a model of what the student knows, what the student is learning and what the student has yet to learn. This enables rich reporting on student competency levels on an objective-by-objective basis and the stimulation of targeted interventions.

Content
I prefer to talk about educational content as learning activities. There are the traditional passive media such as reading, lectures, video and so forth. More engaging are interactive activities like virtual labs, simulations virtual worlds and games. For both active and passive content, education doesn't need special formats. The web content formats managed by the W3C are adequate and well-supported. What is needed is a way to represent the alignment between the content or activities and the standard learning objectives.

The Learning Resource Metadata Initiative (LRMI) is a standard way to describe educational materials including their alignment to standards. It's based on the Schema.org metadata standard adopted by Google, Yahoo!, Bing and Yandex.

LRMI metadata can be shared between systems using the Learning Registry. The inBloom index consumes LRMI data from the learning registry and offers a search service that can find educational content suited to specific student needs.

IMS Global defines standards for packaging learning content for import into learning management systems. However, I prefer the approach IMS uses for Learning Tools Interoperability. Instead of packaging content, this protocol allows content from other sites on the web to be seemlessly integrated into the learning experience. Integration in this way avoids limitations imposed by the packaging format and lets the developers of learning activities collect data about the use and effectiveness of their products.

Assessments
In my opinion, assessments are presently the weakest part of the Personalized Learning Model but that's changing rapidly. Two multistate assessment consortia, Smarter Balanced and PARCC are developing new assessments aligned to the Common Core State Standards. Both are committed to supplying formative and interim assessments in addition to year-end summative exams. CoreSpring is pooling assessments from a more than six different sources to supply a bank of good quality assessments that can be used in class, for quizzes and in interactive learning environments. MOOC developers such as Coursera, edX and Udacity are having to invent new ways to offer interactive assessments at extremely large scale.

In the long run, I expect the line between learning activities and assessment activities to blur. After all, much of learning occurs when the student demonstrates understanding. With adequately instrumented activities, the accumulated data about student competencies should reduce the need for big summative exams at the end of the year.

~ ~ ~

We've come a long way in the last couple of years. Pioneers in this space like DreamBox, Knewton, Read180 and GrockIt had to build a whole infrastructure. But now there's a solid set of building blocks on which developers can build personalized learning applications. I anticipate a lot more innovation at the place where student data and content come together.

Theories of Education Reform

My oldest son was a junior in high school when the standardized tests associated with No Child Left Behind were rolled out. One day, shortly before the exams, he asked me, "Why do we have to take these tests anyway?"

I answered truthfully, "They're not evaluating you, they're evaluating your school."

I found out later that with that information, he and his friends challenged each other to get the lowest scores possible. I sometimes use this story to illustrate broken feedback loops. It was nine months later before the scores had impact. When he returned to school the next fall he found he hand been enrolled in remedial math despite aceing Pre-Calculus the previous year. He had to meet with the counselor to get into the right class.

Today, however, I want to explore the theories of education reform that drove the deployment of these exams. There are three prominent theories of reform with a few variations. Most contemporary efforts to improve education are based on at least one of these.

Theory: Standards and School Accountability
This is the primary theory represented by No Child Left Behind (NCLB). It's based on the broader theory that measuring something and reporting on those measurements will bring about improvement – especially if improvement is incentivized. It also represents the truism that if you don't measure something, you can't tell whether you've changed it for the better.

In order to bring about accountability, NCLB requires states to define learning objectives for each year or grade. These objectives are commonly referred to as the state core standards and each U.S. state has its own set. Furthermore, any public school receiving federal funding must administer a state-wide standardized test to every student in grades 3-9 and at least once in grades 10-12. Student scores are compared with previous years' results to determine whether they have achieved Adequate Yearly Progress (AYP). Certain consequences are tied to individual schools' success or failure to achieve progress for all students.

The core of the theory is this: If we set higher standards, measure against those standards and report performance then learning will improve. Unfortunately, 11 years into this experiment the quality of U.S. student learning is nearly flat.

There are numerous criticisms of standards and testing; but my personal concern is that by themselves they are a blunt instrument. In the absence of a proven formula for improvement the result is a form of natural selection – schools that underperform are taken out (actually they "receive interventions") while better performers survive. Natural selection is proven to work but it takes many generations and a lot of the population are brutalized before measurable improvement occurs.

Despite the lack of success, it's not time to abandon standards or accountability. Prior to 2002 most states didn't have well-defined core standards nor was student performance consistently measured. Now all states have standards, we are measuring regularly and 45 of the states have recently agreed to the Common Core State Standards. While standards and testing are inadequate remedies by themselves, they are important assets on which to build.

Theory: Highly Qualified Teacher
Where the Standards and Accountability theory focuses on school improvement. This theory focuses on teacher improvement. It's certainly intuitive; most of us have had one or more great teachers and we know they make a big difference. It's also justified by the data. Studies confirm that teacher quality is an important factor in student achievement and that the variation in achievement between classes within the same school is greater than variation between schools.

NCLB includes a mandate for states to supply highly qualified teachers to every student but it leaves it up to states to determine what it means to be highly qualified. And that turns out to be a problem. Studies show that certain teachers are consistently more effective than others; value added measures can identify which ones they are (albeit with a moderate error rate); but individual teachers often don't know what they need to do to improve.

In raw form this becomes another application of natural selection. If we reward teachers who perform well and eliminate those who don't then eventually performance will improve – assuming we don't run out of teachers beforehand. But many generations will be required and a lot of brutal actions will be taken in the meantime. No wonder there's so much controversy around teacher evaluations being tied to wages and promotions.

I'm actually in favor of merit pay for teachers so long as good quality performance measures are used. But those evaluations need to be deployed concurrently with professional development that informs teachers on how they are doing and what they can do to improve. Conveniently, resources are emerging to support that. For example, the Measures of Effective Teaching project used the Danielson Framework for Teaching to identify teacher behaviors that are well-correlated with student performance. These and similar frameworks can be used to inform teachers on how they can do better.

Even so, effective teachers alone are not enough. In our current educational system, teachers account for approximately 8.5% of variation in student achievement. School-, teacher-, and class-level factors combined account for about 21%. Meanwhile, background characteristics such as race, parental achievement and family income combine to account for 60% of variation in achievement levels.

So, if every teacher in the country was equivalent to our very best, it still wouldn't be enough to overcome the cycle of intergenerational poverty. To achieve that dream, we have to increase the influence school has over student achievement. That can be done by adapting the learning experience to the needs of individual students.

Theory: Personalized Learning
There's a pattern to these theories: The Standards and School Accountability theory introduces the concept of measurement and uses it to assess whole schools. The Effective Teachers theory takes those same measures and applies them at the teacher level. For this third theory, feedback is applied at the student level.

Personalized learning leverages the same standards as the other theories. It can also incorporate the same measures. However, annual testing alone is insufficient for personalization. Instead, understanding is measured weekly, daily or, in the best adaptive learning systems, continuously. Measurement must happen soon enough and feedback given quickly enough to affect learning activities. A truly personalized system selects activities according to student needs and also adapts to student behavior within an activity.

Bloom's Two Sigma experiments and the follow up work they inspired make me optimistic about Personalized Learning. These and other studies have shown that personalized learning experiences enabled by immediate feedback consistently deliver one to two standard deviations improvement in learning. We believe that is sufficient to overcome background factors thereby enabling a majority of students become high achievers.

Personalized learning is the natural result of 1:1 tutoring which is why tutoring is so effective. To do personalized learning at classroom scale generally requires 1:1 computers and a role change for the teacher as she shifts from "deliverer of knowledge" to "facilitator of learning." As with the other theories, there's a lot of skepticism and resistance to change. But pilot deployments are showing great promise.

Variation: School Choice
School Choice attempts to bring competitive pressure for schools to perform better. In this way, it's a variation on the Standards and School Accountability theory. Like NCLB, School Choice needs standards to be set and school performance must be measured against those standards. Performance is reported to parents who are expected to make an informed choice of which school their students should attend.

Since allocation of school funds is tied to enrollment, the theory is that schools seeking students will compete, not only on standards and their measures, but also on the basis of any other factor that's important to parents and students.

School Choice efforts include charter schools, magnet schools and voucher programs. The idea is to give public and private schools more freedom to experiment thereby accelerating the identification of viable formulas for improved leaning. Studies have shown this to be the case as the average of charter school outcomes is similar to that of public schools while variation among charter schools is much greater. Therefore, some charter schools are substantially better and should be emulated while others are substantially worse and should be shut down or reorganized. It's exactly this kind of variety and freedom that school choice advocates seek.

School choice can incorporate Highly Qualified Teachers and Personalized Learning. Indeed, since both of these theories have been shown to be effective, the expectation is that schools that incorporate these principles will be the best rated and will attract more students.

Variation: Small Classes
The small classes movement is based on studies showing that students learn better in smaller classes – all other factors being equal. But other factors are not equal. Lowering the student:teacher ratio costs a lot of money and other factors such as teacher skill have a greater impact than class size. For example, when California mandated smaller classes they had to hire many more teachers. For at-risk populations, the impact of less-experienced teachers overcame the benefits of smaller classes resulting in lower performance instead of the expected improvement.

Variation: No Excuses
The No Excuses model centers on maintaining high expectations for student performance without making excuses for external issues such as background, troubles at home and so forth. It's associated with charter management organizations such as KIPP and BES. Proponents emphasize pillars such as college expectations, culture of respect, voluntary participation and high discipline. They also have extended hours and extended school years. A key value is the whole school's commitment to each student's success. If a student is struggling or falling behind, they discover that early and engage counseling, tutoring and other supports to ensure the student succeeds.

No Excuses engages all three theories, overall school performance is measured, they hire and train highly effective teachers and they adapt the learning environment to the needs of individual students, albeit most No Excuses schools do adaptation with limited use of technology. Over the last decade, No Excuses schools have demonstrated that background factors can, indeed, be overcome by a supportive school structure. On the other hand, their high reliance on supportive interventions sometimes leaves students underprepared for the independent learning discipline required in college. Recognizing this, No Excuses organizations are updating their practices to better train students to become independent learners.

~ ~ ~ ~ ~

As with the variations listed here, most reform projects mix two or more of these theories. Even NCLB includes a mandate for Highly Qualified Teachers. Personalized Learning efforts are more common at charter schools than conventional public schools.

Education Reform will remain an important part of our civic dialog for a long time. Unsurprisingly, it means different things to different people. For some it's a moral crusade. To those being asked or forced to reform it's more threatening. All too often arguments about reform neglect the research (which is abundant) and fail to fully express the theories on which they are based. That shouldn't be the case as there are decades worth of data and studies behind each of these theories – sufficient for advocates and policy makers to make informed decisions.

The data tells those of us seeking to eliminate poverty that incremental improvement to existing schools is insufficient. Personalized learning with an eye toward training independent learners seems to be the most promising approach. Deploying this at scale requires whole-school changes to the way programs are funded, to the choices of curriculum and technology, and to the roles of educators.

Winds of Change: Higher Productivity in Higher Education

Note: This first appeared last week as a guest post on the Next Generation Learning Challenges Blog. I highly recommend both the blog and the NGLC website.

My first lecture hall experience was American Heritage at Brigham Young University. The course was required for all freshmen and more than 500 of us at a time attended two lectures a week. In a third “lab” period we met with a TA. The professor was charismatic and the instructional design team supplied him with carousels full of colorful slides. Still, a large fraction of the class was asleep at any given time.

Large lecture hall courses are one common method of increasing productivity in higher education. Another is weed-out courses – those designed to convince students that they should choose another, less expensive major. For me the weed-out subject was Discrete Structures. This Computer Science subject is rich with metaphors like trees, maps, chains and links. It can be taught through story, modeling, manipulatives and real-world application. But our version was deliberately dry with an emphasis on precise vocabulary and obscure notational forms. The pass rate hovered near 50% and hundreds of students were convinced that they weren't capable of understanding computer science.

Higher education in the United States is sandwiched between twin pressures, increasing societal needs and expectations on one side with flat or declining funding on the other. To meet this challenge, institutions will have to dramatically increase productivity. But traditional productivity boosts like large lecture halls, weed-out courses or greater admissions selectivity won’t be enough this time around. What’s required is fundamental change to the way we support learning. We need a more personalized approach.

Societal Needs and Expectations

Employment projection is from the Bureau of Labor
Statistics Job Outlook. Supply is based on National
Center for Education Statistics data on annual
Computer Science BS degrees awarded. Attrition
is based on a 40-year career span.

While the U.S. unemployment rate hovers around 8%, there is a shortage of engineers and technicians. In 2012, the unemployment rate for software developers was only 2.8%. An Association for Computing Machinery study indicates that the United States will need more than 150,000 new computer scientists each year through 2020 yet our collective colleges and universities only produce 40,000 degree holders to fill those jobs. Healthcare workers are also in short supply. In 2012 the unemployment rate for physicians was 0.8%. For Physical Therapists it was 2.0% and for Registered Nurses, 2.6%.

At a recent Technology Alliance conference it was noted that colleges and universities in Washington State produce less than half as many engineers, technicians and software developers as the state’s employers consume. The rest have to be imported from other states or countries. A speaker from the University of Washington pointed out that they have increased introductory Computer Science enrollment from roughly 1200 to over 2000 per year. But the Microsoft representative responded that they have 3,600 engineering and computer science openings and they’re competing with Amazon, Boeing and many others to fill those spots.

Of 4.3 million freshmen who started college in 2004, only 2.2 million (or 51%) graduated within six years. This isn’t a perfectly accurate figure. Because of the way records are kept, it’s hard to count students who transfer and complete at a different institution. But inadequate record keeping is another symptom that institutions haven’t focused enough on ensuring their students are successful. Higher completion rates will save a lot of wasted student time.

As we move into the 21st century the fraction of unskilled jobs continues to diminish while those requiring advanced skills increase. It’s no longer appropriate to sort students by “aptitude.” We must give students the support and guidance they need to master advanced subjects.

The Funding Landscape

Education is the largest item in most state budgets. In California it accounts to between 52% and 55% of the state general fund. With the recession hitting state revenues and the expiration of stimulus supplements, state fiscal support for higher education dropped by 4.7% between 2011 and 2012, remaining flat in 2013. Overall, annual support has dropped by 10.8% since 2008. On a per-student basis, state and local financing dropped 24% in the 10 years preceding 2011.

At the same time, tuition is rising much faster than inflation. Tuition and fees at U.S. public universities rose 4.8% for the 2012 school year to an average of $8,655. At nonprofit private colleges tuition and fees rose 4.2% to $29,956. In addition to drops in public funding, the cost to provide education is increasing and the recession has diminished private endowments.

Total student debt in the U.S. now exceeds $1 trillion making it higher than the nation’s credit card dept. Student loans aren't a big problem if they are correlated with significantly higher earning potential. But loan approval is not connected with choice of academic major or the graduation rate of the institution.

Personalized Learning

The demands on higher education are greater than ever. We need more graduates – especially in certain fields. We need better completion rates. We need to support students in tackling challenging subjects. Moreover, we have to do this with flat or declining budgets.

The Bill & Melinda Gates Foundation has assembled representatives from a dozen colleges and universities that are trying new approaches with promising results. The Personalized Learning Network, as it's called, includes innovators like Western Governors University and American Public University; pioneering programs at Arizona State University and UC Berkeley; and NGLC grantees like the Kentucky Community & Technical College System, Rio Salado College and Southern New Hampshire University.

Recently I had the privilege of meeting with this group. There’s a lot of variation in their personalized learning programs but they share these common features:

• Mastery Learning and Independent Pacing: Students have to master the current topic before moving to the next step. Self-pacing grants this freedom and ensures that there aren't gaps in understanding due to bad days or illness. And students don’t waste time on topics that they already understand.

• High Expectations: The institutions make a commitment to support all students sufficiently so that they can master the material.

• Feedback: Students and instructors are constantly informed about conceptual understanding and progress through the material.

• Adaptive Learning: The learning system adapts according to individual student actions and performance.

• Individual Attention: The programs facilitate abundant 1:1 time between students and faculty.

• Motivation: Systems and attitudes that foster student motivation include interesting activities, student autonomy, recognizing good performance and avoiding frustration either due to anxiety or boredom.

All of this is enabled through strategic use of technology. Most use some form of blended online and in-person learning. The key point is not to simply add technology but to apply technology in the service of personalized learning.

Personalized learning programs should be able to address higher education pressures for better success and completion rates. But can they also help educate more students at lower cost? I believe so. Technology can automate many tasks that cost a lot of educator time. Video lectures are a personalization technology because they allow students to view on demand and replay as needed. Not only do they save the time in class but they also save the instructor time preparing the lecture. Objective assignments can be graded automatically and feedback given instantly to the student. Feedback to instructors can help them optimize their interactions with students. Subjective grading, while still consuming human time, can also be made more efficient. All of these factors help institutions increase capacity and reduce per-student costs.

Equally important are the savings offered to students. Immediate feedback helps students learn concepts more efficiently and avoids time wasted on misconceptions. Students can advance immediately upon understanding a concept and get credit for things they learned previously. And authentic learning activities support a better and more complete understanding of each topic. In one study by Carnegie Mellon’s Open learning Initiative they were able to teach students the same material in half the time with better retention.

Changing higher education is like turning a glacier. Features like accreditation, tenure, financial aid, credit transfer, and faculty autonomy interlock to form a seemingly insurmountable barrier protecting the status quo. But the twin pressures of increased expectations and diminishing funding result in an unprecedented incentive for change. Like the Maginot Line, traditional barriers won’t be overcome but simply bypassed.

The Common Core State Standards for Literacy are Two Dimensional

The ideal school librarian would know every student in the school – what their interests are, what their current reading level is and what their teachers will be teaching next. With this knowledge, she would use her comprehensive knowledge of the school's book collection to suggest books or activities that would be both enjoyable and yet challenging to the student's abilities. That is, books that are in the student's Zone of Proximal Development.

It's not really possible for a librarian to have such a comprehensive view of both students and the book collection. But under Race to the Top grants, several states are developing Instructional Improvement Systems that, among other things, will support recommendations like these. Such systems operate at the intersection of student data and content data. And to support them, inBloom (formerly the Shared Learning Collaborative) is deploying student and content data services.

The Common Core State Standards (CCSS) and the Learning Resource Metadata Initiative (LRMI) work together to support the content data side when teaching reading and writing. The CCSS for ELA-Literacy have two dimensions to their basic structure. The grid below shows one way to view the Common Core Standards for Reading. Making up the horizontal dimension are Anchor Standards 1-9. These describe specific skills that the student should be able to apply when reading. The vertical dimension is Anchor Standard 10, the requirement that the other nine anchor skills should be demonstrated against texts of increasing difficulty as the student advances from Kindergarten to 12th grade. Notably, grades 9 and 10 share a level as do grades 11 and 12.

Common Core State Standards for Reading Literature

Here's an example of how this works: Anchor Standard for Reading number 6 states:

• CCSS.ELA-Literacy.CCRA.R.6 Assess how point of view or purpose shapes the content and style of a text.

On the diagram this is marked with a vertical gridline. One of the horizontal gridlines is Reading Literature Grade 4 Standard 10. It's statement is:

• CCSS.ELA-Literacy.RL.4.10 By the end of the year, read and comprehend literature, including stories, dramas, and poetry, in the grades 4–5 text complexity band proficiently, with scaffolding as needed at the high end of the range.

I've marked the intersection of these two with the identifier, "RL.4.6". The statement for Reading Literature Grade 4 Standard 6 is:

• CCSS.ELA-Literacy.RL.4.6 Compare and contrast the point of view from which different stories are narrated, including the difference between first- and third-person narrations.

Notice how this last statement is a refinement of anchor standard 6 targeted at a Grade 4 skill level. So, a source text or learning activity that satisfies RL.4.6 would have a text complexity level in the grade 4-5 text complexity band and it would at least use a first-person or third-person narration. Ideally the activity would include both narration forms and give the student a chance to contrast the two.

So, what are these text complexity bands and how do we tell whether a text is within a particular band? In other words, how do we place a text or learning activity on the vertical dimension?

Appendix A of the Common Core State Standards for English Language Arts describes a three-factor model for measuring text complexity. The qualitative factor refers to levels of meaning, structure and demands for prior knowledge on the part of the reader. "Reader and Task" considerations involve matching texts to the reader's needs or interests and the learning tasks that will be associated with the text. The quantitative factor is a numerical measure that is calculated (usually by computer) from word length and frequency, sentence length, vocabulary and text cohesion. A supplement to Appendix A lists six approved scales for indicating quantitative text complexity for the Common Core. The table below indicates which levels are appropriate for certain grade ranges.

Common Core Band	ATOS	Degress of Reading Power®	Flesch- Kincaid	The Lexile Framework®	Reading Maturity	SourceRater
2nd-3rd	2.75-5.14	42-54	1.98-5.34	420-820	3.53-6.13	0.05-2.48
4th-5th	4.97-7.03	52-60	4.51-7.73	740-1010	5.42-7.92	0.84-5.75
6th-8th	7.00-9.98	57-67	6.51-10.34	925-1185	7.04-9.57	4.11-10.66
9th-10th	9.67-12.01	62-72	8.32-12.12	1050-1335	8.41-10.81	9.02-13.93
11th-CCR	11.20-14.10	67-74	10.34-14.20	1185-1385	9.57-12.00	12.30-14.50

The grid diagram also includes an example of how a source text might be fully aligned to the common core literacy standards. In this case, To Kill a Mockingbird is shown as an appropriate text for teaching standards 1-7 at grades 9 or 10. So, the LRMI metadata for To Kill a Mockingbird would include alignment to standards RL.9-10.1, RL.9-10.2, RL.9-10.3, RL.9-10.4, RL.9-10.5, RL.9-10.6, and RL.9-10.7.

On the vertical dimension, To Kill a Mockingbird is positioned toward the middle of the grades 9-10 range. So, it would be considered moderately advanced for grade 9 and moderately easy for grade 10. To Kill a Mockingbird is rated an 870 on the Lexile scale. A quick glance at the table shows that 870 is in the 4th-5th grade range. The book is positioned higher on the grid than the raw Lexile number would indicate due to qualitative factors such as the complex moral dilemmas posed by the text.

The LRMI metadata schema is designed to be flexible enough to represent all of these dimensions. The AlignmentObject type represents the relationship between a text or learning activity and a node in a framework or taxonomy. The most obvious and common way this is use is with a alignmentType of "teaches" or "assesses" and the target node being a statement in the Common Core State Standards. In the To Kill a Mockingbird example, the "teaches" alignmentType would be used with targets of the six standards (RL.9-10.1 to RL.9-10.7). Any one of these six standards also implicitly brackets the vertical, text complexity dimension. In order to more finely position a resource, LRMI also defines a "textComplexity" alignmentType. Publishers of at least two of the quantitative frameworks listed above are in the process writing guidelines for their use with LRMI. It's also possible to use LRMI to indicate non-quantitative factors. To do so, we would need to define taxonomies for qualitative and "reader and task" factors with appropriate identifiers.

In these examples I've used Common Core Standards for Reading but the writing standards have a similar two-dimensional structure. Overall, it's a rich framework with great promise for improving student literacy.

We have achievement standards (CCSS) and data standards (LRMI). There are emerging services like inBloom that build on these standards. I expect very soon a combination of CCSS, LRMI, open libraries of content and custom recommendation engines will offer students custom reading lists and writing activities tailored to their individual learning needs.

Personal Rapid Transit and Driverless Cars

An ULTra PRT vehicle on a test track. (Wikimedia Commons)

As a teenager in the 1970s I remember reading about Personal Rapid Transit in a number of places including this Popular Science article. Unlike conventional transit like light rail or bus systems, a PRT system uses small, individually switched cars on a specially designed guideway. Upon entering a station, you select your destination on a console. Within a few seconds a 3-6 passenger car arrives and whisks you directly to your destination. At least that was the dream.

Of the dozens of proposals and prototypes, the Morgantown PRT that links WVU campuses is the only one of that era ever to be deployed at scale. The rest were either cancelled entirely or were defeatured into automated people mover systems like you find at many airports.

In 2011 two new systems opened, ULTra PRT at London's Heathrow Airport and the 2getthere system in Masdar UAE. Both are relatively small systems each with fewer than five passenger stations and fewer than 25 vehicles. But the new systems also represent an important departure from previous PRT designs. Both use battery powered vehicles with autonomous control. They run on rubber tires and steer themselves so there's no switching gear on the guideway. They are powered by batteries that automatically recharge when the cars wait at stations. This contrasts with previous PRT designs that used powered guiderails and a central control and switching system.

The primary barrier to PRT systems has been the cost of the tracks or "guideways." It's estimated that it would cost beween $30 million and $40 million a mile to expand the Morgantown system. That's because the guideway has to incorporate precision guide curbs, power transmission, track switches and even a heating system to melt snow and ice to keep it safe in bad weather.

In contrast, the ULTra guideway is estimated to cost between $7 and $15 million per mile. That's because it's a simple concrete pathway with no active systems.

Which brings me to Driverless Cars.

In essence, the ULTra and 2getthere systems are self-driving electric cars in which the environment has been constrained enough to simplify the self-guidance problem. High curbs make it easier for the cars to center themselves in lanes, dedicated roadways minimize pedestrian and obstacle avoidance. Strategically placed charging stations let them be electrically powered using batteries of modest capacity.

Meanwhile, Google's self-driving cars have driven themselves more than 300,000 miles accident-free, on conventional roads, without special infrastructure. Like many, I've wondered why Google is building such cars. They're in the information business, not transportation. A talk by Big Data guru, Ed Lazowska clued me in. Before the Google people let a car drive a route by itself, they first have a human drive the car over the same route. During the trip, its sensors scan the environment, picking out landmarks and obstacles, measuring road conditions and fine-tuning its GPS map of the roadway. Google is interested in supplying data to enable driverless cars and they're doing research to determine what data is needed.

A recent Freakonomics post on the subject suggested that driverless cars will arrive incrementally starting with the already common cruise control, adding adaptive cruise control, collision avoidance and self-parking before fully driverless operation arrives.

But I'm afraid that calling these "driverless cars" is the 21st Century equivalent of calling automobiles "horseless carriages". In each case the focus is on what's missing (the driver or the horse) instead of what new capacity has been introduced. "Horseless carriage" doesn't exactly describe a vehicle capable of sustaining 65 miles per hour with a range of over 300 miles. Nor does it conjure images of the megacities it enables or the endless parking lots it requires.

Consider this possibility: driverless technology enables the PRT dream on existing infrastructure. Instead of dedicated guideways costing tens to hundreds of millions, a PRT system built on driverless technology would rely on GPS and 3G data networks, both of which are already in place. Initial deployments can be restricted to certain neighborhoods that meet high standards of traffic signals, lane markings and crosswalk protection. Even a system restricted to certain lanes and certain streets would offer PRT of greater scale and capacity than anything yet deployed. Yet the investment to get started is regulatory permission, a few vehicles and some signage.

Fancy stations aren't required – only some curb space. Cars would be summoned using smartphones. And it wouldn't just be a peoplemover. Cargo, also, could be sent unattended. Grocery stores could use the same infrastructure for home (or corner) delivery. In the long run, even mail delivery and garbage collection could be automated.

We can learn something from this:

There are some fundamental principles at work here that can be applied to other large-scale problems:

• Infrastructure is usually the most expensive component. Whenever possible, use infrastructure that's already in place and share infrastructure with other projects.
• Push control (or decision making) as close as possible to the application or beneficiary.
• Inform the distributed control with global data.
• Build systems that can be scaled incrementally; where adding capacity is a matter of buying more of the same rather than periodic large investments to get to the next capacity threshold.

Consider the above principles applied to education. (You know I can't resist.) Existing infrastructure includes the internet, inexpensive computers and tablets, content development tools, video standards and so forth. Personalized learning relies on giving more control to the student and teacher to adapt learning to individual needs while being informed by common standards. And web-scale technologies are required if systems are to grow to support millions of students.

The "Wouldn't it be Cool" Department

Walt Disney World has the most heavily used monorail system in the world. They also have a well-maintained private road system connecting their resorts and theme parks. Wouldn't it be cool if Disney deployed a PRT system (based on driverless car technology) to connect their resorts and parks together? Such an attraction would enhance the Disney experience while proving the viability of the concept to the world.

Bloom's Two Sigma Problem Revisited

Benjamin Bloom's Two Sigma Problem has been both a guiding framework and a challenge to educators for more than a quarter century. A bit more than a year ago I wrote about the problem and some of the ways people are approaching it.

Here's the concise version: Bloom and some of his grad students compared classroom teaching with 1:1 tutoring. In both cases they used a mastery-based curriculum. The tutored students performed two standard deviations (two sigmas) better than their conventionally taught peers. While it would be nice to have a 1:1 student:teacher ratio, Bloom acknowledged that it's not practical and he proceeded to research ways to achieve similar results using more scalable means. He published the study in 1984. Since then, the Two Sigma Problem has served as a benchmark of how well students can learn if given the right supports.

A recent meta-study by Kurt VanLehn of Arizona State University compares no tutoring (conventional classroom), computer-based Intelligent Tutoring Systems (ITS), and human tutoring. VanLehn notes that a number of well-known ITS efforts have shown one-sigma improvements over conventional instruction. So, the conventional hypothesis is that computer tutors achieve one-sigma gains while human tutors achieve two-sigma gains as compared to conventional instruction.

VanLehn set out to test that hypothesis. He selected numerous studies that collectively yielded more than 100 comparisons between conventional instruction, three forms of ITS, and human tutoring. The result is surprising: answer-based ITS achieved an improvement of 0.31 sigma over conventional instruction. Step-based ITS achieved 0.75 sigma and human tutors achieved 0.79 sigma.

This is mixed news. On the one hand, the best computer tutors are almost as good as human tutors. That suggests that we can scale up much more effective learning than is achieved in conventional classrooms. On the other hand, VanLehn found no replication of Bloom's 2 sigma results. Is Bloom's goal out of reach or is there another factor involved?

To find out, VanLehn retrieved the dissertations from Bloom's grad students that contributed to the more famous paper. One key experiment yielded an effect size of 1.95 sigma – the probable source of Bloom's Two Sigma challenge. In that experiment both the conventional classroom and the tutors used a mastery learning technique. Whether in class or being tutored, students took a quiz after studying each unit. If their score achieved the mastery threshold, they advanced to the next unit. If not, they studied the unit more and were assessed again. This process was repeated until the mastery threshold was achieved.

The missing piece is that classroom students were required to achieve mastery threshold of 80% before advancing. Meanwhile, tutored students were required to achieve a threshold of 90%. Could it be that  adjusting the mastery threshold could account for a full standard deviation improvement in achievement? If so, numerous online learning systems should be tuned accordingly.

Oleg Bespalov and Karen Baldeschwieler, with their colleagues at New Charter University, have evidence to confirm this hypothesis. In their ITS system, students receive periodic formative assessments in the form of multiple-choice quizzes and self-graded short answer questions. From these assessments they calculate a "readiness score" to help students know when they're ready to advance. Students aren't constrained by the score – merely informed.

This creates a natural experiment in which they can compare student performance on the final exam against individual readiness scores. They discovered that students with a readiness above 90 achieved a 98% pass rate. But for those with a readiness score in the 81-90 range the pass rate dropped to 69%.

Both of these projects indicate that there's a critical threshold somewhere between 80% and 90%. Clearly this is an area deserving of more experimentation and research. But we can already tell that that tuning the mastery threshold is a critical factor for improving student achievement.

Measures of Effective Teaching

The Measures of Effective Teaching Project (MET) released its final reports last week. It got considerable press coverage as the study strives to inform teacher evaluation programs, a subject of considerable controversy.

Most of the stories, like this one from Reuters, focus on the the study's finding that teacher performance can indeed be predicted by performance measures. The best evaluations involve a weighted average of student test scores, teacher observations and student evaluations. Any one of these by itself is a much less accurate predictor.

There are nuances to this that can be gleaned from the project's Policy and Practitioner Brief:

• The different measures (student testing, teacher observation and student evaluation) have some overlap but mostly they measure different aspects of the teacher's skills.
• Different weightings are better predictors of different outcomes. Unsurprisingly, placing greater weight on test results is a better predictor of future student test results. However, equal weighting models or those that emphasize teacher observations are more reliable year over year.
• Effective teacher observations are more than a periodic visit from the principal. Evaluations require a consistent framework and procedure. The MET project used the Danielson Framework for Teaching as a rubric. The reliability of teacher observations is greatly improved by having at least two evaluators.
• When done properly, student evaluations are very reliable and an important component of teacher evaluation. As with observations, the key is to ask the right questions. The MET project used the Tripod Student Survey.
• The "value added" theory is supported. When student scores are compared with the previous year's performance (a value added score) the result is a more consistent predictor of future teacher performance than just the most recent year's scores.

One problem with exclusively using standardized tests to evaluate teachers or schools is that it's a blunt instrument. These tests offer a measure of performance but they offer limited guidance to a teacher or school on how they can improve. Sure, we can fire ineffective teachers and close ineffective schools. But using natural selection to improve schools is slow and costly not to mention cruel. Basically you're just hoping for those teachers and schools that randomly find the right formula for success.

Among the advantages of teacher observations and student evaluations are that they supply rich feedback to teachers to help them improve their practice. Another MET project report, Feedback for Better Teaching, offers guidelines for using feedback. They placed cameras in classrooms and observers codified the techniques used by the teachers. The same video recordings were used by the teachers themselves to observe their own performance – usually with an instructional coach. Processes like these can continuously improve teacher skills and effectiveness.

I've written before about how immediate feedback can help the student learn more effectively. In that context, it's no surprise that feedback to teachers helps them to be more effective. Moreover, it supports the passion that got them into teaching in the first place.

Enterprise-Scale is not Web-Scale

In the 90s, the IT world was talking about Enterprise-Scale. It's not that enterprise-scale was anything new. But up until then, the enterprise was the domain of mainframe and minicomputers. Upstart microcomputers – those with the whole processor on a chip – had previously not been capable of enterprise-scale operations.

It took a lot to achieve enterprise-scale with microcomputers. The leaders included Sun, Oracle and Cisco with Microsoft, Intel, Compaq and others playing fast-follower. They invented RAID arrays, symmetric multiprocessing, storage area networks, network load balancing and much more in the pursuit of five nines of reliability.

As microprocessor-based computers achieved enterprise-scale, pioneers like Google, Amazon, Yahoo!, SalesForce and others pushed right past enterprise into web-scale. User counts were measured in hundreds of millions, storage capacity was measured in petabytes and server arrays numbered in the thousands. Unlike enterprise-scale where key technologies had already been invented by the mainframe world, there wasn't any precedent for web-scale and the pioneers had to invent their own methods. I happened to work at Ancestry.com in the late 1990s/early 2000s and got to participate in some of that invention. But it wasn't until later in the decade before the pioneers started to share what they had learned and build products like Amazon Web Services, Google App Engine and Windows Azure to support the web-scale developer.

This is a important issue for education technology. The education industry is a bit behind the curve in moving to web-scale. For example, most learning management systems are built for enterprise-scale. They are intended to be installed on dedicated servers at a college or university's data center and they're architected to handle tens of thousands of students and teachers.

What happens when you move to the K-12 market or to community colleges? These organizations don't have the data centers and skilled staff needed to deploy, maintain and backup enterprise servers like these. In the past, their data systems only had to handle a few hundred or maybe a few thousand administrators. Teachers and students didn't directly access the district's data systems.

But districts are rapidly bringing all of their students and teachers online. And that means two orders of magnitude more users. Many districts have student counts numbering in the hundreds of thousands. Some get into the millions. And since their technology staffs are already overburdened, they seek hosted solutions, not enterprise-scale servers they have to manage themselves. Hosting a single district might not reach web-scale but a cost effective provider would serve hundreds of districts. And web-scale technologies can reduce the cost to something that districts can afford.

Here are some of the principles of web-scale architecture. For purchasers of products and services, these are the things you need to look for. For developers of those services, these are the principles you need to incorporate into your design.

Always Available
Web scale services use redundant servers to ensure that the service is always running – even during software upgrades and system maintenance. The term "24/7" was invented to describe services that have no weekly scheduled downtime. (And please don't write 24/7/365.)

Billions of Database Records
Today a district might keep a few dozen data points per student per year. In a district with 50,000 students that amounts to around 150,000 database rows per year. Eventually the database might grow to a few million rows total.

But personalized learning applications can collect thousands of data points per student per year. And an online service might serve a few million students. Thus, a web-scale learning service should be designed to store billions of data elements with provisions for orders of magnitude growth beyond that as clickstream data become more important.

Single Sign-On and Identity Management
Today's schools typically run a Student Information System, a Learning Management System and a few custom learning systems for specific subjects. Most of these applications have their own user database mandating separate logins and making requiring a lot of data entry to provision the sytems.

The low-hanging fruit is a single sign-on system that lets students and teachers use the same login account across all systems. But single sign-on is of limited value without automatic provisioning. So it's more important to have an identity management system that automatically shares demographic and enrollment information between the Student Information System and the various learning systems. The long-term need is to integrate the data among all of the systems so that all student performance data is accumulated in a common database.

Services-Integrated Security
Consider security in a Student Information System. The student, her teacher and her parents should have access to her school records, but no-one else (except maybe the principal or a counselor). Enterprise-scale security manages things like this through access control lists (ACLs). Record or element has an ACL granting certain levels of access to specific individuals. For example, the teacher has permission to view and change grades while the student and parent only have permission to view them.

The ACL approach becomes fragile at web scale. With millions of students and parents and thousands of changes to class enrollment it becomes difficult to maintain correct ACLs even when the process is automated. Roles and groups offer some relief but inevitably permission errors creep in and they become a technical support nightmare. Even worse, with regulations like FERPA in place, permission errors can result in significant liability.

Instead, web-scale applications use policy-based permissions. When a student is enrolled in a class, the policy says the teacher should be able to access that student's records. There's no ACL to be updated and permission naturally disappears if the student changes enrollment. The databases of these systems describe the relationships between elements (students, classes, teachers, parents, etc.) and the policies describe how permissions should be granted according to those relationships.

Services-integrated security also means that permissions are enforced at all levels of the system. The UI will control permissions that are offered to the user and the API enforces policy when read or write attempts are made. Thus a rogue or buggy application is still prevented from violating security policy.

Developer Note: Policy-based security can be processor and database intensive. The query to determine whether to permit a particular operation can easily be more expensive than the operation itself. This isn't a reason to reject the approach. Rather, use multiple levels of permissions caching to reduce the database burden.

Linear or Sub-Linear Cost Curves

If you graph number of users on the horizontal axis vs total cost on the vertical axis enterprise-scale systems have costs that grow exponentially. This is because they achieve scale by using progressively bigger and more complex servers and one 32-processor server costs many times more lot more than 32 single-processor servers.

In contrast, web-scale architectures have a linear or sub-linear cost curve. They achieve this feat by using software, database and hardware architectures that spread the load across many commodity-priced servers. As demand increases you simply add servers so variable costs are linear and fixed costs get diluted over a large number of users.

Developing scale-out software like this is complicated and expensive. Because of this, enterprise-scale architectures can cost less in enterprise-scale deployments. But when user counts get into the hundreds of thousands or millions, web-scale becomes more cost effective.

Mashups
Web-scale applications rarely stand alone. In most cases, they are combined with other applications to create a complete solution. This is certainly true of our vision for personalized education. A complete solution includes at least the following:

• Student Information System
• Student/Parent Portal
• Teacher Portal
• Adaptive Learning System (probably multiple subject-specific ones)
• Content Library
• Assessment Bank
• Analytics (Teacher, Department, Schools)
• Interactive Professional Development

Plus, other innovative applications are likely to emerge. In a realistic system, these components will originate from a variety of sources. For time-constrained teachers and students to use them effectively, they will have to be seamlessly mashedup together.

Tools and Techniques for Web-Scale

At Ancestry.com we had to invent many of the web-scale tools we used. But the toolkit has matured in the last few years. The easiest approach is to build on one of the cloud platforms like Windows Azure, Google AppEngine or Amazon Web Services. The downside is that doing so locks you into that vendor's hosting service.

Here are some of the other tools and techniques that help in web-scale development:

• Caching (lots of it and at different levels)
• NoSQL Databases
• Content Delivery Networks
• Network Load Balancing
• Browser-side Code
• Multi-tier Architecture

Ultimately, however, you can use all of these tools and still not have a web-scale application. You have to architect for web-scale from the very start.