A Taxonomy of Education Standards

I've previously posted and updated A Four-Layer Framework for Data Standards. When working with education standards I've also used the following taxonomy that categorizes standards according to their purpose. For convenience, this taxonomy is also available in PDF form under a CC0 dedication.

Types of StandardsThere are three types of standards that are involved educational efforts: Academic Standards, Data Standards and Technology Standards.

Academic Standards include achievement standards like the Common Core State Standards (CCSS) plus curriculum and testing standards. Contemporary practice in the U.S. is to describe academic standards in the form of learning objectives – descriptions of skills that students can acquire or demonstrate. Historically it was more common to describe standards in syllabus form – as a list of subjects to be studied.

Encouraged by the No Child Left Behind Act, the 50 states have each defined core curriculum standards. More recently, the CCSS standards for Mathematics and ELA-Literacy have been adopted by 45 states. Using a similar process, the Next Generation Science Standards have been proposed for multi-state adoption. In higher education there is no such consistency. Some institutions have developed their own sets of standards but most leave the objectives up to the professor. A few industry organizations publish standard sets. These include the AAAS Benchmarks for Science Literacy[3] and the National Center for History in the Schools standards for History.

Data Standards define the data elements and structures used to store and exchange educational information. In the Four-Layer Framework data standards may include layers 1-3 (Data Dictionary, Data Model and Serialization).

For education, the three major domains of data standards are Student Data, Educator Data and Content Data. Important metrics like graduation rate, student financial aid repayment or college-going rate are derived from data sets but aren’t data in and of themselves.

Student Data includes traditional demographic information as well as a student record which includes academic achievements, assessment results, learning activities, attendance and so forth. Educator Data includes information about teachers and staff. It includes qualifying information like academic credentials, a portfolio of creative works and publications and data about teaching performance. Content Data, often called metadata, is information about learning materials including textbooks, assessments, multimedia and digital resources. Content data often indicates the alignment between learning resources and academic standards like the CCSS.

Technical Standards define how systems interoperate. Accordingly, they usually include the protocol layer of the Four-Layer Framework. A wide variety of standards may fit into this category but the majority of education-related technical standards involve Content Packaging Formats, Interoperability Protocols and Data Exchange Protocols.

Content Packaging Formats support the transport of learning content (e.g. text, video, graphics, etc.) and assessments between systems. Examples include IMS Common Cartridge and SCORM.

Interoperability Protocols support interoperability among learning systems. The most common use case is integration of learning tools (like simulations, games or assessments) into learning environments (like a learning management system). Key functions are to identify the user to the learning tool, ensure that they are authorized to access the content, transfer control to the tool, and collect data back. Common examples include OpenID, SAML, OAuth and IMS QTI. Data Exchange Protocols represent layer 4 in the Four Layer Framework for Data Standards. Thus, data exchange protocols are usually paired with a corresponding data standard. Frameworks for setting up data exchange protocols include ESB, SOAP and REST.

Progress Report: The Personalized Learning Model

A bit more than two years ago my colleagues and I at the Gates Foundation came up with the Personalized Learning Model. Eighteen months ago I introduced it on this blog. Two weeks ago, at SXSWedu, we celebrated the launch of inBloom which is a set of services that support the Personalized Learning Model.

The concept of personalized learning was not new or unique to us. Indeed, we chose it because the benefits have been well-proven. Our model was a way to describe how technological supports could be designed to facilitate personalized learning. As we've been working on this for a couple of years now, it's time for a progress report.

Learning Objectives
In 2010 a consortium of states, coordinated by the Council of Chief State School Officers (CCSSO) and the National Governor's Association (NGA), introduced the Common Core State Standards for English/Literacy and Mathematics. They were rapidly adopted by 45 U.S. states. Having common standards across states is, of course, convenient but these standards seek to be an improvement on the previous generation.

The Common Core State Standards were written by building on the best and highest state standards in existence in the U.S., examining the expectations of other high performing countries around the world, and careful study of the research and literature available on what students need to know and be able to do to be successful in college and careers. No state in the country was asked to lower their expectations for their students in adopting the Common Core. The standards are evidence-based, aligned with college and work expectations, include rigorous content and skills, and are informed by other top performing countries. They were developed in consultation with teachers and parents from across the country so they are also realistic and practical for the classroom. (From the CCSS FAQ.)

In August of 2012, the CCSSO and NGA released official identifiers and an XML representation of the Common Core thereby facilitating alignment of digital learning resource to the core standards. Driven by the need to measure and prove coverage of the standards, finer-grained identifiers are being assigned to individual learning objectives within the common core standards.

The Next Generation Science Standards are also under development with an expected release before the end of March. Following their release, state education boards will consider adoption.

Postsecondary education is taking a different approach. There's little formal agreement between colleges and universities on the learning objectives that compose common courses. However, college and university departments are defining the objectives for core curriculum and there is growth in the sharing of these objectives within university systems. Colleges are also considering use of the Common Core for developmental education courses.

Student Data
Common Education Data Standards (CEDS) is a project to create a common data dictionary and logical data model for education data. Applications that align to CEDS use the same definitions for data fields making data exchange easier and increasing fidelity.

The inBloom Data Store uses CEDS for its data model and ingests data in SIF and Ed-Fi data formats. It offers an API through which personalized learning applications can store and retrieve common student data. Security features preserve the privacy of data and ensure that only authorized people can access it.

Newer data stores align student activity and assessment data to standard learning objectives. The goal is derive a model of what the student knows, what the student is learning and what the student has yet to learn. This enables rich reporting on student competency levels on an objective-by-objective basis and the stimulation of targeted interventions.

Content
I prefer to talk about educational content as learning activities. There are the traditional passive media such as reading, lectures, video and so forth. More engaging are interactive activities like virtual labs, simulations virtual worlds and games. For both active and passive content, education doesn't need special formats. The web content formats managed by the W3C are adequate and well-supported. What is needed is a way to represent the alignment between the content or activities and the standard learning objectives.

The Learning Resource Metadata Initiative (LRMI) is a standard way to describe educational materials including their alignment to standards. It's based on the Schema.org metadata standard adopted by Google, Yahoo!, Bing and Yandex.

LRMI metadata can be shared between systems using the Learning Registry. The inBloom index consumes LRMI data from the learning registry and offers a search service that can find educational content suited to specific student needs.

IMS Global defines standards for packaging learning content for import into learning management systems. However, I prefer the approach IMS uses for Learning Tools Interoperability. Instead of packaging content, this protocol allows content from other sites on the web to be seemlessly integrated into the learning experience. Integration in this way avoids limitations imposed by the packaging format and lets the developers of learning activities collect data about the use and effectiveness of their products.

Assessments
In my opinion, assessments are presently the weakest part of the Personalized Learning Model but that's changing rapidly. Two multistate assessment consortia, Smarter Balanced and PARCC are developing new assessments aligned to the Common Core State Standards. Both are committed to supplying formative and interim assessments in addition to year-end summative exams. CoreSpring is pooling assessments from a more than six different sources to supply a bank of good quality assessments that can be used in class, for quizzes and in interactive learning environments. MOOC developers such as Coursera, edX and Udacity are having to invent new ways to offer interactive assessments at extremely large scale.

In the long run, I expect the line between learning activities and assessment activities to blur. After all, much of learning occurs when the student demonstrates understanding. With adequately instrumented activities, the accumulated data about student competencies should reduce the need for big summative exams at the end of the year.

~ ~ ~

We've come a long way in the last couple of years. Pioneers in this space like DreamBox, Knewton, Read180 and GrockIt had to build a whole infrastructure. But now there's a solid set of building blocks on which developers can build personalized learning applications. I anticipate a lot more innovation at the place where student data and content come together.

Theories of Education Reform

My oldest son was a junior in high school when the standardized tests associated with No Child Left Behind were rolled out. One day, shortly before the exams, he asked me, "Why do we have to take these tests anyway?"

I answered truthfully, "They're not evaluating you, they're evaluating your school."

I found out later that with that information, he and his friends challenged each other to get the lowest scores possible. I sometimes use this story to illustrate broken feedback loops. It was nine months later before the scores had impact. When he returned to school the next fall he found he had been enrolled in remedial math despite aceing Pre-Calculus the previous year. He had to meet with the counselor to get into the right class.

Today, however, I want to explore the theories of education reform that drove the deployment of these exams. There are three prominent theories of reform with a few variations. Most contemporary efforts to improve education are based on at least one of these.

Theory: Standards and School Accountability
This is the primary theory represented by No Child Left Behind (NCLB). It's based on the broader theory that measuring something and reporting on those measurements will bring about improvement – especially if improvement is incentivized. It also represents the truism that if you don't measure something, you can't tell whether you've changed it for the better.

In order to bring about accountability, NCLB requires states to define learning objectives for each year or grade. These objectives are commonly referred to as the state core standards and each U.S. state has its own set. Furthermore, any public school receiving federal funding must administer a state-wide standardized test to every student in grades 3-9 and at least once in grades 10-12. Student scores are compared with previous years' results to determine whether they have achieved Adequate Yearly Progress (AYP). Certain consequences are tied to individual schools' success or failure to achieve progress for all students.

The core of the theory is this: If we set higher standards, measure against those standards and report performance then learning will improve. Unfortunately, 11 years into this experiment the quality of U.S. student learning is nearly flat.

There are numerous criticisms of standards and testing; but my personal concern is that by themselves they are a blunt instrument. In the absence of a proven formula for improvement the result is a form of natural selection – schools that underperform are taken out (actually they "receive interventions") while better performers survive. Natural selection is proven to work but it takes many generations and a lot of the population are brutalized before measurable improvement occurs.

Despite the lack of success, it's not time to abandon standards or accountability. Prior to 2002 most states didn't have well-defined core standards nor was student performance consistently measured. Now all states have standards, we are measuring regularly and 45 of the states have recently agreed to the Common Core State Standards. While standards and testing are inadequate remedies by themselves, they are important assets on which to build.

Theory: Highly Qualified Teacher
Where the Standards and Accountability theory focuses on school improvement. This theory focuses on teacher improvement. It's certainly intuitive; most of us have had one or more great teachers and we know they make a big difference. It's also justified by the data. Studies confirm that teacher quality is an important factor in student achievement and that the variation in achievement between classes within the same school is greater than variation between schools.

NCLB includes a mandate for states to supply highly qualified teachers to every student but it leaves it up to states to determine what it means to be highly qualified. And that turns out to be a problem. Studies show that certain teachers are consistently more effective than others; value added measures can identify which ones they are (albeit with a moderate error rate); but individual teachers often don't know what they need to do to improve.

In raw form this becomes another application of natural selection. If we reward teachers who perform well and eliminate those who don't then eventually performance will improve – assuming we don't run out of teachers beforehand. But many generations will be required and a lot of brutal actions will be taken in the meantime. No wonder there's so much controversy around teacher evaluations being tied to wages and promotions.

I'm actually in favor of merit pay for teachers so long as good quality performance measures are used. But those evaluations need to be deployed concurrently with professional development that informs teachers on how they are doing and what they can do to improve. Conveniently, resources are emerging to support that. For example, the Measures of Effective Teaching project used the Danielson Framework for Teaching to identify teacher behaviors that are well-correlated with student performance. These and similar frameworks can be used to inform teachers on how they can do better.

Even so, effective teachers alone are not enough. In our current educational system, teachers account for approximately 8.5% of variation in student achievement. School-, teacher-, and class-level factors combined account for about 21%. Meanwhile, background characteristics such as race, parental achievement and family income combine to account for 60% of variation in achievement levels.

So, if every teacher in the country was equivalent to our very best, it still wouldn't be enough to overcome the cycle of intergenerational poverty. To achieve that dream, we have to increase the influence school has over student achievement. That can be done by adapting the learning experience to the needs of individual students.

Theory: Personalized Learning
There's a pattern to these theories: The Standards and School Accountability theory introduces the concept of measurement and uses it to assess whole schools. The Effective Teachers theory takes those same measures and applies them at the teacher level. For this third theory, feedback is applied at the student level.

Personalized learning leverages the same standards as the other theories. It can also incorporate the same measures. However, annual testing alone is insufficient for personalization. Instead, understanding is measured weekly, daily or, in the best adaptive learning systems, continuously. Measurement must happen soon enough and feedback given quickly enough to affect learning activities. A truly personalized system selects activities according to student needs and also adapts to student behavior within an activity.

Bloom's Two Sigma experiments and the follow up work they inspired make me optimistic about Personalized Learning. These and other studies have shown that personalized learning experiences enabled by immediate feedback consistently deliver one to two standard deviations improvement in learning. We believe that is sufficient to overcome background factors thereby enabling a majority of students become high achievers.

Personalized learning is the natural result of 1:1 tutoring which is why tutoring is so effective. To do personalized learning at classroom scale generally requires 1:1 computers and a role change for the teacher as she shifts from "deliverer of knowledge" to "facilitator of learning." As with the other theories, there's a lot of skepticism and resistance to change. But pilot deployments are showing great promise.

Variation: School Choice
School Choice attempts to bring competitive pressure for schools to perform better. In this way, it's a variation on the Standards and School Accountability theory. Like NCLB, School Choice needs standards to be set and school performance must be measured against those standards. Performance is reported to parents who are expected to make an informed choice of which school their students should attend.

Since allocation of school funds is tied to enrollment, the theory is that schools seeking students will compete, not only on standards and their measures, but also on the basis of any other factor that's important to parents and students.

School Choice efforts include charter schools, magnet schools and voucher programs. The idea is to give public and private schools more freedom to experiment thereby accelerating the identification of viable formulas for improved leaning. Studies have shown this to be the case as the average of charter school outcomes is similar to that of public schools while variation among charter schools is much greater. Therefore, some charter schools are substantially better and should be emulated while others are substantially worse and should be shut down or reorganized. It's exactly this kind of variety and freedom that school choice advocates seek.

School choice can incorporate Highly Qualified Teachers and Personalized Learning. Indeed, since both of these theories have been shown to be effective, the expectation is that schools that incorporate these principles will be the best rated and will attract more students.

Variation: Small Classes
The small classes movement is based on studies showing that students learn better in smaller classes – all other factors being equal. But other factors are not equal. Lowering the student:teacher ratio costs a lot of money and other factors such as teacher skill have a greater impact than class size. For example, when California mandated smaller classes they had to hire many more teachers. For at-risk populations, the impact of less-experienced teachers overcame the benefits of smaller classes resulting in lower performance instead of the expected improvement.

Variation: No Excuses
The No Excuses model centers on maintaining high expectations for student performance without making excuses for external issues such as background, troubles at home and so forth. It's associated with charter management organizations such as KIPP and BES. Proponents emphasize pillars such as college expectations, culture of respect, voluntary participation and high discipline. They also have extended hours and extended school years. A key value is the whole school's commitment to each student's success. If a student is struggling or falling behind, they discover that early and engage counseling, tutoring and other supports to ensure the student succeeds.

No Excuses engages all three theories, overall school performance is measured, they hire and train highly effective teachers and they adapt the learning environment to the needs of individual students, albeit most No Excuses schools do adaptation with limited use of technology. Over the last decade, No Excuses schools have demonstrated that background factors can, indeed, be overcome by a supportive school structure. On the other hand, their high reliance on supportive interventions sometimes leaves students underprepared for the independent learning discipline required in college. Recognizing this, No Excuses organizations are updating their practices to better train students to become independent learners.

~ ~ ~ ~ ~

As with the variations listed here, most reform projects mix two or more of these theories. Even NCLB includes a mandate for Highly Qualified Teachers. Personalized Learning efforts are more common at charter schools than conventional public schools.

Education Reform will remain an important part of our civic dialog for a long time. Unsurprisingly, it means different things to different people. For some it's a moral crusade. To those being asked or forced to reform it's more threatening. All too often arguments about reform neglect the research (which is abundant) and fail to fully express the theories on which they are based. That shouldn't be the case as there are decades worth of data and studies behind each of these theories – sufficient for advocates and policy makers to make informed decisions.

The data tells those of us seeking to eliminate poverty that incremental improvement to existing schools is insufficient. Personalized learning with an eye toward training independent learners seems to be the most promising approach. Deploying this at scale requires whole-school changes to the way programs are funded, to the choices of curriculum and technology, and to the roles of educators.