Of That

Brandt Redd on Education, Technology, Energy, and Trust

20 November 2014

Education Data Standards Update

Over the last couple of years, some colleagues and I have developed several models that are useful for understanding education data standards, where they apply and how they fit together. Many thanks go to host of collaborators who have reviewed and helped with these models.

The first is the Four-Layer Framework for Data Standards. This framework has helped guide decisions about the Common Education Data Standards – what should be the scope and how CEDS should relate to other standards in the space. However, the framework is not limited to education standards. Any organization that's developing specifications for the exchange of data should think of these four layers and try to describe each part semi-independently.

Last year I developed A Taxonomy of Education Standards. This framework categorizes standards according to their purpose or the domain in which they are applied.

These education standards are not exclusively data standards. Academic Standards, which include Achievement Standards and Competency Standards describe skills that students should be able to demonstrate as they achieve certain levels of education. Nevertheless, there are data standards for describing Academic Standards and for aligning content to those standards.

In May of 2013 my friends at SETDA published Transforming Data to Information In Service of Learning. This is an enormously valuable survey of existing data standards with guidance on how organizations can apply them to improve learning and support interoperability of their learning technologies. In doing so, they used both the four-layer model and the taxonomy.

Shortly thereafter, I combined the models into a two-dimensional matrix with the four layers on the horizontal axis the taxonomy on the vertical axis. This allows us to plot existing and proposed standards against the two dimensions to see how they fit together.

At the iNACOL symposium two weeks ago Liz Glowa, Jim Goodell and I presented a workshop on "Competency Education Informed by Data". For that workshop I updated the matrix to reflect changes in the standards landscape over the last year. Here's the updated version:

For that same workshop, Jim Goodell developed a matrix plotting the layers on the vertical axis and the progression from Pre-K to primary, secondary, higher education, and workforce data on the horizontal.


And to tie these all together, here's a translation of the acronyms into the standards with links to their corresponding websites.

AIFAssessment Interoperability Framework
CCSSCommon Core State StandardsBlog Post
CEDSCommon Education Data Standards
Ed-FiEd-Fi Alliance
EDIElectronic Data Interchange
ESBEnterprise Service Bus
IMS CCIMS Common Cartridge
IMS LTIIMS Learning Tools Interoperability
IMS QTIIMS Question and Test Interoperability
LRLearning RegistryBlog Post
LRMILearning Resource Metadata InitiativeBlog Post
NGSSNext Generation Science Standards
OAI-PMHOpen Archives Initiative - Protocol for Metadata Harvesting
OBIOpen Badge Infrastructure
PESCP20W Educational Standards Council
RESTRepresentational State Transfer
SEEDState Exchange of Education Data
SIFSIF Association
SOAPSOAP Protocol
xAPIExperience API (AKA Tin-Can API)

Updated: 25 Nov 2014 to add the OAI-PMH protocol.

30 July 2014

Bitcoin - What Makes a Currency?

Today I'm diverging from the education theme to write about cryptocurrency. I am provoked, in part, by this quote from Alan Greenspan:

“It [Bitcoin] has to have intrinsic value. You have to really stretch your imagination to infer what the intrinsic value of Bitcoin is. I haven’t been able to do it. Maybe somebody else can.”

Now, Greenspan should know better than to say something like that. As a fiat currency, the dollar doesn't have any more intrinsic value than Bitcoin. And that's why I decided to write about this. Most of the supposed "Bitcoin Primers" out there are more confusing than helpful. They don't explain how money works or how cryptocurrencies like Bitcoin satisfy the requirements to become a currency.

What makes a Currency?

Currency is a form of money that accepted by a group of people to exchange value. A functional currency must have three important characteristics:
  • Scarcity - If you have too much of the currency, it's value will plummet toward zero. So, there must be a limited supply.
  • Verifiability - You must be able to verify that a unit or token of the currency is valid and not a forgery or imitation.
  • Availability - Despite scarcity, there still must be a stable supply of the currency to match growth in the corresponding economy.
Precious metals like gold and silver were the first common currencies. They meet all of the foregoing criteria. Gold is scarce; there's a limited amount of it available thereby endowing a small amount of gold with considerable value. It's verifiable; gold has certain characteristics, such as density, malleability and color, that make it easy to distinguish from other materials. And gold is available; while it is not common, gold mines still offer a consistent supply of the material.

One of the difficulties with early uses of gold currency was the complexity of exchange. Merchants had to use a balance or scale to determine how much gold was being offered. To facilitate easier exchange, governments, banks, and other trusted organizations would mint coins of consistent size and weight. This would allow someone to verify the value of a coin without resorting to a balance.

Fiat Currency

"Fiat" means, roughly, "because I said so." Fiat currency has value simply because some trusted entity says it does. It need not have any intrinsic value.

The first fiat money was the banknote. When making a large payment it could be inconvenient or dangerous to move large quantities of coins or bullion. Banks solved this problem for their customers by issuing banknotes. A banknote is a paper that a bank or other entity promises to exchange for a certain amount of coin, gold, or other currency. The bank could keep the corresponding gold locked away in a vault and people could carry more convenient paper certificates.

Beginning in 1863, the United States began issuing gold certificates as a form of paper money or banknote. Certificates like these were backed by stockpiles of gold held in places like Fort Knox. European countries did similar things. With the stresses of late 19th century wars and World War I that followed, countries discovered that they could issue more banknotes than their corresponding stockpiles. This led to a lot of instability until countries figured out how to regulate their currencies. But, by the end of the Great Depression, pretty much every economically developed country had fiat currencies controlled by a central bank. While backed by gold or other reserves, the value of these currencies is not directly tied to the value of gold.

Here's how the U.S Federal Reserve system works: The Federal Reserve Bank creates the money. Money is issued as currency (the familiar U.S. coins and bills) but also simply as bank balances. Indeed, far more money exists as bank records than in actual physical currency. Originally this was done through careful bookkeeping in bank ledgers. Now it's all done on computers. The money is issued in the form of low-interest loans, primarily to banks, which then lend the money to their customers and to other, smaller banks. Other central banking systems like the European Central Bank work in a similar way.

So, how does fiat money meet our requirements for currency?


Scarcity: Only one entity, the central bank, has the authority to create and issue the currency. The central bank limits the issue of money in order to preserve its value.

Verifiability: Coins and paper money are printed or minted using materials and techniques that are difficult for average people to reproduce but are fairly easy for to verify. Money in the form of bank balances is verifiable because each bank or credit union has accounts with higher-level banks ultimately reaching the Federal Reserve. So, when I write a check from my bank to yours, our two banks contact each other and transfer the value sending records up the banking chain until they reach a common parent bank which may be the Fed. Each bank in the chain verifies that the appropriate balances are in place before allowing the transaction to proceed.

Availability: Central banks can create as much money as they think the economy needs. The primary challenge for central banks is manage the money supply - ensuring both scarcity and availability.

Cryptocurrency

Bitcoin is the first, but by no means the only cryptocurrency. The challenge that the pseudonymous creators of Bitcoin tackled was to achieve the three features of currency - scarcity, verifiability, and availability - in the digital realm. They magnified the challenge by prohibiting a central authority like a government or a central bank. Trust, in the case of Bitcoin, is in the system, not in any particular institution.

Scarcity: The "coin" part of most cryptocurrency names is somewhat misleading. Bitcoin doesn't consist of a bunch of digital tokens that are exchanged. If that were the case it would be hard to prevent double-spending of the same token. Instead, cryptocurrencies work more like bank account balances. Bitcoin has is one, big, public ledger that is duplicated thousands of times. All transactions in the ledger must balance - for one account to receive value, another account must be reduced by the same amount. This ledger is called the block chain and it contains a record of every transaction since the creation of the currency.

Verifiability: Cryptocurrences rely on public-key cryptography to ensure that only the owner of a currency balance can initiate its transfer. The bitcoin owner uses their private key to sign the transfer record and then posts it to the network of block chain replicas. Any entity in the network can use that owner's public key to verify that the transaction is valid and that ownership has been transferred.

Availability: Those who host a copy of the block chain have to perform the cryptographic calculations necessary to verify transaction validity and prevent fraud. Those who do this fastest are periodically rewarded through the creation of new Bitcoin balances. Because of the reward, maintaining the block chain is known as "mining" and a small industry of Bitcoin mining software and devices has developed. All users of cryptocurrency benefit from this because the more miners exist, the more secure the currency becomes due to the duplication of records and validation.

This is a tremendously clever scheme because it simultaneously ensures a consistent supply of currency, decentralizes operation, and secures the network against manipulation by creating thousands of replicas of the block chain.

Potential Impact

The true value of any currency is the willingness of a community of people to use it for daily transactions. The three requirements, Scarcity, Verifiability, and Availability combine to cause people to trust a particular currency. When that trust is lost you can get bank runs, hyperinflation, or simple destruction of wealth. Meanwhile, the community rushes to find a new currency.

The advent of the internet with myriad handheld devices capable of initiating transactions makes it possible for multiple currencies to coexist. For the first time in history, people may have a choice among currencies to use in daily transactions. Central bankers, and the sovereign countries that endow them with their power, are appropriately worried. An industry that has historically been immune to competition no longer has that protection.

I think this is a good thing. Just like any other competitive market, competition should incentivize good behavior both from established central banks and from upstart cryptocurrencies.

23 May 2014

Illusions of Success when Inputs are Confused with Outputs

Prosperity has been defined as, "the state of flourishing, thriving, good fortune and / or successful social status." In the United States we tend to measure prosperity in terms of wealth, or lack thereof. Indeed, the U.S. government defines poverty (the lack of prosperity) as having an income below $15,730 for a household of two. The trouble is, that this confuses the output (or outcome) of prosperity with one of its inputs, income (or wealth). And while the two values often correlate, they can be quite different.

In the early 1800's, Georgia gave away millions of acres of land through a series of land lotteries. Nearly everyone who was eligible entered the lottery because an individual had a roughly 1 in 5 chance of winning and a typical parcel was worth about the median net worth of a Georgia resident. A penniless person who entered the lottery had a one in five chance of suddenly becoming wealthier than half of the residents of the state.

When Hoyt Bleakly, of the University of Chicago, and Joseph Ferrie, of Northwestern University, learned of this event they found it to be a convenient natural experiment. Does handing out wealth to random individuals elevate their prosperity and does that prosperity carry over to future generations? The answer, at least in this particular case, seems to be "no." Even though wealth and prosperity are correlated, increasing wealth didn't increase the prosperity of the children. As Bleakley said on a Freakonomics podcast, "Maybe the resources have to come from outside the household, be it say a good public school. Maybe the resources have to come from the parents, but the parents don’t know how to provide it in terms of nurturing, in terms of reading and communicating ideas to their children, etc." In other words, wealth is only one of the contributors to prosperity and it may be among the least important.

Optimizing the Wrong Thing

When two features, like wealth and prosperity, are correlated, and one is easier to measure or influence than the other, a common mistake is to focus on the more convenient factor. The result is a host of unintended consequences.

This is a case where feedback loops offer insight:
A feedback loop with a short-circuit bypassing the system (or student).
In a proper feedback loop, we measure the output, compare it with the reference, and use it to choose the proper input. But when inputs are confused with outputs, the feedback loop is short-circuited – as with the red line in the above diagram. The evidence of this is when we get all kinds of reports showing how good the inputs are. Meanwhile, the real goal suffers.

A Pedagogical  feedback loop measures student outcomes (in the form of competencies or skills), compares them with standards of what students should know, and uses the result to choose appropriate learning activities. But, when inputs are confused with outputs we get reports of good student attendance, appropriate construction of curriculum, the prescribed amount of seat time, properly trained and certified teachers, high quality facilities, and all kinds of other reports about the inputs. Meanwhile, the output, in terms of student skills, remains unimproved.

Here are a few other inputs and outputs to consider:
To be sure, there's correlation in every one of these cases. But, just as with the Georgia Land Lottery, manipulating the input frequently diminishes the correlation and results in a less-than desired outcome. Focusing on, and reporting about the inputs can give the illusion of success. Focusing on the outcome helps identify other factors that contribute to the desired result.

Furthermore, excess focus on inputs results in missed opportunities. As Michael Horne and Katherine Mackey wrote, "Focusing on inputs has the effect of locking a system into a set way of doing things and inhibiting innovation; focusing on outcomes, on the other hand, encourages continuous improvement against a set of overall goals and can unlock a path toward the creation of a student-centric education system."

Incentives are Inputs

Just as mistaking outputs for inputs causes trouble, the reverse is also true. A 2011 study by the Hamilton Project compared incentives tied to inputs with incentives tied to outputs. Groups of students were offered financial incentives tied to input activities such as number of books read, time spent reading, or number of math objectives completed. Other groups were offered incentives tied to outcomes such as high test scores or class grades. The study found that input incentives were much more effective than output incentives. Among their recommendations are:
  • "Provide incentives for inputs, not outputs, especially for younger children."
  • "Think carefully about what to incentivize."
  • "Don't believe that all education incentives destroy intrinsic motivation."
This shouldn't be surprising. Incentives, at least when given to the student, are inputs. Incentivzing outcomes is a different kind of short-circuit in the feedback loop.
Feedback loop with a short-circuit bypassing instructional influence.
In a Pedagogical  feedback loop the instructional system interprets the results of assessment before passing them on to the student. When we incentivize the outcomes (or assessment thereof) we bypass the capacity of the education system to interpret student needs and prescribe the right learning activities.

It's notable that the Hamilton Project study found that incentivizing outcomes was especially ineffective for younger students. Among the goals of any educational system should be to develop students into independent learners. A mature, independent learner has taken on pedagogical skill and responsibility. For independent learners, incentivizing outcomes should be more effective.

Nevertheless, the Hamilton Project study didn't neglect outputs. In every experiment, the effect of the incentives was evaluated according to student outcomes. Only the point of intervention was changed.

Effective Measurement and Improvement

In 2005, New Hampshire abolished the Carnegie unit – a measure of seat time by which most U.S. schools quantify educational credits. "In its place, the state mandated that all high schools measure credit according to students’ mastery of material, rather than time spent in class." Thus, New Hampshire has shifted their fundamental measure of student achievement from an input to an output. Early results of that change are promising.

To be sure, optimizing certain inputs still has a positive impact. Otherwise schools would have completely failed since the institution of the Carnegie Unit in 1905. But shifting the focus from inputs to the outputs we wish to optimize will open the door to greater innovations and more rapid improvements in student achievement.

17 March 2014

Lecture Experiment at Summit Public Schools

A couple of weeks ago I attended the LearnLaunch conference in Boston. In one of the sessions, Diego Arambula from Summit Public Schools told a great story:

In one of their blended learning classes the students were taught by a team of teachers and given flexibility to choose the activities they felt would best help them learn the subject. One of the activities the teachers introduced was optional lectures. Strategically scheduled shortly before tests, the lectures gave students a chance to review material and solidify understanding.

At first, the lectures were quite popular – probably due to their proximity to tests. However, they found that the scores of those students who attended the lectures were not significantly different from those who chose not to do so. The students must have sensed the lack of impact because attendance at the lectures dwindled.

When lecture attendance fell to 3-5 students, scores of those who attended suddenly shot up. Arambula asked the teachers what was happening? The teachers said that with so few students attending, they didn't really deliver a lecture. Rather, they asked the students what areas they were struggling with and they concentrated the time on those particular issues. In other words, the lectures turned into teacher-led study groups or small-group tutoring sessions.

Eventually the teachers abandoned the lecture format and opened a "help bar" at the back of the classroom. Staffed by at least one of the teachers, students could go to the bar just about any time for one-on-one or small group assistance.

There are a bunch of things to learn from this vignette. Here are a few:
  • Summit was prepared to measure the effectiveness of the optional lectures (and presumably any other learning option they offer).
  • The teachers and staff are as much in a learning mode as the students. They discover what works and adjust in those directions.
  • Tutoring and small group instruction is tremendously effective even when it accounts for a small part of the student's learning experience.
Finally, Summit established an environment where innovation like this is natural and encouraged.

27 January 2014

Personalization Relies on Standardization - A Medical Metaphor

In my last post, I wrote about Yong Zhao's observation that the U.S. leads the world in cultivating 21st century skills like Confidence, Risk-Taking, Creativity and Entrepreneurship. Zhao is concerned that the current U.S. "obsession" with standards and assessment will result in reduced appreciation of creative endeavor. Indeed, Zhao's concerns are confirmed by contemporary de-emphasis of arts and humanities education in U.S. public schools.

I share Zhao's concern that today's schools suffer from excess focus on achievement as measured by test scores. I also agree with him that some of this is encouraged by federal programs like No Child Left Behind. However, I disagree with Zhao in that I believe that achievement standards and testing aren't the cause of the problem. Indeed, they're a critical part of the solution.

To explain this apparent contradiction, I’ll borrow a metaphor from Sir Ken Robinson. When I go to my physician, I expect a personalized, custom experience. I expect him to diagnose, treat and prescribe according to my personal needs. In order to do this, however, the doctor will use standard tests. He'll do a standardized exam and ask me standard questions. For example, he’ll measure my temperature in degrees and compare it against 98.6 Fahrenheit. He’ll measure my blood pressure in millimeters of mercury and compare that against standards established by the American Medical Association. Based on those results he may follow-up with custom questions or tests chosen according to my individual needs. But even those follow-on tests will be compared against standards. Finally, he'll prescribe a course of treatment that's customized to my individual needs.

Admittedly, not all doctors handle standards the same way. For example, when my cholesterol tested high, one doctor called in a prescription for Statin drugs without consulting me. This bothered me as I wanted to discuss how serious the problem was and consider alternatives like diet and exercise before simply taking a drug. Indeed, another doctor recommended a Coronary Calcium Scan before going on Statins. The test came out clean and I'm putting additional effort into my exercise.

That’s what standardized testing, properly done, is all about. This school year, the Smarter Balanced Assessment Consortium will test more than three million students in grades 3 to 11. The results from this first year will be used to calibrate the tests and find reasonable benchmarks for student achievement in English and Mathematics. In future years, students’ test results will be used by teachers, students and parents to customize learning activities to the needs of every child.

This isn't a complete solution. We need to actively fight the tendency to teach only what’s going to be tested. Not only is it not good for the child, strangely enough, “teaching to the test” doesn't improve scores as much as a well-rounded education. We also need to resist efforts to standardize curriculum and teaching. Standards belong to measurement of the results of education, not to the inputs.

Doctors can only directly measure a few vital signs and compare them to standards. For more detail they perform or prescribe more extensive tests. Some of these are screenings like the cholesterol test I had with my annual physical. Others are specific to certain problems like the CT scan I had after breaking some ribs. But even the full battery of tests available to a physician can't discover all issues. For the rest, a physician has to rely on interviews, experience, consultation with other doctors and sometimes trial-and-error.

The same is true for education. We can only measure a few of the factors that go into a well-rounded education. The Common Core State Standards only apply to fundamental skills in reading and mathematics. It's a small fraction of all that we hope children will learn. But that doesn't mean we should throw out the standards. Literacy and numeracy are fundamental skills that are prerequisite to every other academic skill we desire students to develop. The mistake is to assume that just because these are the skills that are being measured that they are the only ones that count.

Standards and testing are useful tools – but only when they serve the greater goal of developing confident, creative adults who are capable of a lifetime of self-directed learning.

16 January 2014

Is the U.S. Leading or Trailing the World in Education?

Is the United States leading or trailing the world in education? Unsurprisingly, it all depends on how you measure. And if we emphasize the wrong factor, we risk losing important qualities of the existing educational system.
2012 PISA Rankings
for Mathematics

First, the bad news. The results of the 2012 PISA tests were released in early December 2013. The United States ranks 26th in math and is below the OECD average in all three tested areas: Mathematics, Reading and Science. So, the common narrative that U.S. education trails the economically-developed world seems to be supported.

But if that's the case, how then does the U.S. rank 6th in per-capita GDP, 5th in Global Competitiveness and 2nd in Global Creativity?
Could it be that the U.S. economy is simply coasting based on a previous lead? That doesn't appear to be the case. Previous studies show that the U.S. trailed other industrialized countries in Mathematics, Science and Reading in the 1960s, 1980s, and 1990s. In fact, U.S. rankings relative to other countries have improved somewhat over the last 50 years. It would seem that the U.S. advantage leverages factors not captured by these test scores.

TIMSS is an international test of Mathematics and Science proficiency. In addition to measuring students' mathematical skills it also surveys their attitudes toward mathematics. Yong Zhao, an articulate critic of factory-model education, has drawn some interesting information out of the TIMSS results:

CountryMath ScoresConfidence %
(4th Grade)
Value Math %
Korea61303 (11)14
Singapore61114 (21)43
Chinese Taipei60907 (20)13
Hong Kong58607 (24)26
Japan57002 (09)13
United States50924 (40)51
England50716 (33)48
Australia50517 (38)46

Among countries, there's an inverse relationship between achieving high math scores and either valuing or having confidence in the use of math. Not visible in the table is that the TIMMS results also show that within countries, higher math achievement does correlate with greater confidence and with valuing mathematics. So, while higher skill in math results in greater confidence on an individual level, countrywide programs that result in high math scores do not result in high mathematical confidence or a sense of the value of mathematics.

It also suggests that development of mathematical skill must be combined with gaining confidence in applying mathematics and a sense of the value of mathematics. Mathematical skill alone is not sufficient to develop Numeracy.

Zhao concludes his analysis by pointing out that Confidence, Creativity and Entrepreneurship are key skills that drive U.S. economic leadership. An excessive emphasis on rote learning and test scores, what he calls an "employee-oriented" education, tends to suppress the more "entrepreneur-oriented" skills that are in demand for the 21st century. Rather than praise U.S. education for developing those skills, he simply says that U.S. education "is much less successful in stifling creativity and suppressing entrepreneurship."

I join Zhao and many others in decrying the factory model of education. We can do a lot better than simply "less bad." Our schools should foster more creativity and offer more personalized learning experiences. They should be places where it's safe to fail – especially when taking on a big challenge. And schools should encourage students to pursue studies in individual areas of interest.

Strangely enough, standards and even standardized testing can help with this but only when used properly. I'll elaborate on how that might be accomplished in my next post.