29 May 2013

CTO for the Smarter Balanced Assessment Consortium

I’m way overdue writing this message. About a month ago I started my new job as Chief Technology Officer for the Smarter Balanced Assessment Consortium. I expected to write about it that very week. After all, I had been composing this announcement in my head for a while by then.

That it’s taken so long is an indicator of what a whirlwind this has been. After all, the consortium has been operating for a couple of years now, contracts have been awarded and the work is underway. It’s like boarding a moving train. My new co-workers, partners and vendors have been incredibly gracious as I’m learning this new job.

Smarter Balanced is one of the Common Core Assessment Consortia. It’s a partnership of 25 states and 1 territory most of which have also adopted the Common Core State Standards. The concept is that with common standards for English Language Arts and Mathematics we can collaborate to develop better quality assessments of student skills than individual states could do working independently. Funding is through grants from the federal government and multiple foundations.

Those of you who know my passion for non-summative assessment may wonder whether I’ve gone over to the summative dark side. Summative assessments include the end-of-year assessments given to K-12 students. They may also be final exams or any other tests that come at the end of a course of study.

Non-summative assessments include formative assessments that occur at the beginning of a topic to help students and teachers understand how much they already know, they include daily exercises and assignments, and they include interim assessments. In short, they include any assessment that occurs before the unit or course of study is complete. My previous post on feedback loops and my VSS talk from last fall explain why I believe non-summative assessment has such potential to improve student learning.

As I looked into the Smarter Balanced opportunity I was delighted to find similar passion for non-summative assessment. To be sure, summative assessments are a huge part of our work. But even these exams will use Computer Adaptive Testing technology to accurately place students in a development sequence rather than just determine whether they’ve met some proficiency threshold. And we’re working hard to ensure these assessments are more authentic – that is, activities being measured are closer to the way skills are expressed in the real world.

Moving one step earlier in the learning cycle, Smarter Balanced will also offer voluntary interim assessments that can be used earlier in the year to find strengths and weaknesses in student skills and inform subsequent teaching. They will use the same kinds of questions and skills alignment as the summative exams. I’m exploring how we can offer these interim assessment items in a way that allows other organizations to integrate them into their adaptive learning systems.

And moving to the beginning of the learning process, we are developing a digital library of formative learning materials. These will be teacher-facing content that helps teachers plan and implement formative activities in their classrooms.

For more than two years at the Gates Foundation I studied the state of online assessment and how it fits into the Personalized Learning Model. I’m excited to apply those ideas at that this scale.  At Smarter Balanced we're addressing assessment needs in three ways: with year-end Summative Assessments, Interim Assessments to inform students and teachers as they learn, and with Formative Activities to introduce topics and get a sense of existing understanding. It goes without saying that I’ll write much about that here.

02 May 2013

inBloom - For My Concerned Friends

In my post about the Common Core State Standards I wrote about how concerned pundits have lumped together five related but independent efforts. Today I'm writing about inBloom which I'll contrast Statewide Longitudinal Data Systems – two more of those five.

InBloom is a service designed to help students achieve academic success through personalized learning. Those of us who helped develop the Shared Learning Collaborative (which was renamed inBloom in February) are convinced that personalizing the learning experience is the best way to improve student achievement. Whether personalization is being done by a teacher, an online learning system, or a synergistic combination of the two, it happens when information about what the student needs to learn intersects with information about available learning materials.

With that in mind, we set out to supply teachers and students with the data they need. That's what inBloom does. It taps into existing student data systems at schools, districts and states and makes that data available, in a secure way, to authorized teachers, students and parents. Simultaneously it indexes a library of teaching materials and makes them available to those same individuals.

A lot of work went into preserving student privacy. inBloom requires two things to happen before any student data can be retrieved. First, the application they are using must be authorized by the school district. Second, the individual using the application must be logged into inBloom and be authorized to access the requested data. This protection of student privacy is compliant with and goes beyond the requirements of FERPA and state data privacy laws.

So, who can access student data? Teachers can access data about students who are enrolled in their classes. Parents, if authorized by the school or district, can access their children's data. And students can access their own data. An application, such as a personalized learning system, can only access private student data if an authorized user is logged in to the app.

To match student achievement data against available learning resources, we need a common taxonomy of what it is that students need to learn. It's not sufficient to know that Johnny got an "A" on assignment number 5 but a "C" on assignment number 7. We need to know what learning objectives were represented by each of these assignments. That's why inBloom makes use of the Common Core State Standards. In the data, we can show that assignment 7 was on multi-digit multiplication. And, since it appears that Johnny needs some more practice, we can search the library for multiplication practice that's suitable to his age and preferences.

In a nutshell, inBoom supplies the student and content data needed for effective personalized learning.

Statewide Longitudinal Data Systems

For whatever reason, some people have confused inBoom with Statewide Longitudinal Data Systems (SLDS). The SLDS effort was launched more than a decade ago by the Bush Administration and funded by the Educational Technical Assistance Act of 2002. While a separate statute, it's related to the No Child Left Behind Act of 2001. The official SLDS website describes it this way:
Better decisions require better information. This principle lies at the heart of the Statewide Longitudinal Data Systems (SLDS) Grant Program. Through grants and a growing range of services and resources, the program has helped propel the successful design, development, implementation, and expansion of K12 and P-20W (early learning through the workforce) longitudinal data systems. These systems are intended to enhance the ability of States to efficiently and accurately manage, analyze, and use education data, including individual student records. The SLDSs should help states, districts, schools, educators, and other stakeholders to make data-informed decisions to improve student learning and outcomes; as well as to facilitate research to increase student achievement and close achievement gaps.
Under grants from the SLDS program, 47 states are developing longitudinal data systems that aspire to collect student data from preschool through college and even into workforce placement. Analysis of the data should help researchers understand the impact of different factors and programs on student achievement.

Before being analyzed to find trends, the data is either anonymized or aggregated in order to preserve the privacy of the students. However, the databases themselves necessarily contain personally identifiable information (PII). That's because the data comes from multiple sources: K-12 schools, colleges and workforce databases. In order to connect all of the data about an individual together, you need to be able to match up records and that requires the personal identity information about each individual.

This concentration of individual data spanning decades of educational experiences spooks a lot of people. Two factors help moderate those fears. First, according to federal regulation, data is not combined between states nor is it reported to the federal government. Only aggregate data (sums, averages and so forth) is reported to the federal government. Second, the Family Educational Rights and Privacy Act (FERPA) prohibits the release of any student information without permission from parent. Of course, that doesn't reassure everyone. The mere fact that such databases exist concerns many.

I have a different concern. I've previously written about Theories of Change for educational improvement. In this case, the theory is that over time the collected data will help government officials, education officials, teachers and curriculum developers make better decisions based on what really works. But if we're trying to figure out how a particular curriculum choice in elementary school affects a student's college prospects, it may take 10 years or more to have the data to measure that effect. My concern is that this effort will take a long time to make a difference.

~ ~ ~ ~ ~

inBloom and SLDS both collect student data. Both leverage CEDS definitions for the data fields they collect. But the purposes of the data sets and the people who have access to the data are entirely different. Of the two, I'm more optimistic that inBloom will achieve the impact on student learning that our country needs.