Tuesday, June 7, 2016
Drowning in data. But is it useful? [edited]
There are several challenges to having too much data, and this is not a recent development. It's probably been a decade since I was talking with a friend who was explaining wearily the work she had to do aggregating, disaggregating, and reaggregating data. Why? Data-driven instruction to inform student learning and achievement.
I've talked to teachers recently who say they have far too much data and they have no idea what to do with most of it so they ignore it. Would it be helpful to know how long it took students to answer algebra problems? Yes, it could be. But knowing that it took Student A 7.2 seconds to answer the question while it took Student B nearly 3 minutes to answer the same question is only part of the data picture. Sure, the more data I have the more specific a profile I have of that student, but what if Student B took nearly 3 minutes to answer the question that day because she'd had a fight with her best friend earlier and that was on her mind?
As noted in this article, "What's At Risk When Schools Focus Too Much on Student Data?", an overabundance of data encourages the helicopter parent in the worst possible ways but could also numb students to possibilities. By the same token, because there is much that raw data cannot measure, we run the risk of missing what's really important.
I think we also run the risk of data fatigue. Let's say one of my typically better students has an off day and I'm able to talk with him for a few minutes. He shrugs a lot and answers in non-committal monosyllables. If I know this student at all well, that is data and could signal potential problems that influence his ability to learn. What if the off day shows up in an assessment that is part of the dashboard? What if every time we look at that dashboard we see that off-day grade that serves as a reminder of whatever caused that off day?
There is value in data; I'm not disputing that. At some point, a group of educators at a school need to sit down and do an audit of all of the data they are gathering and figure out what's really useful and what's really helpful and to what extent. Then they have to resist the temptation to codify that information, but to share it with their colleagues and have an open conversation about the data that seems to provide the most value for most teachers and then determine the best way to supplement and amplify that data for the student so the student understands the progress of that student's achievement.
Then, if there are student-teacher conferences, the student could actually participate in the conversation about what he is doing and what he think he needs to improve or continue to grow. When teachers meet with parents, the student could again be part of the conversation to explain things to the parental units and to let those parents/guardians understand what the student believes she needs to improve or continue to grow.
Without context or focus, data is just data and of little use to anyone. And too much data is just too much data.