Friday, September 5, 2014

Standardized tests are bunkum?

Standardized tests are probably bunkum. Many of us have known this for a while. Even if we haven't been able to prove it with barrels of data, we have anecdotal evidence. Administrators and politicians alike chortle when test scores increase even a nanometer. Should the test scores go down, even by the same margin, those same administrators and politicians scurry for excuses and schedule blathering rampages framed as press conferences to divert attention from the test scores to scores of possible scapegoats--usually teachers--to explain away that bad scores. It's never ever the test. It's never ever that that particular test could have been chock full of poorly written or just bad questions. It's never ever that any standardized test measures mostly how well prepared students are to take a standardized test.

Before I pile on (again), I would like to try to make something reasonably clear even at the risk of provoking different howls of frustration from the anti-Common Core throng. Common Core swept
into the spotlight the generally misunderstood concept of "rigor." [Check out Barbara Blackburn's work on rigor, please.] Let me sum up: not "harder" and not more. What has tagged along with Common Core, and nearly unnoticed by many, is the concept of "proficiency." Sure, plenty of teachers are part of the conversation about "mastery" and "'proficiency" and how to grade for either and the complications associated therein. But that's a different topic of conversation. The reason I mention it is this: educators know there is a difference between knowing how to do something and being able to do it.

I had this conversation just last night with a former student of mine who is a teacher and who going through this very same frustrating conversation. It is, for example, one thing to be able to explain how to do a push-up, perhaps even replete with details about muscles and other stuff. It is, however, quite another to be able to complete 10 push-ups. So is the score for mastery being able to do 10 push-ups or knowing how to do a push-up and why it matters? The answer is that it depends on what I'm scoring for mastery and why.

A standardized test asks a student to fill in a bubble for an answer. The student may have guessed or may have tried to complete the task. Unless students write in their test booklets (which they're not supposed to do) or in the margins of their tests, or unless students allowed scrap paper which is collected with their tests and reviewed along with their tests, there is no way to know if a student guessed or tried to complete the task. Even the multi-part multiple choice (selected response) questions designed to require students to provide evidence for one or more prior questions can be gamed. Students can guess at every part and still have a chance of guessing a right answer.

Given that, a standardized test really cannot measure true mastery nor proficiency, however one might define those terms and assess whatever students might do to demonstrate their levels of knowledge and/or skill. But that's just my opinion.

. . . in June 2012, the Texas House Public Education Committee did what elected officials do when they don’t know what to say. They held a hearing. To his credit, Committee Chair Rob Eissler began the hearing by posing a question that someone should have asked a generation ago: What exactly are we getting from these tests? And for six hours and 45 minutes, his committee couldn’t get a straight answer. Witness after witness attacked the latest standardized-testing regime that the Legislature had imposed. Everyone knew the system was broken, but no one knew exactly why.
Except Dr. Walter Stroup, University of Texas College of Education.
Stroup argued that the tests were working exactly as designed, but that the politicians who mandated that schools use them didn’t understand this. In effect, Stroup had caught the government using a bathroom scale to measure a student’s height. The scale wasn’t broken or badly made. The scale was working exactly as designed. It was just the wrong tool for the job. The tests, Stroup said, simply cannot and do not measure what or how much students learn in school.
People have been saying this for decades, and no one has been willing to listen. Why?

In general, the linked article gives some wonderful context for the abysmal performance of standardized tests, but then it becomes a less-than-shocking expose of how Pearson has and continues to strong-arm educators at all levels to accept its will. But it also indicates that as a result of having had the temerity to expose the absurdity of standardized tests and poking the Pearson assessment bear, Walter Stroup was denied tenure.

If this is the case, it is appalling that the University of Texas College of Education took Pearson money and then rationalized its treatment of Walter Stroup. Failing to get tenure is devastating to a college professor, personally and professionally. While I completely sympathize and empathize with Walter Stroup, the point really has to be that standardized tests are still bunkum.

However, if we were to pull the plug on standardized tests, then what? Lee County made history by opting out of all state-wide standardized tests. A few days later it reversed itself. Yes, in our zealousness for whatever (and I really don't know what it is), we test too much. 

Last year CPS reduced the number of tests its students would have to take from 25 to 10. You read that right. They eliminated 15 standardized tests. Fifteen.

I think we think we know why we give students standardized tests. And I think we think we do something with all of that data. But if we also know that the results of the standardized tests don't jibe with what classroom teachers are seeing in their classrooms and across their grades, then something is not right. The confusion of data cannot tell us if student performance in the classrooms is an accurate measure of what they know and what they can do because, along the way, we have learned not to trust the professional judgement of our teachers. That, however, is a topic for another day.

As for these standardized tests, if we insist on giving them, let's be selective about which tests we give and when, let's be realistic about why we give the tests, and then let's be realistic about what we do with that data. Let's be proactive about how we aggregate and examine the data, and how we use that data to inform something that helps teachers do their jobs more effectively and truly helps students demonstrate what they know and what they can do, not how well they can guess.

No comments:

Post a Comment