Tuesday, July 22, 2014

While we're talking about assessments: the absurdity of [standardized] tests

This just in: test scores are no sure guide to what students know. It has to be true because it's a headline in The Wall Street Journal.

The article is positioned about Common Core, sort of. That students in Kentucky and New York who took the tests did not do at all well. In fact, according to the article, the results were dismal.
The results alarmed parents, but the scores on these new tests—just like those on earlier forms of assessment—reveal less about what children know than about the way the test makers decide to measure that knowledge.
The article goes on to address cut scores and benchmarking, ways to identify the levels of proficiency. Well before Common Core, grading and test results has been an issue. What does an 82% on a test really mean? That the student got 82% of the questions right, which means. . . what?

In other words, tests rarely really indicate what students know. Test scores indicate how well students read and understand the questions, how well they are able to discern the correct answer on a multiple choice question or how well they guess, how well they are able interpret what the teacher might want to see in a short answer or essay question. If the questions require students to do something, such as a math or science question, they still need to be able to read and understand the question. And, significantly, while they are studying, they need to be able to discern, guess, and/or anticipate what the teacher might ask on the test and how the teacher might choose to measure what the students have studied and memorized.

In many cases, there is absolutely no proof of learning based on test results. That is, I believe, as much the fault of the test makers as it our expectations of what tests reveal.

So why do we give tests? Habit is my immediate facetious response. Testing is deeply ingrained in the fabric of our educational process.

I'm not saying there is no room for the kind of assessment that looks and feels like a test. But let's be realistic about our expectations of what any kind of test will reveal about student knowledge and learning. Written tests have to be supplemented by other forms of demonstrations of knowledge.

Those who are proponents of PBL, MakerSpace, genius hour, and hackschooling are well aware of the differences. Students demonstrate all kinds of knowledge and learning at a wide range of levels through and because of their experiences with any of these forms of learning and, yes, assessment.

The challenge is how to measure every US student so we can gauge their test-taking skills against those of every other student in the world. We seem to be so obsessed with uniformity and conformity of the assessments so we can be better than mediocre in the world that we've forgotten there are other measures of whatever we mean by student success.

Student success. What does that mean any more? Does it mean a student's ability to reach the highest benchmark of proficiency? And how do we know if that proficiency benchmark is consistent for every other student in every other state? We don't. What teachers in one state think is "well below proficiency" might be different from what teachers in any other state think is "well below proficiency." This could go on and on, and that's my point. Even if we had a national curriculum, we will never have an absolutely uniform and consistent measure of "student success."

I watch cooking competition shows: Chopped, Master Chef, Food Network Star. There are always three judges. Each competitor cooks based on whatever the challenge and each judge samples each competitor's food. Then they discuss the merits of each dish. Every dish is assessed on taste and presentation. It's a simple set of criteria though the nuances of each of those criterion are complex. The judges don't always agree but they come to agreement to determine who moves and who does not.

When I worked in the corporate world, I was assessed based on my performance. I gave an assessment of myself using the company standards and the goals my manager and I had set for me. My manager assessed my performance using those same criteria and his or her interpretation of how well I achieved my goals. It's not as though we talked only twice a year, so there were on-going conversations related to my work and our 1:1 meetings occasionally focused on my progress with my goals. At the end of the day, was I one of the best project managers in the company? in the state? in the country? in the world? Maybe in the company, but the rest? Meh. Who knows? Did I help the company achieve its goals? If I did, I was rewarded accordingly during my merit review.

I mention the cooking competition shows and the corporate world for several reasons.
  • First, as I continue to contemplate how we assess our students and their learning, I'm reaching into other areas to think about how we figure out what our kids know and can do. 
  • Second, when Gordon Ramsey proclaims the next Master Chef, that will be the opinion of three expert judges. Does that mean that home cook is better than every other home cook in the world? Yes. For that moment. Based on the expert opinions of those particular three expert judges.
  • Third, as I think about how we measure "student success" and how we measure ourselves in the vastly diverse country as well as against the rest of the world, I wonder how far off kilter we've gotten in trying to determine educational supremacy.
  • Fourth, it seems to me our priorities for assessment are completely wrong. It is not about the percentage of students who pass the 8th grade assessment at a particular level, which probably tells us more about the days or weeks of test prep leading up to the assessment rather than what students really know.
When we assess student learning, we should be finding ways for students to demonstrate what they have learned and be able to explain how they've learned it. When we assess student learning, it seems to me we should be helping students determine their strengths, their weaknesses, their capabilities and competencies, their passions so they can be successful however they deem success in their lives. Sure, most of them will need some guidance and adult supervision. And many of them will have to be cajoled to do work in areas that seem less interesting to them because they don't yet understand its value. But the big question isn't if students can reach a high level of proficiency on a standardized test. The real question is whether or not students have truly learned anything that will enable them to lead lives as successful, productive citizens. . .however they ultimately choose to define "success."

In the WSJ article, Mr. Rockoff, an economist at Columbia Business School, said, "The deeper question parents ought to be asking themselves is 'Did I know what my kid was learning last year, and if I compare it to the new Common Core curriculum, am I happy or sad?'"

With all due respect Mr. Rockoff, I think that's the completely wrong question. The deeper questions parents should be asking themselves this is this: Did I know what my kid was learning last year? Do I know what my kids are learning this year? Do I understand why my kids are learning what they're learning? More importantly, do my kids understand why they are learning what they're learning and how do they know what they've learned?

More to come.

No comments:

Post a Comment