As a parent, one of the things that I think a lot about is school and the quality of my kid's education. The Pirate will be starting school this fall (THIS FALL!!!) and while I think the elementary schools in this area are ok, I have definitely heard horror stories about testing and grades and failing schools. In a conversation somewhere online (please don't ask where) I saw mention of the book Making the Grades: My Misadventures in the Standardized Testing Industry
by Todd Farley. It seemed like this would be a great way to find more out about how standardized testing was done and I requested it from the library.Farley found his way into the field by chance. A local scoring center was hiring and it was a steady, indoor, good paying job, with no requirements so he took it. Over the next dozen years or so he worked his way up the ladder eventually writing tests and training without any real ambition to do so. Farley eventually quit working in the industry to write his book, which exposes some of the behind the scene stuff.
As it turns out, while the questions asked are standard, the people scoring the tests aren't standard at all. They come from every background and every level of ambition and motivation, and every level of education. They might be high school grads or they might be marking time until law school. They might be hungover all morning and tipsy all afternoon. They might not speak English. Additionally, all the scorers from any one question live in the same geographic location, so if your questions were scored in Iowa, you can expect the scorers to know Iowa slang and not be familiar with what a second grader in New Orleans might say.
It seems as though each question would have a standard "correct" answer, but this is not true. One of his examples is of a question that asks students to draw a picture of a bicycle safety rule, having just read a section explaining the rules. Students got one point for a correct drawing and zero points for an incorrect one. Seems clear, right? But what if the student drew a picture of a kid on a bike wearing a helmet but NOT holding the handlebars? Do they get the point for the helmet? What about the no hands thing? Any answer on any question could have multiple similar answers that the individual scorer was responsible to judging for correctness. Every effort is made to standardize the scorers responses, but because we are all human it is a near impossible task.
The book itself if full of example after example of how the system has very little to do with what the kids know as well as examples of how administrators think when designing the tests. It was a fascinating book, if a little bit poorly edited. By the end of it I felt that it was a little long for a full length book and could have been edited down to a long article, but I'm still glad I picked it up. Did it help me to make any decisions about the state of my own children's education? No, but it does lend some weight to how seriously I take the reported scores of local schools.
How do you feel about standardized testing? Are you a fan of NCLB? Do you think that a schools score accurately reflects what the teachers teach the kids? Does the very idea of teaching to the test make you want to homeschool your children? What has been your experience with testing?
Making the Grades by Todd Farley
Polipoint Press
2009
272 pages
I am an Amazon Associate. As such, any purchase you make at Amazon.com after following a link from this blog will earn me a (tiny) percentage back as income. Thanks.
