Monday, April 22, 2013

How can you tell you fixed it if if it didn't happen?

There's a great deal of discussion in the air now regarding the evaluation process for public school teachers.  The main issue appears to be whether or not it's fair to judge a teacher on the test scores of her students.  One school of thought (espoused by non-teachers, mostly) believes it's the best possible way--outcome-based criteria are rock solid, they contend--as only good teachers can help their students as a group to improve their skills.  Another school of thought (teachers', mostly) focuses on the idea that kids will be kids, and they're as likely to blow off the test because they just don't care, because they're overly stressed, or because they know their much-despised teacher's job depends on their results.  Still another group (philosophers and economists, I suppose) wonder if it's possible to create improvement in a student whose abilities have reached their peak, and whether it's even possible to determine that they have or do not have the ability to move up.  This becomes most pertinent at the upper levels as there's only so much one can do to improve IQ in teens even if one can pry the smart phones from their numb fingers.

At the same time, there's been testy discourse over possible changes in firearm regulations.  The "Let's all go shooting!" club says if everyone owned a gun, there wouldn't be any gun violence because we'd scare each other too much (or something along those lines), while the "Oh, no!  A GUN!" club says where there are fewer guns there are fewer shootings.  In reality it's hard to prove either side because, well, how can you prove that what didn't happen was in any way related to what did?

An area that's been a little neglected is the most esoteric statistical and the most hypothetical (but possibly the most rational) of world views.  How can one say that the teacher who didn't manage to improve her students' test scores didn't prevent them from dropping?  How can one prove that the home invasion where the homeowner didn't get shot was safer because the homeowner was also a gun owner (unless, of course, he shot the invader, which is a different scenario entirely)?  How can you prove what hasn't happened and lay responsibility for the non-event with any accuracy?

It's obvious that there's a connection to horse training in this last question.  When you've spent massed practice time trying to get Widget to stop fussing in the cross-ties, and after six months he still manages to grab your sleeve or leave tooth marks on your glove, who's to say that he wouldn't have been worse if you hadn't done the work with him?

Ethel running alongside the lawn tractor catching grass and bugs
is a learned behavior...or is it?  And what behavior might she
have displayed if she'd never seen a lawn tractor?
We'll never know.

I took this to my Upper Pasture Advisory Council for debate.  This time all hooves pointed at Dakota, the big-butted Appy, who is not a cribber, but a wood-chewer and surface-licker extraordinaire and whose efforts to dissolve, by dint of the longest tongue I've ever seen, the edges of his buckets have left the plastic festooned with gobs of well-chewed sweet feed.  Every anodized gate has a half-moon of rust in the middle of the second rail from the top courtesy of the highly acidic nature of his spit and the devotion to duty that he has displayed over eight years on the premises.

As I repainted the edges of the front wall of the new shed in the pasture where he and Zip had left their dental autographs, I felt hot breath on my neck.  "You've done a pretty shabby job of breaking Slobber Boy, there, of his bad habit, haven't you?"  Leo pointed out.  "He's still as disgusting as he ever was!"  

"But," I said, wiping snot off my hair, "how do you know he wouldn't be worse if I hadn't kept after him?  How do you know he wouldn't have chewed all the way through the fence boards?  In eighteen years I couldn't stop Pokey from licking the oak board on the side of her stall door, and she licked it almost all the way through.  Who's to say he wouldn't have done the same?"

Leo, having had enough of the discussion, snorted and wandered off, but the thought stuck with me.  How can you tell?

Dakota demonstrates his disdain for couture and
possibly a reason for his grudge against wood, metal, plastic
and me.  

The answer is you can't.  The bad part of dealing with live subjects, be they human or otherwise, is that there are no do-overs.  It's not possible to go back in time and readdress the same issue in the same subject under precisely the same conditions to see what changes might have been wrought over time without your intervention.  And there's certainly no way (as I hope I belabored to the point of browbeating in the last two blog posts) to account for random events intervening.  Variables, Man!  There are just so many of them!  For instance, even without my intervention, he might have been kicked in the mouth (Zip, I'm looking at you!) and lost his front teeth.  Or he might simply have lost his taste for rough-cut lumber or shifted his focus to cedar trees--lumber-on-the-hoof, so to speak.  Or perhaps on the day I chose to record his behavior he wasn't really feeling the whole chewing thing, or a mouse distracted him, or he stopped licking because I bleached his bucket and it tasted bad.  Who's to say?

The point, then, is that we can't know what behaviors we prevented from occurring.  We can only watch for long-term changes, ignore the minor interim shifts that are nothing but extraneous information, interesting but useless, and hope that the result we see is actually related to the training (or teaching, or protection) method we pursued.

As far as we horse people go, mostly we can stop being so nit-picky and just enjoy the fact that we're interacting on some level with a creature who is more unlike than like us.  That's total coolness and worthy in and of itself.  The guns and teachers (or even teachers with guns) issues will have to roll on unimpeded by logic and without even a cursory nod to synchronicity or randomness while they're beaten to a pulp by soothsayers and pundits.  That's what we do.

No comments: