Tuesday, August 26, 2008

Meaningful assessment

We're a long way from through the turbulent period that I referred to in my last post, but I just had to get this off my chest.

I have a son who is closing in on his 17th birthday. In the UK, this means that the will be old enough to start learning to drive. This, in turn, means that he is madly trying to learn the highway code for the theory test.

I looked for a few sample tests online for him to try out... and, of course, tried them out myself. Like all drivers of many years' standing, I didn't fare too well! But one of the tests got right up my nose. I got 0/20!

Unlike all the other tests, this was not a multiple choice option. This test presented a picture of a roadsign and asked what it was. Fine, you might think. When I'm driving along the road, I don't get four options under each sign from which to select the right one. So, if I am not able to recognise the sign without prompts, I should probably not be driving, right?

Only, this is where the automated testing process falls down. Because of the limitations of the technology, responses were only marked as correct if the user entered the description verbatim from the highway code.

How is "decrease speed" a more correct response than "slow down"? Why is a "speed limit" not as acceptable as a "speed restriction"?

Even supposing I was able to answer all 20 questions verbatim, how is that any indication that I know what I'm supposed to do?

Surely the true measure of my knowledge of roadsigns is that I know how to respond to them? Note to my son: Shouting out "You're telling me!" does not constitute the required response to this sign.

It all comes back to our knowledge versus behaviour argument.

Then again, even knowing what to do and when to do it is no guarantee that it will be done! I am fairly certain that every single boy racer in the town knows that the speed limit restriction is 30 miles per hour. I'm fairly sure that he also knows what he needs to do in order to observe it. But to what extent does his actual behaviour reflect this knowledge?



Jim said...

a saying about leading a horse and some water comes to mind.

The upsycho said...

@james Too right. The banks are crowded with drowned, yet dehydrated horses. On the other hand, I have been reliably informed that the introduction of a salt tablet between the lower teeth and lip can work wonders! ;o)

Anonymous said...

Well, of course the driver's exam isn't meaningful assessment -- it's easily assessed assessment. The Division of Motor Vehicles hasn't got the time, the manpower, nor (I don't think) the will to individually test each renewing driver in a lifelike way -- either on an actual controlled course, or via a simulation where you'd demonstrate competence by driving close to the speed limit, rather than eliminating choices (b) and (d).

As for the lack of competent answer analysis (speed limit vs. speed restriction), that's just plain dumb. Less so if you figure the contract was let to the highest bidder, and those letting the contract weren't thinking too hard about how to score multiple guess -- I mean, choice -- questions.

In the meantime, I suspect you're doing what will be most effective for your son: taking him on many short driving trips, making him drive you on errands, making him drive over to his buddy's house. I kept a log while doing that with my youngest daughter, and we racked up over 600 miles, probably no more than 5 or 10 at a time.

That's how you present a variety of decisions under a range of conditions (daytime, nightime, rain, bright sun, etc.), and that's how the new driver learns to make increasingly more accurate discriminations.

Knowing how far to park from a fire hydrant is possibly necessary (for passing the written test) but clearly not sufficient.

The upsycho said...

@dave Because he's not yet 17, he isn't allowed to drive, yet, but we will be doing all of the above when that day dawns. With regards to the theory side, we drove to Devon and back over the course of an extended long weekend, and we got him to take his books with him. Then he had to explain the point of every sign encountered for the first time. If he didn't know, he had to look it up. What we forgot to take into consideration was his propensity for falling asleep in cars. He was out like a light after the first hour! Let's just hope he doesn't carry on the tradition when he's behind the wheel, or it will be like that awful joke (I want to die peacefully in my sleep like my Grampa, not screaming in terror like his passngers).

Anonymous said...


Then I won't tell him that in the U.S., he'd typically qualify for a learner's permit around age 15 1/2. There are moves to raise the driving age, which will probably succeed around the time I get my own license revoked for senility...

The upsycho said...

@dave "around the time I get my own license revoked for senility..."

As early as that? ;o)

The age is being raised in the UK. My elder son will be among the last to be allowed to drive at 17. My younger son will have to wait until he's 18, which is the same age as South Africa. I gather from a school friend in Oz that there are states there that allow driving at 15.

V Yonkers said...

Actually, the driving age depends on the state. In NY state, you can get a learner's permit at 16, after taking a test like the one you described. My niece is at this stage and the thought of getting behind a wheel with my son as he learns to drive in two years makes me break out in a sweat (he is good with tests). Drivers cannot get a license until they are 17 and then they are highly restricted (time of day, excluded from some areas such as NY City) unless they have taken a driver's education program. There is also talk that they will not be able to drive with anyone in the car who is under 21 years of age or not a licensed driver (over 21).

Unfortunately, our written tests for the permits resemble the hair splitting tests you describe. I wonder what would happen if the test had questions closer to the reality: e.g. How fast over the speed limit can you go without getting a ticket? When should you stop at a yellow light? When making a left turn, how much distance should you give yourself if a truck is speeding towards you at 60mph, even if you have the left turn arrow?

V Yonkers said...

HA! I was just reading the comment I left yesterday and realized that most in England won't get that left turn signal since you don't have to cross traffic to turn left! I guess it would be a right turn signal in Britain?

The upsycho said...

@v_yonkers True, but we know what you mean.