Unbe-flipping-lievable

A rare emergence from hibernation for me, which just goes to show how strongly I feel about this matter.

I have just received the weekly newsletter from my younger son's school. Said son is in the throes of exams for the lower sixth form (the first of two years of non-compulsory further education which serve as a springboard for those who wish to go on to higher education).

The very first article in the newsletter focusses on the kids the year ahead of my son. In other words, those in the upper sixth. It includes the following:

Students seeking work or higher education places after Sixth Form have never before found the competition so tough. More students across the country are competing for fewer available places and many of our students must obtain A grades in a number of subjects (even A* in some cases) for university entry.

Our recent experience is that universities will not bend the requirements if students miss a grade. The need for students to revise really thoroughly has never been so vital.

It's all about grades! Other things get taken into consideration, sure, but only if you get the grades in the first place.

My son's results earlier in the year were very worrying. His teachers were at a loss: his homework is always submitted on time and is always 100% correct. But his exam results didn't even qualify for the word 'disappointing'. 'Terrifying' would be closer to the mark.

We are currently spending a fortune on private tutors in attempt to help this boy learn some exam skills, so that he can reproduce under exam conditions the levels of competency he demonstrates in the classroom. If he doesn't succeed at that, his lifelong career dreams will become even more difficult to achieve.

How can this be education?

William Nicholson wrote a trilogy called Wind on Fire. The first book starts off in this utterly ludicrous society in which people take regular assessments all the way through their lives. Where they live, what clothes they wear and every single factor of their lives is governed by the cumulative assessment scores of the family.

It is frightening how close we are to that. And I feel quite literally sickened as the parent of two sons who don't 'play the exam game' with any great skill. Nor can I help them there, because I have never been great shakes at it myself.

What kind of a dysfunctional society are we inflicting on these kids? And at what cost?

Assessment model blues

Last night was 'parents' evening' at my son's school. It isn't officially called parents' evening anymore, and hasn't been for some years, because the students themselves are a very important part of the three-way meeting. Nevertheless, the name has stuck on a colloquial level.

Our younger son is a grafter. Always has been. Every single one of his teachers spoke highly of his work ethic. This was no less true in the subjects where he is struggling. His classwork is always done, as is his homework. He contributes to class discussions, and raises excellent questions. He participates wholeheartedly in group work.

For the duration of his school days to date, every teacher (bar one) has found this child a joy to teach.

And it has been a joy to address his education as a parent.

This may have something to do with the fact that he has known since he was just three years old that he wants to be an explosives demolitionist. Of course, when he was three, he didn't know that it was called that. We didn't either. But as his commitment to this career path has remained strong for the past 14 years, we have learnt a lot about it along the way.

The problem for him at the moment lies with the assessment model. Particularly in respect of pure maths. In order to take Maths with Mechanics, which is essential for his future career, he has to take Pure Maths. And he isn't doing terribly well in the tests. In fact, it is fair to say that he is doing poorly.

Part of the problem is that he was placed in a rather low set for maths (over our objections, just by the way) and that set did not cover some sections of the syllabus. As a consequence, what constitutes revision for the rest of the class, is totally new work for those who were in that set.

But the teacher (Ms Verity who has previously made an appearance in this blog), has said she only wishes he could produce the same work in the test situation that he does for homework. His homework, apparently, is excellent.

The difference is that, when it comes to homework, he can open the text book (yes, they still use them) to the example page, and adapt the worked example to the problem(s) at hand. Or he can look it up online. If he struggles to make sense of the example, he can contact a friend via text message or instant messaging and they can work through it together. He has identified a study buddy, and he checks in with this kid, if he's unsure of his workings.

This sounds rather a lot like life, or like the reality of a working environment to me. Don't you agree?

In a test situation, you're deprived of those options. You can't research the solution and then apply it. You can't get a friend to check your work. You can't ask for support from someone with particular skill in this area.

But I think he is demonstrating a very mature, resourceful skill. A skill that will stand him in great stead for the rest of his life. A skill which far outweighs being able to manipulate surds... whatever the heck they are - I don't think they had been invented yet, when I was at school ;0).

Sadly, we haven't yet figured out how to assess that particular skill. We have only figured out how to subject kids to sensory deprivation and expect them to work purely from memory, and fail them on their inability to manipulate surds under these conditions.

You might think that I feel this way just because this is my son we're talking about, here. But I hope that my track record on the subject of assessment stands on its own merit.

In order to better equip my son to meet the current assessment criteria, we have decided to hire a private tutor (as an aside here, I should point out that the school has laid on extra lessons to help kids like ours catch up, but that these clash with rugby practice and we consider his sporting interests to be a valid component of a well-rounded school education).

Oddly enough, the tutor himself has made his own appearance on this blog before! Thanks to his background, we believe that this young man will be the right person to help our son make up the deficit. Of course, this is going to cost a fair amount of money, but our son has amply demonstrated over the years that he is more than prepared to put in the work from his side. Like I said: a grafter. And he needs this, if he is to get into the university he has identified, to get the qualification he has chosen in order one day to "blow stuff up and get paid for it". Because of that, we consider the money well spent.

But it doesn't mean that it sits easily with me. It is a little like training a performing monkey.

But I am satisfied that my son already demonstrates the sort of characteristics that an explosive demolitionist requires: he is sensible, responsible, hard-working, resourceful, determined... Put that on a report card!

Exam time again :o(

With one son busy with GCSE exams and the other with A2 levels, I am surrounded by the detritus of exam-time at the moment. Both my sons' girlfriends are also busy with GCSEs, which simply adds and extra dimension to the stress. Snapping and sniping and the occasional bout of near-hysterical laughter at nothing-in-particular are the order of the day.

I have shared before my antipathy towards exams as a means of testing competency in a subject, and we once again find ourselves reaffirming our every negative feeling on the matter.

My younger son recently sat a maths paper, the first half of which was based around a single scenario. Sadly for him, he didn't understand the initial scenario, so half the total marks for the paper were placed beyond his reach in one fell swoop. Students were presented with a quadrilateral of some kind and given information about a 'transection'. My son is familiar with the word 'transect' in daily language, but assumed that, in this case, it was a mathematical term with associated formulae and methodologies and so forth that he had somehow missed (like integration, for example).

Can you imagine his mounting stress as question after question referred back to this diagram that he simply couldn't fathom? Mentally adding up the marks that he was effectively barred from earning must have been gut-wrenching when he had been doing so well in the subject to date. By the time he reached the first question unrelated to the mystery diagram, his stress levels were through the roof and he could barely think straight. He knew he had to get practically full marks for the rest of the paper to be in with a hope of passing, and this placed him under additional pressure.

My elder son had a very similar experience last year with a statistics paper which centred largely around a single case-study. He got 37% for that paper, after having fared better in stats than any other subject throughout the year. He promptly betook himself to a tutor (okay, we betook him... and paid the extortionate rates) and got an A on the resit. He didn't learn any more about stats from the tutor. He learnt about exam papers... and went from 37% to 80+% in the space of a couple of months.

Now let's think how that scenario might play out in the work place (or anywhere else in 'real life'). Somebody gives you a diagram or a scenario and tells you to perform certain calculations on it which are pertinent to the situation. You can't figure out the diagram/scenario. What do you do? Well, quite clearly, you get some assistance. You ask someone to explain it to you. You look on YouTube or Google or Wikipedia. You look up unknown words in a dictionary. You phone a friend. You ask the audience. Whatever.

Then you perform the calculations and present them back to the person who needed them. Or you buy the floor tiles. Or supply the correct does of the required medication.

Nobody locks you in a sensory deprivation chamber and expects you to do it all on your tod from memory.

Ugh.

So irrelevant.

Reward

The whole theme of reward or cause-and-effect is exercising my mind lately. Mostly it just goes in circles.

I'm not about to disinter Pavlov, but I can't help feeling that we should be looking to reward desired behaviours in education, in the workplace and even in society in general.

Obviously, the nature of the reward would have to be made appropriate to the context. I can't see us giving a newly appointed manager a star for sharing his crayons with the person at the next desk.

Recently, my elder son asked me to intervene in a situation for him. He had lent a sum of money to his younger brother. It seems my younger son was going to repay this debt at £5/month, interest free. But that would take several months, by which time my elder son will have finished school and started his gap year in Australia. So he asked if I would give him the money, and let my younger son repay me at this rate.

I agreed, but I shouldn't have. The thing is, my elder son lent the money to his brother without establishing the terms of repayment first. This is not a very wise thing to do, and in taking over the debt, I have rewarded his poor choice. It isn't a lot of money in the grand scheme of things, so this would have been an ideal opportunity for a bit of tough love. A lesson to be learnt at no great cost.

I'm too much of a softie!

As I see it, the exam system also rewards the wrong behaviours. I know some kids who seem pretty dense, to be honest. They seem to know nothing about anything, and are utterly unable to hold a meaningful conversation about anything of substance. And yet they are in the top set for all their subjects at school, raking in the A grades. If education is preparation for life, the behaviours that should be rewarded are surely things like the ability to find information on a given subject, establish whether information is valid/reliable, apply information to a situation that does not exactly mirror a case study, work as a team, delegate responsibility, meet a deadline, be innovative with limited resources, overcome obstacles. You know... real life stuff.

We're a long way from that, I reckon. Probably mainly because we're a long way from knowing how to assess those skills!

I used to share a house with a few long distance truck drivers when I was a student. These guys were paid a certain fee to haul a load from point A to point B. They would get a bonus for doing it extra quickly. Not only that, but if they managed to shave a day off the delivery time, that gave them an extra day to start earning on the next project. So any time saved gained double rewards, financially speaking. So these guys would drive for 24 hours at a stretch, overdosing on caffeine in attempt to stay awake. There is no doubt that they constituted a hazard to other road users and themselves. In fact, I heard a couple of years back that one of them had been killed in an accident.

But the system was such that they were rewarded for this behaviour.

I have heard many tales of teenage girls in the UK, fed up with their parents restrictive rules, deliberately becoming pregnant in order to qualify for additional benefit payments and perhaps get a council flat. I'm not pretending to know a better way to respond in this situation, but it strikes me that this is another example of rewarding the wrong behaviour.

Like I said, my mind has been going in circles on this subject. I'm not sure I have any constructive answers, but I guess a change to the education system wouldn't hurt!

On exams... again!

I recently posted a little rant about exams. In recent weeks, I have discussed the concept with several people in different spaces. You see, for us in the UK (and probably the USA and Canada), this is exam time, as the academic year draws to a close. When you're the parent of a kid going through it, you might as well be going through it yourself, such is the impact on your life... and stress levels.

One of the analogies I have used is this: you have a glass-bottomed boat. You sail around a certain body of water for a year. Then for 45 minutes you go down and look through the glass bottom. On the basis of this short window (excuse the pun), you make a pronouncement about the state of the ecosystem in that body of water. It's ludicrous.

Today, Harold Jarche pointed me at this post by Daniel Lamire, which in turn pointed me to this post by Jon Dron. Dron's dripping sarcasm is clear in this summary of what the exam process taught him:

• that slow, steady, careful work is not worth the hassle - a bit of cramming (typically one-three days seemed to work for me) in a mad rush just before the event works much more effectively and saves a lot of time
• the corollary - adrenalin is necessary to achieve anything worth achieving
• that the most important things in life generally take around three hours to complete
• that extrinsic motivation, the threat of punishment and the lure of reward, is more important than making what we do fun, enjoyable and intrinsically rewarding
• that we are judged not on what we achieve or how we grow but on how well we can display our skills in an intense, improbably weird and disconcerting setting

He touches on a key issue when he says "I learnt to do exams early in life better than I learnt most of the subjects I was examined on and have typically done far better than I deserve in such circumstances."

And that's it in a nutshell. This is where my elder son falls down. His biology teacher told me earlier this year, "When you speak to him, you think he's really intelligent. But when you see his written work, you realise he isn't really." I think my jaw hit the desk. The PE theory teacher, however, summed it up more astutely: "This is one very clever kid. He just doesn't play the exam game."

Bud on Why our current education system is failing

Thanks to Stephen Downes, I came across this post today. It is written by someone still in high school, and is one of the most articulate, cogent arguments I have read. The debate in the comments is very telling, too!

At this point, consider there to be a section break. The next part of this post goes into a spin-off.

There is one section of his post, in particular, however, that I would like to tease out:

Our top priority must be to instill a passion for reading. The progress of humanity depends on it.

A number of my friends routinely say ” I hate reading”

And I always reply, ” You just haven’t found the right book.”

That 'you just haven't found the right book' argument has been mine, too. Verbatim. I have tended to believe that for every child, somewhere out there is the book that will flip the switch. The book that will take the 'non-reader' label past its sell-by date. I have always told my kids: there isn't a door in the world that is closed to you if you love to read. (Side note: I am trying decide what to make of Bud's view that the 100 books he has read over the past four years constitute a lot of reading, when I read 200 books during my last two years of high school, and my mother before me read 400 over the same period.)

However

I am the mother of a 15 year old son who hates reading. I am also the mother of a 17 year old who enjoys it.

Every night of their early lives, both boys had a bedtime story. At several points during the course of pretty much every day of their lives, someone would read a book to them, or go through a picture book with them. Both boys observed their parents' love of reading. Both boys loved to be read to (said she, ungrammatically).

When, as a baby/toddler, my elder son was 'too quiet' in a way that every parent understands only too well, I would go to investigate, expecting carnage, and find him seated in front of the bookshelf, surrounded by books, 'reading'. I must have repacked those bookshelves 5 times a day when he was little. Quite often, I would find him 'reading' Asterix when the books were longer than his outstretched legs. He liked to be allowed to 'read' an Asterix book in bed... and this was often the result (taken from our family photo album).

And yet. And yet.

I have bust a valve trying to find that 'right book' for my 15 year old. All. His. Life.

Our house has always been full of books. We made regular trips to the library when they were little. We have grabbed reference books to resolve disagreements about trivia at the dinner table. I have come out of bookstores weighed down and broke.

The books on the shelves in our house currently include stuff by Terry Pratchett, JK Rowling, Tolkien and Shakespeare. We have Anthony Horowitz books of all descriptions. We have Captain Underpants. We have Holes. We have The Curious Incident of the Dog in the Night Time. We have Anne MacCaffrey and John Meaney. We have John Grisham. We have exemplars of the juvenile secret agent type that is currently so popular: the Cherub series and the young James Bond stuff (and Horowitz's Alex Rider). We have William Nicholson. We have Christopher Paolini. We have Michelle Paver. Brian Jacques. Trudi Canavan. Frank Peretti. Lloyd Alexander. Sharon Creech. Chris d'Lacey. Roald Dahl. Caroline Pitcher. Colin Foreman. Kevin Crossley-Holland. Justin Richards. Steve Dixon. Jonathan Stroud. Rodman Philbrick.

Nothing. Nada. Not a sausage. Not a glimmer of love for reading.

It is utterly beyond me how anyone can not love to read, but my younger son is one such. He hates to be taken out of the hurly burly of interaction with other people. He loves msn. He loves World of Warcraft. Both these things enable him to engage with other people.

Oddly - and this really gets me - he has the most vivid imagination, and his creative writing is of a high standard. His critical writing, too. His evaluation of the play Blood Brothers read in parts like something I might have written (and before you ask, no, I didn't - I do not, will not do my kids' school work for them). Now, I don't mean to imply that I am a superior writer, but I hope you agree that I'm not bad... and he is only fifteen! His essay about Macbeth was even better. I am at a loss to explain this skill in someone who consumes so little written material from which to draw examples. I am just pleased that he has it!

I have had to learn to just let this one go. Perhaps one day, he will find the 'right book'. Perhaps he never will. I have to let him know that he is not a disappointment to me because he doesn't share my passions.

Have I ever mentioned how much I hate exams?

Later edit: for a far more eloquent look at this same subject from a larger picture perspective, please see Dave Snowden's post.

As you probably know, if you've been here for more than 3 nanoseconds, my elder son is in the throes of school exams. Yesterday, he walked out after the PE theory exam on a pink cloud. He loves PE and the jargon just trips off his tongue in consequence. He knows that stuff.

Today was statistics. All year, his stats teacher (the redoubtable Ms Verity who has featured here before) has told us how well he has been doing. How he just seems to 'get' stats. I was pretty sure there must be career ops for people who combine sport and stats (other than a bookie, of course!).

But it doesn't matter how well you do during the year. It doesn't matter how you perform in formative assessment. You can ace your mock paper. It matters not a jot. It all comes down to a single 90 minute exam... and he is certain he blew it. How can that possibly be right?

He was going great guns, thinking how easy the exam was and how he was going to romp through it, when the rope hit the rudder. About two thirds of the way through, he went blank. He just couldn't do the work. Nothing he had learned seemed to apply to the questions that were being asked.

Today's cloud was black, and he was under it, not on top of it.

I hate that a kid can pwn a subject all year long and then, in a fit of exam nerves, or because the examiner poses a question on a section the teacher glossed over, rather than one s/he majored on, the wheels can come off and it can all go to pot.

Of course, it remains to be seen how badly he really has done, but he is convinced that he has failed, and it has sent him into the Slough of Despond. He has a physics exam tomorrow. This is his worst subject, with his worst teacher (who is rather tellingly my younger son's worst teacher, too, and who has managed to worm his way onto my very short black list). How is he preparing? He is watching Futurama.

I am drained. If that elephant doesn't want to go up that hill, then it can flipping well sit where it is. I give up.

Mom out.

pic of the day - one down...

What a morning we've had! My elder son was having a lie-in because this is an exam week, and his paper was scheduled for this afternoon... or so he thought.

Good thing I checked the schedule!

There's nothing quite like dragging a 17-year old out of bed to go and take an exam... especially when you then discover that he doesn't have pens or pencils, or an eraser, or a ruler... or anything, really. An hour before the exam starts is no time to be discovering these things!

Trying to get him to hurry while also trying not to make him nervous was a balancing act.

It was too late to take the train so I drove him to school. Traffic was dire, and it took all my energy not to give off any signs of mounting stress. Chatting about inconsequential things, opting not to switch off a cringe-worthy interview on the radio, because it had him in stitches...

The relief was so huge when I dropped him off, that my driving skills went all to pot and I nearly had two accidents within 100m of the school.

I prayed all the way home.

Then this text message came through. Okay, so his use of apostrophes sucks, but what the heck!

This afternoon, I will nail him to a chair and make sure he has everything he needs for the rest of his exams. I will also bludgeon him (in the nicest possible way) into having everything ready the night before each paper.

I hate exams, but they're the currency we're forced to work with. We might as well optimise his chances as far as possible, because, with the amount of revising he's doing, the nett result of his AS year looks set to be a level 80 Undead Warlock (actually that's an exaggeration - he has a way to go before level 80). I'm not sure what career opportunities there are for undead warlocks... and I don't recall seeing this criterion listed among the hoops applicants have to jump through to get accepted to any of the universities he's interested in.

Ho hum.

On accreditation

The last two companies I worked for encouraged informal learning and independent study. I mention those as two separate things with good reason.

Informal learning can involve reading a self-help book, reading and commenting on blog posts, joining special interest groups online, etc.

Independent study can be quite formal, you might be studying for your bachelor's degree in your spare time, for example.

Independent study may result in an accreditation. A certificate of some sort, which you can frame and hang on your wall. It's also a lot easier to quantify in a CV when you're applying for a job. The thing is, though, that knowing that a person has completed the requirements for XYZ degree doesn't really tell you anything very much about that person's ability in the workplace.

Jessie (my half-Akita dog) and I are coming to the end of our current series of obedience classes. We have been invited to re-enrol, with a view to reaching a point where Jessie can do her bronze certificate, followed by her silver then her gold.

Let's consider. We would get a certificate, which we could hang on the wall above her bed. Would that make her a better pet? Would she be more obedient? How obedient do we actually want/need her to be?

At the moment, she comes when she is called. She sits, stands, rolls over, gives a paw, waits (for a bit), stays (also for a bit) and goes to her bed when told.

What the bronze certificate test would not reveal is that she is also devious.

• She steals things off the kitchen counter, including sharp knives - how she has managed to escape serious injury is beyond me!
  
• She lies on the couch in the lounge, until she hears anyone coming, then she leaps off and acts utterly innocent, blissfully unaware that the throw retains what we refer to as her hoof-prints, and the cushions are covered in her hair.
  
• She also - and this slays me - pretends to pee. If I take her outside and tell her to 'hurry up', she will squat for a moment and then sit there, grinning and wagging and waiting to be let back into the house.

My point is that the test assesses what a certain group of people have decided is appropriate for this level of obedience. It does not determine what sort of pet she is. It does not establish how obedient she is on other issues.

We want a pet dog that will fit in with our family.

I wonder to what extent the accreditations demanded by recruiters are relevant to the job role the applicant would perform, and the way they would fit in to the team. Must have a first degree. Must be CIPD accredited. Must this. Must that.

I totally understand that the initial slection process is carried out by HR professionals who do not have enough insight into every single job to be able to assess whether a person is likely to be able to handle it based on what they have chosen to share in their CV. So they identify a few key indicators, among which may be some or other form of accreditation. But to what extent are those pieces of paper a reliable indicator? And to what extent are they just like a bronze obedience certificate above the dog's bed?

Data without context

Warning: this post contains some personal revelations you may not be comfortable to learn.

I recently indicated that I was being assessed for early onset Alzheimer's. I won't bore you with the details. Suffice to say that there has been cause for concern.

So I went to see my doctor. She advised me of the possible contributing factors for forgetfulness, such as stress, pain, medication and peri-menopause. Check, check, check and check. But, knowing how worrying uncertainty is, she carried out a blood test. All the key indicators for dementia came back within the 'normal' range. But this is simply data. Without a benchmark, how can anyone spot a trend, or make a prognosis? This is where I miss the old-fashioned concept of a family doctor who is able to see such data within the context of a life well-known.

To give her her due, the doctor followed this up with an aural assessment. This is a list of 30 questions that a doctor asks a patient with possible Alzheimer's. And this was about the most meaningless assessment I have ever done! The questions were so facile, that, in order to 'fail' the test I would either have to be pretty far gone down the road of senility, severely concussed or of low mental capacity. I kid you not. The questions were things like:

• What day is it?
• What month is it?
• What year is it?
• What season is it?
• Spell 'world' backwards.
• Recite the multiples of seven.
  
• I am going to tell you three things. I will ask you them later.
• Fold this paper then put in on the floor.
• This is a pen, this is a watch. Repeat.
• Name three things in this room.
• Name the same three things in reverse order.
• Name the three things I told you earlier.
• etc.
  

The problem with all these tests is that there is nothing to compare the results against. Without that, they are meaningless. Just data. Data doesn't become information until it has meaningful context.

We went through the same song and dance for years with hormonal levels. When I began to suspect that I was perimenopausal, at least a decade ahead of schedule, I went for blood tests. These showed my hormone levels to be at the low end of the normal range. But, once again, this was data without context. Meaningless. And so followed a series of three monthly blood tests which were similarly meaningless, until I just refused to go any more. I figured I had a better idea of what my body was doing than the doctor did, and I didn't need to subject my squeamish self to the ordeal of bloodletting to know the deal. The doctors were basing their judgements on data. I was basing mine on experience. I had the advantage of context.

It took them no less than 6 years to state with certainty what I had known all along.

Some of the quizzes we include in our learning resources are equally meaningless. Let's face it, we're dealing with adults. They know whether they have understood something or not. Surely it would be more meaningful, in a lot of cases to bump the so-called test and ask: have you understood this section? If the answer is no, going back over the same material again is unlikely to change that. So we should offer them a link to an alternative resource; or the contact details of someone 'with skin on' of whom they can ask questions; or the URL for a discussion forum, where user generated content may make the penny drop.

I feel much the same about 'tracking reports'. If you're Joe Bloggs's manager and you receive a report that tells you that Joe visited every single page of the recently published learning resource and that he got 80% on the quiz, what does this actually tell you about what Joe has learned? Many of our clients seem to feel duty bound to include some kind of pass/fail assessment as an indicator of how much a person has learned. C'mon, as Joe's manager you don't need any of that to tell you how well he does his job. You've got far more reliable indicators at your disposal.

Before setting in place a system or process that is going to capture screeds and screeds of data, perhaps we need to push back a little and ask a few key questions:

• What data do you want to extract?
• Why? What purpose do you want it to serve?
• Who is going to analyse that data?
• What do you really want to know?
  
• Are there other ways to discover that?
  
• How is it going to serve the business goals of the organisation?

Criteria for successful completion

A while ago, I was walking my dog with my elder son. We were following the route I normally take when I walk her on my own, when he turned a different way. In a purely interested, non-critical way, I asked his reason for the different route. "It's a short cut," he explained. This of course led to a discussion about the point of the walk - completion versus process.

That conversation has been pottering away in the back of my mind ever since, popping up its head every now and then.

It made me think of the gym, where you will see these two approaches to exercise (among several others, no doubt). Your personal programme may include work on some of the resistance machines. Some people do the exercises with slow control, while others race through. Those who are going slowly have probably understood the point of the exercise is to really work those muscles. Those who race probably have their eye on completion. Get this done. Go home. Have a beer.

My sons have both always had chores around the house (I know, I know, I'm a battle axe and a harridan - I've heard it all before). Variations on this conversation have happened countless times:

Me: Look at the state of this mug, why is it in the cupboard?
Son: It came out of the dishwasher like that.
Me: But it's not clean!
Son: I can't help that. I washed it.

I have lost count of the number of times I've said something like, "When I ask you to vacuum the carpets, the job is done when the carpet is clean, not when you have run the vacuum cleaner once over the whole area."

I related a while back how my elder son claimed to have taught well in a session about javelin, even though his learners hadn't learned anything. His attitude was along the lines of: I have done what I was asked. The fact that it bore no fruit is out of scope for me.

One of the things primary school teachers often relate in respect of set work is that boys tend to complete the set work as quickly as possible, while girls complete the work as correctly and beautifully as possible. I'm not sure to what extent the gender thing really applies, but I certainly noticed the allegedly typical male behaviour pattern in both my sons in primary school. My elder son still tends towards this when it comes to written work, although his verbal reasoning is in a different league. Since reaching high school, my younger son has become more concerned with presentation, but this may have something to do with his interest in visual art, which is one of his chosen subjects.

Surely the completion thing comes down to understanding and buying into the criteria for success?

I'm not sure where I'm going with this line of reasoning, but it's an interesting conundrum, and I don't think the box ticking approach to education/training, whether in school or at work, is going to result in meaningful learning. It strikes me as being little different from the attitude of my elder son that I mentioned a moment ago. But my son is 17 and has yet to gain a sense of perspective. What's our excuse?

Running as assessment?

I used to run. A lot. I don't run anymore. I messed up my knee and the NHS says "We-ell, as we get older..." I don't have private medical. So I don't run anymore.

But I listen to runners talking. Twice a week, as I sit in the clubhouse while my son trains with the club. I hear them discussing this meet and that race.

And I think about running.

If you think about a race, it has a lot in common with summative assessments. You train for weeks on end, then you go and stand on the line with a bunch of other people. When that gun goes off, this is it. This is what all the training and preparation has been about. It doesn't matter how well you did in the training sessions. It matters only how quickly you cross that finish line.

You race against the other runners. You want to do better than them. You race against yourself. You want to do better than before. You're looking for that 'PB' that runners talk about. That personal best.

I'm okay that that's how it works in running.

I'm not okay that that model is applied in education. If I work my tail off all year. If I attend every lesson. If I complete all the set work. If I participate in class discussions and group work. And then I go to pieces in the sepulchral silence and the tension of the exam room, I am considered a loser?

Education is not a race. One brief moment should not outweigh the rest of the year.

In the exam room, I may not talk to other students. On the road, I can talk to other runners. In the exam room, I may not use my phone, or listen to my ipod to soothe my shattered nerves. On the road, I can do both of those things.

Er... no. Not any more.

It seems that in official road races nowadays (as opposed to fun runs) any competitor seen with an iPod is immediately disqualified. Two members of our club fell victim to this ruling at a recent event.

Perhaps they're afraid the person is listening to something which induces the same results as steroids. Perhaps having a rhythm playing in your ears gives you an unfair advantage (Was there ever a song like Faithless's "Mass Destruction" for giving a perfect running rhythm?)

I think we might have started to assess the wrong things out on the road just as much as we have been doing in the exam room.

The tide is moving in the wrong direction!

On playing the exam game

I have just returned from a rather harrowing parents' evening with my elder son. I met with two biology teachers, two physics teachers, two maths teachers and three PE/sports science teachers. We started with biology. The news was not good... and it went down hill from there!

Every single teacher told me that he talks a good talk. That, in conversation, you think you're dealing with a remarkable child, but then you see his written work and you discover he's actually not all that. His predicted grades are shocking: Cs, Ds and Es. They all complained that his homework is poorly done, if at all (one teacher claims never to have received a single piece of homework back from him - my sons swears he has given him homework into his hands on several occasions).

My son has been speaking with easy confidence and obvious enjoyment about what he's learning at school. The teachers think he runs a good bluff, and is gifted at blagging, but then, when it comes down to 'what matters' he can't deliver the goods.

At one stage, I was quite literally in tears. I wasn't sure how much more of this I could take.

But, blessing of blessings, the PE teachers were the last we saw. First up was the sports science teacher who had just given the class a tough assessment in which my son had scored a solid-but-not-stellar result. He said the magic words: "This is one very bright child who doesn't play the exam game. BUT... if he wants to get the results out of this system that will serve his purposes going forward, he is going to have to play the system."

The last two PE teachers told me that my son is scoring stellar results on their assessments. They had been worried that he would prove unable to walk the talk. Unable to reproduce in a written environment what he can do in a spoken one. It seems that, in respect of PE, he has got this sussed. Why? Because he loves the subject. Because anatomy and muscles and training and all that malarkey pushes all his buttons in the way that learning and learners do mine. So he had inadvertently figured out how to take his verbal reasoning skills and put them down in writing. We talked about how he needed to find a way to transfer that magic to his other subjects.

So on the journey back from the school I confessed to my son how much I loathe the assessment process. How misguided and misdirected I consider it. Then I launched into an analogy... as I am wont to do:

When I go to South Africa, I enter the country with Sterling in my pocket. This helps me not at all in South Africa. Even though the Pound is a far stronger currency than the Rand. Even though the British economy is (even now) in far better shape than the South African economy. If I want to buy something in South Africa, I need Rands. So I go to the forex counter and I exchange some of my Pounds for Rands. I use the Rands for as long as they serve my purpose, and then I return home to the stronger economy where I earn and spend Pounds.

This is what my son is going to do going forward. He knows that his Verbal Reasoning currency is stronger than that of Test Scores. He also knows that in the land of Education, the currency is Test Scores. So when he enters the exam room, he will exchange some of his VR currency for TS. He will think about how he would make his case if someone were to ask him the exam question in real life, and he will provide a written version of his spoken answer. He will trade in TS for as long as he needs to. He will use TS to buy what he desires and then he will return home to the land of VR where he excels.

Mercenary, yes. But until they change the system to one which ceases to disadvantage my son and others like him, these are the cards we hold. This is the currency we need. I resent it with every fibre of my being, but I will can't stand by and see his career prospects being limited because 'the system' can't see how bright he is!

Assessments

I'm always conflicted on the subject of assessments. I understand that they serve the purpose of identifying where the learner is strong and where work is required. But this is only true if the assessment mechanism is reliable. A series of yes/no or multiple guess choice questions has limited value, here.

• For a work-based learner, surely the point is for them to know where they need to brush up?
  
• For the university student, is the point for the university to establish whether allowing this person credit their institution with the letters behind his/her name going to bring the university honour or dishonour? Is it a worthiness thing?
• In the case of a school child, I'm hoping the point is that the teacher will get an idea of where the child needs help.
  

But I still think that a reliable method of assessment must revolve around application. For example, ask my sons to spell 'conscience' and they will say out loud "Con. Science," and then write it down correctly. But let them decide to introduce that word into a piece of creative writing or, even worse, an email, or worst of all, a text/IM, and you'll get all manner of interesting variations. So, as far as the teacher is concerned: can they spell 'conscience' or can't they?

Yesterday my younger son took a Spanish test. Of all of us, he is the weakest at languages. I have always had an ear for languages, which my older son has inherited. My husband doesn't have a natural flair, but he developed his ability in order to be able to get by when he arrived in a bilingual English/Afrikaans community in South Africa as a child, able to speak only Swedish. He is the most trilingual of us.

My younger son was woeful at French and, as a consequence, was not permitted to continue with it this September because of an insufficiency of French language teachers at the school. Instead he was moved to Spanish, together with all the other poor achievers.

Yesterday he did a vocabulary test, for which he scored 22/20. I kid you not. The test was for 20 marks, and there was a bonus 4 mark question at the end. He lost 2 marks in the main body of the test, but scored all 4 bonus marks. He is delighted. I wonder at the validity of such a test.

He is also delighted (what teenage boy wouldn't be?) that he learned how to say "did you fart?" in Spanish. I can see how this is going to prove very useful next time we visit Spain on holiday! Mind you, it will probably be a lot more useful than being able to say "I don't have my homework."

I have tried very hard to persuade the child to consider Spanish pod, but he is not motivated to do anything over and above what is on offer at school. So it's fair to say that he is not motivated to learn Spanish. He is merely motivated to pass the assessments.

So I ask again: what is the purpose of assessments and how is it that kids come to view them as the point of the learning, rather than the achievement of a certain level of skill?

One achievement - two perspectives

We had just dozed off last night when one of cats decided to subject us to an oration. She strode around our bedroom, giving voice. On and on it went and no amount of 'polite invitations' to her to "Shut up, stupid cat!" bore any fruit.

It dawned on me that she was probably regaling us with the tale of a successful hunt in which she risked life and limb to protect us from a marauding intruder.

I got up and switched the light on.

She was delighted.

"There. There it is see. I saved you. I protected you. Look at those fangs. See the dripping venom. What do you mean, where? It's right there. Are you blind, woman?"

I flipped back the corner of the rug and there it was.

A moth. About an inch from wing tip to wing tip.

I was unimpressed at the narrowness of my escape. Underwhelmed at the beast's ferocity. It's vicious fangs did not make my heart quail. Her tales of the danger to her own life and limb did not instil in me a sense of awe or wonder. My gratitude knew several bounds.

This was not the reaction Daisy had been hoping for and she proceeded to sulk.

I wonder how many other times I have been guilty of pooh-poohing what someone had thought was a great achievement until I came along to throw water over their campfire.

Hmm.

Meaningful assessment

We're a long way from through the turbulent period that I referred to in my last post, but I just had to get this off my chest.

I have a son who is closing in on his 17th birthday. In the UK, this means that the will be old enough to start learning to drive. This, in turn, means that he is madly trying to learn the highway code for the theory test.

I looked for a few sample tests online for him to try out... and, of course, tried them out myself. Like all drivers of many years' standing, I didn't fare too well! But one of the tests got right up my nose. I got 0/20!

Unlike all the other tests, this was not a multiple choice option. This test presented a picture of a roadsign and asked what it was. Fine, you might think. When I'm driving along the road, I don't get four options under each sign from which to select the right one. So, if I am not able to recognise the sign without prompts, I should probably not be driving, right?

Only, this is where the automated testing process falls down. Because of the limitations of the technology, responses were only marked as correct if the user entered the description verbatim from the highway code.

How is "decrease speed" a more correct response than "slow down"? Why is a "speed limit" not as acceptable as a "speed restriction"?

Even supposing I was able to answer all 20 questions verbatim, how is that any indication that I know what I'm supposed to do?

Surely the true measure of my knowledge of roadsigns is that I know how to respond to them? Note to my son: Shouting out "You're telling me!" does not constitute the required response to this sign.

It all comes back to our knowledge versus behaviour argument.

Then again, even knowing what to do and when to do it is no guarantee that it will be done! I am fairly certain that every single boy racer in the town knows that the speed limit restriction is 30 miles per hour. I'm fairly sure that he also knows what he needs to do in order to observe it. But to what extent does his actual behaviour reflect this knowledge?

Hmmm.

On being an encourager

Some years ago, when my husband and I had been identified as having leadership potential in our local church, we attended a 'school of eldership', where the characteristics of church leadership were explored. The course leader mentioned one thing that has stuck with me to this day.

He spoke of making sure that your life contained a healthy balance of people who played the respective roles of Paul, Barnabus and Timothy. Paul, of course was the one who mentored so many leaders. Having got off to a shaky start, we went on to become one of the most prominent figures of church history, responsible for writing much of the new testament. Timothy was a young man, placed into a position of leadership, but desperately inexperienced and unsure of himself. He needed mentoring and guidance. But it is on Barnabus that I would like to focus today. This is a man whose name is synonymous with encouragement.

Since we moved to the UK, my life has been desperately short of Barnabuses (in the flesh, at any rate) and it has been this lack that has made me realise how vital this often overlooked role is. We have become so accustomed to a society that complains (but usually does little to change things) that we see accusations and complaints where none are intended. Shortly after we moved into our current house (I can't keep calling it 'new' - we've been here since March!), we were standing in the front garden, chatting to our wonderful neighbour - a redoubtable man in his 70s. He has a son who is a musician who tours with a band, so he had noted with interest the arrival of several guitars and amps among our effects. As our elder son came up to join the conversation, Bill said to him, "So - have you been keeping up with your guitar practice then?" My son - not the persecution complex type, but a teenager nonetheless - read a veiled criticism into this and pointed out defensively, "I haven't even been here. I've been staying in Milton Keynes." We had to explain that Bill genuinely wanted to know how the guitar playing was coming along and meant nothing more by the question. To his credit, my son apologised and an enthusiastic conversation ensued.

We live in a culture in which we have become accustomed to criticism. It is so easy to criticise - especially when we can use the hyperbolic words like 'too' (too much, too long, too loud), 'always' and 'never'. On the other hand, it takes thought and insight to be able to offer genuine encouragement. And genuine encouragement needs to be specific, not some vague statement of general postivity. I learnt this from all the child-rearing books I read in the early years of parenting. Telling a child "You are so clever/handsome/wonderful," is all well and good, but genuine encouragement attaches itself to a specific action.

• "Wow! You climbed all the way to the top. Well done!"
  
• "You sang that whole song so beautifully and remembered all the words. That was very clever of you."
• "You solved 14 of these problems correctly. You're really getting much better at this. You have obviously been working very hard. Well done."
  

Recently, as we were driving somewhere, my husband got the mutters. We had passed one those illuminated street signs that flash a reminder at you when you exceed the speed limit. Usually, these switch off when you have dropped below the limit. Some of them even give you a smiley face. This one did neither. It flashed the reminder at my husband, even though he was already travelling below the limit. He is a self-assured man in his late forties, but he wanted his smiley face... or, at the very least, not to be 'told off'. He wanted acknowledgement that he was doing the right thing, not just some blanket admonition that didn't even apply to him.

In a classroom situation (end even with our colleagues), we can take the trouble (a) to offer praise and (b) to personalise it. But it doesn't have to stop there. Even though we are not present when users are using an online learning resource, we have the technology available to us to provide tailored messages of encouragement. There is an episode of Friends where Joey is trying to learn to speak French using a tape. He is delighted that the person on the tape says he is doing very well and, being Joey, he takes the praise to heart. The section I'm referring to can be seen from about 02:56 to 03:56 in this clip.

Few of our users are as undiscerning as Joey. They will want encouraging progress markers, but they will expect them to be relevant. So let's give some thought as to how we could build in some positive feedback.

For me, the first thing is to have some sort of initial assessment that identifies what a user already knows, and to affirm that: "Based on your assessment, you seem to be pretty clued up on x, y and z. If you prefer, you can omit these sections, although you might like to review them out of interest."

You might also choose to include knowledge check questions in the material, which afford you the opportunity to build in some notes of encouragement.

The same approach can be taken if there is to be a final assessment on the learning materials.

Of course, the most individual and sincere form of encouragement still comes from a person-with-skin-on (the online tutor, the line manager, a local champion etc.) but we can make an effort and make a start.

Any other ideas from budding Barnabuses out there?

Assessment must assess!

I am busy with a project at the moment, on which a bit of a sticking point has developed around the assessments. Some of the assessment questions being forward are, well, inappropriate.

One could argue (and I often do) that the most valid form of assessment is the performance management process. One's team leader and colleagues will be best positioned to assess whether your learning has resulted in any changed behaviours, increased skill, etc.

But there are times when compliance requirements must override such intangible evidence, and quantifiable evidence must be recorded. This generates the need for assessment.

Matters are not helped by the fact that the word that has passed into common parlance (thanks to the terminology used in the various applications) is "Quiz". It's hard to place much credence on an assessment that is termed a quiz, but it is what it is, so we'll move past that point.

The expectations of SMEs and sponsoring clients (and now I'm not just referring to my current project) seem often to be for multiple choice questions that are lifted straight from the text of the learning resource.

If it is possible for a person with no prior knowledge of a concept to take the assessment and pass, then I'd question the validity of the assessment. If the question is followed by a reference to page on which the answer may be found, even more so! At the very least, the questions must call for inference. Primary school children do comprehension tests on extracts from literature, where the answers may be found verbatim in the text. Once they get to secondary school, children are expected to be able to draw conclusions and to conjecture based on the given text. Why, when we're dealing with work-based adults, do we feel the need to revert to the verbatim approach?

We should be able to say with confidence that passing an assessment is indicative of something. Based on the fact that Joe Bloggs has passed this assessment, I should be able to assume that he has a good working knowledge of whatever-it-is. If this is not the case, then surely the assessment is a waste of time and resources.

When there is a compliance requirement for an assessment, this usually means that there is a regulatory body in the background. Let's imagine that a representative of this body is visiting our site. Proudly we tell him/her that 97% of our staff have passed the assessment on, say, first aid in the workplace. The rep asks to see the assessment.

Q1. Do you know where the first aid kit is kept?
(a) Yes
(b) No
(Correct answer: yes)

Q2. The position in which we place a person is known as:
(a) The pike position
(b) The first aid position
(c) The missionary position
(d) The recovery position
(Correct answer: d)

I can't imagine that this will fill the rep with any sense of confidence that, were he/she to take a tumble during that visit, any number of employees would be able to provide appropriate first aid. In fact, perhaps it is worth mentioning that I have a certificate that says I am a qualified first aider, yet I faint at the sight of blood - surely the assessment should have taken this small impediment into consideration?

I would also argue that a straightforward multiple choice assessment has limited validity. There are all manner of urban legends that tell of small children/monkeys/random generators that pass such assessments. All these aside, there are limits as to the challenge that such questions can present. Although it is worth bearing in mind that, with the technology at our disposal, there is much that we can do with multiple choice - instead of four text options from which to select, the user might be faced with images, video/audio clips or cartoon strips.

The trouble with assessment questions that go beyond "select the correct answer" is that they are likely to require some human intervention. Even if the answers are a couple of lines of text, someone will have to "mark" them, since the automated options are a bit thin and unreliable on this front. And this is where the rope hits the rudder. Quite often, elearning resources are requisitioned to obviate the need for human involvement in learning and development. I have lost count of the number of times I have suggested a project-based learning-cum-assessment to be submitted to the line manager within a specified timeframe, only to be assured that (a) the learners wouldn't do it and (b) the line managers would never agree to it.

Sigh.

Let's look at it this way:

• Do you really need a formal assessment? If not, skip it!
• Why? Your justification of your requirement will shape the assessment.
• How will it benefit the learner? And yes, this is the point... unless of course, this is purely a box ticking exercise.
• What do you want it to prove? What conclusions should I be able to draw from each person's results on the assessment?
  

Once you have drawn up your assessment, you need to ask yourself whether it serves these purposes. If not, I suggest you go back to the drawing board.

And above all, it must be possible to fail. If you have created an assessment that anybody can pass then what you have is not an assessment at all. It is an attendance register.

Assessing special needs: isn't there a better way?

This is the threatened follow up to yesterday's post, but this one is more directly related to learning, you will be pleased to discover.

Once again, this is not hypothetical.

Once upon a time, there was a baby girl who, at the age of 10 months, was still not able to sit up on her own. Her parents were referred to specialists in Cape Town, where she was assessed for cerebral palsy. The tests came back negative, and it was decided that she was simply atonic (which certainly seems to be the case - today she is somewhat clumsy and inelegant, but not attention-grabbingly so). While she was undergoing tests, she developed an ear infection - something to which she was prone. Later evaluations would indicate that it was likely that the medication she was given for this infection caused her to go deaf, since (a) it was contra-indicated for a child with grommets and (b) she had responded normally to sound stimuli before that time.

The audiologist duly assigned to her case was adamant that she should not be taught sign language. He argued that her poor co-ordination was inadequate to the task and, furthermore, that this avenue would restrict her communication to only those who knew sign language. Since there was no school which taught the approach known as "total communication" in her home town, the child was packed off to boarding school at a specialist school for the deaf 1100kms/700 miles away at the age of 4. This experience was traumatic for the entire family, and she did not progress at all well. After all, what child can learn effectively if she is miserable? Her communication skills did not improve a great deal, so instead of being cut off from hearing people, she was cut off from everyone.

At the age of 7, she was given a cochlear implant. The school felt that this move put her outside of the closely defined demographic they chose to address. She was duly sent back home. Sadly, having spent her formative years in a silent world, she decided that sound was a dreadful thing, and very quickly learnt to turn off the receiver. This meant that the reality of her situation was no different than before, but her eligibility for specialist schools had changed because of the mere presence of the implant - none of the schools were qualified to deal with a situation such as hers, and she was shunted from pillar to post.

It was in selecting a school for her to attend at this stage that her parents found themselves dealing with the inadequacies of tests designed to assess a child's mental capacity and potential. None of the assessments available could be applied. Because of her poor reading age, it was decided that she was very probably dyslexic (I have since learnt that deaf people often have poor reading skills). It was further decided that she had learning difficulties of indeterminate nature. Helpful. She was placed in a specialist school for children lacking in the intellectual capacity to cope with the rigours of the standard curriculum.

Having now left school, she is not qualified to do anything very much and runs the risk of losing her (pitiful) disablity grant, should she ever land a job. Of course, it's too late to change things, but I have often wondered what might have been done.

One thing I noticed, when watching her at play with other deaf children (disclosure: this girl is my niece) is that deaf kids are rough... I mean really physical. For those who work with them routinely, this is par for the course, but imagine a teacher trying to deal with this level of roughness in a mainstream school, where hearing children are not likely to put up with being pushed and pulled, hugged and thumped. Also, when a small deaf child wants something, she takes it. She is not intuned to the idea of manners or asking for something - nor do the wails of dismay from the "victim" alert her to the fact that she has caused any unhappiness. I can't see any mainstream teacher being able to handle this, while trying to teach the curriculum to all 30 children in the class.

Will we ever know what her true potential is? She never acquired the literacy skills to complete a written assessment of her mental capacity, and she doesn't possess the communication skills to complete the verbal tests that are sometimes used as an alternative. How might she have been tested to see if she is, in fact, as intellectually handicapped as has been assumed?

I was very glad when the misery of boarding school was brought to an end, even though this threw her future education into a limbo from which it never recovered. During the time she was a boarder, she used to come to my home for a weekend every month, where it became apparent that the little things that parents do for their small children as a matter of course (I'll spare you the details) were simply going undone, and she spent the entire weekend sitting listlessly on the couch with tears rolling down her face - unable to tell me why she was so unhappy. Sadly, I think that she assumed she was being taken home whenever I came to collect her, which might have had something to do with it.

I apologise for the heartrending sadness of this story, but I wonder if you can suggest what might have been done differently.

Summative assessment - rugby style!

As those who follow me on Twitter will be painfully aware, our house is full of rugby at the moment, due to the fact that world cup is currently taking place in France. The quarter final round is about to begin and South Africa looks set (on paper, at least) to go all the way to the finals. Now before you North American types assume that this competition doesn't involve you, I would like to point out that both the USA and Canada have teams in the competition. Unfortunately both have been knocked out at the group stage, but still...

I have been a rugby fan for as long as I can remember. My father and grandfather both played the game well, and my maiden name was at one stage synonymous with the sport, since an assortment of second cousins x times removed featured prominently in the national team.

So where am I going with this?

During this competition, I have been struck again by the role of the referee in a rugby match. In no other sport that I can think of, does the ref behave in quite the same way. You see, rugby refs (in national and international matches) are miked up so that the television audience can hear what they're saying... and they say an awful lot.

Quite apart from explaining infractions when they have blown their whistle, they also call out things like "Hands off, blue!" "Watch your distance, green!" Constantly making input into the way the game is being played and ensuring that the teams observe the rules and the flow of the game is interrupted as little as possible. To be honest, it is almost as if he's coaching them... but this is a world cup match! And instead of just penalising the infractions, the ref is involved in minimising stoppages. Another way he does this is to "play advantage" - this means, if an infraction is made against a team, but that team maintains possession of the ball, the ref will let the game go on. If possession is quickly turned back over to the offending side, he will then impose the penalty for that infraction. He always indicates when he is playing advantage so both sides know the deal.

It got me to thinking about the extent to which we could apply this approach to situations in which we normally apply summative assessments in learning. An assessment that followed this approach is far more likely to be seen as formative, and yet, here is the rugby world cup being awarded only once every 4 years on the basis of a series of "formative" assessments.

Hmm. I wonder...