Data without context

Warning: this post contains some personal revelations you may not be comfortable to learn.

I recently indicated that I was being assessed for early onset Alzheimer's. I won't bore you with the details. Suffice to say that there has been cause for concern.

So I went to see my doctor. She advised me of the possible contributing factors for forgetfulness, such as stress, pain, medication and peri-menopause. Check, check, check and check. But, knowing how worrying uncertainty is, she carried out a blood test. All the key indicators for dementia came back within the 'normal' range. But this is simply data. Without a benchmark, how can anyone spot a trend, or make a prognosis? This is where I miss the old-fashioned concept of a family doctor who is able to see such data within the context of a life well-known.

To give her her due, the doctor followed this up with an aural assessment. This is a list of 30 questions that a doctor asks a patient with possible Alzheimer's. And this was about the most meaningless assessment I have ever done! The questions were so facile, that, in order to 'fail' the test I would either have to be pretty far gone down the road of senility, severely concussed or of low mental capacity. I kid you not. The questions were things like:

• What day is it?
• What month is it?
• What year is it?
• What season is it?
• Spell 'world' backwards.
• Recite the multiples of seven.
  
• I am going to tell you three things. I will ask you them later.
• Fold this paper then put in on the floor.
• This is a pen, this is a watch. Repeat.
• Name three things in this room.
• Name the same three things in reverse order.
• Name the three things I told you earlier.
• etc.
  

The problem with all these tests is that there is nothing to compare the results against. Without that, they are meaningless. Just data. Data doesn't become information until it has meaningful context.

We went through the same song and dance for years with hormonal levels. When I began to suspect that I was perimenopausal, at least a decade ahead of schedule, I went for blood tests. These showed my hormone levels to be at the low end of the normal range. But, once again, this was data without context. Meaningless. And so followed a series of three monthly blood tests which were similarly meaningless, until I just refused to go any more. I figured I had a better idea of what my body was doing than the doctor did, and I didn't need to subject my squeamish self to the ordeal of bloodletting to know the deal. The doctors were basing their judgements on data. I was basing mine on experience. I had the advantage of context.

It took them no less than 6 years to state with certainty what I had known all along.

Some of the quizzes we include in our learning resources are equally meaningless. Let's face it, we're dealing with adults. They know whether they have understood something or not. Surely it would be more meaningful, in a lot of cases to bump the so-called test and ask: have you understood this section? If the answer is no, going back over the same material again is unlikely to change that. So we should offer them a link to an alternative resource; or the contact details of someone 'with skin on' of whom they can ask questions; or the URL for a discussion forum, where user generated content may make the penny drop.

I feel much the same about 'tracking reports'. If you're Joe Bloggs's manager and you receive a report that tells you that Joe visited every single page of the recently published learning resource and that he got 80% on the quiz, what does this actually tell you about what Joe has learned? Many of our clients seem to feel duty bound to include some kind of pass/fail assessment as an indicator of how much a person has learned. C'mon, as Joe's manager you don't need any of that to tell you how well he does his job. You've got far more reliable indicators at your disposal.

Before setting in place a system or process that is going to capture screeds and screeds of data, perhaps we need to push back a little and ask a few key questions:

• What data do you want to extract?
• Why? What purpose do you want it to serve?
• Who is going to analyse that data?
• What do you really want to know?
  
• Are there other ways to discover that?
  
• How is it going to serve the business goals of the organisation?