Friday, May 19, 2006

Wisdom of crowds

Yesterday, I attended a team development session that focused on group decision making, drawn in part from James Suriowiecki's book "Wisdom of Crowds". In case any of my team who didn't attend are interested, here is a synopsis of what it was about, as well as my own reflections:

Philosophers such as Nietzsche and Thoreau contended that, while individuals may be wise, crowds/groups were not. Surowiecki's book challenges this view and cites research by Belbin and Galton that evidences that group judgements, when aggregated, tend to be wiser or closer to the truth than those of individual experts, without there being a need to start with a particularly smart or well-informed group. The example of "ask the audience" in the TV quiz show Who Wants to be a Millionaire was cited - apparently, over the history of the show, this option has generated more correct answers than the option to "phone a friend" who may be expert in the subject.

Types of crowd wisdom cited were:
Cognition: pertaining to market judgement
Co-ordination: pertaining to behaviour patterns
Co-operation: pertaining to the voluntary suspension of self-interest and/or distrust to form a network

Examples of causes failed crowd intelligence:
Too centralised: Columbia shuttle disaster blamed on the overly hierarchical structure of NASA
Too divided: failure to prevent 9/11 blamed on lack of cohesion between intelligence analysts
Too imitative: the information cascade, resulting in the "everyone's doing it" mentality
Groupthink: a situation where each member of a group deliberately conforms to what appears to be popular consensus. The Bay of Pigs debacle was blamed on this phenomenon in the team of decision makers

We covered some of the experiments that have been conducted to explore things like groupthink, conformity and obedience:
Asch's conformity experiments which investigated the extent to which individuals could be influenced to make a patently incorrect statement so as not to disagree with the majority.
Stanley Milgrim's obedience to authority experiments which investigated what he believed to a be a peculiarly German trait to carry out acts of atrocity in obedience to instruction (he obviously hadn't heard of the Zulu half brothers Shaka and Dingane). To his surprise (and mine), apparently 65% of us are likely to administer a potentially lethal electric shock to a complete stranger on the instruction of an authority figure. Regardless of nationality. I can't help wondering what the long term impact was of this realisation on the naive subjects of this experiment!

The elements required for a wise crowd were identified as:

  • diversity of opinion
  • independence
  • decentralisation
  • aggregation
In order to be able to be an indepent individual in a wise crowd, Surowiecki's recommendation is:
  • keep ties loose
  • expose yourself to as much information as possible

Some recommeded reading to come out of the session:
Wisdom of Crowds by James Surowiecki
Obedience to Authority by Stanley Milgrim
The Toyota Way by Jeffrey Liker
An Intimate History of Humanity by Theodore Zeldin

My reflections
I found the workshop interesting and informative. I do not consider myself sufficiently expert in the subject matter to take issue with most of the allegations or findings, but I did have one major reservation. I was concerned about the use of the terms "crowd" and "group" interchangeably. I consider them very different phenomena and subject to very different psychological influences. To me, a group would be something like the electorate, motorcyclists or television viewers - a disparate collection of individuals loosely connected by a common interest or purpose (my own definition that probably needs work!). A crowd, on the other hand would be an audience, a rally or a congregation - all simultaneously subjected to a set of circumstances. I strongly believe that this is an important distinction that will have enormous bearing on collective behaviour, although the workshop presenter did not agree with me.


David said...

Yes I agree with Karyn it was a very interesting evening. James did an excellent job of pulling together thinking around group dynamics and influence or coersion.

The main point I took away from the session was that a level of conflict can be and is often constructive. James mentioned the devil's advocate approach to stopping Groupthink developing thus ensuring that ideas are fully critiqued and all views taken into account. I see this and other techniques as important in a healthy debate and necessary in organisations. How many organisations or teams do you know where the emphasis is on conformity, agreement and having single way of doing things? -- team days away to bond and develop a single view. Obviously a team need to agree on where they are going, but all too often the methods of getting there may not be the best, or even work.

Unfortunately it is often easier to agree with the consensus. Also the dissenter role may fall to the same person in a team too often which can lead to them being discounted or asked to leave.

Managing the amount and level of information to for consideration is challenging for some teams. I guess we can all think of some situations where decisions are not made because there is always the call for more information or another study. Malcolm Gladwell talks about how this in "Blink: The Power of Thinking Without Thinking" which is worth a read.

A thought provoking evening. Thanks James.

Anonymous said...

For a slightly different perspective, you might appreciate reading Freud's "Group Psychology and the Analysis of the Ego". It has some interesting insights as to the influence of leaders, power, and "hypnosis" on groups and group behaviors.


Anonymous said...

And of course, sometimes a group of "experts" is better than a "random" group...

Plato "Ion"

SOCRATES: And yet surely, my dear friend Ion, in a discussion about arithmetic, where many people are speaking, and one speaks better than the rest, there is somebody who can judge which of them is the good speaker?

ION: Yes.

SOCRATES: And he who judges of the good will be the same as he who judges of the bad speakers?

ION: The same.

SOCRATES: And he will be the arithmetician?

ION: Yes.

SOCRATES: Well, and in discussions about the wholesomeness of food, when many persons are speaking, and one speaks better than the rest, will he who recognizes the better speaker be a different person from him who recognizes the worse, or the same?

ION: Clearly the same.

SOCRATES: And who is he, and what is his name?

ION: The physician.

SOCRATES: And speaking generally, in all discussions in which the subject is the same and many men are speaking, will not he who knows the good know the bad speaker also? For if he does not know the bad, neither will he know the good when the same topic is being discussed.

ION: True.

SOCRATES: Is not the same person skilful in both?


Plato "Georgias"

When the assembly meets to elect a physician or a shipwright or any other craftsman, will the rhetorician be taken into counsel? Surely not. For at every election he ought to be chosen who is most skilled; and, again, when walls have to be built or harbours or docks to be constructed, not the rhetorician but the master workman will advise; or when generals have to be chosen and an order of battle arranged, or a position taken, then the military will advise and not the rhetoricians: what do you say, Gorgias?


Anonymous said...

I think Aristophanes captured the true wisdom of Crouds in his comedy The Clouds.

Better answer the epistemological question as to what "wisdom" is before one attempts to answer the question as to whether crowds possess wisdom.


Anonymous said...

Perhaps the reason why "group judgements, when aggregated, tend to be wiser or closer to the truth than those of individual experts in a democracy can be explained by Nietzsche, "Will to Power":

493 (1885)
Truth is the kind of error without which a certain species of life could not live. The value for life is ultimately decisive.

534 (1887-1888)
The criterion of truth resides in the enhancement of the feeling of power.