Monday, 12 October 2009

When to Be One of the Crowd?

Lemming - public domain image, Argus fin on Wikipedia CommonsThis quote from Terry Pratchett came up on Samizdata recently:

The intelligence of the creature known as a crowd, is the square root of the number of people in it.

and the criticism it received in the comments set me thinking.

Now on the face of it, Pterry does seem to have nodded here. The intelligence of a crowd goes up, according to his formula, with the number of people in it. This doesn't accord too well with what I've seen of mass human behaviour. Such other classical formulations as

The IQ of a crowd is equal to the IQ of its least-intelligent member divided by the number of individuals in the crowd.

whilst maybe harsh, provide a more familiar picture of what happens when humans devolve into a mob.

But two thoughts prevented me from immediately buying into the general criticism. Firstly, Mr Pratchett is, like me, an individualist and honourary member of the great caprine nation. Secondly, he is noticeably smarter than I am. Since this does not, offhand, seem like the sort of mistake he would be apt to make - I wondered under what circumstances he could be right.

And then it hit me, with considerable force, that a crowd is not at all the same thing as a mob. Consider the difference between the crowd in Covent Garden on a balmy summer evening, and the mob that will gather there when Katie 'Jawdown' Price and Peter 'Panopticized' André perform their globally-syndicated reconciliation duet of Darling Let's Have Another Baby in that very same location. In the first case, we have to speak of large numbers of individuals positively revelling in their various opinions and agendas; in the second, only one agenda is in force and only a narrow range of opinions will be held or appreciated. Significantly, only one of these evenings will be crawling with coppers and stewards. It will also be the one that features me shooting myself in the head with the aid of a hastily-procured illegal airgun and a large box of Maltesers, but let that pass.

Can we narrow it down further? Is there more to this than individuals merely being smarter when incidentally crowded together than willingly mobbed-up? Can a crowd in itself be smarter than a mob? Or - as the Pratchett quote demands - actually smarter than its component individuals?

Yes, it can, I say; and it can in principle be very nearly be quantified how much. There is an angry mob waiting with some worse news around the corner, but we'll get to that presently.

If you independently poll a large number of rational agents each with partial knowledge of a subject, and go with the majority result, then you will certainly do better than by asking any one bog-standard agent, and quite possibly will do better than by asking an expert one. This apparently trivial observation is sufficiently counter-intuitive that James Surowiecki's bestselling The Wisdom of Crowds, and the hip modern gurus of crowdsourcing, can dazzle quite remarkably by flashing it about. Their ultra-democratic perspective on decision-making begins as a simple consequence of probability theory, and is powerful right up to the point that the agents cease in any way to be independent.

The Wisdom of Crowds is rather light reading, but it is worth the bother because the author is no mere huckster, and dwells extensively on the Vox Pop Oracle's very special failure mode. This is called an information cascade, and occurs when members of a 'crowd' fail to add their private opinion to the pool, because they prefer to go along with the majority opinion among the prior contributors. So no further information is added, and the crowding of opinions ceases to add up to any further wisdom. At its worst, this may lead to the valid and well-informed opinions of large numbers of people being changed to their hurt, because the snowball got rolling in a benighted region of the Alps where it is unknown how many beans make five, and now of course a million Frenchmen can't be wrong. I suggest that when people get thus swept up by the snowball, they have joined the collective organism known as a mob, which grants not wisdom but stupidity in proportion to its size.

I can't stress enough that this is no mere herd instinct, but an inherent problem amongst groups of rational agents who have no way of knowing how far each other's opinions are really independent. If I have a factual belief on which I will bet that I am correct with estimated probability p, and I believe that some random Joe Sixpack is only likely to be correct with probability q, then there is some number of Joe Sixpacks whose united probability of being correct will exceed my own, and it will be rational for me to defer to their collective wisdom. But this is only true if each additional Joe is bringing his own sixpack of wisdom to the party - rather than just another empty vessel. And if I am not very canny, I may get Joe rather badly wrong.

Add to this the herd or social dynamic of conformity - whereby people adopt dominant positions chiefly for comfort, gain, or fear of harm if they dissent - and the problem of the mob becomes evident with a vengeance. Again, it is idle and foolish to dismiss those so influenced as 'sheeple' or some such blow-off. A lot of people behave this way because, for them, it is usually the rational way to spend their time and clout - and their gut instincts are always ready to remind them of the fact. The reason they have those gut instincts is because they have frequently proven right!

Which is, of course, no defence against marketers, demagogues, and other predators who have learned how to game them.

It seems to me that reluctance to go along with the crowd is generally a conservative gambling strategy. It tends to eliminate the large gains to be made from crowdsourcing one's judgements, but also the large losses from falling over an information cascade without a barrel.

Contrarianism - actively going against the crowd by preference - is not conservative at all: it is betting specifically upon the crowd's ignorance, and implictly on the likelihood that it has devolved into a mob and is doing the funky lemming. Like going with the crowd, it is a high-risk strategy unless the gambler knows something the crowd doesn't. Specifically, whether it is a crowd with common sense or a mob with a common delusion...

I am temperamentally somewhat risk-averse myself, and tend to suspect populations at large of great sheepishness. According to this analysis, I am therefore quite rational in my general stubbornness and slight tendency to the contrarian. The analysis must be right, then!

(La-la-la-LAH-la, I can't HEAR you...)

No comments:

Post a Comment

Note: only a member of this blog may post a comment.