Tuesday, June 19, 2007

Human Applications for Swarm Mentality

Have you ever marveled at how a flock of birds can seem to act as a huge collective organism, wheeling, diving, climbing, and darting in a sky dance that seems impossible to choreograph? Have you ever noticed that these flocks have no particular leader?

This same kind of behavior is exhibited throughout the animal world in various diverse populations, including ant and bee colonies, wildebeest and antelope herds, and fish schools. Scientists have long studied what they call swarm mentality. They want to know how various populations, which individually are relatively unsophisticated and error prone, can somehow act cooperatively without strong leadership to consistently make complex decisions that turn out to be best for the population overall.

The July 2007 edition of the National Geographic Magazine includes a magnificent article on this topic (links will not become active until 7/1/07). The article discusses what scientists have discovered about swarm mentality and how this learning is being applied to address complex issues in the human world. As a computer programmer, this article was particularly fascinating to me because it addresses various computer and technology solutions that are based on swarm mentality.

I once worked for a firm that was in the gasoline and diesel fuel retail business. Although many people are upset at our current high gasoline prices, there is actually a very small margin in the retail end of this industry. The company was continually trying to achieve the lowest cost of product for sale. Sounds simple, right? It’s actually very complex when the multiple variables involved are considered.

The company had long managed its distribution with a small staff of highly knowledgeable experts that acted as much on instinct as they did on information. But with growth, the task was becoming unmanageable. And upper management knew that there were inherent inefficiencies in this centralized approach, regardless of how good the experts were. They knew that if they could cut product costs by even a fraction of a penny per gallon they would increase profits substantially.

Our team of Information System specialists was called in and tasked with studying the feasibility of creating a computer system that would achieve maximum efficiencies in product distribution. Once we understood everything involved, it became clear that we were talking about an artificial intelligence system that would rely on inputs that were virtually impossible to consistently obtain at that point in time. Worse yet, it would require a system that could consistently make good guesses about thousands of daily tanker truck runs.

The NGM swarm article discusses how Houston based “American Air Liquide has been using an ant-based strategy to manage [the] complex business problem” of highly efficient product distribution. The complexities of AAL’s distribution system mimic very closely those of a retail gasoline distribution business. AAL contracted NuTech Solutions, which developed a program that sends out billions of software “ants” daily to delve into various aspects of the complex issues involved. None of the ants is very smart, but each is given a specific type of task and each leaves a software “pheromone trail.” Each day, the program finds the strongest pheromone trails for truck routes. The routes are not always intuitive for drivers, but the savings for the company have been “huge.”

(Contrary to what is portrayed in cartoons, insect colonies do not have strong central leadership. Queens perform a function, but leadership is not a large part of that function. Rather, colonies of simple creatures act collectively to address complex issues.)

The discussion in the article of how bee colonies work through open thought, trials, idea sharing, consensus building, and voting is also quite interesting. Tests repeatedly showed that using these methods allowed bee colonies to consistently select the best site for establishing a new nest. Experts cited in the article tout the effectiveness of this methodology “in boardrooms, church committees, … town meetings,” and faculty meetings.

When it comes to flocks of birds and schools of fish, it turns out that each individual does its own thing while obeying a few very simple rules of thumb.

As I read through the NGM article, I thought that it sounded very libertarian. Libertarian thought contends that maximization of individual political and economic liberty results in the greatest well-being for society. It is asserted, for example, that a mass of individuals acting out of self interest on only local knowledge will invariably act collectively to produce a better overall outcome for society than any kind of centralized approach could ever hope to achieve. But there was very little discussion of political or economic philosophy in the article. It did mention how the futures market and horse race gambling effectively mimic swarm mentality.

The NGM article did include a warning; however, that swarm mentality can break down and produce inferior results. “Crowds tend to be wise only if individual members act responsibly and make their own decisions. A group won’t be smart if its members imitate one another, slavishly follow fads, or wait for someone to tell them what to do. When a group is being intelligent, whether it’s made up of ants or attorneys, it relies on its members to do their own part.”

And that is the flaw in pure libertarianism. It requires responsible individuals that each performs his/her part, and it requires individuals that think and act individually rather than engaging in Groupthink. From a purist perspective, those that do not pull their weight would be treated as are unproductive individuals in ant or bee colonies: they would be killed or otherwise removed from society. Although some claim that we do this today through our justice system and other social structures, I would argue that we generally bend over backward to be merciful to those that make mistakes or that have diminished capacities.

At any rate, we have a significant number in our society that cannot be counted on to act individually and to perform their necessary duties. Thus, pure libertarianism (like pure communism on the other side) is a utopian ideal that cannot exist with imperfect people. This is the reason for many of our current governmental and social structures.

I do believe that swarm mentality offers some good insights into how we can approach solutions to complex issues. But I do not believe that it is the be all and end all of everything. It certainly offers some very interesting prospects from a computer programmer’s point of view.


Anonymous said...

I agree that our society isn’t likely to adopt “pure libertarianism” as a “utopian ideal” but I can’t help but think our country would be better off if more people adopted our nation's founders' idea that the purpose of government is to protect the natural rights of its citizens not to take responsibility for fulfilling their every need. The ever spiraling growth of our “mixed” form of government is just as unrealistic an ideal as the “pure libertarianism” straw man you knocked down in your post.

Your argument that libertarian belief in a limited government is like a swarm mentality is a little lost on me but I agree with you that the implications of these ideas for software development and artificial intelligence are extremely interesting.

Scott Hinrichs said...

The whole idea behind swarm mentality is that individuals carrying out their respective responsibilities and acting in their individual interests, collectively create a society that makes decisions synergystically, much better than the sum of its parts, and much better than could be achieved by centralized planning. The libertarian ideal of the free market works on basically the same principle. However, libertarianism promotes a much higher level of individual freedom and autonomy than seems to be permitted in swarm mentality.

I agree that our mixed form of government is destined for correction at some point. The question is whether that will occur through a series of small corrections or via one huge rude awakening. And although you deride my libertarian straw man, he resembles very closely the view that I get from many libertarian acquaintences.

I agree that government would serve society better by acting as the protector of natural rights rather than acting as a nanny. When you study the actual history of our nation; however, it is shocking how quickly we started down the nanny path. The first nanny provisions (quite rudimentary by modern standards) came in 1790. Thus, we have had elements of mixed government for a long time. But it was only with FDR that we made the formal shift to nannyism. This was later significantly strengthered under Johnson.

And despite the faults I note with libertarianism, it is those libertarian ideals that have slowed our nation's decline into the full nanny state. For this reason, I am glad that Rep. Ron Paul is in the presidential race. He doesn't have the slightest chance of winning (or of becoming VP, for that matter). But he can help influence the debate, and hopefully slow or alter our present course.

Anonymous said...

"For this reason, I am glad that Rep. Ron Paul is in the presidential race. He doesn't have the slightest chance of winning (or of becoming VP, for that matter). But he can help influence the debate, and hopefully slow or alter our present course."

A hearty amen to that!

Anonymous said...

There's a dark side to the human swarm mentality, as well. 4GW insurgents have discovered it is a force multiplier for them when they attack state institutions and achieve unbelievable ROI's (e.g., attack on oil pipeline costs $1,000 but results in $1 million loss to state). John Robb discusses this concept at length on his blog (globalguerrillas.typepad.com) and in his book, Brave New War.

Scott Hinrichs said...

I also thought about insidious uses for swarm mentality. Like any good tool, the bad guys can use it too.