Is Parallel Programming Hopelessly Counter-Intuitive?

Much ink has been spilled over the past few years claiming that parallel programming is hopelessly difficult and counter-intuitive. Although I would never claim that parallel programming is as easy as sequential programming, the large quantity of parallel software currently available should be taken as evidence that parallel programming is not impossibly difficult.

For example, perhaps you have heard of one or more of these products and projects: AIX, Code Sourcery, DB2, GEDAE, HP/UX, JBoss, Linux kernel, MySQL, Oracle's database kernel, PostgreSQL, Rapidmind, Solaris, SQL Server, WebSphere, and Windows, to name but a few? If parallel programming were impossibly hard, these projects (and many more like them) simply would not exist.

Of course, just because parallel programming is possible doesn't necessarily mean that it is any place for “mere mortals”. Except that many very ordinary software engineers were able to pick up parallel programming quite quickly during my time at Sequent and with the Linux kernel community, which indicates that parallel programming just might be open to anyone who takes the time and trouble to learn how it is done in the “real world”. (But wait!!! Aren't some of the same people complaining about the difficulty of parallel programming also decrying the Linux community as a group of unschooled and unskilled hackers? Now, isn't that a very strange combination of views to adhere to?)

Some might object that these two communities are atypical. Now, this objection begs the question of what precisely a “typical” community might be, a question that could be discussed indefinitely. In order to make forward progress, we will instead examine the question of whether concurrency itself is inherently counter-intuitive.

Many decades ago, while I was in my mid-teens, I witnessed a dramatic demonstration of the everyday nature of concurrency. The doorbell rang, and, fool that I was, I answered it. To my astonishment and dismay, five toddlers raced into the house in five random directions, all five of which lead to numerous attractive nuisances best kept well away from toddlers. Their parents rushed into the house, very quickly and efficiently gathering up the toddlers, laughing like hyenas all the while. To my credit, I did manage a semi-reasonable facimile of a greeting, although only after after retrieving my jaw from its new-found home on the floor.

It was the Anderson quintuplets traveling down the west coast of the US, stopping each night to stay with family friends and friends of friends (with my family in the latter category). Their family had a strange aversion to hotels, or perhaps vice versa. Based on my observations during their visit, I am here to tell you that both parents were quite adept at spawning however many threads of consciousness were required by the situation at hand, which was almost always more then one, especially given that they had two children in addition to the quintuplets. (If you never have baby-sat a group of five toddlers, you should try it. It is highly educational — assuming that you survive the experience.)

An obvious objection is that quintuplets are a recent phenomenon, and therefore cannot have had much influence on the evolution of the human mind. However, people have been required to coordinate the activities of groups of people and herds of animals from the beginning. It is hard to argue that coordinating the concurrent activities of CPUs is more difficult, particularly given that people and animals are not always as cooperative as one might wish. In contrast, CPUs are if anything excessively cooperative. In addition, CPUs are considerably simpler than either people or animals, and furthermore CPUs have the great advantage of coming with reference manuals.

But if you still doubt that stone-age human beings needed to handle concurrency, consider fate of Urg's unfortunate acquaintance, shown below.

Concurrency-challenged caveman

If only, like the surviving members of his tribe, to say nothing of today's heavily multitasking teenagers, this poor guy had been able to handle concurrency!!!

In short, given the crucial role of concurrency in everyday life, do you really want to assert that concurrency is counter-intuitive? If so, on what basis do you defend your assertion?

The burden of proof rests firmly on your shoulders.