Hackers, Painters and a Coasy Zeitgeist

Ronald Coase was an economist working in the 1930's and as far as I know was the first to wonder why organisations were the size they were - and he came to the conclusion it was due to transaction costs. If it is more expensive (hassle, price etc) to do a transaction with a 3rd party than supply X in house, then a ompany will do that. If not, it will outsource. This is encapsulated in Coase's Law:

" As transaction costs decrease, the complexity of the firm diminishes. A firm tends to expand till the cost of organizing
that extra transaction within the firm become equal to the cost of organizing the same transaction from the open market. "

I first re-visited this law about 15 years ago, when it became clear that the emerging internet was going to change the transaction costs of business (and much else) by at least one, if not several, orders of magnitude. At that time corporates were large (probably on average larger than today, conglomerates were still unwinding) and the "startup / SoHo" scene was minimal. Articles on electronic cottages, the hollow / shrinking / transparent company etc duly appeared with the zeitgeist of the time, and then suddenly it seemed that everyone was a startup, and the dotcom bubble began.

However, the dotcom bubble popped, but while everyone was watching that a far more rudimentary execution of Coase's law took place - offshoring stuff from high labour to low labour cost countries, as the transaction costs of managing something halfway across the world started to fall. Then from about 2004, ubiquitous broadband and falling hardware costs led to the rise of many Small Office / Home Office (or increasingly No Office) businesses (and I use the term increasingly loosely as increasingly we see ad-hoc collaborations rather than companies doing things), and websites, blogs and organisations sprang up to represent them - Web Workers Daily being a typical example.

What was happening was that broadband penetration and the ability to communicate via the 'Web was massively reducing transaction costs, and I think we are in a situation today where Coase's Law suggests that many organisations are far too large for the new costs of doing business. Things don't all fall apart at once of course, I would argue that Coase's Law works at the edges initially "in the Wild" - as transaction costs go down, various parts of organisations loosen their ties - some secede, some are seceded, not all succeed etc etc.

This was all under the radar of course, people just got on and did what was economically and technically rational - but all things go in cycles, and I have noticed the re-emergence and resurgence, in the last year or so in the academic literature of the "small and independent is beautiful" zeitgeist. Recent books on Starfishes and Spiders and books by Clay Shirky, Charles Leadbeater et al are all signs of this meme climbing back over the parapet.

(Maybe some economist can explain the 10-year cycle in Zeitgeists )

I was discussing this the other day with Nico MacDonald (who hosted the recent lecture by Clay Shirky at the Royal Society of Arts in London) expressing my concern that the "pop-academic-media complex" tends to then grab these things and then charge down a typically (simplified) channel, often not taking into account difficult nuances (or those that are unsupporting of the zeitgeist), and you then wind up with the "Flat Earth News" effect (to bring another recent zeitgeist to my aid), ie a whole lot of self interested parties start bailing onto the zeitgeist and transforming it into a fully blown Bandwagon, and it becomes impossible to have a reasoned discussion about it if you in any way oppose the idea.

(Read the Flat Earth News section on how the Millenium bug was pushed, or the note that in the UK there are about as many PR people as there are journalists providing news, and you start to get the picture)

That this may be happening here, now occurred to met when I say this essay today by Paul Graham, titled “you weren’t meant to have a Boss”

Now don’t get me wrong – I thought Paul’s book “Hackers & Painters” was excellent, and I’m a great fan of his essays, but he has an agenda in this game – he runs an incubator, so the aim of the writer is not so much to explore the issues of working in small vs large organisations, but to persuade as many technical people as possible to become startups despite the evidence. As he notes towards the end:

In an essay I wrote a couple years ago I advised graduating seniors to work for a couple years for another company before starting their own. I'd modify that now. Work for another company if you want to, but only for a small one, and if you want to start your own startup, go ahead.

The reason I suggested college graduates not start startups immediately was that I felt most would fail. And they will. But ambitious programmers are better off doing their own thing and failing than going to work at a big company. Certainly they'll learn more. They might even be better off financially.

Now we already noted last year that what research has been done suggests they will not be better off financially, there is a considerable Founders Discount in fact – but the problem is that the Zeitgeist is ramping up, so inconvenient data like that will increasingly be buried.

And back to my original conversation with Nico…elsewhere Paul notes that:

What's so unnatural about working for a big company? The root of the problem is that humans weren't meant to work in such large groups….

…Companies know groups that large wouldn't work, so they divide themselves into units small enough to work together. But to coordinate these they have to introduce something new: bosses.

These smaller groups are always arranged in a tree structure. Your boss is the point where your group attaches to the tree. But when you use this trick for dividing a large group into smaller ones, something strange happens that I've never heard anyone mention explicitly. In the group one level up from yours, your boss represents your entire group. A group of 10 managers is not merely a group of 10 people working together in the usual way. It's really a group of groups. Which means for a group of 10 managers to work together as if they were simply a group of 10 individuals, the group working for each manager would have to work as if they were a single person—the workers and manager would each share only one person's worth of freedom between them.

In practice a group of people never manage to act as if they were one person. But in a large organization divided into groups in this way, the pressure is always in that direction. Each group tries its best to work as if it were the small group of individuals that humans were designed to work in. That was the point of creating it. And when you propagate that constraint, the result is that each person gets freedom of action in inverse proportion to the size of the entire tree.

There are a whole bunch of things that are being mixed together here, and a promised land is being painted largely by ignoring uncomfortable facts, in true Flat Earth News style. So lets go through the issues raised:

(i) Actually, any evidence from human organisations through the ages is far less kind to Paul’s thesis – humans have worked in large, hierarchical structures for at least 5,000 years. In fact some anthropologists, economists and even mathematicians note that civilisations that failed to be able to aggregate in larger units were pushed out of the more desirable areas by those that did.
(ii) Ditto, even in initially informal human groups hierarchies rapidly emerge – a “boss”, whether de facto or de jure, will arise. In fact studies of most social creatures show fairly rigid hierarchies, pecking orders, etc.
(iii) And in many of these less formal structures, the de facto leaders can be far more capricious, despotic etc than in most countries’ larger organisations, where laws have been developed to reign them in.
(iv) In addition, there has been quite a lot of more recent research into various “modern” organisation structures showing that in actual fact the de-layered, flat, matrixed structure actually has less freedom of action than a more structured approach of yore because its far less clear where boundaries and responsibilities lie – allowing empire builders, politicos and the workshy huge leeway, while causing confusion and demoralisation to the rank and file.
(v) This was put very well in a comment on another blog on the subject - a small startup can build a widget for Facebook, but building the Space Shuttle requires a bit more

I'm not saying that big corporates are not sometimes soul destroying, nor that their structure doesn't make them potentially psychopathic, nor even that working in a startup isn't great fun What I am saying is beware of the opposite pendulum pushers, and be aware that the new new thing won't solve all the problems - and that uncomfortable counterfacts will be buried.

Coasian theory will have its way, the evidence is all around us - companies shedding labour and recruiting contractors, the rise of So/Ho/No, co-working etc are all evidence of this. Large companies will (are) restructuring, but the near anarchic opposite is not therefore the logical solution - as the Starfish guys (implicitly) concede, these sort of structures tend to be best as protest movements, and are usually unable to stay stable in their utopian "all equals" state for long.

And I say this with a slightly heavy heart - I love the idea of small, happy, equal systems pulling down The System. But any quick reading of history tells you that Animal Farm is probably a better model for all Utopii than any other.

I'd also like to bring in something that has worried me for a while - the concept that the DNA of existing businesses is all bad and wrong and crap. If Bizgurus are going to use biological analogies, I'd like to make sure its done correctly and differentiate between the genotype - the blueprint of the organism via its DNA - and the phenotype, which is how it evolves in any specific environment. I think too often the two are confused. As the environment changes, the "DNA" will drive changes in the phenotype, while the DNA stays the same, becaus ethe DNA is from us. As Pogo once said, I have seen the enemy, and he is us".

So net-net the story as it stands now is this:

- Yes, Coasian theory implies that organisations today are probably too large for the emerging transaction costs, and they will start to dis-integrate and re-integrate in new ways. To me this is a more coherent explanation than the argument that the DNA of large companies is inherently wrong, or that the "edge economy" is where its all at. Its more a continuum as transaction costs force structural changes in how and where logical subunits are bonded together across the economy.
- Changes will occur in organisation structure over time, undoubtedly, and it will start at the boundary edges of organisations, especially ones being impacted earlier by the reduced transaction costs (think Media, blogging and UGC for example)
- The optimal structures are still unclear, they won't be what we expect. For example, many people hold up the "Hollywood Model" of ad-hoc teams, while not realising (or conveniently forgetting) that the overall meta-structure within which these teams operate extremely tightly defined by a small number of majors.
- Sadly, Social Networking, like Darwinism before it, will increasingly be bent to all sorts of agendas it was never quite designed to fit (I have noticed the move of the topic from pretty academic tome, eg Barabasi to pop-psychology over the last 5 years, for example)
- Thus, from now on a whole raft of Flat Earthers will be jumping onto the zeitgeist, for a variety of motives, to persuade people that their particular utopia is the right one. They will be beautifully argued for maximum press pickup, but as I've tried to indicate above, the facts that don't fit will be Pan-glossed over - so caveat web worker!

In other words, take a good hard look at where the Proposer of any New Utopia is coming from - if they get to drive the Rolls Royces, then its probably you that's doing the polishing.....

On the Rolls Royce point, I have to mention the irascible Nick Carr's note on Bebo here - kit's too good to ignore :

A little over a week ago, the Birches sold Bebo, the third largest social network, to AOL for $850 million, about $600 million of which will reportedly go into the pockets of their jeans. As for the millions of members who have happily served as sharecroppers on the Birches' plantation, they'll get the satisfaction of knowing that all the labor they donated to their "community" did indeed create something of tangible value.

Now good on the Birch's is my view, but its a good lesson in Rolls Royce spotting!

Update - interesting thought on Zoho blog, ie that transaction costs also define the lifecycle of an organisation.