A model for Open Source/Closed Source co-operation on a single project: Yes Really

I was recently reading a posting by Matt Assay on the Register where he is encouraging open sourcers to unite in a campaign against the evil Apple empire and it’s machine of destruction the Deathstar AppStore. In it he asked “Where’s the rage, open sourcerers”

I responded to Matt with a comment to his post:

“Maybe the throngs of developers developing open source solutions also include a large number happy to finally have an outlet where lower pricing and greater reach mean they can work on their own terms and fairly earn from their work. I'm not saying there are no problems with the Mac AppStore, but the ecosystem is a huge boost to the small developer team and I'm sure, on the Mac at least, will be eating into the share of time spent doing open source projects. Perhaps that's why a revolutionary movement hasn't spontaneously erupted. It appears most end users are very happy that commercial software is being marketed at much lower prices than before ” *

*Spelling error corrected (oh the joys of web comments).

Now I have a lot of sympathy with Open Source projects, love my Drupal installations and think contributing to open source projects is a really good idea. But I do also think there is a sizeable group of radicals in the movement and though radicals shake up the status quo, challenge incumbent models etc., (which is good), some of these radicals have a tendency to overextend the areas to which Open Source software provides a good solution. There are some I have spoken to, who tend to think, in the longer term, open source provides the only solution and see anyone who disagrees with them as something to be opposed.

I don’t think it has to be that way. I think Open Source and proprietary commercial solutions can quite happily cohabit even on a single project and indeed greatly strengthen one another, if only a particular model or pattern is followed. And I have come up with a model which I think supports that solution. But before presenting the solution, let me outline what I think is a not inconsiderable problem area for Open Source.

The simple fact is open source projects are often weak when it comes to human factors design. At least where such projects aren’t being bankrolled by an organisation or billionaire space tourist. Things have improved of course. The recently released Drupal 7 for instance, is leaps and bounds ahead of the previous incarnation. But still the suspicion remains that Open Source projects will always lag behind commercial projects in this regard and the gap was, of late, greatly widened thanks to the high standards Apple has set for iOS devices.

Unfortunately, all too often, the co-operative model conspires to subvert the visual and human factors design process and anyone who has had to do graphic design and deal with customers inputting an opinion before a final result is ready, will know precisely why this is the case. The psychology of the design process simply doesn’t map well to co-operative working. Open source product designers (if that is the right term) also have to be good and co-operative citizens but the very best human factors design has rarely, if ever, been created as a result of this co-operative model (there is co-operation of course, but also the chief designer in charge). This is a fascinating topic deserving of it’s own post and I’m sure there will be a project out there somewhere that runs counter to the generalisation. However, I have yet to see such a counter example, and in any case, even if one does exist - unless there are many many such examples - the generalisation still holds.

Mark Shuttleworth agrees on this point too. It is why he has started the Ubuntu Unity project to overhaul the UI and interaction design of Ubuntu. It is why Unity is being defined by a tight knit team under his direction. Shuttleworth has high ambitions for Ubuntu and he knows that if it is to be universally adopted it must match or even exceed an Apple like level of UI design excellence and that is his stated aim. But Unity project is running into resistance within the Open Source community and the reason for this is by attempting to apply the kind of top-down design approach Shuttleworth knows a best in class UI requires, many of the worker bees feel the hive is being invaded by something foreign and unwelcome. Whether they will swarm and attack is yet to be seen.

So the jury is out, Unity, it may not be. Richard Hillesley gives this excellent analysis of the issues if you haven't seen it already.

This set me wondering if there is a middle way. And having pondered the problem a while, I have come up with what I believe is a solution. I believe there is a licensing model which will allow all many of the commercial ecosystem advantages to be found in e.g. the Apple Appstore whilst at the same time benefiting from the profoundly powerful strengths of Open Source.

The trick, I believe, is to create a license which recognises a different status for UI code and back-end code. A model where user interface code can be closed, but back-end code open, and the two can co-habit quite happily and harmoniously provided there are clear cut criteria that can be applied to qualify any given piece of code as one or the other. And the thing is (at least on my analysis), this can be done. Here’s how I think it could work. For convenience I’m going to call this the Open Interface license:

The unique feature of an Open Interface license is that any modules providing a published public API (e.g. machine to machine interface) are irreversibly open. On the other hand any modules that don’t publish an API, but that nevertheless themselves use an Open Interface public API, may remain closed and may be wholly commercially owned but must still irrevocably carry the Open Interface license designating them as part of an Open Interface project.

But surely, it is natural to object, large swathes of development can still be kept closed? If you think about it, the answer is no, they can’t be. It is impossible to develop any major league software solution if you don’t publish API’s. One of the things that has become very clear is that any commercial software more meaningful than the most basic and simple app, has to provide an API. Accounting packages, Facebook, Word, Photoshop, you name a project and you will confirm this statement. Under this proposed license, any modules such an API is dependent on and the API itself, must be open and public.

The key here is to understand any closed modules can only be kept closed if they don’t provide a machine-to-machine interface or are a build (or execution) dependency for other modules which provide such. If you follow the logic through, it becomes clear the only area it will be possible to have closed software is where that software supplies a user interface. It will be possible to have closed API’s below that, only they are not public and not either a mandatory or optional requirement of public published API’s. As soon as an API is published or made public, the commercial entity then no longer has exclusive rights over that code. This definition of closed versus open is then not a technical definition. It is a social one - and this is a big strength.

I’ve executed a SWOT analysis on this proposal and IMHO this license model comes out looking pretty good:

Strengths:

Provides for more significant commercial ownership model that e.g. mere MVC re-skinning would allow and so provides much more certainty for commercial enterprises to be able to exploit and benefit from open source as equal members of the community but without subverting or compromising the open source contribution.

All project members know where they stand with regard to the other members, what is acceptable on the project vis-a-vis commercial interest and what is not (this is a must for commercial and non-commercial to co-exist)

Provides a clear logical definition of what needs to be open sourced and when (and importantly, legally definable)

Allows for a company to provide an “Apple like” level of experience on top of open source software in a way not to dissimilar to how OS X has been built on top of GNU and allows that company to charge for it without fear they may be forced to give away their investment and commercial advantage. However they know the rules of the game and that this applies only to modules without public API’s.

It maintains a clear pool of open source software where it matters the most and provides the most benefit; backend API’s and system level functions

Open source can be seen as a social benefit. Most if not all the social benefit can be preserved on this model. Pure open source UI’s can also be delivered for any given project.

Reduced maintenance costs

Arguably greater security (open source projects are good at finding and patching security holes)

Greater commercial traction will mean business is more regularly co-opted into supporting the open source level of development

Weaknesses:

Loss of opportunity top to bottom proprietary stack acting as a barrier to entry for competitors (reduced hight of barrier to entry)

Unfortunately and undoubtedly there is a significant disadvantage with this proposal. The license may well be incompatible with existing open source licenses, so it is unlikely it could be retrofitted to current Open Source projects (this is akin to the “Fragile Base Class” problem built into some object oriented languages!). It may be possible to start some new developments on top of some existing open source license code if there is not dynamic loading and linking - but controversy still dogs the distinction between the definition of these two things.

No opportunity to charge for lock-in or maintain lock-in (of course this weakness is a strength from the client or customer standpoint but I put it here in the interest of being comprehensive)

Patent ownership will be incompatible with the open modules parts of any Open Interface license (similar to the last point, this will be viewed as a strength by many)

Opportunities:

As of now potential first mover advantage for projects/companies adopting this model

Savings on maintenance costs can be directed into perfecting UI and other projects

Greater leverage for smaller businesses to compete with big business

Commercial companies can forge closer ties with a user community on a productive and non-confrontational platform

Greater opportunity to acquire customers fearful of lock-in and who also want to deal with well-founded commercial concerns

Threats:

Reduced barrier to entry may result in more entrants after a project succeeds (however remember this works two-ways and there may be opportunities also)

Since the distinction between public API’s and private API’s is whether they are publicly published or not, companies may try to cynically exploit this distinction by declaring private API’s and giving a nudge and wink to select customers that they may be used. It should be noted however, this would not provide a powerful basis for an enduring business relationship and would remain a marginal case (or edge case) albeit an unwelcome one. There could be no legally binding contracts around such API’s if they are not declared public. It would be stipulated in the Open Interface agreement that any API’s made subject to a contract with outside parties, would be by definition open and would count as published.

A business may try to cynically exploit the social definition of a public API’s by forming special closed or secret business relationships. The license would need to include a comprehensive definition of the boundaries of a single commercial entity for the purpose of what can be counted as a private versus a public API. There may again be some unwelcome edge cases around this definition because it is almost impossible to legislate for all eventualities. However that such edge cases may exist wouldn’t invalidate the model in any significant way.

Paul Lancefield on Twitter