Apache considered harmful

"Institutions will try to preserve the problem to which they are the solution." -- Clay Shirky

This is a hard article to write. The way I think about technology, open source, and community has been shaped by Apache and while I appreciate all they have done I no longer believe they embody the values they claim.

Start with the basics. Why does Apache exist?

From apache.org:

The Apache Software Foundation provides organizational, legal, and financial support for a broad range of open source software projects. The Foundation provides an established framework for intellectual property and financial contributions that simultaneously limits contributors potential legal exposure. Through a collaborative and meritocratic development process, Apache projects deliver enterprise-grade, freely available software products that attract large communities of users.

Apache was founded about 12 years ago, a time when companies were still very afraid of open source and many people in the open source community were very afraid of companies. The world hasn't changed that tremendously, big companies still use an open source stamp as a marketing tool, commonly referred to as "open washing", and some in the enterprise are still wary about open source, particularly when it comes to certain kinds of licensing.

But, you would be hard pressed to find a single company that didn't use some amount of open source software nowadays. More importantly, it's practically inconceivable that a technology company would not be using some amount of open source software to build their products. Nobody can afford to license, or develop on their own, all the technology necessary to build a new product, so for some portion of your stack you must turn to open source.

The Apache mission can still be relevant, even if corporations don't have the same biases against open source software, developers still need protections of their own. Legal issues are real and have only gotten worse, especially patent issues. IP protections are more important now than they've ever been for creators of open source software.

But, the interpretation of "Through a collaborative and meritocratic development process [Apache delivers] software products that attract large communities of users", which was created and first realized in a very different world, is what anchors Apache to the past. What exactly is this process and how does it attract large communities?

Ten years ago open source projects faced a long list of barriers to entry. Source hosting was a pain in the ass. Wiki hosting, Mailing List hosting, bug tracking, all of these things we can now take for granted were actually quite hard to set up and maintain as recently as 5 years ago.

There were also conceptual hurdles. If a project became big enough what kept a single company from dominating it? What protected one company from another company taking over a project? For this reason, and others, a series of processes and bylaws were created that governed projects, voting and the election and rights of committers.

Apache became a very political organism and navigating those politics has come to require more and more institutional knowledge over the years. At the same time, its early successes gave new insights in to the future of open source.

A blog of ASF meeting notes and FAQ's maintained by Shane Curcuru is titled "Community over Code". That title is a perfect characterization of the ethos Apache grew into over the years. Open source is about people, people over all else, even code.

The open source section of my brain was seeded and curated by Ted Leung, long time ASF member, and it is this ethos, Community > Code, that I've dedicated a significant portion of my life to. It is this ethos that has led me to the hard conclusion that as the world has changed Apache has become a net negative for its projects.

The GitHub Revolution

For a moment, let's put the git part of GitHub on the back burner and talk about the hub.

On GitHub the language is not code, as it is often characterized, it is contribution. GitHub presents a person to person communication system for contributions. Documentation, issues, and of course code, travel between personal repositories.

The communication medium is the contribution itself. Its value, its merit, its intentions, all laid naked for the world to see. There is no hierarchy or politic embedded in the system. The creator of a project has a clear first mover advantage but the possibility is always there for its position to be supplanted by a fork, creating a social imperative to manage contributions in a satisfactory manor to her community.

GitHub is truly a system of anarchism, in the most classic sense of the term. It is a system of communication and contribution that is without a central organization or institution of governance. Sure, it is hosted, developed, and maintained by someone but they do not enforce any set of governance or process over the users of the system.

It is my belief that we are, right now, in the middle of a very large evolution in the ecology of open source. The language of contribution has infected a new generation of open source contributors. Much of the potential first imagined by open source pioneers is being realized by high school kids on a daily basis who contribute effectively with less effort than has ever been required.

The reason I am so convinced of the importance of this change is so simple it took me nearly a year to identify it. While the ethos of Apache may have been "Community over Code" it required those in the community to understand and internalize that ethos for it to be fully realized. Social problems became political problems because the ethos had to be enforced by the institution.

The new era, the "GitHub Era", requires no such internalization of ethos. People and their contributions are as transparent as we can imagine and the direct connection of these people to each other turn social problems back in to social problems rather than political problems. The barrier to getting a contribution somewhere meaningful has become entirely social, in other words it is now the responsibility of the community, whether that community is 2 or 2000 people.

A system that enforces its principles without intervention is a tremendous achievement and GitHub's adoption trend should not be a surprise to anyone.

Git at Apache

GitHub's decentralized nature is built, in large part, on git. Many of the social principles I described above are higher order manifestations of the design principles of git itself.

While Apache's aversion to git has been known for some time among insiders, it only recently boiled to the surface and became somewhat common public knowledge when the CouchDB project initiated a migration to git from subversion. A lot of the discussion happened on closed email lists, of which I personally had limited access, but I do know that the move was met with fierce opposition from much of the ASF leadership (many of whom are subversion committers).

Prior to this migration I brought up, on more than one occasion, the topic of git with ASF members and was told that the real barrier was just getting someone to maintain the server and that there was no policy barrier that required subversion. Once a project actually attempted to resolve this by maintaing a server and migrating their project to git they were hit with a whole new slew of requirements along with reasons they should be outright denied the right at all.

This is not a move to GitHub, a notion the ASF surely finds ludicrous, this is only about using git rather than subversion to host a projects code on ASF hardware with access governed by its politic. A discussion about why the ASF is still solving a problem projects don't have, source hosting, appears to be beyond the capabilities of the organization.

After a fierce battle CouchDB has been allowed to begin the move to git. The process appears to be going well and is being led by committer Paul Davis.

Enter PhoneGap. The PhoneGap project has been on GitHub for quite a while and already contains an enviable list of contributors. The project has been very successful and the move to Apache is a result of Adobe's recent acquisition of Nitobi, creators of PhoneGap.

By ASF regulations the project must spend time in the "Incubator" even though it has already proven itself as a technology and as a community to the rest of the world. The project requested git as its version control rather than subversion, for obvious reasons. The request was met with some hostility and new pressure has now come down on the CouchDB "experiment".

This is the situation: git must prove itself to Apache. This is akin to Linux being asked to prove itself to Microsoft, an analogy old enough that ASF leadership is sure to understand it.

Apache was not created to host subversion but in those early days its main value proposition for developers, not companies, was removing barriers to entry. Those barriers no longer exist but the ASF is determined to preserve the problem they had been solving rather than turning to their core values and re-structuring the organization to promote their ethos in a changed world.

The problem here is less about git and more about the chasm between Apache and the new culture of open source. There is a growing community of young new open source developers that Apache continues to distance itself from and as the ASF plants itself firmly in this position the growing community drifts farther away.

There are great voices at Apache who understand that the world is changing and are determined to change with it but their pleas for Apache to evolve are drowned in the politics of an aging organization resistant to change.

People have a great capacity for change. Those people can and will continue to lead us as our institutions fail and eventually harm us.


While I was in the process of editing this article my good friend and co-founder Max Ogden told me he had just been asked to take down http://archive.couchdb.org, an IRC log and search utility he wrote for the IRC and Mailing List messages of the CouchDB project. It's unclear which esoteric Apache rule this tool violates, other than being useful and not being hosting at Apache, but he was told it's a "privacy violation" which is odd considering it only logs public IRC and Mailing List messages.

[Update] It has been clarified that the request to take down archive.couchdb.org was made by an ASF committer, possibly with the support of other #couchdb users, but is not a violation of ASF rules and the request was made on their own behalf and not the ASF. The only policy that has been metioned is the freenode guidlines which caution against logging but do not ban it outright.

[Update] Noah Slater has claimed responsibility for the takedown request and has stated his very fair reasoning for doing so in this comment on a github gist https://gist.github.com/1389627#gistcomment-65015.