Software Freedom Day

Software Freedom Day is free and open source's open day, a chance for the general public who might have been curious about open source to come along and learn more. I'll be in Wellington on Saturday, September 20th, for Wellington's event. There'll be: copies of Linux given out and a WellyLUG installfest to provide any help people need installing Linux on their own machines; a SuperHappyDevHouse hack day; and a Bar Camp (which I'm emceeing). It's going to be a heap of fun, and a chance to make a positive difference to software. If you'll be in Wellington on Saturday, swing by for the 12pm kickoff and join the fun. If not, check out the web site to find out what's happening in your town.

Landmark Case Upholds Open Source Licenses

The U.S. Court of Appeal for the Federal Circuit has issued a wondrously clear and unambiguous opinion (pdf) that supports the enforceability of open source and public licenses. It is great news for user and contributor communities, and their lawyers. Nothing that I've seen posted so far actually quotes any of the juicy parts of the opinion, so I've included some of that.

The Back Story:

The software in the case is licensed under Artistic License 1.0, which was written by Larry Wall in the late 80s. The defendants allegedly copied the software into their own products without complying with key Artistic License requirements—they did not include the original authors’ names, copyright notices, references to the COPYING file, information about sources of the original files (e.g., SourceForge), or a description of how the original files had been modified.

The plaintiff asked the court for an injunction prohibiting distribution of the defendants’ products, arguing that by violating these license conditions, the defendants violated the copyright in the software.

In a nutshell (and omitting legal fine points), the decision under appeal denied the injunction, holding that copyright protection is not available for software distributed free-of-charge under an open source license.

This truly shocked the open source, free software, and public license community, and a coalition of groups led by Creative Commons filed a “friend of the court” brief (pdf) in support of the appeal. (In the interests of full disclosure—Allison Randal and I worked on the brief on behalf of The Perl Foundation.)

The Ruling:

The appellate court reversed the lower court’s decision in an opinion that open source lawyers have dreamed about but never thought that we would see.

The court paid tribute to the diversity and importance of the open source, free software, and public license community:

“Public licenses, often referred to as “open source” licenses, are used
by artists, authors, educators, software developers, and scientists who
wish to create collaborative projects and to dedicate certain works to
the public…Open source licensing has become a widely used method
of creative collaboration that serves to advance the arts and sciences in
a manner and at a pace that few could have imagined just a few decades
ago.”

And after noting that “lack of money changing hands” does not equate to lack of economic value, it wholeheartedly endorsed enforcement of the Artistic 1.0 license:

“The clear language of the Artistic License creates conditions to protect
the economic rights at issue in the granting of a public license. These
conditions govern the rights to modify and distribute the computer
programs and files included in the downloadable software package. The
attribution and modification transparency requirements directly serve to
drive traffic to the open source incubation page and to inform downstream
users of the project, which is a significant economic goal of the copyright
holder that the law will enforce.”

Perhaps the happiest aspect of the opinion is the assured and sophisticated discussion of open source processes, projects, and economic value. Although the briefs filed by the plaintiff and by Creative Commons undoubtedly assisted the court’s analysis, it seems safe to conclude that the court was already aware of the significance of open source—yet another sign that the “movement” actually has come of age.

Radar Theme: Open Beyond Source

[This is part of a series of posts that briefly describe the trends that we're currently tracking here at O'Reilly]

The lessons and techniques of open source are applicable beyond source code. Open standards, open hardware, open data, open government are all borrowing from the legal, cultural, and technical toolbox of open source.

Watchlist: Sunlight Foundation, Limor Fried, Change Congress, Wesabe Data Bill of Rights, Creative Commons.

Adhearsion - next killer app for Ruby?

Foo camp attendee Ben Black alerted me to Adhearsion, a framework for developing applications in the VoIP space. Think of it as a Ruby on Rails for telephony. Developed by Jay Philllps who got frustrated by the slow uptake of Asterisk.

Adhearsion is written in Ruby and lets those even without any VoIP experience write applications intuitively and productively or simply download and use a pre-written solution. With the framework extension architecture, VoIP functionality can now be actually traded around - an issue the VoIP industry has always suffered from.

A fresh, standard Adhearsion system out of the box does what many companies spend thousands on. It includes a wide - and growing - set of features that should not have to be rewritten for every business that wants to implement them. And yes, this is open-source.

Considering Microsoft spent around $800M on Tellme, I look forward to see what kind of applications this leads to, and what value they generate. We often forget the enormous market for telephone based services.

Random OSCON Tidbits

Some things I learned about at the Django/Python meetup in downtown Portland during OSCON:

• JS Bridge: a Python to Javascript bridge for all Mozilla applications, still under very active development (i.e., changing daily).
• 960.gs: a grid framework for Javascript (replacing Blueprint CSS) with a naming scheme that makes prototyping designs a lot less painful.
• Dojo has Django Templates: I take my eye off Dojo for a year and it suddenly grows the ability to have full Django templates in the browser. Holy CRAP.

Open Source and Cloud Computing

I've been worried for some years that the open source movement might fall prey to the problem that Kim Stanley Robinson so incisively captured in Green Mars: "History is a wave that moves through time slightly faster than we do." Innovators are left behind, as the world they've changed picks up on their ideas, runs with them, and takes them in unexpected directions.

In essays like The Open Source Paradigm Shift and What is Web 2.0?, I argued that the success of the internet as a non-proprietary platform built largely on commodity open source software could lead to a new kind of proprietary lock-in in the cloud. What good are free and open source licenses, all based on the act of software distribution, when software is no longer distributed but merely performed on the global network stage? How can we preserve freedom to innovate when the competitive advantage of online players comes from massive databases created via user contribution, which literally get better the more people use them, raising seemingly insuperable barriers to new competition?

I was heartened by the program at this year's Open Source Convention. Over the past couple of years, open source programs aimed at the Web 2.0 and cloud computing problem space have been proliferating, and I'm seeing clear signs that the values of open source are being reframed for the network era. Sessions like Beyond REST? Building Data Services with XMPP PubSub, Cloud Computing with BigData, Hypertable: An Open Source, High Performance, Scalable Database, Supporting the Open Web, and Processing Large Data with Hadoop and EC2 were all full. (Due to enforcement of fire regulations at the Portland Convention Center, many of them had people turned away, as SRO was not allowed. Brian Aker's session on Drizzle was so popular that he gave it three times!)

But just "paying attention" to cloud computing isn't the point. The point is to rediscover what makes open source tick, but in the new context. It's important to recognize that open source has several key dimensions that contribute to its success:

1. Licenses that permit and encourage redistribution, modification, and even forking;
2. An architecture that enables programs to be used as components where-ever possible, and extended rather than replaced to provide new functionality;
3. Low barriers for new users to try the software;
4. Low barriers for developers to build new applications and share them with the world.

This is far from a complete list, but it gives food for thought. As outlined above, I don't believe we've figured out what kinds of licenses will allow forking of Web 2.0 and cloud applications, especially because the lock-in provided by many of these applications is given by their data rather than their code. However, there are hopeful signs like Yahoo! Boss that companies are at beginning to understand that in the era of the cloud, open source without open data is only half the application.

But even open data is fundamentally challenged by the idea of utility computing in the cloud. Jesse Vincent, the guy who's brought out some of the best hacker t-shirts ever (as well as RT) put it succinctly: "Web 2.0 is digital sharecropping." (Googling, I discover that Nick Carr seems to have coined this meme back in 2006!) If this is true of many Web 2.0 success stories, it's even more true of cloud computing as infrastructure. I'm ever mindful of Microsoft Windows Live VP Debra Chrapaty's dictum that "In the future, being a developer on someone's platform will mean being hosted on their infrastructure." The New York Times dubbed bandwidth providers OPEC 2.0. How much more will that become true of cloud computing platforms?

That's why I'm interested in peer-to-peer approaches to delivering internet applications. Jesse Vincent's talk, Prophet: Your Path Out of the Cloud describes a system for federated sync; Evan Prodromou's Open Source Microblogging describes identi.ca, a federated open source approach to lifestreaming applications.

We can talk all we like about open data and open services, but frankly, it's important to realize just how much of what is possible is dictated by the architecture of the systems we use. Ask yourself, for example, why the PC wound up with an ecosystem of binary freeware, while Unix wound up with an ecosystem of open source software? It wasn't just ideology; it was that the fragmented hardware architecture of Unix required source so users could compile the applications for their machine. Why did the WWW end up with hundreds of millions of independent information providers while centralized sites like AOL and MSN faltered?

Take note: All of the platform as a service plays, from Amazon's S3 and EC2 and Google's AppEngine to Salesforce's force.com -- not to mention Facebook's social networking platform -- have a lot more in common with AOL than they do with internet services as we've known them over the past decade and a half. Will we have to spend a decade backtracking from centralized approaches? The interoperable internet should be the platform, not any one vendor's private preserve. (Neil McAllister provides a look at just how one-sided most platform as a service contracts are.)

So here's my first piece of advice: if you care about open source for the cloud, build on services that are designed to be federated rather than centralized. Architecture trumps licensing any time.

But peer-to-peer architectures aren't as important as open standards and protocols. If services are required to interoperate, competition is preserved. Despite all Microsoft and Netscape's efforts to "own" the web during the browser wars, they failed because Apache held the line on open standards. This is why the Open Web Foundation, announced last week at OScon, is putting an important stake in the ground. It's not just open source software for the web that we need, but open standards that will ensure that dominant players still have to play nice.

The "internet operating system" that I'm hoping to see evolve over the next few years will require developers to move away from thinking of their applications as endpoints, and more as re-usable components. For example, why does every application have to try to recreate its own social network? Shouldn't social networking be a system service?

This isn't just a "moral" appeal, but strategic advice. The first provider to build a reasonably open, re-usable system service in any particular area is going to get the biggest uptake. Right now, there's a lot of focus on low level platform subsystems like storage and computation, but I continue to believe that many of the key subsystems in this evolving OS will be data subsystems, like identity, location, payment, product catalogs, music, etc. And eventually, these subsystems will need to be reasonably open and interoperable, so that a developer can build a data-intensive application without having to own all the data his application requires. This is what John Musser calls the programmable web.

Note that I said "reasonably open." Google Maps isn't open source by any means, but it was open enough (considerably more so than any preceding web mapping service) and so it became a key component of a whole generation of new applications that no longer needed to do their own mapping. A quick look at programmableweb.com shows google maps with about 90% share of mapping mashups. Google Maps is proprietary, but it is reusable. A key test of whether an API is open is whether it is used to enable services that are not hosted by the API provider, and are distributed across the web. Facebook's APIs enable applications on Facebook; Google Maps is a true programmable web subsystem.

That being said, even though the cloud platforms themselves are mostly proprietary, the software stacks running on them are not. Thorsten von Eicken of Rightscale pointed out in his talk Scale Into the Cloud, that almost all of the software stacks running on cloud computing platforms are open source, for the simple reason that proprietary software licenses have no provisions for cloud deployment. Even though open source licenses don't prevent lock-in by cloud providers, they do at least allow developers to deploy their work on the cloud.

In that context, it's important to recognize that even proprietary cloud computing provides one of the key benefits of open source: low barriers to entry. Derek Gottfried's Processing Large Data with Hadoop and EC2 talk was especially sweet in demonstrating this point. Derek described how, armed with a credit card, a sliver of permission, and his hacking skills, he was able to put the NY Times historical archive online for free access, ramping up from 4 instances to nearly 1,000. Open source is about enabling innovation and re-use, and at their best, Web 2.0 and cloud computing can be bent to serve those same aims.

Yet another benefit of open source - try before you buy viral marketing - is also possible for cloud application vendors. During one venture pitch, I was asking the company how they'd avoid the high sales costs typically associated with enterprise software. Open source has solved this problem by letting companies build a huge pipeline of free users, who they can then upsell with follow-on services. The cloud answer isn't quite as good, but at least there's an answer: some number of application instances are free, and you charge after that. While this business model loses some virality, and transfers some costs from the end user to the application provider, it has a benefit that open source now lacks, of providing a much stronger upgrade path to paid services. Only time will tell whether open source or cloud deployment is a better distribution vector, but it's clear that both are miles ahead of traditional proprietary software in this regard.

In short, we're a long way from having all the answers, but we're getting there. Despite all the possibilities for lock-in that we see with Web 2.0 and cloud computing, I believe that the benefits of openness and interoperability will eventually prevail, and we'll see a system made up of cooperating programs that aren't all owned by the same company, an internet platform, that, like Linux on the commodity PC architecture, is assembled from the work of thousands. Those who are skeptical of the idea of the internet operating system argue that we're missing the kinds of control layers that characterize a true operating system. I like to remind them that much of the software that is today assembled into a Linux system already existed before Linus wrote the kernel. Like LA, 72 suburbs in search of a city, today's web is 72 subsystems in search of an operating system kernel. When we finally get that kernel, it had better be open source.

Marc Fleury and Home Automation

Marc Fleury of JBoss fame blogged about his new project, OpenRemote. OpenRemote aims to build open source middleware, UI, and hardware for home automation while working hard on interoperability with all existing protocols and systems. Also working on the project is Mark Spencer, the creator of Asterisk.

At O'Reilly we're watching the move of computing from desktop computers out into the world, and this encompasses everything from the iPhone-inspired new mobile web to the Nike+ and, yes, home automation. The interest of alphageeks like Mark and Marc (not to be confused with Marky Mark) is a significant sign. Other items of proliferated computing interest seen lately: the projects that everyone recreates when they get into hardware hacking, Zigbee's killer app may well turn out to be smart power meters, detecting ill elders via abnormal power use, and a clever system using hi-res weight as unique primary key. And don't forget IBM "Master Inventor" Andy Stanford-Clark's house, which Twitters its energy use among other things.

OSCON in 37 Minutes

The wonderful Gregg Pollack, of Rails Envy fame, wandered the halls and speaker room at OSCON with his video camera. He asked a pile of speakers to summarize their talks in 30 seconds or less, and has compiled the results into "OSCON in 37 Minutes". It's well worth watching even if you were at the conference—as anyone who's attended knows, it's impossible to see everything unless you have a clone army or a time machine. My favourite so far: R0ml's Exceptional Software Explained (video for the full talk of which is also available).

Perl on App Engine?

I am a Perl hacker. I have written parts of the core, created CPAN modules and written tons of perl code. In fact I am addicted to it ; or rather, CPAN. I have been wanting to play around with Google App Engine, but I haven't had time to get up to speed in Python. Today at OSCON I met up with Brad Fitzpatrick, who told me he had permission from Google to talk about and work on a Perl on App Engine project.

He makes it clear that,

I'm happy to announce that the Google App Engine team has given me permission to talk about a 20% project inside Google to to add Perl support to App Engine. To be clear: I'm not a member of the App Engine team and the App Engine team is not promising to add Perl support. They're just saying that I (along with other Perl hackers here at Google) are now allowed to work on this 20% project of ours out in the open where other Perl hackers can help us out, should you be so inclined.

The plan is to harden Perl (one layer of defense in App Engine's hardened environment); implement Protocol Buffers and stubs of the backend services, so people can write App Engine applications on their local servers.

There is more information at Brad's LiveJournal, as well as the the Perl-AppEngine project. Capturing the creative spirit here at OSCON, Brad and I hacked together a new module that emulates a protected environment, Sys::Protect (generally good idea for any web application).

The Last HOPE

I made the trek to a steamy hot NYC this weekend to attend one day of the three day Last HOPE (Hackers on Planet Earth) conference at the Hotel Pennsylvania. There was too much going to adequately cover it here (or even take it all in), but a few things stood out.

Steve Rambam's eye opening talk on the death of privacy for example. For a solid three hours in front of a standing room only crowd he weaved back and forth between the Orwellian theme of how our privacy is being ripped from us by everyone from Google to Choicepoint and the theme that seemed even creepier to him, self contribution. Over and over he expressed disbelief at how willingly we post our personal details everywhere from Twitter to Facebook while thanking us all the while for making his job as a private investigator that much easier. What the marketers and government don't actively take, we actively give. Naturally I twittered the whole thing.

Cell phone tracking; artificial-intelligence-assisted reality mining; 3000 cameras per square mile in Manhattan; facial, activity, and even gait identification software; government outsourced investigative databases shielded from FOIA requests; UAV-based license plate scanners; beating anonymity by correlating multiple datasets; unanticipated database repurposing; and on and on... Finally I could twitter no more and left the venue hurriedly fashioning a tinfoil hat from a burger wrapper while consigning myself to doubling the dosage on my meds.

I will say this though, there was something deliciously ironic about standing in a room chock full of hackers all listening at rapt attention to a three hour chillingly dystopic harangue on privacy loss while nearly every single one of them was wearing an RFID tag around their necks. Even better, the tag was tracking their every move around the venue and was linked to a comprehensive self-contributed profile.

Moving beyond the privacy nightmare stuff, there was hardware hacking to be found everywhere at Last HOPE. Tables were covered with broken open electronic toys and electronic components and were surrounded by hackers with smoking soldering irons.

Of the completed projects on display, one of my favorites was a something of a hybrid that projected a 3D image onto carefully placed strings.

(continue reading)

RailsConf Europe Early Registration

The schedule for RailsConf Europe just went up last week. It's shaping up to be another great conference. A few sessions and tutorials that particularly catch my eye are David Heinemeier Hansson's keynote on Wednesday morning, "Meta-programming Ruby for Fun & Profit" by Neal Ford, "Offline Rails Applications with Google Gears and Adobe AIR" by Till Vollmer, "From Rails Security to Application Security" by Carsten Bormann, and "I Heart Complexity" by Adam Keys.

The early registration discount for RailsConf Europe ends on July 15th. You can save €150 by registering in the next week.

Nokia to buy and open source Symbian

Nokia have announced their intentions to buy Symbian and open source it. It's being seen widely as a response to Google's Android, also an open source mobile operating system. I think it's easy to confuse "open source operating system" for something that will provide all the benefits of the Linux development model. As always, "open source" covers a wide range of development activities and licenses.

The license chosen for Symbian (the Eclipse Public License) is not the Linux kernel license (the GNU Public License v2.0). I suspect the EPL was chosen precisely for its terms so that handset manufacturers like Nokia are able to have their own proprietary extensions for which they do not have to give away the source. This is similar to Google's license for Android, the Apache license.

Both Google and Nokia are applications companies trying to build a mobile services platform, and they have remarkably similar assets. Google has a cloud computing strength that Nokia doesn't. They both have map information (Google drives the streets, Nokia bought NAVTEQ) and assisted GPS application ability. Nokia has hired some absolute geniuses from the ubiquitous computing world to bring network services into people's lives through the mobile phone, whereas Google's social acquisition, Dodgeball, was a catastrophe. Now they both have a handset platform. The difference is that Google's is built on modern technology and they had a chance to start with a clean slate. Symbian feels very 1990s in comparison.

The real question is what do they both hope to gain by having and open sourcing a handset operating system and its core applications? First, it's defensive (nobody wants someone else to "own the handset" and thus have a competitive advantage). But secondly, it's aggressive. They want handset manufacturers to be able to slap Android/Symbian onto the handsets, no royalties payable, you're welcome, and then ship those handsets to the carriers .... Android's default free web browser, of course, will point to Google. I imagine Nokia's strategy will be similar: they'll open source some compelling standard apps, which are the portals to get more Nokia apps and services onto the handsets.

There's a huge difference between Linux and the handsets, though, and I think it's an important one. Linux's license (the GPL) prevents people who ship Linux from including proprietary extensions. If you ship a modification to Linux, you must release the source. This means there are no privileged applications (the way Microsoft's apps used libraries that third-party apps couldn't), no proprietary competitive advantages in the kernel, and so the rate of improvement of every Linux distribution is maximized.

On Christmas Day 1914, the Germans and British soldiers on the World War I front stopped shooting each other, exchanged presents, and played football together. Essentially, the Linux kernel developer community is the Christmas Truce for the Unix platform developers--a place where they cooperate rather than compete ... because the license dictates that they do it.

Both Google and Nokia, however, have deliberately chosen licenses that don't encourage that kind of cessation to war. Proprietary competitive hardware and software can be put into any Android or Nokia phone at the appropriate level of the stack. I think this will slow down the success of their platforms and means neither will unlock the true potential of an open mobile platform. I believe true demilitarized openness is a necessary, but not sufficient, condition for open mobile platform success.

I'd love to know what you think. Am I off the mark? Have I missed a cunning strategic play? Is this, in fact, open source history being written?

Recent Posts

• Video of Rich Wolski's EUCALYPTUS talk at Velocity | by Jesse Robbins on June 24, 2008
• code_swarm - visualizing the life of open source | by Jesse Robbins on June 18, 2008
• Two new open source projects at Velocity | by Jesse Robbins on June 17, 2008
• CloudCamp gathering after Velocity | by Jesse Robbins on June 13, 2008
• Ignite Boston shows the way to beat commerce interruptus | by Andy Oram on May 30, 2008
• Why search competition isn't the point | by Tim O'Reilly on May 25, 2008
• MicroHoo: corporate penis envy? | by Tim O'Reilly on May 24, 2008
• Yochai Benkler, others at Harvard map current and future Internet | by Andy Oram on May 15, 2008
• Disaster Technology for Myanmar/Burma aid workers | by Jesse Robbins on May 8, 2008
• Maker Faire mimesis and open speculation | by Andy Oram on May 3, 2008
• Mondrian, Just the First Internal Google Tool Be Released Via App Engine? | by Brady Forrest on May 2, 2008
• Missed Twitter Questions from Jonathan Schwartz Interview at Web 2.0 Expo | by Tim O'Reilly on April 26, 2008