The Twitter Gold Mine & Beating Google to the Semantic Web

by Nick Bilton

There's always been jabs at Twitter for not having a viable business model and the chatter has increased in the current economic climate. In a recent interview Evan Williams, Twitter CEO, said "We had planned to focus on revenue in 2010 but that's no longer the case, so we changed the plan quite a bit... We've moved revenue higher on our list of priorities...".

I believe Twitter, potentially, has an incredible business model.

In The New York Times R&D; Labs, where I work, we've been talking a lot about 'smart content', both in relation to advertising, search and news delivery. For the past 157 years (that's how old the newspaper is) we've essentially delivered 'dumb content' to people's doorsteps. You and I, irrespective of interests, location etc. have received the same newspaper on our doorsteps every morning. We're beginning to explore ways to make content smarter, to understand what you've read, which device you've read it on and you're micro level interests—making the most important news find you, instead of you having to find it.

This also changes the advertising model where ads become even smarter. Sure, ads are at about a 1st grade reading level now, with adsense and cookies the ad networks have half an idea of what I'm interested in, but they aren't exactly smart about it. Just because a friend sends me an email about a baseball game, doesn't mean I want to see ESPN ads in my Gmail.

So what does this have to do with a Twitter business model? Twitter, potentially, has the ability to deliver unbelievably smart advertising; advertising that I actually want to see, and they have the ability to deliver search results far superior and more accurate to Google, putting Twitter in the running to beat Google in the latent quest to the semantic web. With some really intelligent data mining and cross pollination, they could give me ads that makes sense not for something I looked at 3 weeks ago, or a link my wife clicked on when she borrowed my laptop, but ads that are extremely relevant to 'what I'm doing right now'.

A quick parusel of my Tweets shows that I live in Brooklyn, NY, I work for The New York Times, teach at NYU/ITP, I travel somewhere once a month for work, I love gardening, cappuccinos, my Vespa , U.I./Design and hardware hacking, I'm a political news junkie, I read Gizmodo & NYTimes.com and I was looking for a new car for a while, but now have a MINI and I'm also friends with these people. That's a treasure trove of data about me, and it's semantic on a granular level about only my interests.

If I send a tweet saying "I'm looking for a new car does anyone have any recommendations", I would be more than happy to see 'smart' user generated advertising recommendations based on my past tweets, mine the data of other people living Brooklyn who have tweeted about their car and deliver a tweet/ad based on those result leaving spammers lost in the noise. I'd also expect when I send a tweet saying 'I got a new car and love it!' that those car ads stop appearing and something else, relevant to only me, takes it's place.

And it doesn't have to be advertising delivered on their site alone. One of the great successes of Twitter has been their API's and the wonderful applications and sites that users have built with them. Why not build out an advertising or search API that delivers the latest micro level tags or ad links of users interests. There's a plethora of opportunity with this data, and if it's done right it becomes enticing and engaging, not annoying, irrelevant and outdated.

Catch 22: Too Big To Fail, Too Big To Succeed

by Joshua-Michéle Ross

Hat in hand the U.S. Auto Industry lined up for their slice of government aid and it appears as of this posting that they will get the money they are asking for. These titans spent years hiding behind the “free market” shibboleth when convenient (the market wants gas guzzling SUV’s) and when punished by that same market we hear that they are victims of factors outside their control and that they are “too big to fail.” It has become a hackneyed expression precisely because it summarizes the situation so well; this is the privatization of profit and the socialization of loss.

The very concept of “Too Big To Fail” points to a deeper truth: the U.S.’s auto industry does not operate within the “free market” at all. Far from it. As their moniker suggests, the “Big Three” are an oligopoly with a long record of eschewing innovation (electric cars, hybrids etc.), killing off alternatives like mass transit and bullying public policy (lobbying against CAFÉ standards, environmental and tax policies [Hummer owners get a $34K tax credit!], the threat of relocating factories etc.) all in an effort to conform the not so “free market” to its lumbering non-strategies of pursuing short-term profit.

Now that their short-term thinking has met with long-term reality we are faced with bailing them out. Fair enough. There are millions of jobs connected to the automobile industry. But do we now trust these same institutions to deliver and execute the plan for a sustainable U.S. transportation industry?

If these are the flaws of the industry, consider their current leadership; The CEOs of these failing behemoths flew in on corporate jets, asked for $25 billion dollars, brought literally not one shred of documentation on what they intended to do differently and couldn’t explain how they arrived at the 25 billion dollar figure in the first place. When asked if they would accept a $1 dollar per year salary (Iacoca style) in exchange responses from GM and Ford ranged from non-committal to sarcastic (“I don’t have a position on that today” - Rick Wagoner of GM, “I think I am OK where I am today.” Ford’s Alan Mulally who earns $22m per year).

Oligopolies like The Big Three thrive on standardization, scale and market manipulation - not innovation. It is precisely their structure, size and leadership DNA that I believe precludes them from any chance of successful innovation. So there is the Catch 22. They may be too big to fail - but they are too big, bloated and corrupt to succeed. If we are the taxpayers funding the bailout, what are the alternatives?

Facebook Growth Regions and Gender Split

by Ben Lorica

Since we began tracking Facebook demographics in late May, weekly growth has held steady, usually in the low single-digits on a percentage basis. More importantly, it's fair to say that the company has successfully expanded overseas. With close to 128M users, the share of U.S. users is down to around 30% from 35% in late May:

Over the last three months, Facebook has added members across all regions, with the strongest growth coming from Europe, South America, and the Middle East/North Africa:

In Europe, growth has been especially impressive in Italy and Spain. I'm not sure when the Italian translation of Facebook launched, but soon after, Italians started signing up in droves. The (crowdsourced) Spanish translation was completed within a month and launched in early 2008. I've read reports that users in Spain have used the site to connect with long lost relatives in Latin America. Venezuela, Argentina, and Uruguay were Facebook's fastest-growth countries in South America. In late May, some Radar readers were highlighting Facebook's growing popularity in Venezuela, Argentina, and Chile.

I don't have any particular insight into how Facebook is growing in the Middle East and North Africa, but the company has added lots of users in Tunisia, Morocco, and Turkey. (I encourage Radar readers from the region to share their thoughts in the comments.)

Having grown up in Southeast Asia, I've been detecting more interest in Facebook among friends in the region. But for now Facebook still lags Friendster and Multiply. In fact Facebook has far less users in all of Asia than users from Canada! Similarly, the U.K. has more than twice the number of Facebook users than all of Asia. Facebook has to contend with homegrown social networks and slightly different online habits: Asian internet users spend more time on gaming and instant messaging. But even with their relatively small user base and amidst a competitive environment, Facebook is growing in Asia (they added 1.5M users from the region in the last 12 weeks).

Another interesting tidbit about Facebook's recent growth, is that the fast-growing regions discussed above are adding teens (13-17) and college-age (18-25) users at a faster rate than North America.

With a commanding share of college-age users in its home country, U.S. growth has been strongest among working age users (26-59). I was expecting stronger growth in the teen market (13-17), but teens remain the slowest growing group in the U.S.

The Gender split has persisted: Females now outnumber Males, 51% to 44%. In late May the Female to Male split was 41% to 34%. The share of users who decline to state their gender dropped from 24% in late May to 5% in early December.

That Females so outnumber Males may surprise people. While the Female/Male distribution has persisted over time, there is quite a bit of variation across regions. The Middle East/North Africa and Africa are the only regions where Male Facebook users outnumber Females.

Open Source Mobile Roundup

by Nat Torkington

Tim sent around a link to the VisionMobile report on open source technology in the mobile space, which I really enjoyed. It covers not just the software used at different layers in the stack, it also covers licensing and governance models. Strongly recommended.

Getting OpenID Into the Browser

by David Recordon

Google Chrome did a smart thing: Less. They unified the search box and address bar, since that's what people do anyway. That gives us back precious pixels for the only thing that's as important to an average web user as where they're going: Who they are. Identity belongs in the browser. Don't just believe me, just this week ReadWriteWeb talks about The End of Online Anonymity and TechCrunch on how Facebook Connect is the Biggest Battle Yet For Social Networks: You, Your Identity And Your Data On The Open Web.

As Web 2.0 took root, the ability to login to a site, store preferences and build a profile became ubiquitous. Beyond reading news or blogs, it's fairly rare that you're on a site where you're either not logged in or don't have the ability to login. The downside is that just about every site requires you to create a new account and have cookies to keep you logged in. Thus when your cookie disappears, you have to login again. Maybe your browser's password manager eases this pain, but there are plenty of people that would be in a world of hurt if their browser every forgot all of their passwords (or they use a friend's computer).

If we remove passwords from the equation and instead use OpenID, there's the notion that upon visiting an OpenID enabled site (now numbering more than 25,000 across the web) you'll most likely submit a form telling that site about your OpenID. I might go to MapQuest and login by typing in my OpenID "https://www.davidrecordon.com/" or Ma.gnolia and clicking a "Sign up with a Yahoo! ID" button. These interactions, with various tweaks around them, are very much the status quo today. If OpenID wishes to see true mainstream adoption, this will need to change.

Imagine if your web browser really knew who you were on the web. Just as you login to your computer, what if when you fired up your browser, it said "Hello Dave" and asked you to "unlock it" as well (Chris Messina was quite influential in my thinking about it this way). In doing so you become securely logged into your OpenID provider (or maybe more than one of them) and as you move around the web your browser takes care of automatically logging you into the sites that you want to be, asking you about others, and helping you register with new ones using your OpenID. Argue as much as you want about the details in making this happen, but I think it's hard to disagree that making it easier for people to manage and use their identity (or identities) online is a bad thing.

There are a lot of proposals around how current OpenID interactions will change - a great summit on OpenID usability was held a little over a month ago - and whether it be more one-click buttons, less buttons, bigger logos, or email addresses I think it's also worth looking at what it will take to really get the browser involved. This certainly isn't a new idea, every major browser has the ability to remember passwords and FireFox even has those pesky user profiles so that people could theoretically have different cookies, bookmarks and other settings.

In the internet identity space this isn't a new idea either. Information Cards (more widely known by Microsoft's CardSpace implementation in Windows) have credit card like rich desktop integration built using WS-* and SAML. Dick Hardt's team up in Canada has built Sxipper for FireFox which helps with both OpenID and normal web forms as well. When I was working for VeriSign, we developed the OpenID Seatbelt which is also a FireFox extension designed to make OpenID easier and prevent phishing by detecting OpenID enabled sites and your provider.

Today, MySpace, Flock and Vidoop released a prototype of their implementation toward this vision with OpenID for Flock. All three of these browser plugins help you manage your OpenIDs, detect when you're on an OpenID enabled site, and then make it easier to sign in. To me, what Sxipper aspires to enable feels the most useful for a mainstream user.

OpenID for Flock is an add-on that polishes previous attempts of putting OpenID into a browser. While the user experience and graphics are quite a bit better than what I helped build at VeriSign, it's lacking the features that help prevent phishing (making sure you're actually logging into your OpenID provider versus a phishing site that looks like it) which is a bit surprising given Vidoop's involvement. That said, OpenID for Flock is Open Source as part of a project dubbed IDentity in the Browser (IDIB) which the same cannot be said for either Sxipper or VeriSign's OpenID Seatbelt. Given that IDIB is Open Source and already written as a Flock add-on, I'd certainly expect to see it ported to FireFox and there be far more community support of it compared to the other add-ons.

So where do we go from here? I don't know how to write great browser plugins so just doing it is out. It's great to see Flock's direct involvement in this Open Source effort as it shows browser vendors innovating and experimenting with how their own products must evolve to support identity. Maybe this will cause the other browser vendors to think seriously about what they too could be doing in future versions to help make identity management easer and more secure on the web.

In my mind, Gears can help us get there. While it started as a project by Google to evolve web browsers faster and add needed features like offline support, it's grown beyond that with offline support now coming in HTML 5 and a new Geolocation API. Today Gears runs on half a dozen different browser/platform combinations including FireFox, Internet Explorer, Safari, Chrome and Android. If there was ever a developer platform to build an Open Source cross browser implementation of what OpenID support might look like, Gears seems like the place to do it. Not only does this mean that we'll need to write less code to have it work in multiple browsers, but ideally if it became mature enough maybe the Gears team would choose to ship OpenID support as well? All of a sudden, the community could be down from a handful of browser plugins to one leading Open Source example.

What do you think? Do you agree that identity is becoming as essential to a browser as location? Should we content ourselves for issues like security to be relegated to a few dozen-pixel lock icon, or have Big-Red-Phishing-Warnings set a standard that important issues deserve significant real estate? Really though, should the browser become more actively involved in how you use the web on a daily basis?

iTunes App Store: The First Five Months

by Ben Lorica

Taking a cue from Raven's recent post announcing the 10,000 iPhone app milestone, I decided to update some charts from earlier posts on the U.S. iTunes app store. First, the weekly growth in the number of apps was slower in November: the number of apps grew less than 10% on a weekly basis for all of November. During the last week of November, there were close to 9,800 unique apps, 22% of which were free.

The average price of a Top 100 paid app continued to decline, falling to a little over $2.60 in the last week of November:

Since high-priced top-sellers actually inflate the MEAN price, I created an alternate chart using the median (the decline in the MEDIAN price is even sharper). The corresponding price distribution continued its downward shift as it becomes harder for high-priced apps to crack the Top 100 paid apps list.

Having more than doubled over the last two months, Gaming remains the largest category accounting for a quarter of all apps. The fastest growing categories were Education and Lifestyle. Medical is the newest app category and as of the end of November there were over 80 medical apps, the 10 most popular of which were free. Among Game apps, Racing, Music, and Sports were the fastest growing Game sub categories.

There has been a slight increase in the proportion of Games priced at 99 cents or lower:

Finally, I computed the share of free apps by category and found that some of the smaller categories have a higher share of free apps. Social Networking apps tend to be apps designed to help users access social web sites from their iPhone, while News apps do the same for news/media sites. In both the Social Networking and News categories, the Free outnumber the Paid apps.

I didn't revisit my previous analysis of the top-selling apps, but I suspect not much has changed since my post at the beginning of November. Among the items I do hope to cover in the future is an analysis of app publishers.

Where 2.0 CFP Closes December 2nd

by Brady Forrest

The fifth Where 2.0 Call For Participation will be closing tomorrow (December 2nd). This year we're going to focus on location-aware technologies and their implications. The iPhone and Android have paved the way for a new breed of app and Where 2.0 will be focused on it. We won't be leaving mapping or geodata behind -- instead we'll look at how new mobile platforms are building on top of them.
If you want to join us on stage submit your talk tomorrow.

Where 2.0 will be held in San Jose, CA from May 19-21, 2009. You can watch all of Where 2.0 2008 online -- for free.

Some of the topics on the radar for Where 2.0 are:

Location-Aware: We will be exploring the implications of our new location-enabled lives, particularly around mobile phones and transponders. What feature is worth sharing your location?

Reality Mining: With the increase in location data come more macro views of our lives. If you want to know where to go in San Francisco, for example, City Sense will show you which parts of the city are hopping. What does this type of information mean for consumers and the enterprise?

Augmented Reality: The location-enabled phone will become a viewfinder for our world. Your phone will be able to tell you what you are looking at. It will also let you leave notes for the next person. What are other cool projects in the works?

Immersive and 3D Imagery: There's an imagery battle happening and consumers are winning. Our world is being documented to an unprecedented degree. While two device manufacturers acquired the mapping data companies, the internet giants have invested in cameras, planes, and satellites. Where will this take the location industry?

Mapping Tables: It's difficult to collaborate in person with an online slippy map; a paper spread out on a table or tacked to a wall is still better. Digital mapping tables are attempting to beat back paper once and for all. By providing everyone the same view and editing capabilities plus the ability to turn on and off layers, will they be able to do it?

Government 2.0: Governments are treasure troves of data. Increasingly they are releasing it online for free. ESRI's release of ArcGIS has also aided the battle by providing municipalities with this ability. This data is aiding both the citizen and Government agencies. How is this critical information being put to use?

Crowdsourcing: Pioneered by OSM, the rest of the mapping industry is catching up. Let's examine where they are taking it.

Disease Awareness: Our increasingly connected world allows diseases to spread in record time. These same networks alert us to outbreaks. We're going to examine new geocentric approaches to epidemiology.

Cartography: Each map has a distinctive look and feel. What are the trends in design and user experience?

Workshops
Back by popular demand, Where 2.0 will have a full day of workshops where participants can dig deep into a range of issues and leave the conference armed with new tools and skills. Workshops are one hour and fifteen minutes in length and will be held on Tuesday, May 19. Topics we'd like to explore include, but are not exclusive to:

Geo Support in Web Application Frameworks: As people design their own mapping applications, there has been a need for built-in geo support. We're looking for workshops that teach about Mapstraction, Modest Maps, Open Layers, GeoDjango, GeoRuby, MapCruncher, and other tools.

Mapping APIs: The location space would not have gotten as far as it has today without all of the innovation in the mapping API space. How can you test the limits of these free resources?

GeoTargeting: Knowing users' locations has never been more important. Identifying it accurately can be difficult and expensive. What are the best methods?

Privacy Implications: As you are collecting user data, keeping track of your users, or collecting geodata, are you aware of the relevant laws? What would you teach others?

GeoBrowsers: Google Earth and NASA WorldWind are both amazing geobrowsers. How can you get the most out of them?

Data Management: Geo applications work with massive amounts of data. What are the tools, tips, and tricks that can be used to manage it?

Protocols and Formats: GeoRSS, GML, KML, EXIF, Microformats, Geo OpenSearch. Which formats are on the way in and which ones are on the way out?

These are just some of the technologies and transformations we've noticed and represent just the starting point for the program. While we'd like you to tap into the theme as your inspiration in writing your proposal, feel free to wander. What are you working on that will change the world, or at least the world you're in? What project is bringing you pleasure, or teasing your brain? Surprise and delight us; shake us out of our assumptions. We're angling for shorter talks with longer breaks so you'll have more time for one-on-one interactions.

10,000 iPhone Apps

by Raven Zachary

Two services that track the iPhone App Store - AppShopper and 148Apps, announced on Saturday that there have been over 10,000 iPhone applications released on the US App Store. The number of currently available applications is just shy of 10,000 due to discontinued apps and a few that have been pulled by Apple (e.g. trademark disputes, terms of service violations, etc.). AppSherpa believes that it will only be a few more days until there are 10,000 iPhone applications available for sale on the US App Store. Total international App Store numbers are not being tracked by anyone outside of Apple, as far as I can tell.

Quick stats highlights from the first 10,000 iPhone applications:

• Games are the leading category, accounting for one in four of total applications. This reinforces Apple's recent marketing campaign around games.
• $0.99 is the most common price point, although one in four applications are free.
• The most expensive application currently for sale is iRa by Lextech Labs for $899.99. This is video surveillance application that integrates with a number of CCTV systems.
• The entire iPhone App Store catalog could be purchased for just over $30,000, although there's only room to fit 129 of them on your iPhone or iPod touch at any given time (148 apps in total, but that includes the default applications from Apple).

Ben Lorica and Roger Magoulas in the Market Research Group at O'Reilly Media have been doing an excellent job tracking the iPhone applications market. You can see some of this work on Ben's Radar posts. I am looking forward to seeing an update from Roger and Ben now that we're crossed the 10,000 mark.

Data Center Power Efficiency

by Jesse Robbins

James Hamilton is one of the smartest and most accomplished engineers I know. He now leads Microsoft's Data Center Futures Team, and has been pushing the opportunities in data center efficiency and internet scale services both inside & outside Microsoft. His most recent post explores misconceptions about the Cost of Power in Large-Scale Data Centers:

I’m not sure how many times I’ve read or been told that power is the number one cost in a modern mega-data center, but it has been a frequent refrain. And, like many stories that get told and retold, there is an element of truth to the it. Power is absolutely the fastest growing operational costs of a high-scale service. Except for server hardware costs, power and costs functionally related to power usually do dominate.

However, it turns out that power alone itself isn’t anywhere close to the most significant a cost. Let’s look at this more deeply. If you amortize power distribution and cooling systems infrastructure over 15 years and amortize server costs over 3 years, you can get a fair comparative picture of how server costs compare to infrastructure (power distribution and cooling). But how to compare the capital costs of server, and power and cooling infrastructure with that monthly bill for power?

The approach I took is to convert everything into a monthly charge. [...]

Why I Love Twitter

by Tim O'Reilly

If you care what I think, you know that Twitter is just about the best way to learn what I'm paying attention to. I pass along tidbits of O'Reilly news, interesting reading from mailing lists and blogs I follow, and of course, tidbits from the twitterers I'm following. These are all the things I could never find time to put on my blog, but that I spray via email like a firehose at editors, conference planners, and researchers within O'Reilly. A lot of my job is, as we say, "redistributing the future" - following interesting people, and passing on what I learn to others. And twitter is an awesome tool for doing just that.

Like a lot of people, I tried out Twitter early on, but didn't stick to it. Most of the early twitter conversation was personal, and I didn't have time for it. I came back when I noticed that about 5000 people were following my non-existent updates, waiting for me to say something. With that many listeners, I thought I'd better oblige. (There are now close to 16,000.) I soon realized that Twitter has grown up to become a critical business tool, ideal for following the latest news, tracking the ideas and whereabouts of people who will shape the future of technology, and sharing my own thoughts and attention stream.

I thought I should outline here some of the specific things I find so compelling about Twitter, with suggestions about architectural features to be emulated by other internet services.

1. Twitter is simple. Twitter does one small thing, and does it well. Folks like Robert Scoble sing the praises of Friendfeed, which you could think of as twitter++. After all, it's got comments and aggregation of data from multiple services. But despite its powerful premise, Friendfeed hasn't dented Twitter's growth. Personally, I don't have time to wade through the comments; for me, Twitter is about quick hits, not about extended discussion. And while I love the promise of service aggregation, I tend to think that trying to marry it to commenting obscures its potential. Less is more. New services like peoplebrowsr are reframing service aggregation in a richer way, as a way of learning more about the people you follow, browsing the social graph. (Peoplebrowsr is still in alpha, but I think it has real potential as a social graph explorer, rather than as yet another people feed-reader.)

2. Twitter works like people do. If I'm interested in someone, I don't have to ask their permission to follow them. I don't have to ask if they will be my friend: that is something that evolves naturally over time. If you're a public figure like I am, the metaphor of mutual "friending" is truly broken. I get tens of thousands of friend requests from people I don't know. Accepting would make it impossible for me to use a social tool to keep in touch with my real friends. Friend groups don't really help.

   Twitter's brilliant social architecture means that anyone can follow me, and I can follow anyone else (unless they want to keep their updates private.) Gradually, through repeated contact, we become friends. @ replies that can only be seen by people followed by both parties to a conversation create a natural kind of social grouping, as well as social group extensibility, as I gradually get more and more visibility into new people that my friends already know. Meanwhile, truly private direct messages are also supported.

   I don't know who first used the term "ambient intimacy" but it's a great description of what begins to happen on Twitter. I know not just what people are thinking about or reading, but enough about what they are doing that our relationship deepens, just like real-world friendships. People who follow me on Twitter learn that I'm making jam or pies, or gardening or riding my bike or feeding the horses, things that I'd never (or rarely, since I'm doing it here) share on my blog. I know a lot more about many of my professional contacts that makes them more into friends. And in the case of my family, who keep their updates private and visible only to a limited group of real friends, we can keep in touch in small ways that mean a lot. I get special moments of my wife or daughters' day that we might not have shared otherwise. It's truly lovely.

3. Twitter cooperates well with others. Rather than loading itself down with features, it lets others extend its reach. There are dozens of powerful third-party interface programs; there are hundreds of add-on sites and tools. Twitter even lets competitors (like FriendFeed or Facebook) slurp its content into their services. But instead of strengthening them, it seems to strengthen Twitter. It's the new version of embrace and extend: inject and take over. (Scoble recently noticed that 60%+ of his friends' updates on Facebook actually came from twitter. And as John Battelle noted in a recent tweet, "I noticed now that my FBook status is updated with Twitter, I get responses in Fbook, but would like to see them here." It might seem like a strength for Facebook to allow Twitter to update its status feed, but not the other way around, but I think Facebook will one day realize that Twitter has taken them over....)

4. Twitter transcends the web. Like all of the key internet services today, Twitter is equally at home on the mobile phone. Even on the PC, I find myself using a separate client (Twhirl is an Adobe Air program) that provides a rich, alternate interface.

5. Twitter is user-extensible.The @syntax for referring to users, hashtags, and whatever you call the use of $ as a special symbol for reference to financial instruments, were all user-generated innovations that, because of Twitter's simplicity, allowed for third party services to be layered not just on the API, but on the content.

6. Twitter evolves quickly. Perhaps because its features are so minimal, new user behaviors seem to propagate across Twitter really quickly. It's a bit like the reason that fruit flies are used for genetic research: the short lifespan compresses the time for mutations to take hold. Perhaps a better analogy would be the speed of cultural evolution among humans compared to biological evolution. The most fascinating evolution happening on Twitter isn't an evolution of the software, but an evolution in user behavior and in the types of data that are being shared.

   I saw this myself with retweeting, a behavior I picked up not from Twitter itself but from twhirl, the Twitter client I use. Because Twhirl actually has a button for retweeting, it seemed the most natural thing in the world to do it. I became one of the most prolific retweeters, figuring that I have more followers than most of the people I know, and that it would be good form to pass on the best of what they post. But it's fascinating to see the growth of retweeting by others, even those not using twhirl.

In many ways, Twitter is a re-incarnation of the old Unix philosophy of simple, cooperating tools. The essence of Twitter is its constraints, the things it doesn't do, and the way that its core services aren't bound to a particular interface.

It strikes me that many of the programs that become enduring platforms have these same characteristics. Few people use the old TCP/IP-based applications like telnet and ftp any more, but TCP/IP itself is ubiquitous. No one uses the mail program any more, but all of us still use email. No one uses Tim Berners-Lee's original web server and browser any more. Both were superseded by independent programs that used his core innovations: http and html.

What's different, of course, is that Twitter isn't just a protocol. It's also a database. And that's the old secret of Web 2.0, Data is the Intel Inside. That means that they can let go of controlling the interface. The more other people build on Twitter, the better their position becomes.

There's a real lesson to Facebook here about giving other services (like Twitter) access to their social graph. They have the best one going, but because they try to keep users coming back to their interface, and even the applications built on their service have to live in Facebook, they end up as a ghetto rather than a true internet service. It's the data, not the interface! Let other people use your data, build on it, and it will still belong to you. Hold it too tight, and they will compete with it.

Lots more to say, but the beach is calling on this sunny Saturday.

Put change.gov Under Revision Control!

by Tim O'Reilly

Last week, the New York Times wrote about Changes at change.gov:

The policy section of the transition site was removed without notice just days after Change.gov went live shortly after the election. At the time a spokesman for the Obama-Biden transition effort said they were “re-tooling” it.

There was an almost instantaneous outcry from bloggers and other advocates of transparency in government who noticed disappearance. At least one site posted a complete archive of the old Agenda pages. (Increasing transparency, by the way, is a key feature of Mr. Obama’s government reform agenda, according to the site’s “Ethics” page.)

The changes, as it turned out, were mostly to tone down the partisan politics of the policy documents as published during the campaign. But the lesson remains: when public documents can be changed without notice, it's essential for the public to be able to see what changed, and why.

There's a profound and simple tool that the Obama administration can use to improve government transparency. It's something that's enabled worldwide collaboration among software developers, and whose relevance for content development has been definitively demonstrated by wikipedia: Revision control. Not only does revision control allow a community to work independently on a common project, it makes it possible to review the changes.

There's a primitive form of revision control in word processing products like Microsoft Word, but we need more than that, especially for documents that bring together the work of multiple independent authors. For change.gov, the wikipedia model might work: logging of every change, with only authorized participants allowed to make changes, but everyone (the public) able to review and comment on associated discussion pages.

The real holy grail, of course, would be to provide revision control on all government regulations, and eventually, on legislation. This would no doubt be fought tooth and nail by lobbyists who don't want their fingerprints on the final result, but that's precisely why it would be such a breakthrough. And that's also why I suggest that the Obama team start with change.gov: demonstrate that the system works, that it has enormous benefits in transparency, and work from there.

Of course, there are major technical and workflow obstacles. Many of the documents in question are probably worked on independently as Microsoft Office files, with bulk merges that obscure the history. What we really need are distributed revision control tools. There's a lot of good work happening in this area in the software development community; it would be fascinating to see it extended to collaborative document development. (Of course, shared editing a la Google Docs is coming to Microsoft Office as well, so perhaps the point of leverage is for Google to improve the revision control capabilities in Google Docs, starting an arms race with Microsoft. Once the tools are in place, the social pressure to use them has a point of leverage.)

Like so many things that go under the rubric of "change," I'm sure that there would be many complications to this proposal, many problems I haven't thought of. Many current assumptions and processes would need to be challenged, and some of the challenges would take us down dead ends. But that's what change is all about. If it was just like the present, it wouldn't be change.

I'd love your thoughts both on the general proposal and specific ideas for implementation.

P.S. I wrote on this same subject about a year and a half ago, in a post entitled Why Congress Needs a Version Control System. It was Karl Fogel who first put this idea into my head, and that post explains his thinking. There are also some comments there from 2007 that may provide more grist for the discussion.

My Web Doesn't Like Your Enterprise, at Least While it's More Fun

by Jim Stogdill

The other day Jesse posted a call for participation for the next Velocity Web Operations Conference. My background is in the enterprise space, so, despite Velocity's web focus, I wondered if there might not be interest in a bit of enterprise participation. After all, enterprise data centers deal with the same "Fast, Scaleable, Efficient, and Available" imperatives. I figured there might be some room for the two communities to learn from each other. So, I posted to the internal Radar author's list to see what everyone else thought.

Mostly silence. Until Artur replied with this quote from one of his friends employed at a large enterprise: "What took us a weekend to do, has taken 18 months here." That concise statement seems to sum up the view of the enterprise, and I'm not surprised. For nearly six years I've been swimming in the spirit-sapping molasses that is the Department of Defense IT Enterprise so I'm quite familiar with the sentiment. I often express it myself.

We've had some of this conversation before at Radar. In his post on Enterprise Rules, Nat used contrasting frames of reference to describe the web as your loving dear old API-provisioning Dad, while the enterprise is the belt-wielding standing-in-the-front-door-when-you-come-home-after-curfew step father.

While I agree that the enterprise is about control and the web is about emergence (I've made the same argument here at Radar), I don't think this negative characterization of the enterprise is all that useful. It seems to imply that the enterprise's orientation toward control springs fully formed from the minds of an army of petty controlling middle managers. I don't think that's the case.

I suspect it's more likely the result of large scale system dynamics, where the culture of control follows from other constraints. If multiverse advocates are right and there are infinite parallel universes, I bet most of them have IT enterprises just like ours; at least in those shards that have similar corporate IT boundary conditions. Once you have GAAP, Sarbox, domain-specific regulation like HIPAA, quarterly expectations from "The Street," decades of MIS legacy, and the talent acquisition realities that mature companies in mature industries face, the strange attractors in the system will pull most of those shards to roughly the same place. In other words, the IT enterprise is about control because large businesses in mature industries are about control. On the other hand, the web is about emergence because in this time, place, and with this technology discontinuity, emergence is the low energy state.

Also, as Artur acknowledged in a follow up email to the list, no matter what business you're in, it's always more fun to be delivering the product than to be tucked away in a cost center. On the web, bits are the product. In the enterprise bits are squirreled away in a supporting cost center that always needs to be ten percent smaller next year.

(continue reading)

Recent Posts

• Why Does Twitter's Business Model Matter to You? | by Sarah Milstein on November 24, 2008
• “Technology is the 7th Kingdom of Life” - A conversation with Kevin Kelly | by Joshua-Michéle Ross on November 24, 2008
• Get One Give One For Christmas | by Nat Torkington on November 23, 2008
• It's Not Over: We are "the change we need." | by Tim O'Reilly on November 22, 2008
• Asynchronous Multiplayer Mobile Gaming | by Raven Zachary on November 22, 2008
• Huffington, Newsom, and Trippi talk politics in a Web 2.0 world | by Sara Winge on November 21, 2008
• My mind is buzzing right now... | by Tim O'Reilly on November 21, 2008
• iGEM Congratulations to Newcastle | by Nat Torkington on November 21, 2008
• Velocity 2009: Themes, ideas, and call for participation... | by Jesse Robbins on November 20, 2008
• Web Meets World: Privacy and the Future of the Cloud | by Nat Torkington on November 20, 2008
• The Commoditization of Massive Data Analysis | by Joseph Hellerstein on November 19, 2008
• DIY Appliances on the Web? | by Jim Stogdill on November 18, 2008