Nominations Open For O'Reilly Open Source Awards 2010

Recognizing community contribution and leadership

The O'Reilly Open Source Awards will be hosted this July at OSCON 2010 in Portland, OR. The awards recognize individual contributors who have demonstrated exceptional leadership, creativity, and collaboration in the development of Open Source Software. Past recipients for 2005-2009 include Brian Aker, Angela Byron, Karl Fogel, Pamela Jones, Bruce Momjian, Chris Messina, David Recordon, and Andrew Tridgell.

The nomination process is open to the entire open source community, closing May 15, 2010. Send your nominations to osawards@oreilly.com.

Nominations should include the name of the recipient, any associated project or organization, and a description of why you are nominating the individual. O'Reilly employees cannot be nominated.

Four short links: 16 April 2010

Deep Web Projects, Industrial Design, EEG Hacking, On Writing

1. Buckets and Vessels (Aaron Straup Cope) -- amazing collection of projects and the cultural shifts they illustrate. Michal Migurski's Walking Papers, software designed to round-trip paper and digital edits to Open Street Map, has recently been used by professors at the University of California’s Berkeley’s School of Information to enable “a sort of psychogeographical dispute resolution between high school students in the town of Richmond marking up maps of their school and neighbourhood with tags like “stoners”, “asian gangsters” or “make-out spot” (https://groups.ischool.berkeley.edu/papermaps/kennedy.html). By allowing participants to manipulate the perception of their environment they are given a sort of bias knob to adjust the psychics and gravity of one space over another and to create a truly personal map of the world. (via auchmill on Twitter)
2. Jonathan Ive on Industrial Design -- fascinating to hear him talk about how he approaches his products; the interplay between materials, manufacturing methods, and function.
3. Hacking Toy EEGs (MindHacks) -- who doesn't want to do this, just based on the title alone?
4. Mamet's Memo to the Writers -- forceful, clear, and commanding. A tremendous insight, in a short period of time, into what good writing is. No idea why it's in all caps. SOMEONE HAS TO MAKE THE SCENE DRAMATIC. IT IS NOT THE ACTORS JOB (THE ACTORS JOB IS TO BE TRUTHFUL). IT IS NOT THE DIRECTORS JOB. HIS OR HER JOB IS TO FILM IT STRAIGHTFORWARDLY AND REMIND THE ACTORS TO TALK FAST. IT IS YOUR JOB. (via Dan Meyer)

Ebook annotations, links and notes: Must-haves or distractions?

O'Reilly editors discuss ebook functionality and connected reading experiences

Liza Daly's recent piece in the New York Times inspired a great back-channel discussion among O'Reilly's editors. The subject: pros and cons of ebook links, annotations, and notes. There was a lot of interesting back-and-forth, so I asked participants if we could publicly share a handful of excerpts.

Mike Loukides on the reading path:

... inasmuch as I have lots of questions when I'm reading, I don't think I'd like to have the tools to answer them right at my fingertips. It's too easy, at least for me, to move from Little Dorrit to the entry on the Marshalsea in Wikipedia to a history of debtor's prisons, and sooner or later: what was I reading?

I suppose it depends on the implementation. The "Annotated" series from the 70s was, I think, just annoying. Better to just read the book and go back later for the commentary, rather than shoving it all in the reader's face.

There's an excellent book titled "What Jane Austen Ate and Dickens Knew" that goes into all the nitty-gritty background: how much was rent, how much did bread cost; if someone has an income of 500 pounds, is that a lot or a little? But it's a good thing that this information is packaged up in a separate book, not embedded into my copies of Dickens' books. But if someone could figure out the right way to build this kind of reading experience in a way that wasn't intrusive, that would be really good.

Adam Witwer on annotations as an option:

As a formerly serious student of literature (I got better!), I couldn't agree with Mike's sentiments more. The more difficult and rewarding stuff that I've read required all of my focus and attention. The only secondary aid I wanted was a dictionary, which is why the built-in simplicity of the iPad dictionary is such a beautiful thing.

Still, there are some texts for which the annotations are an indispensable part of the experience. I would have found "Ulysses" to be nearly impenetrable in places if I didn't have the annotations handy. To have those annotations somehow built in to the ebook so that I could easily flick back and forth between text and annotation sounds very appealing to the grad student in me.

(continue reading)

Four short links: 15 April 2010

Obscurely Secure Data, Bio Data Torrents, Open Knowledge Conference, Library of Twitter

1. Is Making Public Data Available a Threatening Act? (Pete Warden) -- Imagine a thought experiment where I downloaded the income, charitable donations, pets and military service information for all 89,000 Boulder residents listed in InfoUSA's marketing database, and put that information up in a public web page. That's obviously pretty freaky, but absolutely anyone with $7,000 to spare can grab exactly the same information! That intuitive reaction is very hard to model. Is it because at the moment someone has to make more of an effort to get that information? Do we actually prefer that our information is for sale, rather than free? Or are we just comfortable with a 'privacy through obscurity' regime?
2. BioTorrents: A File Sharing Service for Scientific Data -- described in a PLoSone article. BitTorrent for bio datasets. (via Fabiana Kubke)
3. The Open Knowledge Conference -- Saturday 24th April 2010 in London. Check out the programme, killer topics and people.
4. Library of Congress to Archive All Tweets -- Twitter is handing the archive of all public tweets to the Library of Congress, with a search interface. I like this new slant on national libraries' roles as repositories of nationally and historically important digital text.

Twitter By The Numbers

I collected some interesting stats from today's presentations at Chirp. Over a thousand people attended the conference and the numbers below attest to how vibrant the Twitter platform is. Today's announced API enhancements (e.g., user streams, annotations) will make the Twitter ecosystem even more interesting:

1. # of registered users: 105,779,710 (1,500% growth over the last three years.)

2. # of new sign-ups per day: ~ 300,000 (More recently, 60% of new accounts were from outside the U.S.)

3. # of new tweets per day: 55 million

4. # of unique daily visitors to the site twitter.com: ~ 180 million. (That's actually dwarfed by the traffic that flows through twitter's API -- 75% of traffic is through the API.)

5. # of API requests per day: 3 billion

6. # of registered apps: 100,000 (from 50,000 in Dec/2009)

7. # of search queries per day: 600 milion

8. Twitter's instance, of their recently open-sourced graph database (FlockDB), has 300 13 billion edges and handles 100,000 reads per second.

9. # of servers: "... in the hundreds"

10. Blackberry's just released twitter app accounted for 7% of new sign-ups over the last few days

11. A NY Times story gets tweeted every 4 seconds.

Four short links: 14 April 2010

Social Think, Elegant Design, Online Safety, and Music Royalties

1. Designing for Social Interaction -- useful and thoughtful advice for designers of social applications. Some people believe that this is changing, that the web is making us closer to more people. On the contrary, research studies have shown that the vast majority of usage on social networks is between strong ties. As we saw earlier, on Facebook it’s with 4 to 6 people, with phone calls its with 4 people, and with Skype it’s 2 people. When people play online computer games with others, they are mostly interacting and playing with people they know, often with people who live less than a few miles away. This pattern of technology being used for strong tie communication is not new. When the telephone was invented, it did more to expand and strengthen strong ties than to weaken them. A study in the 1970s showed that the majority of phone calls were to people who live within five miles of the caller’s home. (via bokardo on Twitter)
2. EveryTimeZone -- beautiful HTML timezone visualiser.
3. Teaching About Web Includes Troublesome Parts (NYTimes) -- Common Sense’s classes, based on research by Howard Gardner, a Harvard psychology and education professor, are grouped into topics he calls “ethical fault lines”: identity (how do you present yourself online?); privacy (the world can see everything you write); ownership (plagiarism, reproducing creative work); credibility (legitimate sources of information); and community (interacting with others).
4. How Much Do Music Artists Earn Online? -- an astonishingly depressing chart of how uneconomic traditional online sales are for artists.

Web operators are brain surgeons

Our increased reliance on web-based intelligence makes speed and reliability even more important.

As humans rely on the Internet for all aspects of our lives, our ability to think increasingly depends on fast, reliable applications. The web is our collective consciousness, which means web operators become the brain surgeons of our distributed nervous system.

Each technology we embrace makes us more and more reliant on the web. Armed with mobile phones, we forget phone numbers. Given personal email, we ditch our friends' postal addresses. With maps on our hips, we ignore the ones in our glovebox.

For much of the Western world, technology, culture, and society are indistinguishable. We're sneaking up on the hive mind, as the ubiquitous computing envisioned by Mark Weiser over 20 years ago becomes a reality. Today's web tells you what's interesting. It learns from your behavior. It shares, connects, and suggests. It's real-time and contextual. These connected systems augment humanity, and we rely on them more and more while realizing that dependency less and less. Twitter isn't a site; it's a message bus for humans.

The singularity is indeed near, and its grey matter is the web.

Now think what that means for those who make the web run smoothly. Take away our peripheral brains, and we're helpless. We'll suddenly be unable to do things we took for granted, much as a stroke victim loses the ability to speak. Take away our web, and we'll be unable to find our way, or translate text, or tap into the wisdom of crowds, or alert others to an emergency.

(continue reading)

The iPad isn't a computer, it's a distribution channel

"You don't want your phone to be an open platform..." and with that brief statement, Apple justified the closed iPhone and then quickly followed it with the monitored and controlled app store. But Steve, the iPad isn't a phone at all so why not open it up again? If people are concerned about the safety of their apps or need you to protect them from porn, you can do an "app store approved" program or something can't you? And really, do we even need an app store to tell us which apps are good in an era of ubiquitous user feedback and preferential attachment?

The thing is, Jobs' argument was always a bit disingenuous. Closed follows from his brain architecture, not from an argument on behalf of his customers or their network providers. Those are post facto justifications supporting an already-held point of view. And the reason the iPad is going to stay closed isn't because it is good for users, it's because it is good for Apple.

The bottom line is that the iPhone was a relatively open phone and we accepted it, but the iPad is a relatively closed computer, and that's a bummer. Jobs probably believes that he is doing it for the users, finally giving them a post-crank-the-handle-to-start-it experience, but it doesn't take a genius to see how it benefits Apple. Beliefs and self interest usually go hand in hand and here's what I think is really happening...

Microsoft in the 1980s was the perfect business. The kind of business every MBA would like to invent. It had network effects to drive adoption, products with near-zero marginal cost, and a distribution channel that was controlled and constrained enough in the days of the floppy disks in boxes to enforce direct monetization. In short, it had leverage. The kind of leverage that delivers a very steep-sloped relationship between enterprise valuation and market penetration. Or, put the way an economist would put it, the kind of leverage that captures greater than economic rents. We paid more than we had to for Windows and Office during all those years, but at least we can take some comfort in the fact that a big part of it turned out to be an involuntary tithe for Bill's charitable efforts.

The music industry worked the same way. The constrained distribution channel of the vinyl record gave artists and the music companies that distributed their work tremendous leverage, and get them paid for it. Even if a lot of that economic surplus ended up the noses of label executives...

(continue reading)

Government transparency: Using search data to connect with your audience

A couple of weeks ago at Transparency Camp, I gave a talk on using search data to help ensure that the information the organizations in attendance were opening up could be found by the right audiences. It's awesome that organizations like the Sunlight Foundation, Open Congress, and Follow the Money are making details about government actions easily accessible by citizens. And the government itself has made great strides in opening up data with sites such as recovery.gov and data.gov.

But the truth is that most people don't keep a full list of web sites that provide this information handy. Or, actually they do. It's called Google.

When Americans want to know about health care reform, they don't go to opencongress.org and search for "H.R.3200" or H.R.4872". They go to Google and type in "health care reform". One key to making sure that the information you are working so hard to surface makes its way to the citizens who are looking for it? Use free search data to find out the language people are using to refer to that information. At Transparency Camp, I demonstrated a number of these tools, which I'll outline below.

Open Congress and Effective Use of Search Data
But first, does this method of understanding how your audience searches and ensuring your content is visible in search engines really make that much of a difference to the transparency in government and open data initiatives? Just ask OpenCongress.org. (Thanks to David Moore, of the Participatory Politics Foundation and Open Congress for taking the time to talk to me about how Open Congress uses search data to better connect with audiences.)

Open Congress is a non-profit, non-partisan organization that aggregates official government information with conversations about those issues. Their goal is to provide American citizens with visibility into what's happening in Congress. And how to people find them? 70% of their traffic comes from search engines (and 75% of those visitors are from Google). Below is recent data from Google Analytics that shows the distribution of traffic sources over the course of a month.

Open Congress's pages are visible to searchers because they pay attention to what people are searching for. They use free tools from Google, such as Webmaster Tools, Analytics, and Insights to learn about how people are searching and they craft their titles and headlines accordingly. When they had new information about H.R.3590, they checked Google Trends and found that "health care bill" was a trending topic. People were interested in the information they had, but would never find it if it were titled "H.R.3590".

By paying attention to the language of their audience and caring about search visiblity, their overview pages for the three main health care bills have had 1,929,332 views and the pages with the text of those bills have had 1,128,570 views. (You can find more details about the visitor stats for their health care bill pages here.)

Let's see this in action. The site ranks number one on Google for [health care bill text].

(continue reading)

Four short links: 13 April 2010

Find the Pretty, Win the Prize, Manage the Data, and Model the Temple

1. 0to255 -- simple cute colour-generator. (via Hacker News)
2. ProPublica Wins Pulitzer Prize (NYTimes) -- important landmark in the rise of online journalism. The award is a landmark for ProPublica, founded in 2007, and the other digital news outlets that have sprouted around the country. Over the last few years, the Pulitzer Prize board has relaxed the eligibility rules, allowing news sites to submit work published only online; this year there were many such submissions.
3. Big Data Workshop -- unconference at the Computer History Museum in Mountain View. (via jchris on Twitter
4. 3D Machu Picchu, Created With LIDAR -- viewable in Google Earth, took over 1,200 hours of work. (via skry on Twitter)

The missing link in Twitter's ad program

A future version of Promoted Tweets needs to include the user base

The "Twitter Business Model Watch" is officially (and thankfully) over.

Announced today, Twitter's Promoted Tweets advertising program is supposed to help companies offset the limited window of attention that's common to the real-time web. The new ads, which have the look and behavior of normal tweets, will float like corks at the top of Twitter search results. The feature will roll out to user accounts down the line. (Advertising Age has an example of a Promoted Tweet.)

I've run a few non-scientific tests on the shelf-life of a tweet, and most burn bright for three or four minutes and then fizzle into obscurity. It's tough to build a brand campaign around that. So the "cork" innovation is quite clever.

Linking ads to search queries is smart, too. We're all aware it often yields good results. And I'm intrigued by the assortment of views, clicks and retweets that will influence Twitter's new "resonance" metric. Clearly, there's been a lot of deep thought in the halls of Twitter.

But there's something missing here.

If Twitter really wants to emulate Google, as this New York Times article suggests, it needs to empower the little guy. Not the little advertiser. The little user.

(continue reading)

Grumpy old men, the "Inmates" and margins

iPad, iPhone and the future of computing

As the iPad descends upon us, it is fair to ask, "Is this the beginning of the end, or the end of the beginning?" Depending upon whom you ask, the conclusions widely vary.

For example, RealNetworks' Rob Glaser forcefully argues that Apple's vertically integrated model "Must be stopped." He cautions: "If that's the way the industry plays out -- and there are a couple of vertical stovepipes that are closed -- A: we will have a much slower pace of innovation than we've ever had and B: there will be a tremendous loss in terms of value creation versus it being more horizontal."

Meanwhile, science fiction writer, blogger and tech activist, Cory Doctorow, recently made waves when he asserted in Why I won't buy an iPad (and think you shouldn't, either) that, "If you can't open it, you don't own it. Screws not glue." He concluded:

The real issue isn't the capabilities of the piece of plastic you unwrap today, but the technical and social infrastructure that accompanies it. If you want to live in the creative universe where anyone with a cool idea can make it and give it to you to run on your hardware, the iPad isn't for you. If you want to live in the fair world where you get to keep (or give away) the stuff you buy, the iPad isn't for you. If you want to write code for a platform where the only thing that determines whether you're going to succeed with it is whether your audience loves it, the iPad isn't for you.

And don't even get me started on the legions who dismiss Apple's end-to-end approach with an "Apple's Evil" slap, or more stridently, paint the story as "destined" to play out as things did in the PC Wars, with arrogant Apple racing to an early lead, only to get its head handed to it in the end.

I won't spend a lot of time bringing to the fore the masses that see the Apple model in more favorable terms, as the numbers speak for themselves across just about any metric that matters:

• 85 million iPhones/iPod Touches/iPads sold
• 185,000 applications built
• 100,000 developer ecosystem
• 4 billion application downloads
• 15 billion iTunes media sold
• JD Power Award for Customer Satisfaction
• Ungodly operating margins/cash flow

So how to reconcile the animus with the market's clear directional momentum? Read on ...

(continue reading)

Recent Posts

• Citizens as public sensors | by James Turner on April 12, 2010
• Four short links: 12 April 2010 | by Nat Torkington on April 12, 2010
• The Wellington Declaration | by Nat Torkington on April 11, 2010
• We are iPad. Resistance is (not) futile | by Brett McLaughlin on April 9, 2010
• Games & Entertaiment account for half of all iPad apps | by Ben Lorica on April 9, 2010
• Four short links: 9 April 2010 | by Nat Torkington on April 9, 2010
• Brian Aker on post-Oracle MySQL | by James Turner on April 8, 2010
• Who is the iPad for? | by Jason Grigsby on April 8, 2010
• Four short links: 8 April 2010 | by Nat Torkington on April 8, 2010
• Stop fishing and start feasting: How citable public documents will change your life | by Silona Bonewald on April 7, 2010
• The iPad and computing's middle ground | by Marc Hedlund on April 7, 2010
• Four short links: 7 April 2010 | by Nat Torkington on April 7, 2010