The App Store and the Long Tail Part 2: The Real "DRM" At Stake

Andrew Savikas
August 10, 2009 | Permalink | Comments (5) | Listen

Note there's a lot of images in this post, so if you're reading it via RSS, you may want to click through to the original post if you can't see the images.

A few weeks ago I wrote about how the small number of sales from many different countries were adding up to more than the large number of sales from the US in the App Store for our books. That trend has continued (and accelerated), and right now about 60% of our App sales are coming from outside the US:

When I've talked with other publishers about our success with iPhone Apps, they typically discounted what I said because I was talking about iPhone: The Missing Manual, a title particularly suited to the device. And to a degree, that's a fair argument, and I don't expect very many other books-as-apps to sell as well as that one. But the results for the next batch of 17 titles is instructive. For the two-week period of July 20 to August 2 (the first two calendar weeks the apps were on sale), five of the 17 titles sold more units as iPhone apps than via print (as measured in Bookscan). Here's a comparison across all 17 titles:

That got me wondering why there's not stronger interest from other publishers, especially trade publishers, in iPhone apps (besides concerns about pricing and the approval process). Then as I was looking at rankings for some of the top paid book apps, I spotted a possible answer.

In the App Store, each country has its own top 100 lists (overall and for each category, and for free as well as paid). Something that's #1 here in the US may not even register on the top 100 in another country. Here's the current (as of this writing) worldwide rankings for the "Classics" App, the #1 paid book app right now:

Classics is one of the most popular paid book apps in nearly every country with iPhone service (the list actually goes further down than shown above).

Now here's the current (as of this writing) worldwide rankings for "Twilight" which has been holding steady in the top 25 paid apps here in the US:

Yup, that's it. Just the US. Presumably this is a rights issue -- Hachette either doesn't have the rights to sell this book as an App anywhere else, or they're choosing not to. But taken in light of our own sales of nearly 2/3 outside the US and the data from Classics, that means a publisher who can't (or won't) sell their app outside the States is missing a lot of the market. Here's the current rankings for the "A Twilight Trivia" app, which is ranked above Twilight in the US (and is not affiliated with Hachette or Stephenie Brown):

So there's clear interest in the Twilight content on the iPhone outside the US -- enough interest to keep this app well into the top 100 paid book apps in dozens of countries.

Perhaps the most important "digital rights management" at stake right now is that of the rights to sell digital content globally.

If you're planning to attend the Frankfurt Book Fair, producing and selling digital and mobile content from a global perspective will be a big part of the program at TOC Frankfurt on Oct. 13.

Does Digital Cannibalize Print? Not Yet.

Andrew Savikas
August 7, 2009 | Permalink | Comments (10) | Listen

One of the big risk factors publishers think about when it comes to digital books is that they will cannibalize print sales. Factor in the lower prices we're seeing for ebooks, and it's a quite reasonable concern.

Looking at data on sales from our website, at first glance that would appear to be exactly what's happening:

Over the past 18 months, we've gone from print outselling digital by more than 2:1 to just the opposite.

But that's not the full story. If there really was cannibalization happening, you'd expect to see our print sales underperforming the overall computer book market, but that's not what's happening. Here's a comparison of how our sales (as measured by Bookscan) stack up against the broader computer book market. The data here is normalized (the first period in the graph is set to 100, and subsequent results are calculate relative to that period):

Roger Magoulas, who heads up our Research Team (which is doing some way cool stuff with App Store data) put it this way in a recent backchannel email covering this as part of a larger analysis:

By looking at the data and these charts we infer that while O'Reilly physical book sales are down compared to last year, this seems more the result of the drop in demand for computer books since the financial meltdown than the impact of ebook sales. Since O'Reilly is a relatively prolific publisher of econtent we would expect that ebooks would affect O'Reilly's physical book sales more than other publishers and we don't see that evidence in these results. Even if ebooks are taking a bite out of O'Reilly physical book sales, we see no negative effect on O'Reilly's slightly increasing share in the physical book market nor on how O'Reilly's sales correlate with the overall market for physical computer books.

So, for now, if what we infer is correct, you can put away your exorcism crosses, ebooks seem more a legitimate expanded market opportunity than a projectile vomiting Linda Blair wannabe.

Two Cool New Bookworm Features

Andrew Savikas
July 30, 2009 | Permalink | Comments (0) | Listen

There's no question that there's plenty of room for improvement among EPUB readers. From simple things like poorly handling multiple author names to more complicated issues like CSS support, readers (the people, not the software) deserve better. That's one reason we've been sponsoring the open-source Bookworm reader, which is among the best ebook readers around (and looks great on mobile browsers, including Kindle's).

Liza Daly (@liza) has a post over on the O'Reilly Labs blog covering two very cool new Bookworm features:

• Feedbooks integration: one-click import from the most popular Feedbook titles, localized to the language set in your Bookworm preferences
• Add to Bookworm: any website can use the same technique to create an "add to Bookworm" button

Bookworm is currently available in seven languages: English, German, Danish, Finnish, Italian, Spanish, and Japanese (if you're interested in helping to translate Bookworm into more lnaguages, let us know). Report bugs and request features over at the public issue tracker. (And if you want to contribute to the project, visit the project page over on Google Code.)

Anderson: "It's All About Attention"

Andrew Savikas
July 29, 2009 | Permalink | Comments (0) | Listen

Over on Spiegel Online, Chris Anderson does a great job responding to nearly all of the standard old-media responses to new media. Unsurprisingly (I'm sure Wired would have done the same) they pulled one line from a lengthy response to create the provocative title "Maybe Media Will Be a Hobby Rather than a Job." The full passage is much more useful and nuanced:

In the past, the media was a full-time job. But maybe the media is going to be a part time job. Maybe media won't be a job at all, but will instead be a hobby. There is no law that says that industries have to remain at any given size. Once there were blacksmiths and there were steel workers, but things change. The question is not should journalists have jobs. The question is can people get the information they want, the way they want it? The marketplace will sort this out. If we continue to add value to the Internet we'll find a way to make money. But not everything we do has to make money.

The complete interview is worth a read.

Would an Apple Tablet be an Ereader? Yes and No.

Andrew Savikas
July 28, 2009 | Permalink | Comments (4) | Listen

Last Friday the latest round of rumors of an Apple Tablet swelled considerably after a piece from Apple Insider asserted the device is now on the 2010 product roadmap:

However, the past six months have reportedly seen the critical pieces fall into place. Jobs, who's been overseeing the project from his home, office and hospital beds, has finally achieved that much-sought aura of satisfaction. He's since cemented the device in the company's 2010 roadmap, where it's being positioned for a first quarter launch, according to people well-respected by AppleInsider for their striking accuracy in Apple's internal affairs.

That means that the device, which is expected to retail for somewhere between the cost of a high-end iPhone and Apple's most affordable Mac notebook, is bound to turn up any time between January and March, should there be no last minute setbacks. Analyst's following the Cupertino-based company may consider factoring first full-quarter sales of the device into their models for calendar Q2.

The news sparked considerable interest among publishers, who apparently see this development as a "Kindle killer" that will upset Amazon's apparent dominance of the ebook ecosystem. It's understandable from the perspective of a publisher, but if this device actually exists, it's doubtful anyone at Apple sees it as an "ereader" any more than it sees the iPhone as "a GPS device." (The speculation stems from a piece in the Financial Times quoting an anonymous "publishing executive" and saying Apple has been talking to publishers.) Apple also talked to major newspapers before the iPhone launched, but they didn't build the iPhone as a mobile newspaper.

Some have been speculating about whether Apple would ink deals with aggregators like OverDrive or Ingram Digital to secure ebook content for a tablet. But that assumes that Apple sees a need to directly deal with ebooks the way they deal with music, and I don't believe that's likely. While it's possible they'd beef up the native PDF capabilities in a larger device, I think it's much more likely they'll establish the market (the App Store) and the platform (some variant on the iPhone SDK), and let developers and content creators take care of the rest, the way they have already on the iPhone with games.

Seeking Alpha has a nice analysis of Tablet Fever, and correctly places any discussion of news or books in the context of the App Store, where it firmly belongs:

Steve Jobs has mentioned that he has never seen anything like success of the App Store in his career. If he is saying that, then I'm saying that this 9.7 inch iTouch that has been designed to optimally utilize the apps will become the flagship Apple product... The order of operations for the iPhone are phone first, iPod second, Apps third, and Internet browser fourth. This new iTouch is principally designed to take advantage of the App Store gaming, books, news, entertainment, social networking, etc...

Wanted: Proposals for TOC 2010

Andrew Savikas
July 23, 2009 | Permalink | Comments (0) | Listen

If you follow us on Twitter, you already know that the Call for Proposals is now open for the 2010 Tools of Change for Publishing Conference (Feb. 22-24, 2010 in New York).

One of our main themes last year was that all publishing is now digital publishing, and that's becoming clearer with each new announcement about a new device, reseller, smartphone, or other new way to reach an audience -- an audience that is now often walking around with a bookstore in their pocket or briefcase. Here's some of the topics we're looking for:

• Reaching mobile readers: when your customers carry bookstores in their pockets
• Pricing and packaging digital books
• Case studies of successful (or unsuccessful!) new publishing and digital initiatives
• Case studies from implementing lessons learned at a previous TOC Conference
• Strategies and tactics for incorporating print-on-demand into a supply chain
• Moving beyond books: selling merchandise, community, experience, and other scarce goods in a world of "free"
• Tools and challenges for an efficient all-digital workflow
• Revising your P&L's for the economics of digital publishing
• Understanding and responding to the changing retail landscape
• Using the web to find and promote the original people behind "user-generated content"--authors
• Best practices for working with Amazon, Google, and other big internet players
• How to capture and analyze web metrics of interest to publishers
• Best new practices and tools for working with and supporting authors during editorial, production and/or marketing phases
• Systems and devices for displaying digital copy (demos welcome)
• Business models for delivering and/or receiving material via new devices
• New copyright clearing, assertion, and determination mechanisms
• XML, EPUB, RDF, and other TLA's (three-letter acronyms) decoded and explained
• Using open-source tools to assemble a digital publishing workflow

If you have an idea for a session, tutorial, or "lightning demo" we want to hear from you.

And if you're making plans for the Frankfurt Book Fair, don't forget to sign up for TOC Frankfurt, a one-day conference looking at these issues from a European perspective.

Long Tail Evidence from The App Store

Andrew Savikas
July 20, 2009 | Permalink | Comments (5) | Listen

Last week we released 16 of our books as iPhone Apps (and on Saturday added The Twitter Book), and there's some interesting Long Tail data coming in. We've seen Long Tail behavior in the data from Safari Books Online and from Google Book Search, though in this case it's about geography: even though regions like Colombia, Belgium, and Greece are individually generating a small number of sales, together they add up to more than the total number sold in the US:

Content is a Service Business

Andrew Savikas
July 13, 2009 | Permalink | Comments (25) | Listen

I've been a fan of Trent Reznor's music since first hearing Pretty Hate Machine in junior high school, but in the past few years I've been increasingly impressed by his attitude and approach to the economics of the modern digital media business. His release of Ghosts I-IV is a case study in how to do exactly what Kevin Kelly outlines in Better than Free : "When copies are free, you need to sell things which cannot be copied." Notice that even though the Free Download option is right there at the top, the $300 "ultra-deluxe" version is sold out (and was sold out within 24 hours of being released).

On his forum a few days ago, Reznor posted advice to aspiring young musicians eager to make it in the music business, and the advice is just as applicable to writers and other artists working in almost any digital medium and attempting to compete with the vast content available on the Web:

[W]hat you NEED to do is this - give your music away as high-quality DRM-free MP3s. Collect people's email info in exchange (which means having the infrastructure to do so) and start building your database of potential customers. Then, offer a variety of premium packages for sale and make them limited editions / scarce goods. Base the price and amount available on what you think you can sell. Make the packages special - make them by hand, sign them, make them unique, make them something YOU would want to have as a fan. Make a premium download available that includes high-resolution versions (for sale at a reasonable price) and include the download as something immediately available with any physical purchase. Sell T-shirts. Sell buttons, posters... whatever. [emphasis added]

This is not just about using free digital content to sell physical goods. It's an acknowledgment that what you're selling as an artist (or an author, or a publisher for that matter) is not content. What you sell is providing something that the customer/reader/fan wants. That may be entertainment, it may be information, it may be a souvenir of an event or of who they were at a particular moment in their life (Kelly describes something similar as his eight "qualities that can't be copied": Immediacy, Personalization, Interpretation, Authenticity, Accessibility, Embodiment, Patronage, and Findability). Note that that list doesn't include "content." The thing that most publishers (and authors) spend most of their time fretting about (making it, selling it, distributing it, "protecting" it) isn't the thing that their customers are actually buying.

Whether they realize it or not, media companies are in the service business, not the content business. Look at iTunes: if people paid for content, then it would follow that better content would cost more money. But every song costs the same. Why would people pay the same price for goods of (often vastly) different quality? Because they're not paying for the goods they're paying Apple for the service of providing a selection of convenient options easy to pay for and easy to download.

This is not new to digital content. Why would the price of admission to see a given year's Razzie Award winner be equivalent to the price of admission to see the year's Best Picture? Because the price of admission is not for the content. It's for the privilege of seeing it early, and doing so on a big screen in a social environment -- movie patrons pay for the service provided by the theater, not for the movies themselves (here's a counterpoint on movie pricing). That's the point that Reznor and Kelly are making: think long and hard about what your customers want, and provide the service of giving that to them.[1]

"But people are still buying content when they buy a book or an album," the argument goes. Yes, they are. The same way that you're buying food when you go to a restaurant. You are purchasing calories that your body will convert to energy. But few restaurants (especially those you visit frequently) have ingredients any different from those you can get yourself at the corner store, for much less money. So it can't be true that your primary goal is to purchase food; you're purchasing a meal, prepared so you don't have to, cleaned up so you don't have to, and done so in a pleasing and convenient atmosphere. You are paying for the preparation of the food and the experience of eating it in the restaurant, not the food itself [2] (beyond the raw cost of the physical ingredients, which in the case of digital content is effectively zero).

This came up during a discussion on Peter Brantley's email list recently, in the context of what someone is paying for when they buy one of our Cookbooks (which contain "Recipes" for how to accomplish specific tasks with a particular computer language or technology, often culled and curated from material and techniques previously published in blog posts, mailing lists, or help forums). I asserted that rather than the content itself, people are paying for the preparation of that content, to the extent that it helps them solve their problems more quickly and conveniently. When you think about what we do as a service business, then it makes perfect sense: readers are paying us for the service of finding a bunch of great and interesting stuff, and putting it together in a convenient package. It's the convenience of not having to find it themself, and the concise package that saves them from having to dig through a bunch of web bookmarks or search results. I didn't buy "Home Buying for Dummies" last year because I wanted a book on home buying; I bought it because I didn't want to screw up something really important (buying a house) and was willing to pay someone to spell out all of the stuff I needed to worry about in one place. People don't buy Jim Cramer's books because they want Jim Cramer's content -- they buy his books because they think it will help them get rich, and they think paying him is a great shortcut alternative to acquiring his knowledge (knowledge, not "content") themselves. These are services, not products.

The recent (and absurd) notion put forward by European publishers to "strengthen copyright protection as a way to lay the groundwork for new ways to generate revenue online" is intimately tied up with this issue of the value of content (and therefore the value of various players in the content value chain, like authors, publishers and the latest bogeyman, aggregators and search engines). Arguing that you need to beef up copyright protection to make sure there are ways to generate revenue online incorrectly assumes that what people are paying for is the copyrighted content itself. People do not care about content, they care about themselves and their problems.

You don't get an "A" for effort just by spending time and money creating content (and you are not entitled to your business model -- you have to earn that money every day by doing something that people find worth paying for -- and they decide it's worth paying for, not you). Content only has value to the extent that someone will pay for it because it accomplishes something they'd rather exchange money for than do themselves -- and when was the last time you said "Gee, I really need some content. I could write some of it for myself to read today, but I'd rather pay someone else to do it." [3] Google and other aggregators haven't stolen any value from the creators of the content they are aggregating -- they have done what intermediaries have always done, which is create new value based on doing for customers what those customers cannot or do not want to do themselves -- the service of sorting through all that content to find the thing that solves their problem. (I use "problem" loosely -- it may be boredom, loneliness, a tax audit, an idea for a first date,...) Again I'll return to Kevin Kelly, who elucidated the role of aggregators in relation to content creators far more eloquently than I ever could:

The giant aggregators such as Amazon and Netflix make their living in part by helping the audience find works they love. They bring out the good news of the "long tail" phenomenon, which we all know, connects niche audiences with niche productions. But sadly, the long tail is only good news for the giant aggregators, and larger mid-level aggregators such as publishers, studios, and labels. The "long tail" is only lukewarm news to creators themselves. But since findability can really only happen at the systems level, creators need aggregators. This is why publishers, studios, and labels (PSL) will never disappear. They are not needed for distribution of the copies (the internet machine does that). Rather the PSL are needed for the distribution of the users' attention back to the works. From an ocean of possibilities the PSL find, nurture and refine the work of creators that they believe fans will connect with. Other intermediates such as critics and reviewers also channel attention. Fans rely on this multi-level apparatus of findability to discover the works of worth out of the zillions produced. There is money to be made (indirectly for the creatives) by finding talent. For many years the paper publication TV Guide made more money than all of the 3 major TV networks it "guided" combined. The magazine guided and pointed viewers to the good stuff on the tube that week. Stuff, it is worth noting, that was free to the viewers. There is little doubt that besides the mega-aggregators, in the world of the free many PDLs will make money selling findability -- in addition to the other generative qualities.

I love his metaphor of the internet machine ("a very large device that copies promiscuously and constantly"), and it's one worth keeping in mind if you think you're in the business of selling "content," because you are probably wrong.

Update: Jim Lichtenberg kindly reminded me he gave a presentation [PPT] on the same topic at the 2008 TOC Conference. Worth a read.

1. Many publishers have actually been doing the same thing for years with hardcover, trade, and mass-market editions of the exact same content at different prices.

2. This is why celebrity chefs aren't particularly worried that doing TV shows and selling cookbooks describing exactly how to make the food they serve in their restaurants will harm business.

3. There are people who do in fact want to pay someone to write content for them as a service. They're called publishers.

What Ebook Resellers Should Learn from Scribd

Andrew Savikas
July 7, 2009 | Permalink | Comments (12) | Listen

Scribd made a splash when they opened up a "Scribd Store" for selling view and download access to documents. Their terms (80% to the document publisher) are quite generous, though one reason publishers keep so much is that most of the merchandising (including pricing) is self service -- Scribd could learn a lot from other media retailers if they're interested in really promoting document sales.

But one area where existing ebook resellers could really learn from Scribd is in terms of data access and reporting. During one particularly frustrating conversation with an ebook reseller just last week, I learned that we'd be lucky to get sales reports nearly 6 weeks after any sales. These are digital sales. On the Web. Paid by credit card. No inventory to track, no shipping, no check or invoice processing.

Compare that to Scribd, where I get an email every time a document is sold telling me how much it sold for, and the total lifetime earnings for that document. I can also view a graph showing document views over time:

And every single day I get a detailed summary of document activity (this is a very small excerpt):

Here's your daily summary of what's happened with your Scribd account since 
you last checked out the site.
 ------------------------
      Someone liked your document entitled "Apache 2 Pocket Reference by O'Reilly Media"!     
 7 minutes ago
 ------------------------
      Someone liked your document entitled "JRuby Cookbook by O'Reilly Media"!        
 7 minutes ago
 ------------------------
      Someone liked your document entitled "Analyzing Business Data with Excel by O'Reilly Media"!    
 31 minutes ago
 ------------------------
Maria added your document "Tomcat: The Definitive Guide by O'Reilly Media" to their list of favorites
        
 39 minutes ago
 ------------------------

There's also quite a variety of reporting formats and interfaces in the wild among ebook resellers, meaning lots of time is spent by either IT staff or accounting staff (or both) parsing and processing each flavor of report. Though there is a Digital Sales Report format, I haven't heard about any reseller actually using it (if you know of one, tell me in the comments).

Success on the Web requires nearly realtime analytics, and that's one area of ebook retailing Scribd seems to really understand.

"Being wrong is a feature, not a bug"

Andrew Savikas
July 2, 2009 | Permalink | Comments (2) | Listen

A thoughtful piece from Michael Nielsen on the disruption of the scientific publishing industry includes a lot that's very relevant to other publishers and media companies. For example:

In conversations with editors I repeatedly encounter the same pattern: "But idea X won't work / shouldn't be allowed / is bad because of Y." Well, okay. So what? If you're right, you'll be intellectually vindicated, and can take a bow. If you're wrong, your company may not exist in ten years. Whether you're right or not is not the point. When new technologies are being developed, the organizations that win are those that aggressively take risks, put visionary technologists in key decision-making positions, attain a deep organizational mastery of the relevant technologies, and, in most cases, make a lot of mistakes. Being wrong is a feature, not a bug, if it helps you evolve a model that works: you start out with an idea that's just plain wrong, but that contains the seed of a better idea.

Around here we like to say "fail forward fast," and it's an acknowledgement that we will learn much more by trying and doing (and probably failing) than by planning. The real challenge with that is to make those experiments as cheap (financially and otherwise) as possible.

TOC Coming to Frankfurt

Andrew Savikas
July 1, 2009 | Permalink | Listen

I've had the opportunity to speak with quite a few of my industry colleagues in Europe during the past year, and it became increasingly obvious there was an opportunity to bring the Tools of Change for Publishing message to a European audience. So we've teamed up with the Frankfurt Book Fair to put on a special one-day TOC Frankfurt on Tuesday October 13, the day before the Book Fair begins.

Many of the topics (and some of the speakers -- including Tim O'Reilly, Cory Doctorow, and Sara Lloyd) will be familiar to TOC New York attendees, but tuned for a European audience. And while the program is still in development, we're also trying to include some fresh voices who can bring a more global perspective -- such as Kotobarabia's Ramy Habeeb and Guardian Media Group's Simon Waldman.

If you have your own ideas for a session, speaker, or topic, you can submit it right here (just a simple Google Form).

TOC blog readers get a discount on registration by using the code TOC09BL when registering.

New on O'Reilly Labs: Open Feedback Publishing System

Andrew Savikas
May 21, 2009 | Permalink | Comments (2) | Listen

O'Reilly engineer Keith Fahlgren has formally launched our new Open Feedback Publishing System over on O'Reilly Labs:

Over the last few years, traditional publishing has been moving closer to the web and learning a lot of lessons from blogs and wikis, in particular. Today we're happy to announce another small step in that direction: our first manuscript (Programming Scala) is now available for public reading and feedback as part of our Open Feedback Publishing System. The idea is simple: improve in-progress books by engaging the community in a collaborative dialog with the authors out in the open. To do this, we followed the model of the Django Book, Real World Haskell, and Mercurial: The Definitive Guide (among others) and built a system to regularly publish the whole manuscript online as HTML with a comment box under every paragraph, sidebar, figure, and table.

You can see the system in action at the site for our upcoming book Programming Scala.

Undocumented Kindle "Clippings" Limit?

Andrew Savikas
May 20, 2009 | Permalink | Comments (2) | Listen

O'Reilly author Shelley Powers is a heavy user of Kindle's "clipping" feature, and has run into an apparently undocumented clipping limit imposed by Amazon:

I tried to find information about the clipping limit in the Kindle TOS or User Guide, but nothing was covered. I also tried to find out if one can "delete" items from the existing clipping file, in order to replace with other clippings at a later time, but once the limit is reached, nothing associated with the book can be added to the clipping file, not even a highlighted sentence.

Shelley also notes that the clipping limit applies to DRM-free books as well, which definitely doesn't make much sense.

Amazon's Physical vs. Digital Dissonance

Andrew Savikas
May 18, 2009 | Permalink | Comments (4) | Listen

In March of 2008, I wrote about the frustrating experience of trying to get this blog added to Kindle. Fourteen months later, apparently that "rather large ingestion queue" is still full, because the blog never showed up, and I never heard another peep about it. (There is now a self-publishing feature for blogs, but as with their self-publishing book feature (known as DTP), the standard terms of service you must accept to participate aren't something many commercial publishers will be willing or eager to swallow.)

As you might expect, Amazon is one of our biggest customers, and our relationship with them is an important one. They give us far more (virtual of course) "shelf space" than most retailers could possibly provide, and their lean ordering systems mean much less exposure to the risk of significant returns. But much of the efficiency and innovation that is the hallmark of their physical-goods business doesn't seem to be translating into their newer digital programs.

Cory Doctorow has a post over on boing-boing venting his own frustrations with trying to get answers from Amazon:

I love Amazon's physical-goods business. I buy everything from them, from my coffee-maker to my DVDs. I love their consumer-friendly policies, and their innovative business practices. I just wish their electronic delivery business was as good as their physical goods side.

(For the record, we're the "major publisher" Cory references -- I passed his questions along to my own contacts on the Kindle team, and despite repeated attempts haven't been able to get a response either.)

I do understand that many of these are new products and systems, and it's inevitable that there will be glitches and problems; it's often important to be willing to be "good enough" in order to move quickly. But some of these things are bordering on the absurd (like the 14-month wait for ingestion of this blog...). For example,  while we were thrilled they worked quickly to help us get The Twitter Book up for sale on Kindle, for more than two weeks (until just last Friday) the product page for the print version not only didn't show the Kindle version as available, it actually included a link saying "Tell the Publisher! I'd like to read this book on Kindle." Sigh.

In the wake of releasing about 200 of our books onto Kindle, more than one customer complained that the Preview wasn't up to par:

@timoreilly I love how the Kindle sample of the Twitter book doesn't even get past the preface for the book. Not much of a sample.

Turns out the default preview percentage is 5% of the book, so we asked if we could dial that up to 20% (in line with the amount included in a preview of one of our books on Google Book Search). The response? Since we're the only publisher that's asked for it, it's not a high priority change they're prepared to make right now. (Note to other publishers: please let Amazon know you'd like the option to increase the preview percentage on your Kindle books.)

Amazon is a business like any other, and they're entitled to prioritize as they see fit. And I hope that all of the new vendors, sites, and services popping up (or ramping up) to sell ebooks create some urgency for Amazon to improve their own programs so they're as efficient in the digital supply chain as they are in the physical one.

Scribd Store Sets New Standard for Ebook Ecommerce (and 650 O'Reilly Ebooks Included)

Andrew Savikas
May 17, 2009 | Permalink | Listen

There are more than 650 (DRM-free of course) O'Reilly ebooks now on sale in the new Scribd store, which officially launches Monday morning. Full details over on O'Reilly Radar:

For a publisher (and I use the term loosely) the terms for the Scribd store are impressive -- publishers set the sale price directly, and keep 80% of the revenue (compare that to Amazon's DTP program, where the standard terms are that Amazon gets to set the actual price, and the publisher only gets 35% of their "suggested" price). There's also an interesting "automated pricing" option in Scribd, which uses an (unspecified) algorithm to set the sale price. But the pieces of the Scribd store I'm most excited about is the real-time reporting (compared with a lag of a month or more with most ebook resellers, including Amazon), the option to easily provide free updates to existing content, and the variety of adjustable display options -- like preview amount, refreshingly optional DRM, and purchase-link images. Administering and understanding your sales in Scribd is downright delightful compared with the same for Kindle.

Ebook Piracy is Up Because Ebook Demand is Up

Andrew Savikas
May 12, 2009 | Permalink | Comments (8) | Listen

My email, twitter, and "real-world" information stream is abuzz today with references to a New York Times story about the increase in piracy of ebooks:

“It’s exponentially up,” said David Young, chief executive of Hachette Book Group, whose Little, Brown division publishes the “Twilight” series by Stephenie Meyer, a favorite among digital pirates. “Our legal department is spending an ever-increasing time policing sites where copyrighted material is being presented.”

John Wiley & Sons, a textbook publisher that also issues the “Dummies” series, employs three full-time staff members to trawl for unauthorized copies. Gary M. Rinck, general counsel, said that in the last month, the company had sent notices on more than 5,000 titles — five times more than a year ago — asking various sites to take down digital versions of Wiley’s books.

The reason there's an "exponential" increase in piracy of ebooks is because there's an exponential increase in demand for ebooks:

That's not a bad thing! It's an indicator of unmet demand (and in particular for non-DRM encrypted content). I know I have no interest in buying an ebook that's locked to a single vendor or device, and I'm sure many of these "pirates" feel the same. This is a good time to revisit Tim O'Reilly's seminal Piracy is Progressive Taxation, which includes the following lessons:

1. Obscurity is a far greater threat to authors and creative artists than piracy.
2. Piracy is progressive taxation.
3. Customers want to do the right thing, if they can.
4. Shoplifting is a bigger threat than piracy.
5. File sharing networks don't threaten book, music, or film publishing. They threaten existing publishers.
6. "Free" is eventually replaced by a higher-quality paid service.
7. "There's more than one way to do it."

I'm not suggesting publishers stop sending those DMCA notices; but 3 full-time staffers? Putting those resources toward building new ways to meet that demand is a much better investment.

Coincidentally, our research report Impact of P2P and Free Distribution on Book Sales is now available.

Authoring Tools from Alpha Geeks

Andrew Savikas
May 11, 2009 | Permalink | Comments (7) | Listen

Cory Doctorow (@doctorow) has posted a nice article covering some of the tools he's built or borrowed to make his writing life more manageable. I'm especially intrigued by the Flashbake project, which augments simple use of version control (something many of our authors have been using for years, and which we use extensively in our production toolchain) to automatically capture contemporaneous data about the writing process:

Now, this may be of use to some notional scholar who wants to study my work in a hundred years, but I'm more interested in the immediate uses I'll be able to put it to — for example, summarizing all the typos I've caught and corrected between printings of my books. Flashbake also means that I'm extremely backed up (Git is designed to replicate its database to other servers, in order to allow multiple programmers to work on the same file). And more importantly, I'm keen to see what insights this brings to light for me about my own process. I know that there are days when the prose really flows, and there are days when I have to squeeze out each word. What I don't know is what external factors may bear on this.

Thinking about content like code opens up a wealth of tools and techniques for managing that content. After all, programmers spend more time than just about anyone doing what can very easily be called "creative writing" with text, so it's no surprise they've built tools to make their lives easier and more productive. We're getting ready to announce a new project over at O'Reilly Labs, one also built on top of version control (Subversion in our case) and another example of using software tools to improve the writing (and in this case reading) experience.

Over 160 O'Reilly Books Now in Kindle Store (without DRM), More on the Way

Andrew Savikas
April 17, 2009 | Permalink | Comments (24) | Listen

I'm happy to announce that more than 160 O'Reilly books are now available on Kindle (both Kindle 1 and Kindle 2), and are being sold without any DRM (Digital Rights Management). Though we do offer more than 400 ebooks direct from our website, the number for sale on Kindle will be limited until Amazon updates Kindle 1 to support table rendering ("maybe this summer" is the most specific they would get). The text-to-speech feature of Kindle 2 does work with these books. A list of currently available titles is below.

There's a lot of overlap between the kind of early-adopter crowd likely to buy a Kindle and the audience for our books. So it's no surprise that we received a lot of requests to add O'Reilly books to the Kindle store, and it's great to finally be able to get those readers the books they want. We expect to add another 100 or so titles in the coming weeks; those have needed a more detailed analysis of the table content to identify good candidates.

There were two main reasons we held our books back from sale on Kindle:

1. Poor rendering of complex content. Kindle 1 was optimized for the simple text of mainstream trade books (think airport-bookstore fiction and non-fiction), and lacked support for properly displaying tables or computer code, two very common elements in O'Reilly books. We knew customers would be disappointed to find much of the content of our books unusable (and likely to complain to us about it, rather than to Amazon). In this case, Amazon actually agreed with us, and after they saw how those tables looked on a Kindle 1, told us they weren't comfortable selling many of our books until they've updated Kindle 1. (More details below the fold).
2. Compulsory DRM. We strongly believe DRM (Digital Rights Management) encryption adds unwelcome cost and complexity to any digital system, frustrates legitimate customers who respect copyright and want to pay for their content, and is demonstrably ineffective at preventing unauthorized copying -- much of it done by people who either (a.) wouldn't otherwise pay, or (b.) resort to piracy when there's no legitimate sales channel. Other publishers are free to make their own decisions on DRM, but Kindle's compulsory DRM was inconsistent with our views on digital distribution.

Although we've been working for some time with Amazon to resolve these issues, as a stop-gap we'd been directing Kindle owners to oreilly.com, where all of our "ebook bundles" include a Kindle-compatible .mobi version that can be uploaded or emailed to your Kindle. While the table and code issues remained, readers at least had the other, richer formats (EPUB and PDF) for reference. We've now updated all of the .mobi files for sale at oreilly.com to display properly on Kindle 2 (basically undoing many of the hacks we'd done to get something passable the first time around). If you own a Kindle and have purchased ebooks from oreilly.com, visit oreilly.com/e from the Kindle browser to download the updated .mobi files directly to your Kindle. While we will also update our ebooks with Amazon as changes are made and errors fixed, they currently have no way of updating that content for customers who already purchased it.

While the rendering in Kindle 2 still leaves a bit to be desired, we felt it was an acceptable baseline, and look forward to continuing to work with them to improve the display of technical content on Kindle. (Ironically, the Kindle 2 web browser displays complex content like tables and code quite well -- check out the Bookworm mobile version if you have a Kindle.)

Our thanks do go to Amazon for working with us on this. They're a favorite target of criticism (often right here, and often for good reason), but this is a good step and they do deserve some kudos. While we'd prefer that Amazon directly supported the open EPUB standard, this is real progress in giving readers easy access to digital books without locking them in to a single vendor.

If you want to tell Amazon to hurry up and update your Kindle 1, or to improve their rendering of technical content to match Sony Reader, Stanza, Bookworm, Calibre, and others, you can drop them a line at kindle-feedback@amazon.com.

Current Available Titles

(As of April 16, 2009)

• 97 Things Every Software Architect Should Know
• Access Data Analysis Cookbook
• ActionScript 3.0 Design Patterns
• Adding Ajax
• Ajax Design Patterns
• Ajax on Java
• Ambient Findability
• Analyzing Business Data with Excel
• AppleScript: The Missing Manual
• ASP.NET 2.0: A Developer's Notebook
• Asterisk: The Future of Telephony
• Beautiful Code
• Beginning Perl for Bioinformatics
• Building a Web 2.0 Portal with ASP.NET 3.5
• Building Scalable Web Sites
• Commercial Photoshop Retouching: In the Studio
• CSS: The Missing Manual
• Database Nation
• Designing Gestural Interfaces
• Designing Interfaces
• Designing Web Interfaces
• Devices of the Soul
• Digital Identity
• Digital Photography Pocket Guide
• DNS and BIND
• Dreamweaver 8: The Missing Manual
• Dreamweaver CS3: The Missing Manual
• Dreamweaver CS4: The Missing Manual
• Dreamweaver MX 2004: The Missing Manual
• eBay: The Missing Manual
• Eclipse
• Enterprise JavaBeans 3.0
• Enterprise Rails
• Enterprise SOA
• Essential PHP Security
• Excel 2003 for Starters: The Missing Manual
• Excel Scientific and Engineering Cookbook
• Facebook: The Missing Manual
• Ferret
• FileMaker Pro 10: The Missing Manual
• FileMaker Pro 8: The Missing Manual
• FileMaker Pro 9: The Missing Manual
• Flash 8: The Missing Manual
• Flash CS3: The Missing Manual
• Flex 3 Cookbook
• FrontPage 2003: The Missing Manual
• Google Apps: The Missing Manual
• grep Pocket Reference
• Hackers & Painters
• Hardcore Java
• Hardening Cisco Routers
• High Performance Linux Clusters with OSCAR, Rocks, OpenMosix, and MPI
• High Performance MySQL
• Home Networking Annoyances
• Home Networking: The Missing Manual
• Integrating Excel and Access
• Intermediate Perl
• iPhone Forensics
• iPod: The Missing Manual, 6th Edition
• iPod: The Missing Manual, 7th Edition
• iWork '05: The Missing Manual
• Java Generics and Collections
• Java Message Service
• Java Servlet & JSP Cookbook
• Java Web Services: Up and Running
• JavaScript Pocket Reference
• JavaScript: The Good Parts
• JavaScript: The Missing Manual
• JBoss: A Developer's Notebook
• JRuby Cookbook
• Just a Geek
• Learning Flex 3
• Learning JavaScript
• Learning Perl
• Learning Perl
• Learning Rails
• Linux Device Drivers
• Linux Kernel in a Nutshell
• Linux System Programming
• Mac OS X Leopard Pocket Guide
• Macintosh Troubleshooting Pocket Guide for Mac OS
• Making Things Happen
• Managing Projects with GNU Make
• Mastering Oracle SQL
• Mastering Perl
• Maven: A Developer's Notebook
• Microsoft Project 2007: The Missing Manual
• MySQL Pocket Reference
• Network Troubleshooting Tools
• Network Warrior
• NUnit Pocket Reference
• Objective-C Pocket Reference
• Office 2008 for Macintosh: The Missing Manual
• Open Sources 2.0
• Oracle Essentials
• Oracle Essentials
• Oracle PL/SQL Best Practices
• Oracle Regular Expressions Pocket Reference
• Oracle RMAN Pocket Reference
• Oracle SQL Tuning Pocket Reference
• Painting the Web
• Photoshop CS4: The Missing Manual
• Photoshop Elements 3: The Missing Manual
• Photoshop Elements 4: The Missing Manual
• Photoshop Elements 6 for Mac: The Missing Manual
• Photoshop Elements 6: The Missing Manual
• Photoshop Elements 7: The Missing Manual
• PHP Pocket Reference
• PowerPoint 2007 for Starters: The Missing Manual
• Practical mod_perl
• Practical RDF
• Process Improvement Essentials
• Producing Open Source Software
• Programming .NET 3.5
• Programming .NET Components
• Programming Flex 2
• Programming Python
• Programming Web Services with SOAP
• Python Cookbook
• QuickBase: The Missing Manual
• QuickBooks 2005: The Missing Manual
• QuickBooks 2009: The Missing Manual
• Quicken 2006 for Starters: The Missing Manual
• Quicken 2009: The Missing Manual
• Rails Cookbook
• Rails: Up and Running
• Real World Haskell
• Real World Web Services
• Ruby on Rails: Up and Running
• sendmail Cookbook
• SharePoint Office Pocket Guide
• SharePoint User's Guide
• SOA in Practice
• Spam Kings
• Spring: A Developer's Notebook
• SQL and Relational Theory
• SQL Cookbook
• SQL Tuning
• Subject To Change: Creating Great Products & Services for an Uncertain World
• The Art of Application Performance Testing
• The Art of Capacity Planning
• The Art of Lean Software Development
• The Canon EOS Digital Rebel XS/1000D Companion
• The Cathedral & the Bazaar
• The Internet: The Missing Manual
• The Myths of Innovation
• The Photoshop CS4 Companion for Photographers
• The Ruby Programming Language
• Time Management for System Administrators
• UML 2.0 in a Nutshell
• Understanding Open Source and Free Software Licensing
• Unit Test Frameworks
• Unix for Oracle DBAs Pocket Reference
• Using Moodle
• Using SANs and NAS
• Visual Basic 2005 Jumpstart
• Visual Basic 2005: A Developer's Notebook
• Visual C# 2005: A Developer's Notebook
• We the Media
• Web 2.0: A Strategy Guide
• Web Security Testing Cookbook
• Wikipedia Reader's Guide: The Missing Manual
• Wikipedia: The Missing Manual
• Windows 2000 Pro: The Missing Manual

The Tables problem

Here's some screenshots showing the table problem:

How Kindle 1 (mis)handles tables:

The same table on Kindle 2:

Google's Distribution Advantage Has Its Limits

Andrew Savikas
April 13, 2009 | Permalink | Comments (1) | Listen

Scott Karp has an insightful (and provocatively titled) piece over on the Publishing 2.0 blog about just how deeply Google has inserted itself in Web distribution of content. While much of the piece is about linking, one paragraph in particular is worth calling out for traditional publishers (emphasis added):

If media companies want to compete with Google, they need to look at the source of its power — judging good content, which enables Google to be the most efficient and effective distributor of content. They also need to look at Google’s fundamental limitation — its judgment is dependent on OTHER people expressing their judgment of content in the form of links. Above all, they need to look at sources of content judgment that Google currently can’t access, because they are not yet expressed as links on the web.

"Content judgment" is a neat way to put it, reinforcing that when there's already more than 1 trillion web pages in Google's index value is shifting away from more content toward better filtering and curating of what's already there. (Or as Clay Shirky says, it's not information overload, it's filter failure.) While many publishers fret about customers no longer paying for content, they may miss the boat by not realizing that customers will pay for packaging and convenience (which often means judgment and filtering). For example, at the same time the market for our printed reference books has declined, our Safari online subscription service has steadily grown at a double-digit pace, in part because those subscribers value the implicit filtering of the library.

Open Publishing Distribution System -- an Open-Standards Catalog Format

Andrew Savikas
April 8, 2009 | Permalink | Comments (2) | Listen

It's no secret we're big fans of the iPhone/iPod reading app Stanza. While the Kindle App has overtaken Stanza for the top-spot among free book apps in iTunes, Stanza offers a much better reading experience than the Kindle App (for example, by supporting standard formatting like tables and whitespace-preservation) (Update: You can use the latest version of mobigen.exe to get better whitespace-preservation (from <pre> and friends) on the Kindle.) And I'm not the only one who feels that way: "Stanza is hands-down the best e-book reader for the iPhone and iPod Touch, and its free. Go. Get it now." (Wired.com).

But more than the quality of the software, the major reason I'm so bullish on Stanza is their willingness to experiment. When our own Keith Fahlgren suggested they use the standard Atom format for their catalog system, they responded:

We wound up taking your advice and implemented support for Atom for Stanza's catalog format. Thanks for the suggestion! Using the Atom standard is much better than using our own custom format (although we may need to eventually extend the custom format with our own tags).

And when we proposed using Stanza to create a standalone book app (for iPhone: The Missing Manual), they were eager to dive in head first, and we both learned a lot in the process.

That Atom-based Online Catalog feature turned out to be an interesting prototype for a distributed digital discovery and ecommerce system, and it's awesome to see them willing to embrace the potential for such a system well beyond the boundaries their own product, and to join with Peter Brantley and the Internet Archive in laying the groundwork for what's being called the Open Publication Distribution System:

Users of compatible Reading Systems, in addition to being able to access content they have previously acquired or acquire via other means, are also able to access a catalog (list of online sources of content). Typically, the catalog offers a number of free titles, which may be hosted by the Reading System vendor and/or other sites, as well as the opportunity to purchase or borrow paid content from stores and libraries. Additional stores and libraries may be added by the user to their personal catalog. The mechanism through which compatible Reading Systems access the distributed catalog has three components: eBook content, XML catalog metadata, and an HTTP transport for the catalog. The remainder of this document will discuss each of those components in turn.

One of the reasons we've thrown our support behind the Bookworm online ebook reading system as part of O'Reilly Labs is to help support the development and testing of new standards like this one, and we're excited to contribute to this new initiative. It's also great to see Adobe support this as well, and is a nice follow on to our work with them on EPUB output for the open-source DocBook XSL stylesheets.