CARVIEW |
Ben Lorica

Ben Lorica is the Senior Analyst in the Market Research Group at O'Reilly Media, Inc.. He has applied Business Intelligence, Data Mining and Statistical Analysis in a variety of settings including Direct Marketing, Consumer and Market Research, Targeted Advertising, Text Mining, and Financial Engineering. His background includes stints with an investment management company, internet startups, and financial services. At O'Reilly, Ben works on custom research and consulting projects, open source data warehousing and analytics.
An ex-academic, he was an Assistant Professor at U.C. Davis and was the founding Department Chair for Statistics and Mathematics at C.S.U. Monterey Bay.
Fri
Aug 15
2008
The U.S. iTunes App Store
With the iTunes App store now over a month old, I decided to look closely at data from the U.S. store over the last three weeks. While sales numbers are not publicly available, Apple publishes overall as well as category-level rankings. There are currently just over 1,800 (paid and free) applications in the App store, double what it was three weeks ago. Games is the largest category with about 500 applications (roughly 27% of all apps), up 87% from three weeks ago. Puzzles, Arcade, and Board games are the three largest Gaming subcategories:

The fastest-growing category, Education, more than tripled over the last three weeks.
The average price per paid app is around $5.50, with 94% of apps priced at $10 or less. Prices vary considerably by category with expensive apps skewing the average price in a category: a single application priced at $449 drove up the average price of Finance apps to more than $22. Excluding the top and bottom 1% priced apps, the average price of an iPhone application is about $5.20. Similarly, by removing the top and bottom priced app in each category, we get a more reasonable estimate of the average price per app within a category (click here for details).
The Book category is comprised mostly of ebooks and while there are over 150 such "apps", it was the only category not represented in the Top 100 rankings:

In contrast, more than 1 in 10 of all Music apps were among the Top 100 Paid Apps:

Looking beyond the Top 100 paid apps to all paid iPhone applications, the best-performing categories (in terms of popularity) are Music, Weather, Navigation, Lifestyle, and Entertainment (click here for details).
On average, app providers have slightly over one app each, with 25 (out of the close to 1,100) providers accounting for about 21% of all paid apps:

Most providers had 0 or 1 app listed in the Top 100 Paid Applications, with the following exceptions: Hottrix, Pangea Software, Inc., Phase2 Media, telience.com, and Electronic Arts all had 2 apps in the Top 100 list. For now, the cohort of web developers who dominate the Facebook application platform have been unable to make similar inroads in the iPhone platform. Perhaps it's time to brush up on Cocoa and Objective-C?
tags: apple, iphone
| comments: 7
| Sphere It
submit:
Mon
Jul 21
2008
Facebook Growth By Country and the Slowdown in App Usage
With the Facebook Developers conference slated for later this week, I thought it would be a good time to give a brief update of a previous post on Facebook demographics. What follows are recently published number of users by country and region, along with growth rates for select regions and countries. Over the last four weeks, the fastest growing regions were South America, Central America and the Carribean:

While Facebook grew double-digits in Asia it did so from a relatively small base (approx. 3.7 million users), in a region with hundreds of millions of potential users. Of the countries in South and Central America, Chile is worth highlighting (up 67.5% from four weeks ago). As several Radar readers predicted, Facebook has grown steadily in Chile where it now has over 2.2 million users (around 14% of the population). In other parts of the Americas, Hi5 and Orkut remain the largest social networks:

Looking closely at the top 30 countries, a few European countries have grown more than ten percent over the last four weeks (France, Spain, Germany, Italy), with France having the most number of users (approx. 2.5 million). Skyrock remains the largest social network in France. Norway saw a decline but is still home to more than a million Facebook users. We will continue to track how Facebook is doing vis-à-vis other leading regional social web sites and whether their disputes with other companies affect their growth rates.

As far as recent trends in the Facebook app platform (the subject of this week's f8 conference), we have detailed reports (here and here) on the subject. At the last Graphing Social Patterns conference, Roger Magoulas provided highlights of our most recent findings. The number of published apps continues to grow steadily (to over 32K) but total usage remains flat. Besides the fact that the top 10% of apps account for 98% of total usage, aspiring Facebook app developers should know that only about 6% of apps average at least 500 active users per day:

(For specific tips on how to launch and build successful Facebook apps, consult this O'Reilly Radar Report.) Finally, as I noted in a previous post, the most popular applications on the Myspace platform continue to account for slightly less users than their Facebook counterparts.
tags: facebook, facebook reports, myspace, platforms
| comments: 13
| Sphere It
submit:
Tue
Jul 15
2008
Developer Interest in the iPhone, Android, and Symbian
With several hundred applications now available in the iTunes App store, I decided to consider alternate ways of gauging interest in the platform. Using MarkMail, one can quickly scan thousands of mailing lists and restrict the results to those related to software development. Based on the number of posts to (MarkMail) mailing lists, Linux-based alternatives generate considerably more email chatter than the iPhone:

Staying with the previous metric (posts to mailing lists), there does seem to be growing interest in the iPhone among developers. Since the launch of Android (November 2007), the number of iPhone related messages has grown at a faster rate than those for its competitors:

Other online tools suggest growth in the number of job postings that mention the iPhone. But while a majority of the most recent iPhone related job postings were posted by Apple (making the recent growth in job postings less impressive), Android jobs postings came mostly from outside Google.

For now the launch of the iPhone puts the spotlight on Apple's App store and platform. The reality is that the mobile landscape is evolving rapidly and with Android yet to launch, the previous numbers will change dramatically over the next months. We will continue to monitor developer interest in the different mobile platforms using a variety of indicators.
Yet another option lurks, one already familiar to web developers and users. At last weekend's Foo camp, I attended a session on the mobile web and left convinced that with access to the right hooks into mobile devices, web developers can deliver equally cool apps through mobile browsers. Which mobile platform are you most excited about?
tags: android, iphone, mobile
| comments: 22
| Sphere It
submit:
Tue
Jul 8
2008
Please Update Your Browser
A research study released last week measures the proportion of web users running the most updated and secure browsers. With drive-by-downloads increasingly popular with malware distributors, web surfing with an older version of a browser is getting riskier. The study is based on data from Google's search and web application server logs over an 18 month period (Jan-07 to Jun-08), with browser versions lifted from the HTTP USER-AGENT header field found in the server logs.
The researchers assumed that "... most updates and patches for existing Web browser technologies (both the core browsing engine and third-party plug-ins) increasingly incorporate new and vital security fixes": so for the purposes of the study the latest version or update of a browser was considered the "secure" version. The share of users running the latest major release varies over time, with Firefox users much more likely to be using the most secure version:

Overall, 45.2% of Internet users were not using the most secure browsers. The results were on the optimistic side since the researchers were unable check for out-of-date and vulnerable browser plug-ins, nor go back in time and adjust for the many zero-day attacks aimed at browsers.
Firefox's auto-update mechanism resulted in most of its users updating to a new version within three days of a new release. Opera's "manual update & download reminder" approach meant it took about eleven days before most of its users updated to a new release. The researchers found that it took 19 months before 53% of IE users updated to IE7, in contrast, 92% of Firefox users were already using version 2. I agree with their recommendation that the other major browsers follow Mozilla's (auto-update) lead:
While Microsoft’s operating system auto-update functionality encompasses the Internet Explorer update mechanism even if the browser is not in use, the fact that patch updates (for both Internet Explorer 6 and 7) are typically only made available on a monthly basis means that updates are released less frequently (when compared to Firefox), which can result in a lower short term patching effectiveness.Based upon our findings, we strongly recommend that software vendors embrace auto-update mechanisms within their products that are capable of identifying the availability of new patches and installing security updates as quickly and efficiently as possible - ideally enabled by default and causing minimal disruption to the user. We also recommend that these same auto-update mechanisms are capable of alerting the user of any plug-ins currently exposed through the Web browser that have newer and more secure versions available.
They actually go further and envision a "best before" dating system, akin to what the food industry adopted years ago to help consumers evaluate the likelihood of spoilage. I'm not crazy about the analogy (food and Internet browsing safety) but some form of aggressive notification may encourage users to update their browsers quickly.
What I like about this study is that the resulting data-gathering systems should be able to provide regular updates and over time we can monitor how browser users and makers adapt. Other notable comprehensive security studies include Google's automated system for uncovering web-based malware, and RobotGenius' ongoing automated analysis (using multiple commercial scanners and a behavioral AV detector) of every Windows executable available for download. But while good data sources help determine the scope of a problem, in the case of computer security, bridging the cultural divide that exists between web developers and their Black Hat counterparts may prove just as important.
tags: malware, security
| comments: 7
| Sphere It
submit:
Thu
Jul 3
2008
Seesmic Starts Adding Features
Seesmic is a company built specifically to encourage asynchronous video conversations. We spent a few hours recently with Seesmic founder and CEO Loic Le Meur, who kindly gave us an update on the company. Four weeks after opening its service to the public, Seesmic recently announced a product roadmap heavily influenced by users.
After focusing on making sure the service scales, the company is now ready to add features including private groups, the option to block individual users from your Seesmic player, and letting users flag offensive content (e.g. porn). Search is a currently a big problem for them, and according to Loic they plan to address search in several ways: (1) give users the option of adding meta-data to their videos (description, tags, etc.), (2) employ automated audio-to-text software to create transcripts, and (3) since Seesmic videos are already on Google, use Google Video search. With apps for both Facebook and OpenSocial slated to be released in August, Seesmic hopes to draw more teen and college-age users.
One of the problems with following conversations on Seesmic is that unlike text, there isn't a way to skim through video. Some people just take longer to get their point across. Assuming a 2-minute per video average, a conversation involving 60 posts/replies would take two hours to view from start to finish. Board member Pierre Omidyar started a Seesmic thread on the possibility of limiting videos to 30 seconds (a la twitter), but for the moment, there are no plans to limit the length of videos. However, the company plans to provide tools to filter out long videos and to display limited portions for faster viewing.
One month after their public launch, here are some key metrics
- 23,000 unique users from 25 countries (about 50% are from the U.S.)
- 3,000 videos are uploaded each day (total of slightly more than 300K videos)
- average length of a video is 2 minutes
- 30 million page views (doesn't include videos viewed through their API)
The Seesmic community not only provides valuable input for their product team, some users have put together impressive mashups and visualizations. My favorites so far are a Youtube and Seesmic mashup for people conversing in sign language, and a visual of conversations related to the recently released French hostage, Ingrid Betancourt. If you download the PicLens Firefox plugin, a Seesmic user created a fun tool to help you quickly navigate all the videos posted by a particular user: try this sample search ("deepakchopra") and set options to 3D Wall.
As to the inevitable question of business models, Loic is mulling a few possibilities: text ads similar to Google AdSense, premium membership, white labels, and customized players for companies, just to name a few. For now, their recent round of funding gives them the luxury of focusing on growing their user base and improving their service. It remains to be seen whether or not asynchronous video conversations catch on in a massive way. Video may never appeal to the many netizens adept at communicating through text. However, the more time you spend on Seesmic, you start seeing why Loic believes that there will be a market for video conversations. By default, the Seesmic community is defining how that market evolves, and four weeks after launching, they seem to be doing just fine.
tags: seesmic, social networks
| comments: 5
| Sphere It
submit:
Tue
Jul 1
2008
On Friends, Followers, and the Top Twitter Users
An easy way to increase Twitter's signal-to-noise ratio is to follow less people. I'm sure you've heard of Twitter users who follow several thousand twitterers. How they keep up with that many micro-blogs is beyond me. Unfortunately, spammers have discovered that to increase their "following", they simply follow thousands of other users, a small percentage whom will politely start following them. Granted, number of followers is not as informative as the number of conversations a twitter user starts or posts that get retweeted, it is currently the only metric that is generally available.
The Twitterholic ranking does not appear to have been overrun by spammers, although some of the top users follow a substantial number of twitterers. According to Twitterholic, the top users averaged around 10,200 followers. Assuming that each of the top users follows a fraction (say 5%, or 1 in 20) of those following them, that translates to the top users following about 514 other twitterers on average. It turns out about half of the top twitterers follow a larger fraction than that. In the graph below, the green line represents 5% of the number of followers (friends-to-followers ratio equal to 5%). As per Twitterholic, I use the term friends to refer to the number of users a twitterer is following:

I highlighted the users who follow more than 15,000 other users, including one user ("ringernation") who follows more than 100,000 twitterers. 100K is about one-tenth of the twitter user base! Over the second half of June, ringernation increased his following by 61%, going from a followers-to-friends ratio of 3.8% to 6%.
Needless to say, when someone is following that many other users, chances are he is no longer paying close attention to what others are saying: just because Scoble is one of your "followers" doesn't mean he is even speed reading what you're tweeting. As Tim observed a few weeks ago:
For example, among the top twitterers, it's pretty clear that many of them are simply following anyone who follows them, which drives their "popularity." But that makes clear that they aren't actually following any of those people -- the volume is just too great. So ironically, if you follow everyone, you follow no one. (Unless you "friend" them, and only really follow your friends.)
So you can see that there are three categories of twitterers: those who use it for its original purpose, by following and being followed by a small group of friends; those who use it for marketing, by broadcasting to many but following none; and those who recognize the asymmetry, and are followed by many, but follow fewer.
Finally, in case your wondering, 42% of the top users follow more than 666 other twitterers.
tags: social networking, twitter
| comments: 5
| Sphere It
submit:
Mon
Jun 30
2008
Evil GIFs: Partial Same Origin Bypass with Hybrid Files
Many web sites allow users to upload different types of files, in particular GIF and other image files. During a recent webinar to promote the upcoming Black Hat briefings in Las Vegas, a group of hackers announced the creation of a hybrid file that can potentially bypass a browser's same origin policy. They created a GIF file that also happens to be a JAR file ( a "GIFAR" file). Once uploaded onto a web site, and assuming the web server runs a JVM, it allows one to run a malicious java applet on someone else's web server.
Details were not provided, since the hackers claim that Sun is still working on a patch. For more on hybrid (image) files as attack vectors, go to minute 41:23 of the webinar.
tags: black hat, security
| comments: 2
| Sphere It
submit:
Thu
Jun 26
2008
The Suspended Facebook App Top Friends
Techmeme and CNET are reporting that one of the most popular Facebook apps has been "suspended" due to security concerns uncovered by a user:
Until Facebook suspended the Top Friends app, created by Slide, anyone could browse partial profiles of anyone else on Facebook who had added Top Friends to their page. CNET News.com confirmed that the security hole exposed the birthdays, gender, and relationship status of strangers, including Facebook executives, the wife of Google co-founder Larry Page, and one profile that seemed to belong to Paris Hilton that used her middle name "Whitney."
According to our research data, Top Friends has been among the Top 3 most used applications pretty much since the Facebook platform launched. Since early April 2008, it has averaged around 1.7M active users and has been the third most popular application:

With close to 30K Facebook applications now in existence, I'm sure many others suffer from similar security problems.
tags: facebook, facebook reports, security
| comments: 7
| Sphere It
submit:
Tue
Jun 24
2008
BarCamp Nairobi Technology Survey
BarCamp Nairobi took place this past weekend and several bloggers estimated that there were over two hundred participants. As part of BarCamp, Erik Hersman, kindly conducted a simple survey for us. In this short post, I will give a brief summary of the results of the survey. For more details on BarCamp Nairobi, consult Erik's blog and flickr pages.
The goal of the short survey was to get a feel for the technologies favored by the attendees. Of the 52 completed surveys, 21 respondents (40%) cited php as one of their primary programming languages.

27 (or 52%) cited one of the common scripting languages (perl, python, php, ruby). The fact that c# was more popular than python, ruby, and perl, is probably indicative of the local IT job market as well. Windows and Linux garnered almost the same number of users:

In a previous post, I mentioned a paper Erik wrote that outlines the importance of cell phones in Africa (“Africa’s PC”). Not suprisingly more than half the BarCamp attendees develop for mobile phone platforms: 27 responded Yes (52%) when asked whether they do any mobile phone application development. At the end of the camp, one of the mobile phone app developers was interviewed by a reporter for the NY Times.
The results of the survey are available as a Google Doc.
tags: africa, barcamp
| comments: 0
| Sphere It
submit:
Fri
Jun 20
2008
Malware Centers and Offshoring
Most studies place China, Brazil, and Russia among the leading sources of conventional and web-based malware. Depending on the type of malware involved, there is a good chance that one of these three countries is among the leading suppliers. Malware from these countries reflect local Internet usage patterns. In Brazil, 75% of regular Internet users access online banking services so Brazilian malware tends to target financial transactions. In China, instant messaging services and online gaming account for several hundred million active users, and close to a billion dollars per year in virtual goods and currencies. Thus malware targeting onling gaming and IM credentials are common in China. Organized crime syndicates in Russia have steered resources towards the theft of credit/bank account numbers, botnets and phishing.

Why is fellow BRIC nation India not a malware center? While cyber laws and their enforcement are important, cyber law enforcement is weak in lots of countries not known for producing malware. The most common response I got from people I queried is that crimeware centers need a steady supply of skilled workers, and the criminal know-how to identify opportunities and evade prosecution. Here are three ingredients that may be crucial to nurturing a malware industry:
1. High-standard of basic education, large supply of technical workers
2. Strong presence of traditional organized crime
3. Widespread poverty and lack of employment opportunities for recent (technical) college graduates
Compared to Brazil and Russia, where organized crime syndicates are involved in the malware industry, the many amateurish Chinese hacker groups maintain public web sites and give interviews to the press. In contrast, the strong presence of organized crime in Brazil and Russia may explain the profit-making focus and relatively low-profile of digital miscreants in those countries. Over the past few years the sphere of influence of Russian criminal groups has slowly widened to include some hacker groups in the rest of the FSU.
Contrary to the common perception that jobs are easy to secure in China, many technical graduates in China face a challenging labor market. A 2005 survey by McKinsey indicated that multinationals were reluctant to hire graduates of second-tier universities in China. Similarly, a 2006 Chinese government study (National Development and Reform Commission) estimated that 60% of that year’s university graduates would be unable to find employment in their preferred fields. The government attributes the reduced quality of many technical education programs to the rapid growth in enrollment.
Unlike its BRIC peers, India has a technology sector that can't seem to get enough workers. Along with the usual focus on law enforcement, strengthening the IT job market in the other BRIC nations would go a long way towards weakening the crimeware industry in those places. You give people good jobs and they are less likely to work for local criminal syndicates. A good reason to not reflexively oppose IT offshoring.
tags: bric, malware, security
| comments: 7
| Sphere It
submit:
Recent Posts
- When Micro-blogging Grows Up on June 12, 2008
- Twitter Availability & Response Times: A Mixed Bag on June 5, 2008
- Africa's Energy Deficit: Energy Hacks Can Make A Difference on June 4, 2008
- Where Does Facebook Grow From Here on May 27, 2008
- Myspace/Facebook App Platforms & Total Installs on May 19, 2008
- Where 2.0: Satellites and The Public Interest on May 14, 2008
- Where 2.0: Eye-Fi and Dash Navigation Apps on May 14, 2008
- Where 2.0: EveryScape Crowdsources Streetview on May 13, 2008
- Where 2.0 Keynotes: EveryBlock, Nokia, FortiusOne on May 13, 2008
- Macs in the Enterprise on May 7, 2008
BENS'S TWITTER UPDATES
BUSINESS INTELLIGENCE
RELEASE 2.0
Current Issue

Velocity: Web Operations & Performance
Issue 2.0.9
Back Issues
More Release 2.0 Back IssuesCURRENT CONFERENCES

RailsConf Europe is the largest gathering of the European Ruby on Rails community. This is your chance to meet, connect, and collaborate with other Rails programmers and developers. Read more

New York has long been where the world's biggest industries go online, and as Web 2.0 grows up and gets serious, the time is right to convene the East Coast web communities under the umbrella of the next generation web. Read more
O'Reilly Home | Privacy Policy ©2005-2008, O'Reilly Media, Inc. | (707) 827-7000 / (800) 998-9938
Website:
| Customer Service:
| Book issues:
All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners.