Four short links: 25 August 2010

Narrative and Structure, Teaching Science, Time-Series Statistics, and Who Benefits from Open Source

1. Why Narrative and Structure are Important (Ed Yong) -- Ed looks at how Atul Gawande's piece on death and dying, which is 12,000 words long, is an easy and fascinating read despite the length.
2. Understanding Science (Berkeley) -- simple teaching materials to help students understand the process of science. (via BoingBoing comments)
3. Sax: Symbolic Aggregate approXimation -- SAX is the first symbolic representation for time series that allows for dimensionality reduction and indexing with a lower-bounding distance measure. In classic data mining tasks such as clustering, classification, index, etc., SAX is as good as well-known representations such as Discrete Wavelet Transform (DWT) and Discrete Fourier Transform (DFT), while requiring less storage space. In addition, the representation allows researchers to avail of the wealth of data structures and algorithms in bioinformatics or text mining, and also provides solutions to many challenges associated with current data mining tasks. One example is motif discovery, a problem which we recently defined for time series data. There is great potential for extending and applying the discrete representation on a wide class of data mining tasks. Source code has "non-commercial" license. (via rdamodharan on Delicious)
4. Open Source OSCON (RedMonk) -- The business of selling open source software, remember, is dwarfed by the business of using open source software to produce and sell other services. And yet historically, most of the focus on open source software has accrued to those who sold it. Today, attention and traction is shifting to those who are not in the business of selling software, but rather share their assets via a variety of open source mechanisms. (via Simon Phipps)

Four short links: 24 August 2010

Android Chaos, Open Source Briefings, Scripting nmap, Russian Cybercrime

1. The Dirty Little Secret About Google Android (ZD Net) -- By some reports, the Open Handset Alliance is in now shambles. Members such as HTC have gone off and added lots of their own software and customizations to their Android devices without contributing any code back to the Alliance. Motorola and Samsung have begun taking the same approach. The collaborative spirit is gone — if it ever existed at all. And, Google is proving to be a poor shepherd for the wolves-in-sheep’s-clothing that make up the telecoms and the handset makers in the Alliance. The mobile phone industry is as messed up as enterprise Unix was in the 80s. (via Hacker News)
2. MilOSS Working Group 2 Wrap Up -- A challenge was issued at the barcamp lunch in response to the need for a canonical set of briefing charts detailing the value of open source software for the military, from security to basic definitions to legal issues. All-in-all, about 100 briefing charts were created and will soon be made available to the community to use/modify/tweak as needed. (via johnmscott on Twitter)
3. nmap Scripting Engine -- Lua embedded in nmap lets you automate a lot of network-related tasks. (via Slashdot)
4. Russian Cybercrime: Geeks not Gangsters -- “Basically, from what we’ve seen on the forums much of what goes on with the sales of services is much more petty criminal activity, or crimes of opportunity,” Grugq said. “Often poor students who like to hack for fun will sell access to a server they’ve owned. Many don’t even realise that this is an illegal activity. This sale will be for $20 or $30 (£!3 or £19), which is a lot of money for a poor student in Russia, but for a hardened criminal mastermind bent on destroying Western civilization — not so much.” We need to launch a distributed denial of students attack on Russia. (via jasonwryan on Twitter)

Cost is only part of the Gov 2.0 open source story

Washington, D.C. CTO Bryan Sivak adds realism to his open source advocacy.

Bryan Sivak, chief technology officer for the District of Columbia and a speaker at the upcoming Gov 2.0 Summit, has smartly mixed healthy realism with enthusiastic support for open source in government. The result is a message that resonates beyond open source evangelists.

For example, here's what he recently had to say about the allure of open source cost savings:

"I don't think cost savings of open source is the panacea that everyone thinks it is. It's true that there's no upfront licensing cost, but there's cost in figuring out the appropriate implementation strategy, making sure you have the people with the right skills on staff, and making sure you're able to maintain and manage the system. You need to put a lot into how you implement it."

(continue reading)

Four short links: 23 August 2010

Crowdsourced Architecture, Lego Timetracking, Streaming Charts, and The Deeper Meaning of School

1. Open Buildings -- crowdsourced database of information about buildings, for architecture geeks. A sign that crowdsourcing is digging deep into niches far far from the world of open source software. (via straup on Delicious)
2. Lego-Based Time Tracking -- clever hack to build physical graphs of where your time goes. (via avgjanecrafter on Twitter)
3. Smoothie Charts -- a charting Javascript library designed for live streaming data. (via jdub on Twitter)
4. The Big Lie (Chris Lehmann) -- why school is not only about workforce development: I think - I fear - that the next twenty or thirty years of American life are going to be difficult. I think we're going to have some really challenging problems to solve, and I think that we're going to be faced with hard choices about our lives, and I want our schools to help students be ready to solve those problems, to weigh-in on those problems, to vote on those problems. It's why History and Science are so important. It's why kids have to learn how to create and present their ideas in powerful ways. It's why kids have to become critical consumers and producers of information. And hopefully, along the way, they find the careers that will help them build sustainable, enjoyable, productive lives. Also read Umair Haque's A Deeper Kind of Joblessness which Chris linked to.

Four short links: 20 August 2010

Case Study, Promise Transparency, Scriptable Browsing, Open Science Data Success

1. Case Study: Slideshare Goes Freemium (Startup Lessons Learned) -- I love case studies, they're the best part of every business degree. The MVPs were tricky to implement for emotional reasons, too. Because the SlideShare team was used to giving away a high-value product, engineers balked at charging for a clearly imperfect product. The analytics package, for instance, launched in what Sinha calls “a very crude version; we started off and sold it before we were comfortable with it."
2. Guardian's Pledge Tracker -- keeping track of the pledges and promises from the new UK government. (via niemanlab)
3. luakit browser framework -- script WebKit using Lua. (via ivanristic)
4. Sharing of Data Leads to Progress on Alzheimers (New York Times) -- The key to the Alzheimer’s project was an agreement as ambitious as its goal: not just to raise money, not just to do research on a vast scale, but also to share all the data, making every single finding public immediately, available to anyone with a computer anywhere in the world. No one would own the data. No one could submit patent applications, though private companies would ultimately profit from any drugs or imaging tests developed as a result of the effort.

On re-reading Steven Levy's "Hackers"

Why the "Hackers" thesis still holds. Plus: How hyperlinks created new context in the ebook edition.

When O'Reilly Media bought the rights to "Hackers" for its 25th Anniversary re-release, we knew this classic needed no gilding. The characters are just as astonishing, the anecdotes as gripping, the analysis as pertinent as when the book was released in 1984. O'Reilly publisher Dale Dougherty recently conducted a video interview with "Hackers" author Steven Levy about the book.

Nevertheless, spiffing it up for the ebook version has paid off by delivering a new dimension to the book that readers are reporting back on favorably. After I offer my reactions to re-reading the text after 25 years -- stronger reactions than I had expected -- I'll finish with a discussion of the links we added to the electronic version.

(continue reading)

Open source givers and takers

Taking without giving isn't the problem. We need better open source contribution metrics.

Dana Blankenhorn's recent ZDNet blog points to Accenture's "hockey stick for open source" and notes that while 69 percent of the companies Accenture surveyed plan to increase their open source investment in the next year, only 29 percent plan to contribute back to the open source community. That sounds very plausible. But is it a problem? I'm not so sure.

First, I don't think "all take and no give" is a failure. Or even a problem. If you're giving, you shouldn't be surprised if people take. If you're taking something that's been freely given, you shouldn't feel obliged to give back. If you do, that's great. And if you're a giver, you should be glad that people are taking, whether or not you're getting something back in return.

(continue reading)

Four short links: 6 August 2010

Amazon Margins, Crowdsourced Science, Data Tool Opensourced, Document Splitting

1. AWS: Forget the Revenue, Did You See the Margins? (RedMonk) -- According to UBS, Amazon Web Services gross margins for the years 2006 through 2014 are 47%, 48%, 48%, 49%, 49%, 50%, 50.5%, 51%, 53%. (these are analyst projections, so take with grain of salt, but those are some sweet margins if they're even close to accurate)
2. Science Pipes -- an environment in which students, educators, citizens, resource managers, and scientists can create and share analyses and visualizations of biodiversity data. It is built to support inquiry-based learning, allowing analysis results and visualizations to be dynamically incorporated into web sites (e.g. blogs) for dissemination and consumption beyond SciencePipes.org itself. (via mikeloukides on Twitter)
3. ScraperWiki Source Code -- AGPL-licensed source to the ScraperWiki, a tool for data storage, cleaning, search, visualization, and export.
4. Doc split -- a command-line utility and Ruby library for splitting apart documents into their component parts: searchable UTF-8 plain text via OCR if necessary, page images or thumbnails in any format, PDFs, single pages, and document metadata (title, author, number of pages...)

Four short links: 5 August 2010

Delicious Graphs, Charities and Data, Climate Psychology, Data Structure Portability

1. Delicious Links Clustered and Stacked (Matt Biddulph) -- six years of his delicious links, k-means clustered by tag and graphed. The clusters are interesting, but I wonder whether Matt can identify significant life/work events by the spikes in the graph.
2. Open Data and the Voluntary Sector (OKFN) -- Open data will give charities new ways to find and share information on the need of their beneficiaries - who needs their services most and where they are located. The sharing of information will be key to this - it’s not just about using data that the government has opened up, but also opening your own data.
3. Cognitive and Behavioral Challenges in Responding to Climate Change -- At the deepest level, large scale environmental problems such as global warming threaten people's sense of the continuity of life - what sociologist Anthony Giddens calls ontological security. Ignoring the obvious can, however, be a lot of work. Both the reasons for and process of denial are socially organized; that is to say, both cognition and denial are socially structured. Denial is socially organized because societies develop and reinforce a whole repertoire of techniques or "tools" for ignoring disturbing problems. Fascinating paper. (via Jez)
4. Blueprints -- provides a collection of interfaces and implementations to common, complex data structures. Blueprints contains a property graph model its implementations for TinkerGraph, Neo4j, and SAIL. Also, it contains an object document model and implementations for TinkerDoc, CouchDB, and MongoDB. In short, Blueprints provides a one stop shop for implemented interfaces to help developers create software without being tied to particular underlying data management systems.

Four short links: 4 August 2010

Python Reasoning, Learning the Right Way, Curated Folksonomy, Arduino Image Correction

1. FuXi -- Python-based, bi-directional logical reasoning system for the semantic web from the folks at the Open Knowledge Foundation. (via About Inferencing)
2. Harness the Power of Being an Idiot -- I learn by trying to build something, there's no other way I can discover the devils-in-the-details. Unfortunately that's an incredibly inefficient way to gain knowledge. I basically wander around stepping on every rake in the grass, while the A Students memorize someone else's route and carefully pick their way across the lawn without incident. My only saving graces are that every now and again I discover a better path, and faced with a completely new lawn I have an instinct for where the rakes are.
3. Stack Overflow's Curated Folksonomy -- community-driven tag synonym system to reduce the chaos of different names for the same thing. (via Skud)
4. Image Deblurring using Inertial Measurement Sensors (Microsoft Research) -- using Arduino to correct motion blur. (via Jon Oxer)

Lessons learned building the elmcity service

Jon Udell's reflections on mashing up software cultures and calendar data.

In 1995 I started writing a column for BYTE about the development of the magazine's website, plus some early examples of what we now call web services and social media. When I started, I knew very little about Apache, Perl, and the Common Gateway Interface. But I was lucky to be able to learn by doing, by explaining what I learned to my readers, and by relaying what they were teaching me. Because I came to the project with a beginner's mind, the column became a launchpad for a lot of people who were just getting started on web development.

Nowadays I'm working on another web project, the elmcity calendar aggregator. And I came to this project with a different kind of beginner's mind. I had built a first version of the service a few years back in Python, on Linux, using the Django framework. After I joined Microsoft I decided to recreate it on Azure. I started in Python -- specifically, IronPython. But Azure was brand new at the time, and not very friendly to IronPython. So I switched to C# and .NET. I knew more about that environment than I had once known about Perl and CPAN, but not a whole lot more. That inexperience qualifies me to write another series of learning-by-doing essays, and that's what this will be.

The code, which is under an Apache 2.0 license, will live on github. I'll discuss it in detail over on O'Reilly Answers. In this space, I'll reflect on larger themes: building and operating a cloud service in 2010, in a way that cooperates with other services and straddles two different cultures.

(continue reading)

Four short links: 3 August 2010

Structured Data, Graph Tools, Photo Lives, Prize Theory

1. OpenStructs -- an education and distribution site dedicated to open source software for converting, managing, viewing and manipulating structured data.
2. TinkerPop -- many (often open source) tools for graph data.
3. Polaroid a Day -- a moving human story told in photographs.
4. Prizes (PDF) -- White House memorandum to government agencies explaining how prizes are to be used. The first part, the why and how of contests and prizes, is something to add to your "here, read this" arsenal.

Recent Posts

• Four short links: 2 August 2010 | by Nat Torkington on August 2, 2010
• Four short links: 30 July 2010 | by Nat Torkington on July 30, 2010
• Four short links: 28 July 2010 | by Nat Torkington on July 28, 2010
• Four short links: 27 July 2010 | by Nat Torkington on July 27, 2010
• Four short links: 23 July 2010 | by Nat Torkington on July 23, 2010
• VistA scenarios, and other controversies at the Open Source health care track | by Andy Oram on July 22, 2010
• Four short links: 22 July 2010 | by Nat Torkington on July 22, 2010
• Four short links: 20 July 2010 | by Nat Torkington on July 20, 2010
• Four short links: 19 July 2010 | by Nat Torkington on July 19, 2010
• Report from 2010 Community Leadership Summit | by Andy Oram on July 18, 2010
• Four short links: 16 July 2010 | by Nat Torkington on July 16, 2010
• Four short links: 15 July 2010 | by Nat Torkington on July 15, 2010