Open government is a mindset

Analysis: Connections are forming between social media, open government and e-government.

I recently talked about the role social media can play in open government at Social Security's Open Government Employee Awareness Day. My presentation is embedded below:

As I said in my talk at the agency, what I've seen in my reporting over this year suggests a nascent connection between the evolution of social media, open government and e-government. The economic meltdown of the past few years has pushed state governments to do more with less. The federal government has explicitly -- and sometimes implicitly -- endorsed the use of several types of online social software as tools for open government. The top-down open government directive has come at a time when there is an active network of civic hackers finding innovative ways to use free services, open data and partnerships with social entrepreneurs.

(continue reading)

Data as a service

A look at how services and widgets are democratizing data and visualization.

The last two months have seen some important developments in the way data is made available. First, Infochimps created a web API for publishing data. The number of datasets is relatively limited; there are five available now, of which four have to do with Twitter data, and one maps IP addresses to census data (and that one appears not to be available yet). Their site allows you to request (or vote on requests) for new datasets. Pricing is reasonable. You can do significant experimentation, or even run a useful low-volume application, without running up any charges.

"Data as a service" is not a new term, by any means. There have been any number of data services over the years. But this is something different from the many services that have sold data -- or even the more recent services that have sold data via the Internet. Data as a service is another part of the cloud computing alphabet soup, on par with "infrastructure, software, or platform as a service" (IaaS/SaaS/PaaS). Infochimps makes possible applications where data lives in the cloud. Granted, you're not going to access terabyte datasets over the Internet. But neither do you have to download (or have shipped) a giant dataset for the few Kilo- or Megabytes that interest you. Infochimps is pushing a bit beyond simple data access. Their Twitter APIs aren't raw data, but implement trust metrics, influence metrics, and more. So perhaps it's better to call this "algorithm as a service" (AaaS), not unlike the Prediction API (machine learning using Google's algorithms) that was announced at Google I/O.

(continue reading)

Detroit 2.0: Motor City to Maker City

Maker Faire Detroit opens this coming weekend at The Henry Ford in Dearborn. Our goal is to create a fun, family-friendly event and showcase talented makers from Michigan and aroundthe Midwest. I also think the event gives us an opportunity to consider ways that makers can be part of re-inventing Detroit from the ground-up. This Thursday, just before Maker Faire Detroit, we've put together a special program Can Do Camp to explore what makers are doing and what can be done.

One of the presenters will be K. Venkatesh Prasad, Technical Leader, Infotronics in the Research and Advanced Engineering Group at Ford Motor Company. At Maker Faire in the Bay Area, Prasad spoke about about Automakers 2.0, (see links below) which was the idea that there would be a new generation of people working on cars and thinking of the car as a technology platform.

Here is a brief sketch of Prasad's ideas for a new Detroit that leverages its capacity for building physical things but also extends into a digital world where more of those things are connected.

(continue reading)

Redesigning the New York City subway map

The long and complicated path that led to Eddie Jabbour's KickMap.

Note: The field of data visualization is much broader than most people conceive of it, and exploring this breadth was one of our primary goals in compiling the projects described in "Beautiful Visualization." In the following excerpt, KickMap designer Eddie Jabbour explains the complexity he faced and the trade-offs he made while reinventing one of the most iconic maps in the world.

What follows is Eddie Jabbour's story, as told to Julie Steele:

Maps are one of the most basic data visualizations that we have; we've been making them for millennia. But we still haven't perfected them as a tool for understanding complex systems -- and with 26 lines and 468 stations across five boroughs, the New York City subway system certainly is complex. The KickMap is the result of my quest to design a more effective subway map, and ultimately to encourage increased ridership.

The need for a better tool

I was born in Queens and raised in Brooklyn. The first subway map I saw was my father's, circa 1960. It made a vivid impression on me because it intimidated me. I saw a gray New York with red, green, and black lines running all over it like a grid (see Figure 5-1), and hundreds of station names attached (1). It reminded me of a complex electrical diagram that I couldn't understand; it looked very "adult-serious" and even a little scary. I hoped I'd never have to deal with it.

(continue reading)

Four short links: 27 July 2010

Preservation, Scaling Social Networks, Monetizing Music, and Android Unopened Source

1. Digital Continuity Conference Proceedings -- proceedings from a New Zealand conference on digital archiving, preservation, and access for archives, museums, libraries, etc.
2. What Are The Scaling Issues to Keep in Mind While Developing a Social Network Feed? (Quora) -- insight into why you see the failwhale. (via kellan on Twitter)
3. Fan Feeding Frenzy -- Amanda Palmer sells $15k in merch and music in 3m via Bandcamp. Is the record available on iTunes yet? Absolutely not. We have nothing against iTunes, it’ll end up there eventually I’m sure, but it was important for us to do this in as close to a DIY manner as possible. If we were just using iTunes, we couldn’t be doing tie-ins with physical product, monitoring our stats (live), and helping people in real-time when they have a question regarding the service. Being able to do all of those things and having such a transparent format in which to do it has been a dream come true. We all buy stuff on the iTunes store - or AmazonMP3 or whatever - but it’s not THE way artists should be connecting to fans, and it’s certainly not the way someone is going to capture the most revenue on a new release. (via BoingBoing)
4. Sad State of Open Source in Android Tablets -- With the exception of Barnes & Noble’s Nook e-reader, a device that isn’t even really a tablet, I found one tablet manufacturer who was complying with the minimum of their legal open source requirements under GNU GPL. Let alone supporting community development.

Wrap-up of the health care IT track at O'Reilly's Open Source convention

The first health care track to be included in an O'Reilly conference covered all three days of sessions at last week's Open Source convention and brought us 22 talks from programmers, doctors, researchers, corporate heads, and health care advocates. We grappled throughout these three days--which included two popular and highly vocal Birds of a Feather gatherings--with the task of opening up health care.

It's not surprising that, given this was an open source conference, the point we heard from speakers and participants over and over again was how critical it is to have open data in health care, and how open source makes open data possible. Like most commercial fields, health care is replete with managers and technologists who don't believe open source software can do the job of powering and empowering busy clinicians in high-risk situations. Some of the speakers spent time challenging that view.

I decided over the course of the week that the health care industry has two traits that make it more conservative than many fields. On the one hand, the level of regulation and certification is mind-boggling. Hardly any technical job can be taken without a particular course of training and a certificate. Privacy regulations--which are interpreted somewhat differently at every clinic--get in the way of almost anyone doing anything new. Software has to be certified too, not something that software firms in most domains are accustomed to. All these controls are in place for good reason, and help you feel safe proffering your arm for a needle or popping the pills each day your doctor told you to take.

Paradoxically, though, the health care field is also resistant to change because the actors in it are so independent. Health care is the most fragmented industry in the country, with 80% of medical practices consisting of one or two physicians.

(continue reading)

Why fragmentation is a good sign for Android

Author Marko Gargenta on Android's growth and app compatibility.

Like every popular open source project, Android is dogged by the fragmentation threat. It's the nature of openness: the lack of a formal hierarchy means many different flavors of an operating system or package could appear on many different devices.

At OSCON, I asked "Learning Android" author Marko Gargenta if fragmentation is a looming problem for Android. He doesn't see it that way. Fragmentation, to him, is a side effect of accomplishing a goal:

Keep in mind, when Google started the Android project, their goal wasn't a specific device. Their goal was many, many different devices, with many, many different companies adopting this platform. So, fragmentation is sort of built into the nature of the project. There are going to be different flavors of Android out there. That's not necessarily a bad thing. What's been bad is for a consumer not knowing what application is going to be able to run on what device. And to address that, recently the Android project has released a definition of what compatibility means. [Emphasis added; question asked at the 3:24 mark in this video.]

(continue reading)

Four short links: 26 July 2010

Maturing Wikileaks, Connectivity as a Right, Music from Proteins, Preserved Source

1. Is Wikileaks Growing Up? -- I linked earlier to FAS commentator Steven Aftergood, who had ripped Wikileaks as irresponsible and dangerous. The latest leaks, however, get grudging respect. "the latest dump deals with a perfectly newsworthy topic and -- judging from my initial glances at the news coverage -- Wikileaks itself has acknowledged the necessity of withholding certain portions of the documents that might endanger individuals who are named in them. If so, that is commendable." (via jayrosen_nyu on Twitter)
2. Open Connectivity and Open Data -- is access to the Internet a human right? Video of a presentation by Jon Penney, the InternetNZ CyberLaw Fellow at Victoria University of Wellington, New Zealand.
3. ProteoMusic -- twisted music inspired by genomes and proteins. (via christianbok on Twitter)
4. MacPaint and QuickDraw Source Donated to Computer Museum -- source is as much a historical artifact worthy of preservation as hardware, and will be increasingly so. Should Library of Congress require submission of distributed computer code the same as for published books? (via Andy Baio)

Web 2.0 risks and rewards for federal agencies

Potential security and privacy issues balance gov. innovation and cost savings.

The nature of record keeping and government transparency in the information age is rapidly changing. Officials can text, tweet, direct message, send "Facemail," IM or Skype, all from a personal smartphone. That's why yesterday's testimony of David Ferriero, Archivist of the United States, at a hearing on "Government 2.0: Federal Agency Use Of Web 2.0 Technologies" was both critically relevant and useful. (It's embedded below, after the jump.)Officials are "free to use external accounts as long as emails are captured into records management systems," he said. "Every new technology provides new challenges to what is a record." Ferriero said that new guidance on government use of social media will be released this fall, updating the 2009 guidance issued by the National Archives and Records Administration (NARA).

The biggest challenge, said Ferriero, is whether the record is the whole site or just a portion. "Web 2.0 offers opportunities unimagined a decade ago," he said.

David McClure, associate administrator for citizen services and innovative technologies at the General Services Administration, echoed that sentiment in his testimony."Web 2.0 isn't fundamentally about the technology itself but how people are coming together to achieve extraordinary results," he said, pointing to uses for idea management, ranking or ranking ideas, communication and more. "From an efficiency perspective, a lot of software meets those needs without the need for the agency to build tools, when the market is as robust as it is today."

More on the House subcommittee hearing on Government 2.0 after the jump, including a United States General Accountability Office (GAO) report on Web 2.0 and security in government and videos.

(continue reading)

Four short links: 23 July 2010

Reputation Systems, Faceted Search Tutorial, Video Utility, and Chinese Slang

1. 5 Reputation Missteps (and how to avoid them) (YouTube) -- a Google Tech Talk from one of the authors of the O'Reilly-published Building Web Reputation Systems.
2. Solr on EC2 Tutorial -- the tutorial shows how to index Wikipedia with Solr. (via Matt Biddulph)
3. clive -- a command line utility for extracting (or downloading) videos from Youtube and other video sharing Web sites. It was originally written to bypass the Adobe Flash requirement needed to view the hosted videos..
4. ChinaSmack -- how to talk smack online in Chinese. (via BoingBoing)

VistA scenarios, and other controversies at the Open Source health care track

The history and accomplishments attributed to VistA, the Veterans Administration's core administrative software, mark it as one of the most impressive software projects in history. Still, lots of smart people in the health care field deprecate VistA and cast doubt that it could ever be widely adopted. Having spent some time with people on both sides, I'll look at their arguments in this blog, and then summarize other talks I heard today at the Open Source Convention health care track.

Yesterday, as I described in my previous blog, we heard an overview of trends in health care and its open source side in particular. Two open source free software projects offering electronic health records were presented, Tolven and openEMR. Today was VistA day, and those who stayed all the way through were entertained by accolades of increasing fervor from the heads of vxVistA, Medsphere, and ClearHealth. (Anyone who claims that VistA is cumbersome and obsolete will have to explain why it seems to back up so many successful companies.) In general, a nice theme to see today was so many open source companies making a go of it in the health care field.

(continue reading)

How open source can improve health care

Three perspectives on the links between health care, government and open source.

The Health IT track at OSCON this week brought together three of the thought leaders working to create a Nationwide Health Information Network (NHIN). They are:

David Riley, head of the CONNECT initiative for the Federal Health Architecture (FHA) Program. Riley is responsible for creating the product direction and overseeing product development for CONNECT.

Brian Behlendorf is a collaboration advisor for the CONNECT project, and he co-founded the Apache Web Server Project.

Arien Malec is the coordinator for NHIN Direct, where he manages the activities of a public-private group that's developing specifications for data transport in support of the meaningful use rules.

(continue reading)

Recent Posts

• Four short links: 22 July 2010 | by Nat Torkington on July 22, 2010
• Day one of the health care IT track at O'Reilly's Open Source convention | by Andy Oram on July 21, 2010
• Detroit Can Do Camp - July 29 | by Dale Dougherty on July 21, 2010
• Where Facebook's half a billion users reside | by Ben Lorica on July 21, 2010
• Why software startups decide to patent ... or not | by Pamela Samuelson on July 21, 2010
• Four short links: 21 July 2010 | by Nat Torkington on July 21, 2010
• Beware the march of the IP trolls at the House Committee on Small Business | by Andy Oram on July 20, 2010
• Four short links: 20 July 2010 | by Nat Torkington on July 20, 2010
• Social Security in the Gov 2.0 age | by Alex Howard on July 19, 2010
• In defense of games in the workplace | by Mac Slocum on July 19, 2010
• Four short links: 19 July 2010 | by Nat Torkington on July 19, 2010
• Report from 2010 Community Leadership Summit | by Andy Oram on July 18, 2010