Four short links: 27 August 2010

Audio API, Book Search Helps Publishers (Gasp!), Tracking Antiquities, Guaranteeing Diversity Fail

1. Working Audio Data Demos -- the new Firefox has a very sweet audio data API and some nifty demos like delay pedals, a beat detector (YouTube) and a JavaScript text-to-speech generator. (via jamesaduncan on Twitter)
2. Estimating the Economic Impact of Mass Digitization Projects on Copyright Holders: Evidence from the Google Book Search Litigation -- [T]he revenues and profits of the publishers who believe themselves to be most aggrieved by GBS, as measured by their willingness to file suit against Google for copyright infringement, increased at a faster rate after the project began, as compared to before its commencement. The rate of growth by publishers most affected by GBS is greater than the growth of the overall U.S. economy or of retail sales.
3. In History-Rich Region, a Very New System Tracks Very Old Things (NY Times) -- Getty built a web database to help Jordan track its antiquities sites (and threats to them) with Google Earth satellite images. (via auchmill on Twitter)
4. What Women Want and How Not to Give it To Them -- thought-provoking piece about the ways in which corporate diversity efforts fail. Must read.

Earthquakes are HUGE on Data.gov

Checking in on Data.gov roughly one year later

After launching just over a year ago with only 47 data sets, the "Raw Data Catalog" catalog on Data.gov now has 2,326 entries that have been collectively downloaded almost three-quarters of a million times. Of course, even these sizable download counts understate the actual impact of this data, which is being embedded in a variety of sites and apps, like those being developed for the Health 2.0 Developer Challenge.

The big Data.gov winner so far? The Department of the Interior's "Worldwide M1+ Earthquakes, Past 7 Days" data set. My guess is that there is some great app or visualization out there making daily use of this file -- if you know what it it is, report it in the comments.

The top 10 data sets by download count are:

1. Worldwide M1+ Earthquakes, Past 7 Days. 122,888 downloads. Real-time, worldwide earthquake list for the past 7 days. Department of the Interior.
2. Latest Volumes of Foreign Relations of the United States. 10,090 downloads. The feed for the latest ten volumes of the official historical documentary record of U.S. foreign policy in the Foreign Relations of the United States series. Department of State.
3. U.S. Overseas Loans and Grants (Greenbook). 6,670 downloads. These data are U.S Economic and Military Assistance by country from 1946 to the present. US Agency for International Development.

---

Interested in making sense of your data, or teaching others how? The O'Reilly Stata Conference: The Business of Data, is happening 1-3 February, 2011, in Santa Clara, CA.

---

4. Child-Related Product Recalls. 2,784 downloads. Lists recalls from CPSC, the agency charged with protecting the public from unreasonable risks of serious injury or death from thousands of types of consumer products. US Consumer Product Safety Commission.
5. Airline On-Time Performance and Causes of Flight Delays. 2,716 downloads. On-time arrival data for non-stop domestic flights by major air carriers, as well as additional items, such as departure and arrival delays, origin and destination airports, flight numbers, scheduled and actual departure and arrival times, cancelled or diverted flights, taxi-out and taxi-in times, air time, and non-stop distance. Department of Transportation.
6. 2005 Toxics Release Inventory data for American Samoa. 2,628 downloads. The Toxics Release Inventory (TRI) is a publicly available EPA database that contains information on toxic chemical releases and waste management activities reported annually by certain industries as well as federal facilities. Environmental Protection Agency.
7. OSHA Data Initiative - Establishment Specific Injury and Illness Rates. 2,588 downloads. The data used by OSHA to calculate establishment-specific injury and illness incidence rates. Department of Labor.
8. 2001 Federal Register in XML. 2,506 downloads. The official daily publication for rules, proposed rules, and notices of Federal agencies and organizations, as well as executive orders and other presidential documents. National Archives and Records Administration.
9. 2007 National RCRA Hazardous Waste Biennial Report Data Files. 2,266 downloads. Data on the generation of hazardous waste from large-quantity generators and on waste management practices from treatment, storage, and disposal facilities. Environmental Protection Agency.
10. Residential Energy Consumption Survey (RECS) Files, All Data, 2005 2,000 Downloads. Data on the use of energy in residential housing units including physical housing unit types, appliances utilized, demographics, fuels, and other energy-use information from the Residential Energy Consumption Survey (RECS), which is conducted every four years. Department of Energy.

Here's a breakdown of the contributions by agency:

Finally, here's a link to the data.gov catalog that includes the number of times the set has been downloaded. (If you're interested in how this was done, check out Use BeautifulSoup to parse data.gov over on O'Reilly Answers).

Congrats to everyone at data.gov for creating this incredible resource for developers-at-large.

Four short links: 26 August 2010

Economic Growth Without Copyright, Ebook Numbers, Hypothesis Analysis Tool, Who Pays for Open Data?

1. Germany's Industrial Expansion Fueled by Absence of Copyright Law? (Der Spiegel) -- fascinating article about the extraordinary publishing output in 1800s Germany vs other nations, all with no effective and enforceable copyright laws. Sigismund Hermbstädt, for example, a chemistry and pharmacy professor in Berlin, who has long since disappeared into the oblivion of history, earned more royalties for his "Principles of Leather Tanning" published in 1806 than British author Mary Shelley did for her horror novel "Frankenstein," which is still famous today. Books were released in high-quality high-price format and low-quality low-price format, and Germans bought them in record numbers. When copyright law became established, publishers did away with the low-quality low-price version and authors complained about the drop in revenue.
2. Cheap Ebooks Give Second Life to Backlist -- it can't be said enough that dead material in print can have a second life online. Here are numbers to make the story plain. (via Hacker News)
3. Competing Hypotheses -- a free, open source tool for complex research problems. A software companion to a 30+ year-old CIA research methodology, Open Source Analysis of Competing Hypotheses (ACH) will help you think objectively and logically about overwhelming amounts of data and hypotheses. It can also guide research teams toward more productive discussions by identifying the exact points of contention. (via johnmscott on Twitter)
4. Economics of Scholarly Production: Supplemental Materials -- scholarly publications include data and documentation that's not in the official peer-reviewed article. Storing and distributing this has been the publication's responsibility, but they're spitting the dummy. Now the researcher's organisation will have to house these supplemental materials. If data is as critical to science as the article it generates, yet small articles can come from terabytes of data, what's the Right Thing To Do that scales across all academia? (via Cameron Neylon)

The Big Picture: What are we making in school?

Elliot Washor of Big Picture Learning organized an educational symposium during Maker Faire Detroit. The symposium brought together educators and practitioners who explored engaging the hands and minds of students, sometimes called thinkering. As a group, they experienced Maker Faire and then met to discuss "how making can be an integral part of how young people figure out who they are in the world." This is a really key idea, I think: what we can learn by making is a process of discovering what we can do, and we begin to participate in making and changing the world around us.

Elliot has shared his thoughts in a Huffington Post article, Making Their Way: Creating a New Generation of Thinkerers. Here is an excerpt:

Making provides opportunities for young people to use their hands and their minds together. Untold numbers of youth are messing around with all manner of tools to create, in tangible form, what's on their minds. Equally important, the maker movement nurtures communities of practice that bring adults and young people together around common interests. Thus, to visit the Maker Faire or a community-based fab lab is to see an aspect of our young people that we seldom witness in schools.

Sadly, however, to observe these young "thinkerers" is to be at least temporarily deluded into believing that this is what many of our young people are all about. Not so. Unfortunately, most young people do not experience making in their schools or in their lives. Literally and figuratively, most of our young people are not at the Faire. Research reveals that the vast majority of them are not into making at all and instead are frittering away their time in a variety of wasteful and unproductive learning activities.

Making is a celebration of an alternative and powerful way of knowing and of thinking things through. Consequently, making is typically antithetical to what traditional schools are all about. That is why the communities of practice that come together at Maker Faires and fabrication labs usually--some would say thankfully--flourish outside of schools.

A few educators, however, are circling these making places to determine where and how they fit in schools, if at all. Educational historian Larry Cremin once wryly noted, that educators respond to a new area of learning by creating a course in it. Recall how schools responded to technology by creating a course "down the hall at fifth period" without ever thinking about changing every course because technology existed. Similarly, educators run the risk of demeaning hand and mind work by creating separate courses for making rather than bringing making into all aspects of the school curriculum and thereby thoroughly reconstituting it.

Recently I learned about a East Bay School for Boys, which is opening this Fall. Incoming sixth graders begin by building their own desk, which according to a consulting teacher David Clifford, gets them involved in creating their own learning environment. In a video on the EBSfB site, one of the organizers of the school said that students can learn through "Play, Practice and Production." That's a really nice framing of how we naturally learn to do things, whether we're talking about soccer, music or robotics.

Will schools find ways to integrate making into the educational experience of students or will students continue to have to look for this experience outside of school -- seeking patchwork alternatives in the community or at home?

Tracking the signal of emerging technologies

The first NASA IT Summit featured deep views into the future.

Last week the words of science fiction writer William Gibson ran rampant over the Twitter back channel at the inaugural NASA IT Summit when a speaker quoted his observation that "The future is here. It's just not evenly distributed yet." That's a familiar idea to readers of the O'Reilly Radar, given its focus on picking up the weak signals that provide insight into what's coming next. So what does the future of technology hold for humanity and space flight? I've been reading the fiction of Jules Verne, Isaac Asimov, David Brin, Neal Stephenson, Bruce Sterling and many other great authors since I was a boy, and thinking and dreaming of what's to come. I'm not alone in that; Tim O'Reilly is also dreaming of augmented reality fiction these days.

Last week I interviewed NASA's CIO and CTO at the NASA IT Summit about some of that fiction made real. We discussed open source, cloud computing, virtualization, and Climate@Home, a distributed supercomputer for climate modeling. Those all represent substantive, current implementations of enterprise IT that enable the agency to support mission-critical systems. (If you haven't read about the state of space IT, it's worth circling back.)

Three speakers at the Summit offered perspectives on emerging technologies that were compelling enough to report on:

• Former senior technology officer at the Defense Intelligence Agency Lewis Shepherd
• Gartner VP David Cearley
• Father of the Internet Vint Cerf

(continue reading)

The cut-free autopsy

How data and digital imaging are shaping virtual autopsies -- an excerpt from "Beautiful Visualization."

The field of data visualization is much broader than most people conceive of it, and exploring this breadth was one of our primary goals in compiling the projects described in "Beautiful Visualization." In the following excerpt, Anders Persson of Linköping University in Sweden explains how radiological digital imaging methods allow medical practitioners to conduct "virtual autopsies" without the use of a scalpel or any other invasive instrument.

Warning: Some readers could find the forensic illustrations in this post too graphic for their tastes. You might want to skip this one if you're squeamish.

The following was written by Anders Persson:

This chapter's topic is extremely important to those who work in the field of medical information visualization. Emerging technologies are enabling visual representations and interaction techniques that take advantage of the human eye’s broad-bandwidth pathway into the mind, allowing users to see, explore, understand, and validate large amounts of complex information at once.

A striking feature of both clinical routine and medical research today is the overwhelming amount of information -- particularly, information represented as images. Practitioners are dealing with ever-larger numbers of images (hundreds or thousands rather than dozens) and more complex, higher-dimensional information (vectors or tensors rather than scalar values, arranged in image volumes directly corresponding to the anatomy rather than flat images). However, they typically still use simple two-dimensional devices such as conventional monitors to review this overflow of images, one by one. As the bottleneck is no longer the acquisition of data, future progress will depend on the development of appropriate methods for handling and analyzing the information, as well as making it comprehensible to users. One of the most important issues for the future is the workflow. The entire chain from the acquisition of data until the point at which the clinician receives the diagnostic information must be optimized, and new methods must be validated.

Normally, performing this validation process on living patients has its limitations. It can in some cases be impossible to know if the acquired diagnostic information is correct as long as the patient is alive; the real gold standard is missing. Postmortem imaging has the potential to solve this problem.

The methodology of autopsy has not undergone any major transformation since its introduction in the middle of the 19th century. However, new radiological digital imaging methods, such as multidetector computed tomography (MDCT) and magnetic resonance imaging (MRI), have the potential to become the main diagnostic tools in clinical and forensic pathology in the future. Postmortem visualization may prove to be a crucial tool in shaping tomorrow’s healthcare, by validating new imaging technology and for quality assurance issues.

(continue reading)

Four short links: 25 August 2010

Narrative and Structure, Teaching Science, Time-Series Statistics, and Who Benefits from Open Source

1. Why Narrative and Structure are Important (Ed Yong) -- Ed looks at how Atul Gawande's piece on death and dying, which is 12,000 words long, is an easy and fascinating read despite the length.
2. Understanding Science (Berkeley) -- simple teaching materials to help students understand the process of science. (via BoingBoing comments)
3. Sax: Symbolic Aggregate approXimation -- SAX is the first symbolic representation for time series that allows for dimensionality reduction and indexing with a lower-bounding distance measure. In classic data mining tasks such as clustering, classification, index, etc., SAX is as good as well-known representations such as Discrete Wavelet Transform (DWT) and Discrete Fourier Transform (DFT), while requiring less storage space. In addition, the representation allows researchers to avail of the wealth of data structures and algorithms in bioinformatics or text mining, and also provides solutions to many challenges associated with current data mining tasks. One example is motif discovery, a problem which we recently defined for time series data. There is great potential for extending and applying the discrete representation on a wide class of data mining tasks. Source code has "non-commercial" license. (via rdamodharan on Delicious)
4. Open Source OSCON (RedMonk) -- The business of selling open source software, remember, is dwarfed by the business of using open source software to produce and sell other services. And yet historically, most of the focus on open source software has accrued to those who sold it. Today, attention and traction is shifting to those who are not in the business of selling software, but rather share their assets via a variety of open source mechanisms. (via Simon Phipps)

Four short links: 24 August 2010

Android Chaos, Open Source Briefings, Scripting nmap, Russian Cybercrime

1. The Dirty Little Secret About Google Android (ZD Net) -- By some reports, the Open Handset Alliance is in now shambles. Members such as HTC have gone off and added lots of their own software and customizations to their Android devices without contributing any code back to the Alliance. Motorola and Samsung have begun taking the same approach. The collaborative spirit is gone — if it ever existed at all. And, Google is proving to be a poor shepherd for the wolves-in-sheep’s-clothing that make up the telecoms and the handset makers in the Alliance. The mobile phone industry is as messed up as enterprise Unix was in the 80s. (via Hacker News)
2. MilOSS Working Group 2 Wrap Up -- A challenge was issued at the barcamp lunch in response to the need for a canonical set of briefing charts detailing the value of open source software for the military, from security to basic definitions to legal issues. All-in-all, about 100 briefing charts were created and will soon be made available to the community to use/modify/tweak as needed. (via johnmscott on Twitter)
3. nmap Scripting Engine -- Lua embedded in nmap lets you automate a lot of network-related tasks. (via Slashdot)
4. Russian Cybercrime: Geeks not Gangsters -- “Basically, from what we’ve seen on the forums much of what goes on with the sales of services is much more petty criminal activity, or crimes of opportunity,” Grugq said. “Often poor students who like to hack for fun will sell access to a server they’ve owned. Many don’t even realise that this is an illegal activity. This sale will be for $20 or $30 (£!3 or £19), which is a lot of money for a poor student in Russia, but for a hardened criminal mastermind bent on destroying Western civilization — not so much.” We need to launch a distributed denial of students attack on Russia. (via jasonwryan on Twitter)

Cost is only part of the Gov 2.0 open source story

Washington, D.C. CTO Bryan Sivak adds realism to his open source advocacy.

Bryan Sivak, chief technology officer for the District of Columbia and a speaker at the upcoming Gov 2.0 Summit, has smartly mixed healthy realism with enthusiastic support for open source in government. The result is a message that resonates beyond open source evangelists.

For example, here's what he recently had to say about the allure of open source cost savings:

"I don't think cost savings of open source is the panacea that everyone thinks it is. It's true that there's no upfront licensing cost, but there's cost in figuring out the appropriate implementation strategy, making sure you have the people with the right skills on staff, and making sure you're able to maintain and manage the system. You need to put a lot into how you implement it."

(continue reading)

Four short links: 23 August 2010

Crowdsourced Architecture, Lego Timetracking, Streaming Charts, and The Deeper Meaning of School

1. Open Buildings -- crowdsourced database of information about buildings, for architecture geeks. A sign that crowdsourcing is digging deep into niches far far from the world of open source software. (via straup on Delicious)
2. Lego-Based Time Tracking -- clever hack to build physical graphs of where your time goes. (via avgjanecrafter on Twitter)
3. Smoothie Charts -- a charting Javascript library designed for live streaming data. (via jdub on Twitter)
4. The Big Lie (Chris Lehmann) -- why school is not only about workforce development: I think - I fear - that the next twenty or thirty years of American life are going to be difficult. I think we're going to have some really challenging problems to solve, and I think that we're going to be faced with hard choices about our lives, and I want our schools to help students be ready to solve those problems, to weigh-in on those problems, to vote on those problems. It's why History and Science are so important. It's why kids have to learn how to create and present their ideas in powerful ways. It's why kids have to become critical consumers and producers of information. And hopefully, along the way, they find the careers that will help them build sustainable, enjoyable, productive lives. Also read Umair Haque's A Deeper Kind of Joblessness which Chris linked to.

Opensource and Javascript: Polymaps Used To Make PrettyMaps

SimpleGeo, geo cloud services and data provider, and Stamen, creators of many beautiful data visualizations, have teamed up to release Polymaps. Polymaps is an opensource Javascript mapping framework. It's been on Github for a while, but they are finally announcing it.

Out of the gate, Stamen has also launched a great example application, PrettyMaps, combining Natural Earth, OSM and Flickr layers. The effect is, well, pretty. Here is the continental United States at zoom level 3:

And here is Los Angeles at zoom level 8:

(continue reading)

Space IT, the final frontier

Exploring open source cloud computing, virtualization and Climate@Home at NASA's first IT Summit.

When people think of NASA and information technology in 2010, issues like the future of manned space flight, the aging space shuttle fleet or progress on the International Space Station may come to mind. What casual observers miss is how NASA is steadily modernizing those systems, including developing open source cloud computing, virtualization, advanced robotics, deep space communications and collaborative social software, both behind the firewall and in the public eye.

NASA has also earned top marks for its open government initiatives from both the White House and an independent auditor. That focus is in-line with the agency's mission statement, adopted in February 2006, to "pioneer the future in space exploration, scientific discovery and aeronautics research," and it was on display this week at the first NASA IT Summit in Washington, D.C.

(continue reading)

Recent Posts

• Be innovative, but don't use that word | by Mac Slocum on August 20, 2010
• Four short links: 20 August 2010 | by Nat Torkington on August 20, 2010
• The software behind the VA health care transformation | by Andy Oram on August 19, 2010
• Four short links: 19 August 2010 | by Nat Torkington on August 19, 2010
• Facebook Places plays nice with Yelp, Foursquare, Gowalla and Booyah | by Brady Forrest on August 18, 2010
• Linked data is opening 800 years of UK legal info | by Alex Howard on August 18, 2010
• The laws of information chemistry | by Jon Udell on August 18, 2010
• Thousands of workers are standing by | by Mac Slocum on August 18, 2010
• Four short links: 18 August 2010 | by Nat Torkington on August 18, 2010
• Tracking the tech that will make government better | by Alex Howard on August 17, 2010
• On re-reading Steven Levy's "Hackers" | by Andy Oram on August 17, 2010
• Four short links: 17 August 2010 | by Nat Torkington on August 17, 2010