Credit card company data mining makes us all instances of a type

The New York Times has recently published one of their in-depth, riveting descriptions of how credit card companies use everything they can learn about us. Any detail can be meaningful: what time of day you buy things, or the quality of the objects you choose.

The way credit collectors use psychology reminds me of CIA interogators (without the physical aspects of pressure). In fact, they're probably more effective than CIA interogators because they stick to the basic insight that kindness elicits more cooperation than threats.

So who gave them permission to use our purchase information against us? What law could possibly address this kind of power play?

There's another disturbing aspect to the data mining: it treats us all as examples of a pattern rather than as individuals. Almost eleven years I wrote an article criticizing this trend. The New York Times article shows how much we've lost from what we consider essential to our identity--our individuality.

Update

This article drew six comments in a few hours--thoughtful and valid comments, which have made me set down attitudes into words. Now we can look put the attitudes under a light and see what makes sense, or doesn't, to readers.

The article contained two levels of criticism: a criticism of data mining to build up composite pictures of individuals, and a criticism of the use of data accumulated from routine transactions to manipulate those individuals.

Building up a composite picture

Of course, a company that reaches out and does any marketing has to target people. Someone who bought the O'Reilly book Even Faster Web Sites (sorry about the plug) might appreciate a notification about our upcoming Velocity conference, which was founded by the book's author and covers the same topics. Someone who bought a book on a totally different subject wouldn't want or respond to the notification. O'Reilly does this kind of targeting, like most companies, and until everybody participates in truly frictionless information exchanges, companies will have to continue doing it.

Aggregated information is useful too. Organizations that mine public data for evidence of health epidemics can identify likely sites and investigate them further. The data mining is understood to provide an approximation of the truth.

Where I see a problem is when the increasing quantity of constant information refinement shades over into a qualitative change. There's a difference between a campaign targeted to 500 likely customers and a campaign targeted to one.

At some point the composite portrait starts to look so much like a person that corporate decision makers can begin to believe it is the person. The portrait becomes like a replicant, or like the statues that came to life in myths from Pygmalion to Pinocchio.

Joseph Weizenbaum, creator of the classic Eliza program, was shocked to see that people treated his "doctor" program like a human interviewer. There were plenty of computer programs that prompted the user with questions and gave varied responses based on the answers, but none had imitated a person so realistically.

Nowadays, nobody would be drawn in by Eliza. And perhaps companies and customers alike will get used to composite portraits. Perhaps the companies will send their composite to each of us and we can update it to make it more accurate. That will be a very different world, though.

Now we can turn to the next level, manipulation.

Manipulation

I've read numerous accounts in biographies and articles about interrogations, and talked to a couple people who have undergone interrogations. I haven't been on either side of an interrogation, but I've been deposed for a court case. All these situations remind me vividly of the exchanges reported in the New York Times article.

In these exchanges, a well-armed caller is laying, like a silkscreen, a composite over the real person and trying to manipulate the result. It's not exactly a case of asymmetric knowledge (because at least in theory, a customer could also learn a lot about a company and use that knowledge to manipulate it). It's more insidious: an employee carrying out a precise initiative on behalf of a company--a machine in the service of a goal--approaching the targeted customer in an informal manner that brings out a natural, human, empathetic reaction in customer.

Interrogation always takes place in the context of an open or implied threat--there would be no reason for making the contact otherwise--but as I mentioned in the article, the interrogation goes best when the threat is raised only rarely and strategically. A feigned sympathy and heart-to-heart engagement is the path to the most desired outcome.

In a sense, now, the employee has become the replicant. He is using a careful counterfeit of human responses to induce the behavior he or she is paid to induce. This is ethical when dealing with a criminal, although even then US law limits (based on the Fourth Amendment) the gathering of relevant information by the interrogator beforehand. I question how ethical it is in a business situation, especially when exploiting information given by the customer for entirely different purposes.

Encouraging results from Peer-to-Patent

Congratulations to the organizers of Peer-to-Patent, which is carrying off one of the most audacious experiments in Internet activism in our day. A lot of ink has been spilled about Barack Obama's application of social networking techniques to presidential campaigning (and to Ron Paul's successful fund-raising before that) but Peer-to-Patent makes those achievements seem entirely run-of-the-mill.

The premise behind Peer-to-Patent, which many observers called impractical, was that thousands of experts in technical fields would flock to the site to read patent applications (if you've ever read one, you'd hike the stakes against success several notches right there) and would find prior art that would lead to rejection or restrictions on patent claims.

Well, it's working. A report released by the non-profit project in PDF format reports the data from surveys and an analysis of patents handled during the first year of the project. The sample is small (23 patents) but bears some impressive fruit.

(continue reading)

Hacking TCP/IP To Support Location Aware Services

I just received a simple proposal (which is usually the best type) from Brian McConnell, an O'Reilly author and old phone hand who has founded several telecom companies. His proposal, which follows, represents a creative linking of the GPS/location domain and TCP/IP. If you thought there was no use for IPv6, read on (but it could work with IPv4 now).

(continue reading)

Ignite Boston shows the way to beat commerce interruptus

I felt like was I drifting back to the dot-com boom last night during Ignite Boston. Movements that I saw getting stalled seven years ago seem to be finding their way forward again.

Ignite Boston, a party held every few months by O'Reilly, draws people from around the region who are interested in technology and socializing. Last night, the approximately 325 attendees packed two floors of a bar, and it's a good thing the street outside was closed off because there were plenty of celebrants out there as well, escaping the noise inside to have a conversation.

(continue reading)

The wiretapping accusation against P2P and copyright filtering: evidence that we need more user/provider discussion

I would by no means argue with celebrated law expert Paul Ohm when he suggests that cable companies and other ISPs might be breaking the federal wiretap law by doing deep packet inspection. This was the recent news from a WIRED reporter blogging from Computers Freedom & Privacy.

I will leave it up to the lawyers to decide whether the wiretap law was passed with the intent to keep providers from reducing traffic that strains their bandwidth, or from complying with requests from movie studios to prevent the unauthorized exchange of first-run films. I'll also let lawyers decide whether the ISPs are shielded by exemption that allows them to protect their service.

But I can't help observing that the same kinds of deep inspection that Ohm decries (and that permits China and other governments to censor content) is also used for spam and virus filtering. Superficial traffic analysis could perhaps, someday, identify spam and viruses, but it's currently critical to check for the signatures of malicious content. Would Professor Ohm like to personally handle the 2000% increase in email he'd get if he forced his ISP to stop filtering?

On the other hand, I wonder whether web mail services such as Hotmail, Yahoo! and Google would be guilty of wiretapping if they check traffic. After all, they are not delivering traffic to another system as Comcast is; they are terminating the traffic on their own systems, where their users access it. I'd think they have a much stronger defense, partly because the data is technically on their own systems, and partly through the claim that they need to run filters to protect these systems from viruses, or even just excessive traffic.

These dilemma suggest to me that the relationship between ISPs (or mail service providers) and customers has to change, and perhaps that the wiretap statute has to adapt. What we want is that most perplexing of legal solutions: to screen out malicious behavior and impacts that users don't like, while leaving positive and desired behavior alone.

(continue reading)

Yochai Benkler, others at Harvard map current and future Internet

Harvard's world-renowned Berkman Center for Internet & Society is celebrating its tenth anniversary with a conference called Berkman@10. I'll report here on today's sessions, which were organized as a fairly conventional symposium (although as loosely as one could run it with 450 attendees). Tomorrow will be set up as an unconference, where the audience defines most of the topics and self-organizes into small-group discussions.

• Whither peer production--wither peer production?
• The conscious commons
• What will a Harvard for six billion people look like?
• Designing for cooperation
• Seeding and modeling
• The dilemma of openness
• The Berkman Center broadens

(continue reading)

Google Friend Connect and limits to sharing

We're all tired of acquaintances tugging on us to sign up for new social networks, and of the torque we feel bouncing between the networks we're on if we can't resist the herding instinct that brings us to join them. But we wouldn't want to have just one big social network, either. That would inhibit innovation and prevent people from enjoying a site's special features and cultural uniqueness.

Google's Friend Connect, which was announced on Monday and covered by Radar as well as other sites, represents a small step toward a middle ground. It could be considered the natural succession to Google's OpenSocial, also discussed extensively on Radar. The OpenSocial API forms the basis for communications between Friend Connect widgets and the site hosting them, using lightweight Ajax and JSON protocols. Friend Connect uses the APIs provided by other sites for communication with them.

I had a little tour of Friend Connect last night at the party celebrating the opening of Google's new Cambridge office, covered in another blog.

(continue reading)

Maker Faire mimesis and open speculation

O'Reilly's Make magazine and the Maker Faire that we're hosting today and tomorrow in San Mateo, California have been described in many ways, ranging from a revival of the mid-20th-century love for Popular Mechanics magazine to an exciting new impetus for teaching children about science. During my six hours there today, I noted its strong connections to powerful and fundamental human urges toward creation, mastery, and the reproduction of our own culture.

Some of the Maker Faire centers are devoted to the kind of do-it-yourself projects shown in our magazine. Anyone from a four-year-old to a mechanically adept adult can find challenge and satisfaction at these tables. Projects in another building took a big step up, showcasing the brain children of engineers who devoted their spare time to building games and toys or aiding their communities with research projects. A number of the booths seemed to be run by Renaissance men and women who were making a living from their creative combinations of art and technology.

(continue reading)

Book review: "The Future of the Internet (And How to Stop It)"

Most of us in the computer field have heard more than our fill about the free software movement, the copyright wars, the scourge of spyware and SQL injection attacks, the Great Firewall of China, and other battles for the control of our computers and networks. But your education is stifled until you have absorbed the insights offered by comprehensive thinkers such as Jonathan Zittrain, who presents in this brand new book some critical and welcome anchor points for discussions of Internet policy. Now we have a definitive statement from a leading law professor at Harvard and Oxford, who combines a scholar's insight into legal doctrines with a nitty-gritty knowledge of life on the Internet.

You can read Zittrain for cogent discussions of key issues in copyright, filtering, licensing, censorship, and other pressing issues in computing and networking. But you're rewarded even more if you read this book to grasp fundamental questions of law and society, such as:

• What determines the legitimacy of laws and those who make and enforce them?
• What relationship does the law on the books bear to the law as enforced, and how does the gray area between them affect the evolution of society?
• What is the proper attitude of citizens toward law-makers and regulators, and how much power is healthy for either side to have?
• How can community self-organization stave off the need for heavy-handed legislation--and how, in contrast, can premature legislation preclude constructive solutions by self-organized communities?

Core questions such as these power Zittrain's tour of technology and law on today's networks. "The Future of the Internet" takes us briskly down familiar paths, offering valuable summaries of current debates, but Zittrain also tries always to hack away at the brambles that block the end of each path. Thanks to his unusually informed perspective, he usually--although not always--succeeds in pushing us forward a few meticulously footnoted footsteps.

(continue reading)

To be free, information has to be smart (comments on Chris Anderson's "Free!")

WIRED Magazine's editor in chief Chris Anderson, following up on the popularity of his Long Tail meme, theorizes in the March 2008 issue of WIRED about the modern tendency to put information online at no cost. I'll start this blog with the implications of offering free information in the computer field, and build from there to what I agree and disagree with in Anderson's article.

Anderson's taxonomy of "free" contains six models that justify giving the information away. The idea of "free as in freedom" (that is, open source information in the GPL or Creative Commons style) doesn't enter at all into his article. Is that important, given that the article is economic rationale for business? I think it's a crucial omission.

(continue reading)

Recent Posts

• O'Reilly Radar, other O'Reilly efforts win JOLT awards at SD West on March 5, 2008
• Network neutrality: code words and conniving at yesterday's FCC hearing (Part 2 of 2) on February 26, 2008
• Network neutrality: how the FCC sees it (Part 1 of 2) on February 25, 2008
• Rating the ratings, and the end of neutrality on February 8, 2008
• Educating computer users: the need for community/author collaboration (Part 2 of 2) on February 7, 2008
• Educating computer users: the need for community/author collaboration (Part 1 of 2) on February 6, 2008
• Developing an improved online environment for educating computer users on February 3, 2008
• Two tools we need to improve online information on January 31, 2008
• Survey: what benefits do online communities bring? on January 23, 2008
• Privacy 2007: Hiding in the Crowd on December 28, 2007