Cross-posted at Immizen.com as a more in depth, history/technology-focused expansion of two previous DK diaries, one by peggy and one by chicago DEM
Reported as early as Feb, 2012 in a Slate article titled “Obama’s White Whale”, project Narwhal, Obama’s top-secret campaign project until then, featuring high tech data integration and data mining techniques, promised to change the ways in which campaigns are fought and won.
It started off as part of the 2008 campaign, when early on, during the first initial steps of the campaign in 2007, each Obama campaign department started creating their own data repositories for the election and voter data they collected. They realized, albeit too late for the 2008 elections, that they were collecting unprecedented volumes of political information, but had no way to study the data together to yield the maximum benefit, which in this case meant, targeting voters in the most effective way and getting them to cast their vote. In 2008:
Even as the outside world marveled at their technical prowess, Obama campaign staffers were exasperated at what seemed like a basic system failure: They had records on 170 million potential voters, 13 million online supporters, 3 million campaign donors and at least as many volunteers—but no way of knowing who among them were the same people.
So what changed between 2008 and 2012? The implementation of two new projects:
Narwahl and Dreamcatcher.
Project Narwahl relied on a real time full data integration technology that allowed the campaign to target voters in ways previously only imagined. As t
he Slate article reported in Feb, 2012:
Permanently linking the campaign’s various databases in real time has become one of the major projects for Obama’s team this year. Full data integration would allow the campaign to target its online communication as sharply as it does its offline voter contact. When it comes to sensitive subjects like contraception, the campaign could rely on its extensive predictive models of individual attitudes and preferences to find friendly recipients… that might mean pulling email addresses only for those who had identified themselves as women on their registration forms and whose voter records included a flag marking them as likely pro-abortion rights.
The project also helped target undecided voters, as reported in the Applied Data Labs blog:
The project also helps logistically; last time around when volunteers canvassed neighborhoods they simply went door to door. Now armed with detailed maps about neighborhood residents, canvassers can skip the hardcore supporters on each side and get right to those illusive undecided voters.
As for
project Dreamcatcher, the Obama campaign was particularly secretive about this one. From the tids and bits we hear about it, the project seemed to be grounded in
Association Rule learning, an algorithm that is also
used by popular online stores such as amazon.com to make recommendations to their customers :
Hopes and dreams, and fears and frustrations, these are the marketers goldmine. They are the difference between telling you why you should buy something and making you feel that you need to buy it. You see it in the change from a laundry list of features to making a product sexy. This change in marketing is the goal of “Project Dreamcatcher”.
The article by the Applied Data Labs blog finishes with these words:
Although this article focuses on Obama’s Campaign, you need to realize the Romney camp is trying to do the same thing. The data mining of politics and personalization of political messages is just part of the larger trend in todays society.
Unfortunately for Romney, as we learned just a few days ago, his attempt at anything similar to Narwhal and Dreamcatcher,
project ORCA, was a
COMPLETE FAILURE.
Aren’t we happy we did not get THAT president?
If you want to read and commiserate with Romney for such a failed attempt at reproducing or beating Obama’s system, please read the blog entry by John Ekdahl, a web developer and blogger who signed up to be a Project ORCA volunteer, basically a Republican Romney supporter, and the story by another Romney ORCA volunteer, Fat Dave, who called his experience:
“A failure and an embarrassment. And I sensed it the night before the election, when I called the 800 number for our final conference call and got a busy signal.”
In the
blog entry by Ekdahl, comment #567 by a GOP official is an especially devastating account of the failures of Romney's project ORCA:
The Congressional campaign tried using this list to get voluteers and GOTV, but got many hangups and "why are you calling me, I'm a democrat!" I begged RPV for the usual list of registered voters sorted by precinct, but was told all of that was coming from Romney. It never did.
ORCA is a telling election-day failure for Romney that points to his managerial incompetence.There is no point in discussing the technologies of a failed system, we will therefore not indulge in trying to understand the technology behind Romney’s system, but you are free to
read more about it in this ABC article if only to feel pity for the guy or feel relieved that he did not get elected.