In my May 13 diary, "What does Daily Kos accomplish?", I began with these questions:
- "What purposes does DK achieve?"
- "How well does it succeed in them?"
- "How can we quantify that?"
- "Which purposes are in tension with each other?"
- "Which ones are most important?"
saving all but the first and third questions for later. As luck would have it, on the same day, another diarist simply declared "DailyKos sucks". Well, that's straightforward, but a bit unscientific.
What purpose does DK achieve? In my last diary, I came up with 46 possibilities, from "allow people to rant" to "convince people to run for office". One commenter turned to Daily Kos's mission statement, which is to help elect Democrats. I'd want to at least qualify "Democrats" with "progressive", but I responded as follows:
Okay, so does it actually help elect Democrats and if so, how do we know? That's where the measurement issues come in. How do we know that this site is effective at recruiting canvassers or donations? How do we measure the effect that our strategy discussions have on the actual world? And if this site's mission is electing Democrats, do any of the other things that it does interfere with that mission?
And now I'll answer myself: Given the tools we have, and the limited use we put to them, it's impossible to determine to what extent the existence of Daily Kos makes a difference in electing Democrats. It's quite possible that DK turned the tide in, say, Jim Webb's squeaker victory, both by coaxing a few canvassers out the door and, earlier, by publicizing Allen's racism. But we can't know. One could imagine setting up an interface that facilitates tabulation of doors knocked or money raised, but it doesn't exist yet.
So electing Democrats is supposedly our central purpose... and yet, we have no way of measuring our effect? We don't know whether we're getting people into office, and we don't know how many people we're putting out on the streets?
And what if our other goals interfere with our main one? For instance, we pride ourself on building a lively, albeit rowdy, community here, with the help of "pootie" pictures, collages, and in-jokes. But what if these actually worked against helping to get progressive Democrats elected? I'm not motivated by puritannical thinking, but by our limited amount of time and real estate, which forces us to do some zero-sum thinking. If slapstick satire diaries and filler comments consistently overwhelm action diaries and informative comments, is the site really working? Sure, it's comforting to tell oneself that satire serves a purpose, perhaps drawing people who wouldn't visit otherwise. But without measurements, or defining goals, it verges on the Texas sharpshooter fallacy, where someone fires a gun and then draws targets around the bullet holes, and the person stumbling across the holes later concludes that s/he must have been quite a good shot.
Last year, stephdray called the community out with the bold declaration DailyKos Fiddles while America Burns. I'd love to just quote the whole thing, since she's one heck of a writer, but I don't have the space. To summarize, she pointed out that while we were focusing on the salacious details of Cheney's hunting accident, we had lost the opportunity to push for a filibuster to block Alito's nomination. She held the leadership of the blog especially responsible for this lost opportunity, but she also suggested that individuals should consider the impact of their diaries on the real world when deciding what to write about.
Unfortunately, two other excellent writers used their gifts to blunt Steph's points. Hunter, in I, Hunter, Am Your Leader, suggested, with a heavy dose of snark, that any attempt at leadership would be rejected by the community. Jerome a Paris, in DailyKos is doing fine, took a more serious tone, and brought up some worthwhile distinctions between the relatively few frequent visitors and the hordes of less-frequent readers. But his main purpose was to demonstrate that Steph's objections could be laid to rest, and in doing so, he turned here and there to mild Panglossianism (multiple recommended diaries on the same subject serve as a useful mirror of our interest) or unproven assertions ("the collective mind of dKos... flags the best diaries or the most important information") that made no attempt to define criteria: How do we know that the best diaries are being brought to people's attention when we aren't even defining criteria to measure "best"?
I think it does make sense for each of us, individually, to define explicit criteria for judging the diaries we read or write, and then determine how well those criteria are met for the diaries on the recommended list, or on the rescued list, or on the diaries that we end up opening for whatever reason.
Some criteria are easily quantifiable. One might call for X letters to the editor, or Z dollars. I think such goals are great. You might exceed your goal, you might make it, you might fall short, but you learn something in the process.
Last year, I conducted an interview with Paul Jay of Independent World Television, a network developing a daily news show, The Real News. IWT is determined to reach a mass audience with news that really matters, and by refusing to take money from corporations or government agencies, it addresses our chief complaints about media bias. In 2005, there had been a number of diaries on IWT, two of which drew hundreds of recommendations and enthusiastic comments, initiating a wave of publicity that IWT hadn't been expecting.
Great setup, right? Once again, bring the good news about IWT to the blogosphere, and prove the power of the blogosphere to IWT. So I put a lot of work not only into the interview, but the marketing. I knew that worthwhile diaries often disappeared, and even auxiliary listservs (e.g., election integrity and reform) never attracted enough visitors to push a diary onto the recommended list. So I looked through every Daily Kos diary ever written on IWT and harvested the e-mail addresses of the people who had posted diaries, comments, recommendations, or tips, and had listed addresses in their profiles. This was a laborious, nonautomatable process (perhaps not a bad thing). I informed people on the list that unless they wanted to be removed from my mailing list, I would inform them when each installment was up. I had very few "remove" requests, and they were fairly polite. Of course, many people never wrote back. But I did get many enthusiastic replies.
I posted the five installments of the interview, "Go big or go home", during the first week of 2007, and analyzed whether the series had satisfied my criteria for success.
In terms of content, I was satisfied that my diary:
- offered new information , since no one in either the blogosphere or the mainstream media had written about IWT in months
- offered the words of a compelling spokesperson; Jay knows how to captivate an audience
- addressed a subject that had been shown to be of interest to the Daily Kos community
- suggested an action as simple as visiting the IWT site and possibly making a donation
- contained one picture per diary to attract attention
That much I could control. Now did the attention paid to the diaries meet my criteria (goals)? Not really:
- They didn't land on the recommended list, and only the last installment landed on the rescued list.
- They attracted perhaps 20% of the comments and recommendations that the highest-impact 2005 IWT diaries did.
- They attracted perhaps 5% of the attention paid that week to (a) a well-known blogger for her defense of an alleged troll (b) that blogger's later mea culpa, (c) another well-known blogger's humorous observations on the whole exchange.
- They didn't have a huge amount of reverberation beyond Daily Kos or within it (didn't see many comments that reference IWT or my diaries)
However, they certainly got a wider readership (both numerically and in a geographic sense) than they would have received had I posted them on a physical billboard or handed them out to passersby. jotter's diaries showed that:
- one of the installments was read by 205 registered users
- 4 of the 5 installments landed on the middle of the high-impact list, somewhere around the 60th most noticed diaries for the day of publication
And I did get e-mail from one writer for Mother Jones, which I found very exciting until I learned that his editor wanted him to sit on the story for as long as a year.
I've had similar experiences with my ongoing series on progressive talk radio (e.g., Grassroots Meet Rainmakers: Boston Progressive Radio).
Now, I would have never heard of IWT or progressive radio at all if it hadn't been for DK, so clearly the site is useful. But since the site is only achieving a fraction of its potential success in getting new, "actionable" information to people, I wonder if we can make our site better, both through our individual behavior and through technical changes to the site.
I propose some effectiveness tips that I myself try to adhere to:
- If you're not registered, register. It takes two minutes, you lose no privacy, and when you read a diary, your visit increments the number of registered users who've visited it.
- If you are registered and don't have your e-mail address in your profile, add it (in spam-proof format: mine is alanfordean AT-SIGN yahoo DOT com). If you don't want to make your main address public, you can take 15 minutes to obtain a free dedicated e-mail account for this purpose. The effect on your privacy is negligible, and now you can actually be part of a social political network.
- Add yourself to the Frappr map for Daily Kos so that people can get a hold of you if they're looking for someone in a particular area. Kagro X has done this, and I have as well.
- Try cutting down on the number of diaries you visit just to see whether they'll confirm your suspicion (based on the title or author name) that they'll be truly awful. If you do visit them, don't spend much time there.
- If a diary looks like it's about to be a rant that won't teach you anything, try looking elsewhere to see whether there's something with more value.
- If a diary has already attracted some threshold number of comments, think twice about adding another unless it contains important information.
- Specialize in a subject, and scan the diary list for that subject. You'll be doing your part to make yourself an expert on that subject and make sure that it gets the attention it deserves.
As for technical improvements, I'd love it if the admins would listen to the most frequently raised request and expand the recommended list. For all intents and purposes, it has stayed the same size as the readership has increased by a tremendous amount. There can't be any technical reason why going from a list size of ten to twenty could be a problem.
I could go on, but I've already written at length (despite much pruning), and I'm very curious to hear what your responses are.