Skip to main content

300 diaries a day. At maybe 500 words per diary average, that's 150,000 words every day, a mid-sized book's worth. And that's not even counting the comments, which add hundreds of thousands of words. Every day.

Truly, trying to keep up with Daily Kos is like trying to take a drink from a firehose. And that pales in comparison with the fun job of trying to find something that was written yesterday, last week, or six months ago, that just might be worth looking at today.

This diary is here to help.

Update [2006-9-13 10:39:45 by dmsilev]: Adding tool #6. Thanks to jotter in the comments for reminding me about the RSS feeds.

Update [2006-9-16 19:20:23 by dmsilev]:: This diary has been converted to a page on dkosopedia: DailyKos_Search

(nb: This is a revised and expanded version of this diary, posted over the weekend)

I'm going to talk about 5 general categories of ways to cope with the mass of content we all collectively generate: the diary list, the hotlist, diary compendia, tagging, and searching. I'll talk about how each is useful (or not) for both filtering the daily diary deluge and for looking for information after the fact.

One, the diary list. We all use this, pretty much without thinking about it. Chances are, if you are reading this, it's because you saw it on the diary list. The diary list generally shows the last 20-50 (depending on your settings) diaries posted. It can also show the last 20-50 diaries that use a specific tag (more on this below). The diary list is a pretty crude filter; the only guide to content is the title of each diary. It's also useless for searching for anything more than a couple of hours old. Beneath the front-page list of diaries is a link to the Recent diary list, which is a list of the 50 most recent diaries, plus a link to the 51-100 most recent, etc. This is a good way to step back through the last day or so's worth of diaries.

The diary list is a great way to see what is being talked about Right Now, but if you are interested in specific subjects, it is less than ideal.

Two, the hotlist. Every user has a hotlist, which keeps track of three things. The first, the hotlist per se, is a list of diaries which you have flagged as interesting (for easy future reference). To flag a diary, open the diary and click on the little '+' icon after the diary's title. The diary will now show up in your hotlist (click the '-' icon to remove it). The second thing you can use your hotlist for is to subscribe to authors. To subscribe to an author, open up any of their diaries and click on the 'subscribe' link next to the author's name right below the diary title. Now, any time that author writes a new diary, a link to that diary will show up in your hotlist. The third part of the hotlist, New Replies, shows any comments which are replies to comments that you have made. This is entirely automatic, and doesn't need any manual setup.

Three, diary compendia. There are several compendia of diaries, many of which are published daily. Some of the most well-known:

  1. Jotter's high impact series, posted daily, is a list of the previous day's diaries sorted by essentially how much attention they've received. This is a great way to see at a glance the pulse of DailyKos, one day removed. It's also wonderful for ego-boosting (or crushing) to see how high your latest masterpiece has climbed on the charts.
  2. Susan G's Diary Rescue, posted daily, lists "underappreciated" diaries. These are entries which, in the opinion of the rescuers, deserve a second look. Rescued diaries are usually accompainied by a sentence or two of description, making it easy to scan for topics of interest.
  3. Sidinny's Election Race Diary Rescue is a daily list of election-related diaries. Most valuably, it is broken down first by state, and then (when appropriate) by Congressional district. Want an easy way to find out what is being written about your district? This should be your first port of call.

In addition to these daily compendia, there are several weekly and semi-regular compendia, focusing on a variety of specific topics. A (somewhat) complete list can be found in the ongoing diary series section of the FAQ.
(nb: in the comments, please list other compendia series, and I'll update this list). As with the diary list, the compendia are primarily useful for overviews of what is going on at the moment. They're not so useful for finding that one diary from 2 months ago that has the crucial bit of information you need. For that, we turn to the fourth and items on our list...

Fourth is tagging. Tags are a way of adding subject markers to a diary to make life easier for future searchers. Some tips for tagging to make it easier for people to find stuff:

  • Use first and last names, not just last names. 'Bush' isn't a very useful tag; 'George W. Bush' is more useful.
  • For diaries covering elections, tag with the standard ID for that election. Standard ID is the state abbreviation (CA, IL, etc), a dash, and then either 'Sen', 'Gov', or a number (for Senate, Governonr, and Congressional elections respectively). For Congressional elections, use 'IL-07' rather than 'IL-7'; the leading zero helps people who make sorted lists.
  • Use both generic and specific tags. If something nasty happens in a particular Iraqi city, tag with 'Iraq' and with the name of the city.
  • Tags should be seperated by commas, not spaces.
  • Readers should add additional tags that they feel are missing

A reader can get a quick list of the last several diaries to use a specific tag by typing it into the 'Tag' field above the Recent Diaries list, clicking the 'sort by Tag' button, and then the 'Set' button. This is great for getting an overview of recent uses of a tag, but for anything more sophisticated, the Search engine is needed.

Fifth, we come to Search. The Search engine is the go-to place if you're looking for stuff. Search is great for just typing in phrases and seeing what comes back, but here I'm going to talk about fancier uses for the engine.

First, a way to "subscribe" to multiple tags, ala the hotlist for subscribing to diaries. Pick a tag that you're interested in, say 'IL-06', for example. Now, go to the search link, type 'Tag="IL-06"' into the search field, change the sort order to 'Time', set the search type to 'Stories and Diaries', the 'From' field to 4 weeks, and hit Search. Now you have a list of diaries from the last month about IL-06. Now, and here is the key step, bookmark that URL. Save that long complicated URL; give it a nice name like 'IL-06 diaries', and hit that bookmark every time you want to see what's new. Since we've changed the sort type to 'time', the newest entries are on top. The beauty of this approach is that you can "subscribe" to as many tags as you want, and most web browsers will let you keep your bookmarks in a nice easily-organized place. Just make a subfolder for "DailyKos bookmarks" or whatever, and you're good to go.

Next, combining searches. Let's say that you're overdosing on Lieberman-itis, but you like the take a particular author has on the race. To grab all of the stories that, for example, mcjoan has written on the CT Senate race, use the search term 'author=mcjoan tag=CT-sen' to get this list. Bookmarking searches of this type is a good way to subscribe to an author when you aren't interested in everything they write, but just particular subjects. Say, for example, you like the meta diaries that I write, but either can't stand or don't care about anything else I've written. Rather than simply subscribing to me and manually separating the wheat from the chaff, you could search on 'tag=meta author=dmsilev' to get this list of recent meta diaries by yours truly. As in the previous paragraph, just bookmark this URL, and your "subscription" is all set.

You can exclude search results based on tags as well. Say, for example, you are following the Montana Senate race, and want to see a list of stories that are just about Jon Tester, and not Senator Burns. You would use the search term 'tag="Jon Tester" NOT tag="Conrad Burns"' to get these diaries about Tester himself. Notice that the tags are in quotation marks; that's needed because the tags have spaces in their names. Similarly, if there's an author that you like to read on most subjects, but can't stand on one particular subject, you can subscribe to a filtered version of that author. Using myself as an example (so I don't piss off anyone else...), say you don't like my repetive FAQ diaries. You would search on 'author=dmsilev NOT tag=FAQ' to get this. Again, bookmark this and you'll never have to read another of my FAQ diaries ever again...

You can also limit your search to diaries that have been chosen for Diary Rescue; these are all tagged with 'Rescued'. So, to see all of the CT-Sen diaries that have been rescued, use 'tag=CT-Sen tag=Rescued' to get this list. Fascinating: only one entry in the last four weeks. Most of the diaries that appear on the Recommended list get tagged with 'Recommeended'; this allows you to limit your search only to Recommended diaries; 'tag=Recommended tag=CT-Sen' nets this list. And, just for giggles, this search, on 'tag=Rescued tag=Recommended' plucks out all of the diaries which were rescued, and subsequently rode that rescue onto the Recommended list. Or, perhaps I should say 'the diary', since it seems to have only happened once in the last several months.

A fast way to check whether a particular story in the media has been diaried is to use the 'site' search term. This searches for any diaries that use a URL from a particular website. For example, say some columnist for the Chicago Tribune has just written a scorching anti-Bush column. You can quickly check to see if anyone has diaried it by searching on '', to see a list of diaries that refer to the Trib. Now, this isn't a useful technique for checking up on stories off the AP wire, etc. (because those stories are hosted by many different sites). For those, your best bet is to do a regular text search on some of the key words in the story. Or just scan the titles on the Recent list.

Sixth is an RSS Feed. (this section is verbatim from a comment by search-meister jotter.)

Another option worth exploring in this area is RSS (really simple syndication).

RSS is a kind of publishing mechanism, such that you can subscribe to sources you like in a more fine grained way. For instance, at daily kos you can subscribe to RSS "feeds" (as they are called) for each and every kog! For example, if you give this url to a news reader, it will inform you whenever a new diary by our host, dmsilev, appears.

(I've ommitted the leading http slash slash so you can see the full url)

Some browsers (Safari, Firefox) are RSS ready, and some very good web based and commercial software is available as well. I use Netnewswire, now owned by newsgator, and find it very useful for keeping up with sites and authors.

Now, to end this longer-than-I-thought-it-would-be diary, a plea. The fancy search tricks that I describe depend on good-quality tagging to work their magic. Please, please, please put a few seconds of thought into the tags that you choose. As outlined above, mark election-related diaries with the standard tag (or tags) for that election. If the diary is about a specific person or several people, use their name(s) as tags, but please doublecheck the spelling.


Originally posted to dmsilev on Wed Sep 13, 2006 at 06:16 AM PDT.

Your Email has been sent.
You must add at least one tag to this diary before publishing it.

Add keywords that describe this diary. Separate multiple keywords with commas.
Tagging tips - Search For Tags - Browse For Tags


More Tagging tips:

A tag is a way to search for this diary. If someone is searching for "Barack Obama," is this a diary they'd be trying to find?

Use a person's full name, without any title. Senator Obama may become President Obama, and Michelle Obama might run for office.

If your diary covers an election or elected official, use election tags, which are generally the state abbreviation followed by the office. CA-01 is the first district House seat. CA-Sen covers both senate races. NY-GOV covers the New York governor's race.

Tags do not compound: that is, "education reform" is a completely different tag from "education". A tag like "reform" alone is probably not meaningful.

Consider if one or more of these tags fits your diary: Civil Rights, Community, Congress, Culture, Economy, Education, Elections, Energy, Environment, Health Care, International, Labor, Law, Media, Meta, National Security, Science, Transportation, or White House. If your diary is specific to a state, consider adding the state (California, Texas, etc). Keep in mind, though, that there are many wonderful and important diaries that don't fit in any of these tags. Don't worry if yours doesn't.

You can add a private note to this diary when hotlisting it:
Are you sure you want to remove this diary from your hotlist?
Are you sure you want to remove your recommendation? You can only recommend a diary once, so you will not be able to re-recommend it afterwards.
Rescue this diary, and add a note:
Are you sure you want to remove this diary from Rescue?
Choose where to republish this diary. The diary will be added to the queue for that group. Publish it from the queue to make it appear.

You must be a member of a group to use this feature.

Add a quick update to your diary without changing the diary itself:
Are you sure you want to remove this diary?
(The diary will be removed from the site and returned to your drafts for further editing.)
(The diary will be removed.)
Are you sure you want to save these changes to the published diary?

Comment Preferences

, Meteor Blades, arlam, Mark Sumner, Ed in Montana, opendna, dwellscho, Maccabee, coral, RF, Nathan in MD, Mogolori, jotter, dalemac, natasha, casamurphy, AlanF, Tulip, left of center, yerioy, Joan McCarter, musing85, jk, gaspare, RunawayRose, bosdcla14, Robespierrette, Avila, krwada, RAST, Debby, Heather in SFBay, SanJoseLady, wintersnowman, rhubarb, bawbie, Raven Brooks, marjo, SallyCat, Carnacki, reklemrov, twistandshout, Poika, Jerome a Paris, Plan9, sardonyx, kissfan, smintheus, Hose B, 88kathy, Babsnc, litho, mentaldebris, Shadan7, nyceve, Loquatrix, CoolOnion, Rupert, Glic, ksh01, Scoopster, mkfarkus, roses, michelle, javelina, MissAnneThrope, Ignacio Magaloni, jukeboxgrad, Bearpaw, MJB, ornerydad, Swordsmith, FlyLeft, Cedwyn, jted, lirtydies, high uintas, Alna Dem, litigatormom, Chrisfs, Redfire, mayan, dejavu, annan, rcvanoz, sockpuppet, jakyra, missliberties, Chicago Lulu, grayslady, churchylafemme, homo neurotic, dwahzon, On The Bus, Steven Payne, Tillie630, horsewithnoname, RenaRF, Eddie Haskell, 313to212, 4jkb4ia, walkshills, One bite at a time, bwintx, schuylkill, SanDiegoDem, Marianne Benz, HK, DarkSyde, American in Kathmandu, boran2, thereisnospoon, Spam Spam Bacon Spam, oortdust, Gowrie Gal, memophage, polydactyl, MichDeb, Fabian, maybeeso in michigan, DCleviathan, Bluesee, farleftcoast, 3goldens, kingubu, Green Zombie, enough, arnott, PBen, Jashugan, Militarytracy, Luetta, Far left coast, Ajax the Greater, Brooke In Seattle, Bouwerie Boy, Sweet Georgia Peach, lennysfo, lcbo, dunderhead, GreyHawk, Ambrosius, annefrank, evilpenguin, JellyPuddin, bmaples, JenThinks, flernk, wulidancer, wiscmass, mbcarl, Lisa Lockwood, sbdenmon, word is bond, pdknz, dsteffen, Team Slacker, doe, JanL, proudprogressiveCA, Land of Enchantment, melvin, Asinus Asinum Fricat, lgmcp, SSMir, motherlowman, occams hatchet, Coherent Viewpoint, mspicata, pico, dannyinla, trashablanca, nyarlahotep, Do Tell, tommymet, sherlyle, NOLAWitch, BlueInARedState, emeraldmaiden, cazimi, Ellicatt, Gorette, demondeac, rl en france, cookseytalbott, borkitekt, Texas Blue Dot, Gasonfires, kck, Irishkorean, SherriG, birdbrain64, global citizen, paiges, StrayCat, Lashe, arbiter, imabluemerkin, condoleaser, plf515, armadillo, dirtfarmer, a small quiet voice, Timothy J, Clive all hat no horse Rodeo, va dare, MarketTrustee, Dreaming of Better Days, kurt, scoff0165, wiscow, Lew2006, Darrell J Gahm, Granny Doc, kidneystones, Temmoku, slksfca, Callandor, Pandoras Box, SC damn yankee, Monique Radevu, turquoise, FoundingFatherDAR, Mike McL, jennyjem, Polacolor, anotherdemocrat, Cronesense, sistersilverwolf, Cat Whisperer, oscarsmom, JESchwartz, FWIW, whl, offgrid, DrWolfy, Allogenes, Mary Mike, Ticonderoga, Jimdotz, ilex, joyful, Blue South, Buffy Orpington, mandot, lizpolaris, stratocasterman, Heyroot, mcc777c2, KenLeft

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site