the tags that defined us

We've had tags since October 10, 2005. Heading the list of most commonly used tags are "George W. Bush" and "Iraq". This is no surprise; dKos is a political blog whose stated purpose is to help get Democrats elected, and "George W. Bush" and "Iraq" are the two central reasons why it's important to do so. While these two tags reflect our focus over the entire time period since October 2005, how have our foci changed? I've broken down tag counts on a monthly basis since October 2005 (up to Feb 10, 2007, when I got my last tag dump), and present them below.

Each table starts with the year and month in yyyymm format, and the number of qualifying diaries for that month. I only ran things for tags used 10 or more times, and a diary qualifies if any of its tags pass that threshold. Each row contains a count and a percentage of total diaries tagged with that row's tag. Note that the "recommended" tag is excluded, and I'll talk about that later..

200510    5997 diaries

1	1416	23.61	george w. bush
2	1194	19.91	plamegate
3	1070	17.84	patrick fitzgerald
4	1062	17.71	iraq
5	990	16.51	valerie plame
6	986	16.44	karl rove
7	954	15.91	scooter libby
8	768	12.81	harriet miers
9	724	12.07	dick cheney
10	702	11.71	supreme court
11	480	8.00	republicans
12	418	6.97	judith miller
13	394	6.57	joseph wilson
14	380	6.34	2006
15	372	6.20	bush regime
16	370	6.17	democrats
17	344	5.74	media
18	320	5.34	indictments
19	306	5.10	tom delay
20	300	5.00	cia
21	300	5.00	elections
22	284	4.74	fitzmas
23	274	4.57	senate
24	268	4.47	hurricane katrina
25	266	4.44	bush administration

Right off the bat, you can see what held our attentions. While George is still leading the way, Iraq has dropped to 4, as Plamegate and Harriet Miers tags are strong. Karl Rove is up there too, which, as you'll see, holds true for most every month. Note that Hurricane Katrina is at 24. Tags didn't start until October 2005, but that doesn't mean that you can't tag diaries written before that date. And many did. A total of 706 diaries written in September 2005 received after-the-fact tags; let's look at them real quick.

200509     706 diaries

1	560	79.32	hurricane katrina
2	172	24.36	new orleans
3	156	22.10	george w. bush
4	148	20.96	fema
5	74	10.48	media
6	56	7.93	bush regime
7	52	7.37	iraq
8	50	7.08	michael brown
9	36	5.10	elections
10	34	4.82	hurricane ivan
11	34	4.82	congress
12	34	4.82	2006
13	32	4.53	cheers and jeers 07-09 2005
14	32	4.53	cindy sheehan
15	32	4.53	cheers and jeers
16	30	4.25	emergency preparedness
17	30	4.25	energy
18	28	3.97	patrick murphy
19	28	3.97	supreme court
20	28	3.97	rick santorum
21	28	3.97	9-11
22	26	3.68	bob casey
23	26	3.68	tom delay
24	26	3.68	republicans
25	22	3.12	john roberts

Heckuva job there, Brownie...

200511    7442 diaries

1	2410	32.38	iraq
2	2210	29.70	george w. bush
3	696	9.35	dick cheney
4	566	7.61	republicans
5	556	7.47	torture
6	546	7.34	war
7	514	6.91	democrats
8	458	6.15	2006
9	434	5.83	john murtha
10	426	5.72	congress
11	400	5.37	iraq war
12	388	5.21	samuel alito
13	388	5.21	senate
14	336	4.51	supreme court
15	336	4.51	scooter libby
16	326	4.38	lies
17	326	4.38	karl rove
18	306	4.11	media
19	296	3.98	plamegate
20	290	3.90	valerie plame
21	272	3.65	elections
22	264	3.55	bush administration
23	262	3.52	cia
24	258	3.47	california
25	250	3.36	2008

In November the war jumps to the forefront again, and the torture issue becomes prominent. Harriet is gone, replaced by Alito, and Plamegate lingers. Well, I'm not going to give you a full running commentary for each month, you can do that yourself. Here are the rest of the months, with an occasional comment.

200512    6489 diaries

1	2232	34.40	george w. bush
2	1526	23.52	iraq
3	632	9.74	nsa
4	438	6.75	torture
5	406	6.26	christmas
6	396	6.10	spying
7	390	6.01	congress
8	382	5.89	republicans
9	376	5.79	2006
10	370	5.70	war
11	368	5.67	wiretaps
12	356	5.49	impeachment
13	314	4.84	democrats
14	306	4.72	patriot act
15	282	4.35	domestic spying
16	282	4.35	senate
17	280	4.31	dick cheney
18	254	3.91	bush administration
19	248	3.82	fisa
20	248	3.82	iraq war
21	236	3.64	war on christmas
22	230	3.54	tom delay
23	222	3.42	constitution
24	220	3.39	elections
25	214	3.30	media

Domestic spying, liberals hating christmas.

200601    7925 diaries

1	2376	29.98	george w. bush
2	2298	29.00	samuel alito
3	1004	12.67	iraq
4	968	12.21	jack abramoff
5	950	11.99	filibuster
6	828	10.45	democrats
7	782	9.87	supreme court
8	624	7.87	nsa
9	498	6.28	republicans
10	484	6.11	2006
11	428	5.40	senate
12	386	4.87	media
13	364	4.59	congress
14	348	4.39	bush administration
15	344	4.34	corruption
16	332	4.19	tom delay
17	316	3.99	constitution
18	310	3.91	2006 elections
19	292	3.68	iran
20	288	3.63	scotus
21	288	3.63	impeachment
22	280	3.53	al gore
23	278	3.51	war
24	274	3.46	john kerry
25	260	3.28	spying

Oh, lookey there. Iran.

200602    7159 diaries

1	1382	19.30	george w. bush
2	1116	15.59	dick cheney
3	972	13.58	iraq
4	708	9.89	bush
5	548	7.65	2006
6	532	7.43	democrats
7	424	5.92	nsa
8	376	5.25	republicans
9	352	4.92	congress
10	332	4.64	2006 elections
11	316	4.41	media
12	314	4.39	senate
13	292	4.08	sotu
14	280	3.91	bush administration
15	272	3.80	iran
16	268	3.74	uae
17	240	3.35	paul hackett
18	214	2.99	ports
19	214	2.99	jack abramoff
20	212	2.96	elections
21	208	2.91	house
22	208	2.91	war
23	202	2.82	ohio
24	202	2.82	karl rove
25	194	2.71	harry whittington

Dead-eye Dick. Hackett. Ports. Hey, Iran again.

200603    7780 diaries

1	2150	27.63	george w. bush
2	1570	20.18	iraq
3	642	8.25	russ feingold
4	608	7.81	democrats
5	546	7.02	censure
6	500	6.43	republicans
7	464	5.96	2006
8	386	4.96	war
9	362	4.65	2006 elections
10	356	4.58	congress
11	352	4.52	senate
12	342	4.40	immigration
13	336	4.32	abortion
14	328	4.22	media
15	314	4.04	iran
16	308	3.96	dick cheney
17	292	3.75	impeachment
18	260	3.34	iraq war
19	256	3.29	bush administration
20	242	3.11	nsa
21	214	2.75	9-11
22	210	2.70	elections
23	202	2.60	politics
24	192	2.47	economy
25	186	2.39	corruption

Russ. Censure.

200604    7759 diaries

1	2182	28.12	george w. bush
2	1408	18.15	iraq
3	912	11.75	iran
4	686	8.84	immigration
5	532	6.86	democrats
6	492	6.34	war
7	472	6.08	dick cheney
8	448	5.77	republicans
9	438	5.65	2006 elections
10	430	5.54	bush administration
11	418	5.39	2006
12	386	4.97	donald rumsfeld
13	352	4.54	congress
14	342	4.41	oil
15	338	4.36	valerie plame
16	326	4.20	tom delay
17	298	3.84	media
18	282	3.63	iraq war
19	272	3.51	environment
20	264	3.40	senate
21	258	3.33	scooter libby
22	250	3.22	economy
23	230	2.96	karl rove
24	228	2.94	impeachment
25	220	2.84	republican party

Iran up to number 3.

200605    8363 diaries

1	2170	25.95	george w. bush
2	1092	13.06	iraq
3	964	11.53	nsa
4	710	8.49	immigration
5	592	7.08	democrats
6	562	6.72	2006 elections
7	498	5.95	republicans
8	470	5.62	iran
9	434	5.19	congress
10	414	4.95	al gore
11	402	4.81	karl rove
12	402	4.81	war
13	386	4.62	2006
14	354	4.23	cia
15	344	4.11	dick cheney
16	330	3.95	media
17	322	3.85	bush administration
18	278	3.32	hillary clinton
19	270	3.23	stephen colbert
20	270	3.23	elections
21	268	3.20	2008 elections
22	262	3.13	senate
23	258	3.09	meta
24	244	2.92	corruption
25	230	2.75	impeachment

Meta pops up, and so begins our navel-gazing. By the way, notice who's regained the top spot and held it the past half year?

200606    8797 diaries

1	1922	21.85	iraq
2	1628	18.51	george w. bush
3	906	10.30	yearlykos 2006
4	644	7.32	democrats
5	638	7.25	republicans
6	628	7.14	2006 elections
7	538	6.12	2006
8	494	5.62	congress
9	434	4.93	media
10	434	4.93	meta
11	434	4.93	war
12	422	4.80	senate
13	390	4.43	karl rove
14	386	4.39	elections
15	374	4.25	ann coulter
16	364	4.14	al gore
17	358	4.07	dick cheney
18	320	3.64	gay marriage
19	294	3.34	environment
20	284	3.23	global warming
21	284	3.23	iraq war
22	280	3.18	politics
23	268	3.05	daily kos
24	262	2.98	joe lieberman
25	260	2.96	2008 elections

Yearlykos, and, oh no, it's Joe.

200607    8596 diaries

1	1960	22.80	george w. bush
2	1780	20.71	israel
3	1350	15.70	iraq
4	1340	15.59	lebanon
5	728	8.47	joe lieberman
6	716	8.33	war
7	694	8.07	2006 elections
8	622	7.24	hezbollah
9	590	6.86	2006
10	582	6.77	middle east
11	562	6.54	ned lamont
12	538	6.26	democrats
13	526	6.12	ct-sen
14	490	5.70	senate
15	472	5.49	republicans
16	436	5.07	iran
17	362	4.21	palestine
18	358	4.16	meta
19	342	3.98	congress
20	326	3.79	lieberman
21	310	3.61	elections
22	306	3.56	connecticut
23	302	3.51	media
24	268	3.12	politics
25	264	3.07	snark

Israel-Lebanon. Ned.

200608    9543 diaries

1	2050	21.48	george w. bush
2	2022	21.19	joe lieberman
3	1854	19.43	ned lamont
4	1710	17.92	iraq
5	1210	12.68	ct-sen
6	1198	12.55	2006 elections
7	912	9.56	israel
8	864	9.05	2006
9	722	7.57	lebanon
10	722	7.57	senate
11	700	7.34	democrats
12	658	6.90	republicans
13	630	6.60	terrorism
14	558	5.85	connecticut
15	516	5.41	war
16	482	5.05	hurricane katrina
17	460	4.82	iran
18	444	4.65	hezbollah
19	444	4.65	congress
20	408	4.28	dick cheney
21	394	4.13	elections
22	374	3.92	lieberman
23	374	3.92	george allen
24	348	3.65	snark
25	348	3.65	meta

Ned and Joe, of course.

200609   10396 diaries

1	2952	28.40	george w. bush
2	1748	16.81	iraq
3	1536	14.77	torture
4	1392	13.39	2006 elections
5	1372	13.20	9-11
6	1070	10.29	abc
7	878	8.45	democrats
8	784	7.54	republicans
9	778	7.48	senate
10	758	7.29	2006
11	756	7.27	the path to 9-11
12	694	6.68	disney
13	672	6.46	congress
14	564	5.43	terrorism
15	490	4.71	george allen
16	488	4.69	war
17	478	4.60	bill clinton
18	444	4.27	media
19	438	4.21	house
20	430	4.14	iran
21	418	4.02	elections
22	412	3.96	dick cheney
23	408	3.92	propaganda
24	398	3.83	jim webb
25	396	3.81	war on terror

The Path TO 9-11, and all sorts of election tags.

200610   12807 diaries

1	2476	19.33	2006 elections
2	2256	17.62	george w. bush
3	2224	17.37	iraq
4	1870	14.60	mark foley
5	1608	12.56	2006
6	1230	9.60	republicans
7	1128	8.81	house
8	1064	8.31	senate
9	1062	8.29	congress
10	938	7.32	democrats
11	890	6.95	dennis hastert
12	826	6.45	elections
13	530	4.14	election
14	520	4.06	republican party
15	514	4.01	foley
16	484	3.78	polls
17	482	3.76	jim webb
18	476	3.72	war
19	474	3.70	meta
20	446	3.48	torture
21	438	3.42	north korea
22	436	3.40	george allen
23	434	3.39	snark
24	402	3.14	va-sen
25	402	3.14	corruption

Elections takes top spot. Note that if you add the "Mark Foley" and "Foley" tags, they jump up to number 2. The Foley tags have since been combined.

200611   11470 diaries

1	2726	23.77	2006 elections
2	2278	19.86	iraq
3	2162	18.85	george w. bush
4	1150	10.03	democrats
5	966	8.42	senate
6	752	6.56	republicans
7	750	6.54	2006
8	750	6.54	2008 elections
9	674	5.88	congress
10	602	5.25	nancy pelosi
11	596	5.20	election
12	588	5.13	house
13	564	4.92	elections
14	558	4.86	meta
15	468	4.08	jim webb
16	434	3.78	iraq war
17	418	3.64	war
18	410	3.57	snark
19	406	3.54	dick cheney
20	402	3.50	howard dean
21	400	3.49	gotv
22	358	3.12	media
23	350	3.05	john kerry
24	338	2.95	election integrity
25	328	2.86	va-sen

Pelosi. We won...

200612    7989 diaries

1	2566	32.12	iraq
2	2028	25.38	george w. bush
3	930	11.64	2008 elections
4	526	6.58	barack obama
5	502	6.28	meta
6	490	6.13	war
7	482	6.03	iraq war
8	442	5.53	impeachment
9	412	5.16	john edwards
10	374	4.68	democrats
11	358	4.48	iraq study group
12	348	4.36	snark
13	344	4.31	president
14	336	4.21	congress
15	324	4.06	iran
16	320	4.01	community
17	310	3.88	hillary clinton
18	304	3.81	christmas
19	290	3.63	dick cheney
20	286	3.58	media
21	276	3.45	republicans
22	274	3.43	global warming
23	266	3.33	israel
24	266	3.33	al gore
25	250	3.13	open thread

...and the run-up to 2008 begins.

200701    9459 diaries

1	3732	39.45	iraq
2	3198	33.81	george w. bush
3	1074	11.35	2008 elections
4	910	9.62	iraq war
5	884	9.35	iran
6	744	7.87	surge
7	684	7.23	escalation
8	670	7.08	barack obama
9	660	6.98	war
10	640	6.77	hillary clinton
11	544	5.75	congress
12	526	5.56	dick cheney
13	500	5.29	president
14	480	5.07	democrats
15	474	5.01	impeachment
16	462	4.88	sotu
17	442	4.67	global warming
18	432	4.57	john edwards
19	432	4.57	meta
20	386	4.08	health care
21	350	3.70	nancy pelosi
22	350	3.70	israel
23	350	3.70	republicans
24	340	3.59	media
25	338	3.57	john mccain

The war/surge/escalation, and, hey, over there, it's Iran.

200702    2859 diaries

1	966	33.79	iraq
2	468	16.37	george w. bush
3	344	12.03	iran
4	308	10.77	john edwards
5	266	9.30	meta
6	234	8.18	barack obama
7	218	7.63	iraq war
8	210	7.35	war
9	194	6.79	2008
10	194	6.79	hillary clinton
11	174	6.09	2008 elections
12	166	5.81	global warming
13	164	5.74	congress
14	156	5.46	dick cheney
15	148	5.18	senate
16	136	4.76	democrats
17	122	4.27	al gore
18	116	4.06	republicans
19	114	3.99	wesley clark
20	104	3.64	climate change
21	102	3.57	israel
22	102	3.57	media
23	98	3.43	environment
24	96	3.36	joe biden
25	92	3.22	community

And this is about where we are today.

Recommended
I mentioned that I didn't include "Recommended" tags. Over the past year, the number of "Recommended" diaries has been about 10 percent of each month's total, and I wanted to do a little study of what kinds of diaries make the Recommended list. In reading the tagging / folksonomy / IA literature, I found a paper by one of the guys at Raw Sugar, which was a site that tried to... well, I'm not sure what, exactly, their business plan was, and others didn't either, because it went out of business. But their idea was that they wanted to use clustering techniques to impose a hierarchy on tagged resources to implement "directed search."  Well, it's not important now, but it did lead me to break down tags into what they call "cotags," and which I will call "tag pairs," or "tag bigrams." Again, that's not important now. What it does, though, is use relative tag pair counts to determine an appropriate cutoff point for tag clustering, and it's something that might be very useful for dK4. I'll talk a little bit about this later this week. For now, though, take a look at one of Raw Sugar's demo sites before it goes away.