new tag and tag search pages working now

A quick announcement before I drive off from Ann Arbor to Toronto, eh, to see my nephew's hockey tournament.

For those of you like me who are becoming increasingly obsessed with tags, ontologies, taxonomies, folksonomies, and their application to dailykos, I have a couple of new pages for you at meta.dkosopedia.com.

First, each day's tag data run now produces a new tags page, available here.

Second, you can find the beginnings of a tag search facility here. It's a little raw right now (and that page's nav bar is broke), but is still pretty useful. I'll juice it up in the next few days, adding tag counts and maybe some limited regex capability. I'll also try returning "related" tags, for some definition of related.

In any case, have fun...

•  How flippin' cool! Thanks. n/t

  by Gasonfires on Fri Dec 29, 2006 at 11:01:11 AM PDT

•  Go well, Centerfielder

  and travel safely. I like the looks of the little tag search tool! Thanks.

  by monkeybiz on Fri Dec 29, 2006 at 11:01:20 AM PDT

•  Lookin' good.

  ... now I feel like one of my clients, having been given something useful, demanding something more.

  If the tag sorts could return things in date order, newest first, that would be soooo cool.

  by BlaiseP on Fri Dec 29, 2006 at 11:14:26 AM PDT

  •  That's

    something I can't easily do right now, but it'll be a piece of cake in dKos4. Good suggestion.

    Although...

    I guess I can take an old db dump, and the daily processing, and kinda sorta merge them, and...

    Ok, my interest has been piqued, and while it'll be a back-burner thing, it'll be in my queue.

    by The Centerfielder on Fri Dec 29, 2006 at 06:33:21 PM PDT

•  good job

  already bookmarked.

  Now we have to spread the word! Any time someone says "I looked for a diary on this but I couldn't find one" give 'em these links. They work great!

  by dougymi on Fri Dec 29, 2006 at 11:45:22 AM PDT

•  Awesome!

  Speeds up my Tagging escapades immensely.

  Maybe one-offs should be excluded altogether from this search function, since they could swarm some searches, and we want to discourage their re-use, in favor of the 'popular' tags with the same meaning, or more accepted terminology. The lists brought up with this search don't differentiate. Including the (xxx) after each, or listing them in order of popularity, not alphabetically, may help with this.

  This is a fantastic tool that I'll be linking to in comments. Thanks CF!

  by Halcyon on Fri Dec 29, 2006 at 12:58:52 PM PDT

•  There may be a glitch.

  I searched for 'Saddam', then clicked on a few obvious one-offs, removed/corrected them, then searched for 'Saddam' again, and those removed tags came up again on the search.

  by Halcyon on Fri Dec 29, 2006 at 01:50:19 PM PDT

  •  This isn't dynamic

    It's based on the alltags page that I download and process each night, so changes won't be reflected until the next night. It's how it has to work for now.

    Yes, I intend to include each tag's diary count on a next pass. And while I like the results sorted alphabetically, I'll try to add a checkbox to return results sorted by frequency. At the very least I can either 1) segregate one-offs to a separate section at the bottom of the page, or 2) add a "min diary count" parameter, and only return those results which are greater than or equal to that min.

    by The Centerfielder on Fri Dec 29, 2006 at 06:18:19 PM PDT

    •  I've been using your search for 5 hours

      now (yeah, I read here most of the day; sick), and it's wunderbar. I figured shortly after posting the above that you were working off of your daily database.

      The depressing thing about it is when I search for a term like '2008' and find that there are already 15 different tags for  the equivalent of '2008 presidential candidates'. It's amazing that people come up with perhaps every possible variant in such short order. (I think '2008 presidential primaries' should be reserved for use in 2008, starting with Iowa caucus & NH, or whatever the new order is, or for diaries about the changes of dates/sequences of the state primaries.)

      Next we will need th remapping software to use along with your search. We can then copy and paste tags from a search into a remapper. It will make it easy to work on a group of tags on one subject, and quickly winnow them down.

      Happy New Year.

      by Halcyon on Fri Dec 29, 2006 at 07:40:48 PM PDT

    •  Wow, you're up early, and busy!

      Fantastic!

      Next improvement suggestion: instead of having the polls above the fold, put the Tags above the fold. It makes it easier to correct them (although sometimes one should actually read the diary first ;–), and it eases the researcher's task because he can see the set of Tags to learn more about diary content without having to click and scroll through it. Seeing the combination of Tags narrows down the search parameters. Of course it would be great, eventually, to be able to search for a diary using combinations of Tags. Anyway, just a thought, since most polls are snark and of little value.

      by Halcyon on Sat Dec 30, 2006 at 05:06:23 AM PDT

      •  Again I agree

        and that's a ct thing. I'll suggest that in my next email to him. Thanks.

        What do you think we should do about disambiguating the two Barbara Bushes? Cute as it is, I don't think "Not Jenna" is the best solution. Perhaps mom Barbara should be "Barbara (Babs) Bush", but that's asking alot for people to remember.

        by The Centerfielder on Sat Dec 30, 2006 at 06:57:10 AM PDT

        •  How about Babs/Not Jenna: Barbara (Mrs.) Bush?

          And 'Barbara (twin) Bush, or 'Barabra (Jr.) Bush? And if Jr. marries and changes her name? What then?

          [My sister bears the same name as our mother. My sister uses her middle initial, which mom never did. 'Jr.' is often used by people in conversation to clarify, since they both are in the same profession. Even so there are inaccuracies on the internet, where others have confused them.]

          It's likely one we'll be having to sort out behind the scenes on a continuing basis, unless the search function can eventually incorporate a method for indicating the Approved Tag, as in, e.g. the Tag search brings up 'Barbara Bush' and points to the two separate Tags that distinguish between them in a second column, with "}" between, like an outline?
          .......
          Is it a feasible idea to create a remapping program that could be used in combination with your search, having them in separate tabs, and cutting and pasting sets of tags that will all be remapped onto one approved tag, then hitting 'enter' and voila. Could an ajax editor do this, and only be accessible to Tag editors?

          There are many cases in which there are multiple variants which could be changed without needing to read all of the diaries, and it would be a heck of a lot quicker and easier to just change them from the Tag database. But, there would have to be a way to change the tags on each of the diaries to which they are attached. Is this possible via software, or will we have to manually change on each diary?
          ......
          OK, back to adding 'Saddam's execution' to diaries today. Check out how many there are when you update tomorrow!

          by Halcyon on Sat Dec 30, 2006 at 07:28:54 AM PDT

          •  Yes

            I've been thinking long and hard about the remapping program. One consideration is authority and responsibility. I was trying to use the dKosopedia "tags:" namespace as a place for people to make pages a program can parse up. That way their dKosopedia usernames are associated with changes, but it's a cumbersome way of entering data, and no one seemed anxious to use that method. However, I think I can create a form for all sorts of tag mapping/remapping and require a login using the dKosopedia user table. That way there's still an audit trail of who did what.

            That's actually my next big task to tackle, and the results of that can be used as input to an ajax tag entry editor, though that's a bit more down the road.

            by The Centerfielder on Sat Dec 30, 2006 at 10:39:48 AM PDT

        •  Years of birth: Barbara Bush (#### -)

          by nu on Sat Dec 30, 2006 at 02:05:43 PM PDT

•  wasn't there a widget in the works too? nt

  by borkitekt on Fri Dec 29, 2006 at 02:50:27 PM PDT

  •  gee

    you remember I said that, huh?

    Yes, I have that almost ready to release. There's a problem with php session ids appearing in the url instead of in cookies, and that's something I have to hammer out with my ISP, but it's only a problem when you want to edit a dKosopedia page from within the widget. I can release it with caveats, I suppose.

    by The Centerfielder on Fri Dec 29, 2006 at 06:27:00 PM PDT