Skip to main content

View Diary: Public Release: USAGate DOJ Email Database (176 comments)

Comment Preferences

  •  this database isn't big so it's not a big (4+ / 0-)

    problem.  the search pulls up nothing when i try searching with too many terms.

    Synthetic search helps with that.  It gives you more search results and gives you a significance percentage score.

    So if I search for 'email server offsite', it will give me results that don't exactly match that, but come close. The current search says nothing found.

    Synthetic search costs money.  This database is fairly small, so the cost would be really small.  I know somebody who works there if you are interested.

    Don't fight it son. Confess quickly! If you hold out too long you could jeopardize your credit rating. --Brazil (1985)

    by hypersphere01 on Mon May 21, 2007 at 01:31:04 PM PDT

    [ Parent ]

    •  Def interested (4+ / 0-)
      Recommended by:
      bwren, dangangry, 4Freedom, moosely2006

      Tech guy emailing you as we type.
      Thanks for helping.

      •  You know, with the level of competence and (1+ / 0-)
        Recommended by:
        drational

        commitment those involved are revealing here, if I were the subject of some of these investigations, I would be shaking in my boots right now.

        This is an amazing and wonderful effort. It may change the way political research is done. This much talent looking into issues this critical is bound to uncover damaging communications that will help make the case against the majors.

        When the fruits of all this labor are fully assimilated by media and all interested parties, some very bad people will be quaking and shaking even more.

        Many, many thanks to all.

        New national holiday: Freedom Day - January 20th, 2009

        by 4Freedom on Tue May 22, 2007 at 06:49:44 AM PDT

        [ Parent ]

    •  simple boolean operations would be nice (3+ / 0-)
      Recommended by:
      4Freedom, drational, moosely2006

      "full text search" is a technical term, at least in database speak, and that's not doing it.

      Much to my horror, I'm pretty sure it's not using MySQL which I think is the heart of the problem.

      email sent which was probably more critical sounding than I intended.

      If it's not, I strongly recommend consulting a database design expert and getting the content into a proper relational database so that the data is actually useful once it's in there. Start with database design and let someone who does does database design for a living do that and tell you how to set everything else up based on that.  Otherwise, the volunteers are wasting their time.  The point of this kind of project is to allow complex algorithmic analysis.  People should be able to use the raw data as a starting point for academic research.  The technology currently being employed makes that impossible.

      This post brought to you by George Soros and the vast left wing conspiracy

      by VelvetElvis on Mon May 21, 2007 at 03:32:21 PM PDT

      [ Parent ]

      •  More on The Way (4+ / 0-)
        Recommended by:
        Ckntfld, 4Freedom, drational, moosely2006

        Currently, the search is limited to 'simple' searches.  This was the quickest version to get to market.  However, much more is on the way.  The data set is currently indexed in Verity and 'explicit' searching will soon be possible (working out bugs when not writing this).  We intend/hope to go way beyond what now exists and provide the tools you mention and many more.

        It would have been nice to have all the time in the world to get all those things working before we gave it the public launch, but we decided that there would be real value in what we already have.

        How about offering to help?  You seem to know something about this.

        •  what technology is being employed? (2+ / 0-)
          Recommended by:
          4Freedom, drational

          I guess I shouldn't jump to the conclusion that it's all  all Microsoft just because it's .asp running on IIS.

          Can you run MySQL or PostgresSQL or anything like that on with asp on IIS?

          This post brought to you by George Soros and the vast left wing conspiracy

          by VelvetElvis on Mon May 21, 2007 at 05:23:13 PM PDT

          [ Parent ]

          •  MS SQL Server Back End, Cold Fusion Up Front (2+ / 0-)
            Recommended by:
            4Freedom, drational

            Well, you are half correct on the assumptions.

            The website could handle MySQL, but I can't.  We've got some data in MySQL that we are going to port over to MS SQL Server here tomorrow.

            This is an iterative process.  As we get things better organized and built, the ability to search and use the data will grow.

            •  one question (0+ / 0-)

              do the people who volunteered their time know that a $5000 software license is required to access the data offline and do they approve?

              This post brought to you by George Soros and the vast left wing conspiracy

              by VelvetElvis on Mon May 21, 2007 at 05:53:49 PM PDT

              [ Parent ]

              •  All the data is in excel spreadsheet and .txt (1+ / 0-)
                Recommended by:
                skywriter

                so I am not sure why anyone would have to pay a fee.
                If they can run excel, they can have the data.
                If they want to do more complex analysis, they will have to use software they pay the fee for.

            •  i'll volenteer (0+ / 0-)

              and do what I can to find help if you stop locking the data up.  I've not used windows in ten years.  I have absolutely no way to work with that data unless I go out and spend a few thousand bucks.  The whole point of a project like this is to facilitate access to information.  Locking it up in proprietary format is contrary to that goal.  

              This post brought to you by George Soros and the vast left wing conspiracy

              by VelvetElvis on Mon May 21, 2007 at 05:58:06 PM PDT

              [ Parent ]

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site