Average American has 6,000 page NSA 'phone' file

I did a rough but reasonable calculation to come up with the figure in the title, making just a couple assumptions. Primarily, 2 assumptions: the size of the database, and the number of Americans spied upon.

Database size:
If the NSA phone database is "the largest database ever assembled in the world," and the largest database that I can find was approaching 900 TB, as of 2 years ago at Stanford University, then I make the assumption that the NSA's database is 2,000 TB. It could be larger or smaller.

Number of Americans: I'll just pull the number '150 million' out of my ass. The true number could be larger or smaller.

Divide the database size by the number of Americans, and you get 13.3 MB per American in the database. I opened a spread sheet file on my computer, and had 2,000 characters on 1 printed page. Using that as another assumption, 13.3 MB divided by 2,000 characters per page gets you 6,650 pages for just 1 American!!

Please jump the hump...

This is important to the discussion, because we can be reasonably certain there is still a great deal that Bush is not telling us about what is in this database. All the people I've called for 3 years would not make a 6,650 page document. Further food for thought: let's bracket the range of the average file size per spied-on American:

If you push the numbers in one direction (say, 1,000 TB database and 200 million Americans) you'd get
5 MB per person, or 2,500 pages.

If you push the numbers in the other direction (say, 5,000 TB database and 20 million Americans) you'd get 250 MB per person, or 125,000 pages.

Whichever scenario you pick, something still doesn't add up with what we've been told.

I have a problem with just accepting what this Administration says. We're supposed to sacrifice our freedom because we're in a War on Terror, and the terrorists hate our freedom. After 9-11, Bush didn't want to investigate how the attack happened, but instead his gut told him to rapidly start gathering information on tens of millions of innocent Americans. After 9-11, Bush wanted to attack Iraq instead of getting who attacked us. Now we have Americans fighting and dying abroad in Iraq, based on lies, so that we can sacrifice freedom here at home.

Bush expects us to believe he is sincere and intense about protecting American lives. I have a question for him he won't answer: Why the hell was Bush's choice for FEMA, an agency involved in Life and Death decisions, a man who had just been fired from a horrible 10 years at the Arabian Horse Association? Bush surveyed the people available and Mike Brown was the right man for the job? How the fuck do you explain that, Mr. Bush? If Bush was sincere about protecting us, he'd pick people with experience and competence, instead of incompetant failures.

Why is Bush's first instinct to sacrifice our freedoms, before doing the obvious protections, such as securing the borders, or screening more than 5% of incomming cargo?

•  Maybe Bush thought his oath was to

  'execute' the Constitution and defend his office against all enemies, foreign and domestic.

  by bejammin075 on Fri May 12, 2006 at 10:31:37 AM PDT

•  He is the fucking worst there is...n/t

  by jdmorg on Fri May 12, 2006 at 10:31:58 AM PDT

•  well

  mine is full of stuff about my 15-year-old cat's diarrhea, and which toys his 2-year-old sidekicks like to carry around.

  Stuff like that.

  Have fun, NSA.

  by Plutonium Page on Fri May 12, 2006 at 10:34:16 AM PDT

  •  We have ways of making your cat's diarrhea talk.

    by bejammin075 on Fri May 12, 2006 at 10:43:09 AM PDT

•  Excel probabaly isn't the best program to use

  to determine the size of nformation.

  There are factors of how many tables, how many fields in each record, what the datatype and size of each field is, and how many records there are.

  Of course, this is based on the idea that "the largest database every created" is really true.

  by FleetAdmiralJ on Fri May 12, 2006 at 10:42:49 AM PDT

  •  The point of this was to get an estimate

    into the discussion. We will likely never know the 'true' size of the NSA database, but we can make estimates.

    by bejammin075 on Fri May 12, 2006 at 10:45:44 AM PDT

  •  For example

    If the average record in this database is 10KB in length, and the database is 1 PB (1 petabyte, or 1000 terabytes), then one is looking at 100 billion records...which if they're tracking nearly every phone call in america may be a good guess.

    So how much data is in 10Kb? Since a lot of it will probably just be text, try opening notebad and filling it with text until its 10kb in size. I bet it wont be small...and its probably enough to strongly suggest that they're not just tracking who called whom.

    by FleetAdmiralJ on Fri May 12, 2006 at 10:53:39 AM PDT

•  Very Interesting Analysis!

  Thanks for laying out the possibilities.

  Also, it is unfathonmable why you would "protect" Americans by employing the most obscure method while leaving the obvious and more critical solutions, such as securing ports and nukes, in a total mess.

  Nice diary.

  Thanks,

  LesIsMore than upset today. Wake up America!

  by LesIsMore on Fri May 12, 2006 at 10:42:58 AM PDT

•  it isn't the phone numbers

  suppose you were to record the phone calls to and from all people in the USA.

  Each record is about 24 bytes. 10 + 10 + 4 for the time.

  330 million people, an average of (say) thirty a day? (remember I'm counting 2 year olds in this). Think about how many calls are on your monthly statement if you get call detail.

  10 billion calls a day? By 24 bytes.

  240 billion bytes a day.

  I HAVE 1/20 OF THAT ON THE LAPTOP I'M TYPING THIS COMMENT ON

  20 hard drives a day... is that a bleeding edge database?

  They aren't recording the numbers.

  They are recording the calls.

  by Robert Ullmann on Fri May 12, 2006 at 11:19:11 AM PDT

  •  Not only that potentially.

    Phone companies can work out the locations of each cellphone to near GPS accuracy - it's trivial to do.

    Wanna bet that data's not finding its way into the NSA database as well.  Phones ping the cell tower every minute.  Say a 100 byte GPS location code per minute, for each cellphone in the US (and 70% of us have 'em) = around 28 TB per day, which seems in the ballpark to make the database large enough.

    by magnus on Fri May 12, 2006 at 11:51:06 AM PDT

  •  your laptop has a small harddrive

    if you consider high end hardware, say 400gb harddrives even with data back up systems etc thats less than a harddrive a day.

    if were looking at 1<>2 Petabytes thats 13<>26 years worth of data.  

    HOWEVER

    people recoment 20kbps for mp3s of phone recordings, SO using that assumption and assuming an average phonecall length of ... 20 Min (total guess here)  thats 24000kb or 2.9MBytes per call, 87.8MB per person thats close to 21.6 PETA bytes per day, far more than was discussed here.  

    Now the NSA may have much better bitrate encoding schemes for audio maybe even 1kbps or 1000 bits per second, but even then we are still talking a database the size of the LARGEST currently know database EVERY DAY

    by haagmm on Fri May 12, 2006 at 11:55:30 AM PDT

•  I'd wager that your numbers are off

  With a scant hundred thousand of these terabyte drives the NSA could keep more than half a gigabyte of data on 180 million people.  That's the kind of storage you need to store everything including the full audio of all calls.

  They probably have much better stuff than this.  This is just the current consumer stuff.

  by Capn Guts on Fri May 12, 2006 at 11:30:30 AM PDT

  •  yes, but it is a challenging database

    I pushed my comment as a diary as well.

    You are quite correct, anything that NSA and companies like Verizon find to be a "challenging" size is more than sufficient to record the content of all the calls.

    Robert

    by Robert Ullmann on Fri May 12, 2006 at 11:43:07 AM PDT

  •  not to nitpick

    but those laci drives are really just two harddrives stuck together as far as i remeber. parousing websites the largest harddrives i can find are 500gb per physical harddrive, and those are not designed to meet the needs of having litterally 100 thousand of them.  The technical issues with addressing THAT MUCH space are not simple.  the larger enterprise solutions tend to be huge collections of SCSI disks for there relaibility and speed, and as such alot of the high end enterprise hardware and software is optimized for them.   High end scsi drives are closer to  160gb max.

    I am not saying the NSA isnt ahead of technology, but i would suspect that the technical holdups on larger harddrives exist for them as well.  

    by haagmm on Fri May 12, 2006 at 12:04:53 PM PDT

•  why do you think they aren't monitoring computers

  people keep saying phones
  how do you know they don't monitor what websites you go to, what passwords you use...like Dailykos?

  by ProgressiveFighter on Fri May 12, 2006 at 12:03:54 PM PDT

  •  us hackers have long know of this

    i use the term "hacker" here very lightly.

    in short go google for ECHELON, a project created long ago to monitor all signals traffic.  It has sence extened to include web traffic and email.  

    It was supost to be used EXTERNAL only, though the fact that it consisted of several contries, the US the UK Australia and New Zealand and consiquenty country A woud have Country B spy on country A citizens.

    by haagmm on Fri May 12, 2006 at 12:17:18 PM PDT

•  hmm..what software exists to manipulate this data

  I remember a story on Democracy Now a few months ago.  Amy's guest surmised that Poindexter's Total Information Awareness program was simply moved off the books.  He thought the new code name was 'basketball', I remember.

  If this is true, and 'basketball' overlaps with the NSA program, this may account for the extra storage (perhaps credit info, airline tickets, addresses, etc).

  What I wonder is, who writes database software capable of dealing with this data?  Is it difficult, or are they just using a 'Google' like approach?

  In other words, are they just creating a big brother version of Google?  Any combination of data available in a split second?

  Anyone want to wager this isn't what is being developed?

  by themank on Fri May 12, 2006 at 12:14:48 PM PDT

  •  here is the link to the Democracy NOW show

    http://www.democracynow.org/...

    by themank on Fri May 12, 2006 at 12:21:20 PM PDT