This is only a Preview!

You must Publish this diary to make this visible to the public,
or click 'Edit Diary' to make further changes first.

Posting a Diary Entry

Daily Kos welcomes blog articles from readers, known as diaries. The Intro section to a diary should be about three paragraphs long, and is required. The body section is optional, as is the poll, which can have 1 to 15 choices. Descriptive tags are also required to help others find your diary by subject; please don't use "cute" tags.

When you're ready, scroll down below the tags and click Save & Preview. You can edit your diary after it's published by clicking Edit Diary. Polls cannot be edited once they are published.

If this is your first time creating a Diary since the Ajax upgrade, before you enter any text below, please press Ctrl-F5 and then hold down the Shift Key and press your browser's Reload button to refresh its cache with the new script files.


  1. One diary daily maximum.
  2. Substantive diaries only. If you don't have at least three solid, original paragraphs, you should probably post a comment in an Open Thread.
  3. No repetitive diaries. Take a moment to ensure your topic hasn't been blogged (you can search for Stories and Diaries that already cover this topic), though fresh original analysis is always welcome.
  4. Use the "Body" textbox if your diary entry is longer than three paragraphs.
  5. Any images in your posts must be hosted by an approved image hosting service (one of: imageshack.us, photobucket.com, flickr.com, smugmug.com, allyoucanupload.com, picturetrail.com, mac.com, webshots.com, editgrid.com).
  6. Copying and pasting entire copyrighted works is prohibited. If you do quote something, keep it brief, always provide a link to the original source, and use the <blockquote> tags to clearly identify the quoted material. Violating this rule is grounds for immediate banning.
  7. Be civil. Do not "call out" other users by name in diary titles. Do not use profanity in diary titles. Don't write diaries whose main purpose is to deliberately inflame.
For the complete list of DailyKos diary guidelines, please click here.

Please begin with an informative title:

Since the disclosure of the PRISM program by Edward Snowden, debates on what, why and how the NSA does what it does, and the appropriateness of it, have cranked-up the volume across the internet including here.

As an IT industry worker bee that holds these issues near and dear, I've found myself increasingly distressed at the content of many arguments on this site I find to be either ill-informed, aiming to deny facts or mis-direct from them, or simply off-track of essential issues.

My purpose here is to share information I think would help to re-frame the debate with facts about what the NSA (and their ilk in the US and elsewhere) now do to collect, analyze and act on information, and to pose some basic questions for consideration.

If you think it's possible that:

- countries, including the USA, spy on other countries and their own citizens
- the internet and other digital communications facilitate data collection & analysis
- people have become conditioned to accept increasing surveillance and less privacy
- that cyber warfare has become a core mission of the (global) MIC
- that governments use fear to manipulate public opinion and gain acceptance of intrusion

... see you after the fold.


You must enter an Intro for your Diary Entry between 300 and 1150 characters long (that's approximately 50-175 words without any html or formatting markup).

What the NSA does

The NSA was founded in 1952 as a secret organization under the DoD to acquire, analyze and report foreign intelligence, for the use of, primarily, the Executive Branch and DoD under it, and to develop technology to execute its mission including hardware and software. For the first 2 decades of it's existence, it was largely unknown to outsiders in government and the public, and largely stuck to it knitting, mission-wise.

Gradually, as data processing and electronic eavesdropping technology became more sophisticated and important to intelligence, the size and budget of the organization grew to the point it became more visible as an entity if not transparent as an organization, eventually becoming a larger organization then the CIA (today an estimated 40,000 employees verses 20,000 employees, respectively) and developing mission creep.

As an organization with a secret budget and little oversight, it's impossible to provide much accountable detail but Wikipedia, the ultimate authority on all things, provides some basics.

However, in the past 30 years, diligent investigative journalism and disclosures by current and former members of the organization present a more complete picture for our purposes and I will link various articles you should read to understand, particularly the more recent work of James Bamford available on the internet and disclosures by William Binney and Thomas Drake.

How the NSA acquires information

From it's founding the NSA has leveraged various means to acquire intelligence but particularly focused on electronic eavesdropping by tapping into communications networks including telephone, satellite and data networks from the US, listening posts abroad and through the use of various surveillance assets including friendly foreign governments, some of the foundation programs being MINARET, SIGINT, MASINT, ECHELON and the like, which involved various branches but leveraged by NSA.

Then, the internet changed everything, turning a well-monitored trickle of data into a high pressure firehose surveillance organizations rushed to capitalize on but found overwhelming in volume, rapidly falling behind both in technology, capacity and funding.

Again, 9/11 changed everything, when failing in their basic mission to intercept a threat, the organization got the argument and unquestioning compliance of the Executive and Congressional branches to fund an unprecedented expansion, including the justification to expand surveillance domestically.

Sucking From The Hose

As telephone systems including mobile became all digital and traffic merged with the internet, the means to tap into the data stream and store records became a more simple and automated if larger scale and more daunting task, but an irresistible one to those convinced more is better, and empowered by the Patriot Act, the wheels turned.

Starting in 2001, the NSA initiated various programs to tap into landlines, cellular and internet data streams including Stellar Wind, Trailblazer, PSP and the Terrorist Surveillance Program, which quickly blurred the lines between foreign surveillance authorized for the agency and domestic that is not.

Regardless of where one stands on the issue, we should recognize the fundamental problem digital communications, particularly internet, pose in filtering data and also the opportunity to do so if massive data is acquired and stored for later use, which is exactly the path chosen and the present situation.

The NSA did and does tap into primary data streams to acquire and store data for future use and is raising the level of sophistication from the early methods such as Room 641A to it's current expansion of resources in both Fort Meade, Maryland (HQ) and Bluffdale, Utah (see The NSA Is Building the Country’s Biggest Spy Center).

While you, I and Webster's might define this activity as "acquisition" since it involves accumulation, filtering and categorization (analysis), and storage, the NSA does not consider it "acquired" until it is used, presumably with a FISA order if needed. We might question "How do you know what to request a FISA ruling on if you haven't got it?" but that could be foolish.

Eventually, the programs came to light and faced both Congressional enquiries and lawsuits, but the work proceeded undeterred and spawned such programs as Turbulence, Boundless Informant and the now infamous PRISM.

How Is It Done?

Fans of spy movies know you can pull this stuff right out of the air, and while that is true, the difficulty, bandwidth and capacity is pretty poor, so the sensible thing to do is to tap into Central Office Switches, Fiber Optic Land Lines and Trans-Oceanic Cables and Satellite Communications Base Stations, which the PRISM Powerpoint shows as a map with a startling resemblance to that shown in NSA's Lucky Break: How the U.S. Became Switchboard to the World.

It went public with Room 641A when whistle blower Mark Klein disclosed the existence of an NSA tap on an AT&T Switch in San Francisco complete with photos. The EFF filed suit, but Hepting vs AT&T was eventually dismissed based on Congress granting retroactive immunity, a case covered in the NOVA episode The Spy Factory. So much for accountability and oversight.

Since then, NSA has pursued the infrastructure to realize to its objective of Total Information Awareness which is best described as a smart storage tank for the contents of a fire hose, with the critical point that data streams are collected. Please read:

Connecting the Dots on PRISM, Phone Surveillance, and the NSA’s Massive Spy Center
Secret To PRISM Program: Even Bigger Data Seizure
The NSA Is Building the Country’s Biggest Spy Center (Watch What You Say)
What the NSA can do with “big data”
Big Brother on a budget: How Internet surveillance got so cheap

And then, perhaps most importantly, Bamford's latest article for Wired, The Secret War, an expose on how General Keith Alexander has accumulated unprecedented resources and power, including crossing the line from surveillance to command and control. Supporting evidence can be found in  Inside the NSA's Ultra-Secret China Hacking Group (no comments yet from the flies on the wall during Obama & Xi's weekend retreat but I'm sure it was interesting).

Leaks Below Deck

By the mid 00s, dissatisfied with the direction of domestic surveillance and corruption in procurement, several NSA members of long standing left the organization and attempted to rally DoD and Congressional support for investigation, becoming whistleblowers.

First, and most significantly, William Binney, a long serving and highly placed technology officer resigned in 2001 and spend years working the halls of Congress and various agencies before going public after the New York Times published an expose in 2005, and was ultimately the target of a 2007 FBI investigation. Binney, who warns the US is on the road to a totalitarian state, has continued to work with investigative journalists and supporting other whistleblowers while continuing to speak out, often in contradiction to standing officials. He will not shut-up and has much to say.

Another significant case was that of Thomas Andrews Drake, administrator turned whistleblower, who disenchanted with the direction and the financial waste he witnessed on the ill-fated "Trailblazer" program while the NSA scaled-up in the mid-00s, took the case public and was prosecuted under the Espionage Act, recounted in Is Thomas Drake an enemy of the state?

Defended by fellow kossack Jesselyn Radack, charges were eventually dropped when it was demonstrated the information disclosed by Drake was, in fact, in the public domain as it had been released before in filings by the NSA. Drake has continued to speak out, most recently supporting Snowden in the article Snowden saw what I saw: surveillance criminally subverting the constitution. I can recommend Radack's recent diaries here.

So Where Does It Lead Us?

Does this information help to re-frame the picture? Can we distinguish between the dangers of massive data acquisition put to what future use we don't know and splitting hair about how exactly NSA gets data from Google or Facebook for analysis by PRISM (just an app, folks)?

Can the commentary by these whistleblowers help us to focus here? Does their direct approach to the implications for society ring some bells?

Living in a society less free than the USA, my personal comment is people need to get focused on where this can lead, the message it sends the world and the down-line implications.

This is not setting the example for the world to follow. Or as one Chinese critic recently put it, NSA surveillance: the US is behaving like China, only in a sense, I think it is worse because it is more secretive and less transparent than the heavy-handed censorship and surveillance we must tolerate daily and so severe criticized by Clinton and Obama is their infinite wisdom (ouch, but we can now say that without hesitation).

Make your own sense of this. Please.

I don't think the issue is Google this or Microsoft that.

It is what is being done in your name and done to your future, we are told for your safety. Do you buy that?  Has the "right balance" been struck as Mr. Obama suggests?

How can you know that if you don't have the basic facts?

Thanks for reading & thinking.

Update 2013.06.17 CST 19:18

I just wanted to add a link to these excellent USA Today interviews with Binney, Drake, Wiebe and Radack discussing the Snowden disclosures.

Extended (Optional)

Originally posted to koNko on Sun Jun 16, 2013 at 07:10 AM PDT.

Also republished by DFH Local No 420.

Your Email has been sent.