OK

This is only a Preview!

You must Publish this diary to make this visible to the public,
or click 'Edit Diary' to make further changes first.

Posting a Diary Entry

Daily Kos welcomes blog articles from readers, known as diaries. The Intro section to a diary should be about three paragraphs long, and is required. The body section is optional, as is the poll, which can have 1 to 15 choices. Descriptive tags are also required to help others find your diary by subject; please don't use "cute" tags.

When you're ready, scroll down below the tags and click Save & Preview. You can edit your diary after it's published by clicking Edit Diary. Polls cannot be edited once they are published.

If this is your first time creating a Diary since the Ajax upgrade, before you enter any text below, please press Ctrl-F5 and then hold down the Shift Key and press your browser's Reload button to refresh its cache with the new script files.

ATTENTION: READ THE RULES.

  1. One diary daily maximum.
  2. Substantive diaries only. If you don't have at least three solid, original paragraphs, you should probably post a comment in an Open Thread.
  3. No repetitive diaries. Take a moment to ensure your topic hasn't been blogged (you can search for Stories and Diaries that already cover this topic), though fresh original analysis is always welcome.
  4. Use the "Body" textbox if your diary entry is longer than three paragraphs.
  5. Any images in your posts must be hosted by an approved image hosting service (one of: imageshack.us, photobucket.com, flickr.com, smugmug.com, allyoucanupload.com, picturetrail.com, mac.com, webshots.com, editgrid.com).
  6. Copying and pasting entire copyrighted works is prohibited. If you do quote something, keep it brief, always provide a link to the original source, and use the <blockquote> tags to clearly identify the quoted material. Violating this rule is grounds for immediate banning.
  7. Be civil. Do not "call out" other users by name in diary titles. Do not use profanity in diary titles. Don't write diaries whose main purpose is to deliberately inflame.
For the complete list of DailyKos diary guidelines, please click here.

Please begin with an informative title:

No Fly List is a subset of FBI's Terrorist Watch List. That system is not up to commercial standards. Several new parts, called RAILHEAD, are flat-out busted: NY Times "the flaws discovered are mind-bogglingly basic."

Obama needs to move mountains to repair the No Fly List system. Otherwise, come the next airliner attack, he gets rebranded:

-- Another George "Mission Accomplished" Bush

-- Another Michael "Loosey-Goosey" Dukakis

-- A poster child for weak management

How's that for politics ???

A weak-defense terrorism disaster is the GOPers' big hope for return to power.

As of today, prime contractor Boeing keeps the RAILHEAD contract despite $500,000,000 down the drain. Nobody gets fired. Boeing beats its own record of $10,838,231,984 in one year from the Federal government.

Newspaper lead: "The White House circles the wagons...."

Next thing, they'll be handing out Presidential Medals of Freedom -- like for George Tenet and Paul Bremer. It's as dumb as Icebergslim's box of rocks.

For an Action Plan -- plus, an intro to how these projects work -- go BTF :::

Intro

You must enter an Intro for your Diary Entry between 300 and 1150 characters long (that's approximately 50-175 words without any html or formatting markup).

"Yep, son. we have met the enemy and he is us."

  --  Walt Kelly, for the first Earth Day

That keeps coming back and coming back.

al-Qaeda has its own business model. But damn, we can make it easy for them.

The No Fly List is reported to have between 3,400 and 14,000 hard-core terrorists. That is not enough. al-Qaeda recruits the mentally ill -- DSM IV Axis I Depression victims, specifically. There have to be far more young Muslims who have fallen for al-Qaeda craziness, one time or another. If and when one of these kids go down mentally ill, you want him tagged for "special processing" physical searches and likely visa cancellation before he can blow up an airliner.

CIA remains adept at lying its ass off:

"Abdulmutallab's father didn't say his son was a terrorist" when he visited the U.S. Embassy in Nigeria, "let alone planning an attack. Not at all," one U.S. intelligence official said. "I'm not aware of some magic piece of intelligence that suddenly would have flagged this guy — whose name nobody even had until November — as a killer en route to America, let alone something that anybody withheld."
Read more: http://www.denverpost.com/...

Father came in to the Embassy twice. Told everything to State and again to CIA. Followed up in writing and with telephone calls. Couldn't have given more warning, if he'd brought in a jihadist suicide-video.

The TIDE database is reported at 400,000 to 570,000 likely-terrorists on the list. That is not enough. Go for an Identity Management approach and there's no problem tracking billions of people with sensible STATUS functions to track what-all information comes through.

Once you have decided that a John Doe is not worth tracking, you have already made a mistake. False acceptance, matched now with false rejection. Statistically certain. The Identity Management approach avoids both certainties.

All these organizations have Informatica PowerCenter software so there is no technical problem -- no reason whatsoever to fail at verifying&validating to cancel a visa at Department of State.

PRIORITY ONE

is fixing the major scoping and personnel problems with the RAILHEAD PROJECT. That is name of the overall Boeing contract "vehicle" for all things related to the No Fly List and other terrorism database work.

Take care of the worst people problem: eliminate corruption. Then the technical problems will fall in line.

-- The No Fly List system, itself, called TIDE, is repairable.

-- A fast and low risk architecture will use existing software, FBI's Office of Intelligence for expertise, and a cutey-pie bridge tool built by a tiny company called Triple Hop.

-- The Terrorist Watch List and other associated lists can be upgraded with Intelligence Community resources, but also with auxiliary very-large-scale commercial Identity Management data sources.

-- No problemo.

I will show how the pieces fit together, below.

NEW YORK TIMES PROJECTILE BARFS AT BOEING

From the New York Times:

"That Troubled Terrorism List"
Published: August 24, 2008

The new program, known as Railhead, is intended to fix the problems with the current outmoded (TIDE) program. That database — begun as an urgent priority after the Sept. 11 attacks — has been bedeviled by an array of problems, including the inability to do basic searches to find suspects’ names.

Bush administration officials have been pronouncing Railhead a success. But the investigation by a House Science and Technology subcommittee found it crippled by serious design flaws, management blunders and runaway contractors. Hundreds of private contractors from dozens of companies involved were recently laid off as government managers finally ordered a fresh overhaul in the face of "insurmountable" problems.

Some of the flaws discovered are mind-bogglingly basic. The Railhead database, it seems, also has fundamental problems with its search function. It failed, for example, to handle multiple word searches connected by "and" and "or," and it could not offer matches for slight misspellings of suspects’ names.

Worst of all, RAILHEAD has been trashing the No Fly List and the other Action Lists for two years.

THE MANAGEMENT CHALLENGE

I want you to read through this section. Some day you might find it useful.

Fixing the No Fly List, today, is more difficult for management than for the technical work. "All problems are 85% management." Several of the most critical management decisions will have to come down from the White House. No temporizing.

Arguably, the very worst of George Bush was that he never fired people for raw incompetence. Crisis management at the Departments was nonexistent.

Bush Administration screwed up the economy, Katrina response was blocked by Michael Chertoff and his DHS top management crew, the counterfeiters at DoD concocted the bogus Iraq War WMD threats, S.E.C. let Wall Street and the mortgage brokers operate as an Organized Crime crew, and on and on and on. Political loyalty and Rovian "message discipline" were all that mattered. They had no standards for integrity.

Meanwhile, here, the standard for protection is simple:

"I do solemnly swear that I will support and defend the Constitution of the United States AGAINST ALL ENEMIES FOREIGN AND DOMESTIC; ...."

From here on with Obama, game goes to hard work. Elbow grease. Reading people carefully. Doing humility, so you can finish things. Integrity. Of the factors, integrity is the deal maker:

"The integrity of a process determines the quality of the product."

  --  W. Edwards Deming, operations research innovator

Churchill made a related comment that has its place: "In wartime, truth is so precious that she should always be attended by a bodyguard of lies." Churchill did well enough during WW II.

Churchill not withstanding, when you need to go build something, Deming's advice is the best starting point. Especially applicable for this situation, where Obama needs to avoid a horrible, airliner-killing disaster. The processes for upgrading TIDE and supporting the No Fly List mo' bettah (that's Hawaiian slang from WW II era) will need to be governed with integrity at every milepost.

KICK ASS AND TAKE NAMES

The main perp has to be Michael Leiter, Esq. Director of the National Counterterrorism Center (NCTC.) Magna cum laude in 2000, President of Harvard Law Review following in the tracks of Barack Obama.

Brilliant man. But still this is a striving lawyer.

Today Leiter is the leading denier for thousands of No Fly List errors. A coward for not squaring up, sinking teeth into Boeing.

Then there is Boeing, itself. The prime contractor. The $10,838,231,984-a-year Boeing. The #2 Federal contracting company overall.

The "connected" Boeing.

Boeing took the FBI's TIDE (Terrorist Identities Datamart Environment) system, written using the Oracle database. Boeing "upgraded" the TIDE database design, reportedly, by quadrupling the overall size. Also by replacing the Oracle commercial off-the-shelf (COTS) search engines with a vaporware piece of dysfunctional fraud.

Boeing's vaporware search engine for RAILHEAD never worked.

The TIDE system works sensibly enough. Upgrade is straightforward.

Most all of the main organizations involved with NCTC have Informatica's PowerCenter tool installed. This tool specifically handles transfers of messages and data files. That list of PowerCenter users includes Department of Justice, Department of State, Department of Transportation, and many others. The original specification for the TIDE upgrade focused on augmenting information feeds from older legacy systems in the intelligence community -- exactly the data migrations where PowerCenter is a default federal standard.

Boeing ignored the K.I.S.S. solution. Ignored PowerCenter.

Boeing did everything they could to avoid running a simple project with a dozen stages to arrange PowerCenter-to-PowerCenter message and file transfers. The Mid-Atlantic Region area has roughly ten-thousand experienced PowerCenter developers. Boeing would have had one helluva lot of trouble spending $50,000,000 upgrading TIDE honestly.

BOEING FAILS AT COMPLEX WORKFLOW SYSTEMS

RAILHEAD is not the first major software failure by Boeing.

This is the Boeing that couldn't do its own corporate Sarbanes-Oxley financial compliance system. Boeing makes nice sharp realtime systems for its airliners, but apparently they cannot consolidate accounting and Workflow processes.

Boeing never could get clear the distinctions between Financial Accounting and Managerial Accounting. Despite four tries at reading the Sarbanes-Oxley requirements -- stated in S.E.C. Implementation Directives and translated to English in 2002 by F.I.E.C.C. for their own financial examiners -- Boeing failed miserably.

Boeing's software culture refuses to read/understand/apply specific elements. Well-defined Workflow controls; Check List hierarchies; and as well the need to test corporate systems for degraded data, response to Known Error test events, and escalating-for-special-handling every problem that goes bad in the workplace. Boeing is in love with ROI (Return on Investment) analysis -- trying to apply ROI to micromanage system design.

Boeing's failure at building a corporate Sarbanes-Oxley compliance system reflects point-by-point the failure and the eventual damage that Boeing inflicted on the No Fly List support software. Can't do for themselves, can't do for NCTC.

HOW THE NO FLY LIST GOT BUSTED

Michael Leiter and Boeing took the RAILHEAD PROJECT and ballooned it into a $500,000,000 boondoggle. Simple upgrades to support the No Fly List have not been applied directly to TIDE. The commercial Identity Management approach -- which works at millions of companies -- was never considered.

Boeing went mad with greed. 814 contractors at a recent head count. $100,000,000 blown for Gold Plate security at one Boeing office building. No sense of integrity. No quality control.

Upgrades for TIDE should have run to simple goals and mileposts:

-- Migrate the core application to the new generation of Oracle database

-- Expand functionality to utilize new features

-- Devote resources to specialty software: the Oracle Text linguistic search engine; new Application Server technology to support web pages; and the Built-In Packages that Oracle shops use to save man-hours for common development tasks.

-- Integrate with new releases of software such as the Informatica PowerCenter message and file transfer software. Consider new performance and security features.

-- At least take a look at Identity Management commercial resources. (Pssst... wanna find somebody ???)

Instead, Boeing went all vaporware. $500,000,000 for fraud:
-- Swap out Oracle for a non-existent fantasy system

-- Fake the original list of legacy system transfers. They still don't know what fields have what values in the dumps from the legacy systems. And too arrogant to ask for help.

-- Produce multiply redundant subtasks, and

-- Fake both formal tests and informal demonstrations for NCTC training

Of course, these actions were largely criminal. The Bush Administration took their campaign contributions and jobs-for-wives and structured bribes and never prosecuted a damn thing.

John Michael McConnell was the Director of National Intelligence during the period where this RAILHEAD PROJECT software went through implementaiton and user-acceptance testing. This Mike McConnell and his ODNI crew were responsible for quality control.  

A SIMPLE PROTOTYPING DEMONSTRATION

How hard is it to fix TIDE to do the RAILHEAD expansions ???

I took the RAILHEAD/TIDE/No Fly List problem and prototyped it using Oracle 11g and the Oracle Text linguistic search engine.

I build databases, data warehouses, and text warehouses. Identity Management systems. I also install Oracle, Linux, Sun Solaris, and lots of other database tools. We got the basic commercial telephone and marketing info on disk.

The Jets game and the Dallas game first-half were enough to get a 1-terabyte, 1-billion row system built using real data on people in Africa-EU-Middle-East. A standard base for a compliance system. Lots of text fields, indexed with Oracle Text CONTEXT indexes.

One trick is to pull the descriptives to DIMENSIONS. There's another Oracle feature: INDEX_SET that makes queries go fast.

First test query: find Mr. Umar Farouk Abdulmutallab without using his name. His father had gone in the the Embassy in Nigeria and told our people what he knew about his son. He also identified himself.

Look at what we have coming Out-of-Africa from Department of State or CIA:

-- (a name string that we are discarding as unusable, possibly bogus)

-- Father's name and identifying document

-- Father's residence, date of birth, general descriptors

-- The indication that the individual had a prior U.S. Visa (has_a Visa application)

-- Date of Birth

-- Place of Birth

-- Dates related to earlier presence in a NATO country

This Out-of-Africa descriptive data proved overwhelming.

Any four items get Mr. Abdulmutallab to the top of my match system. # 1 in the list of match-candidates.

BTW: Run the query as "Umar Farouk" and the real Mr. Abdulmutallab popped up as my # 3 match-candidate.

The drill-across fields on the screen blink for the hits with Father and Visa. Giving TIDE access to these drill-across DIMENSION database elements is many steps short of rocket science.

This would have come out even better if I'd had an INPUT STATUS option in the first prototype to tell the system that the name fields might be degraded.

But, still, I've been a pro at this for ever. My system worked first time out.

WHERE DOES BARACK OBAMA NEED TO GO WITH TIDE-RAILHEAD

RAILHEAD is a failure. Another NORAD. Another SNAFU throw-out project. Another FBI-meets-IBM/Documentum ultraturd.

We have never seen a commercial search engine -- not even the first, not-ready-for-beta Oracle text "Cartridge" code -- that was as bad as RAILHEAD. The one-liner for RAILHEAD is that Boeing blew $500-million for a throw-out.

K.I.S.S.

Give TIDE-for-expansion back to FBI and the Office-of-Intelligence shop for the new data migration features.

Budget $25-million and allow no more than 50 people to work on the project. Have new sources do their own publishing projects, writing to Interface Tables that PowerCenter migrates to a PowerCenter install at TIDE.

The one piece of new technology I recommend has to do with building indexes remotely, to avoid having to copy terabytes of text data in to Tide and/or NCTC.

Remote index building.... that is the neat trick that Triple Hop perfected. Oracle bought Triple Hop. So you gotta get this technology, this time around, out of Oracle. Again, no problemo.

Combine the Triple Hop indexing technology with Informatica PowerCenter for data acquisition and migration management -- first rate tools for doing the TIDE/NCTC input problems.

Any and all of the input Departments and Agencies will be happier not exporting whole copies of their data. What they will have to do is to put up mirror servers inside their own network DMZ's. Then give access to these servers to NCTC.

Minimum intrusion. Minimum security risk. Minimum cost. Minimum performance impact.

Easy to maintain.

Once the TIDE system gets better at identifying people -- using place-holders, artificial identities for aliases and the like, and other common Identity Management tricks -- you can then increase the No Fly List to something like 50,000 or 100,000 identified individuals.

There's way more radical Muslim than that, you betcha.

No problemo.

Barack Obama gets the maximum chance to escape this NCTC RAILHEAD PROJECT disaster with his presidency intact.

Avoiding being the next George Bush and Michael Dukakis -- priceless.

And if I can do this problem -- the whole fully-realistic problem -- in an afternoon, then you know that FBI's Office of Intelligence can do it right.

There's probably fifty different commercial shops that can also do this project with first-rate final product. Shops that never screw over the client.

Now... let's see if the Obama White House can knock off a mini-Katrina. Get in there fast. Take charge. Fix the mess.

Odds, anybody, on the outcome ??? Mo' bettah... hopefully.

Extended (Optional)

Originally posted to vets74 on Fri Jan 15, 2010 at 02:44 AM PST.

Poll

If Obama leaves the No Fly List busted, sooner or later he's screwed. Another Michael Dukakis....

36%4 votes
0%0 votes
0%0 votes
0%0 votes
0%0 votes
0%0 votes
0%0 votes
0%0 votes
0%0 votes
54%6 votes
9%1 votes

| 11 votes | Vote | Results

EMAIL TO A FRIEND X
Your Email has been sent.