Skip to main content


NSA: The False Balance Between Security and Privacy

By J. Kirk Wiebe
January 21, 2014

“Nothing goes wrong quite so dramatically as a disastrous military expedition” - from the introduction to Arrogant Armies: Great Military Disasters and the Generals Behind Them by James Perry

Judging by the number of current and past high-level government officials associated with intelligence telling us that “we must find the right balance between security and privacy,” you’d think it would be true. But, as we’ve learned, it has become all too rare that a government official truly knows of what he or she speaks.

The truth is that there is no need for a tradeoff between security and privacy, if the intelligence analysis process is set up correctly. 

Unfortunately for the victims of the Boston Marathon bombing and other victims yet to be, it isn’t. NSA won’t admit it, and Congress and the President don’t know, but the fact remains that there is a better way to analyze “Big Data” than what the NSA is currently doing - a way that requires no bulk collection of the data of innocent people – domestic or foreign, and a way that uses automation and business rules to focus the minds of analysis so that they always pay attention to the information having the highest probability of containing usable intelligence.

How did we get to this point? The answer begins with this question: what really led to the events of 9-11?

Many have asked that very question any number of times since those indelible, tragic events of more than a decade ago.  Opinions have run the full spectrum of possibilities, with some having come close to the truth, while others believed the whole matter was a conspiracy hatched by the U.S. Government itself.

Allow me to whet your whistle. Unless they have possessed a Top Secret/Special Intelligence security clearance, most people do not know that polygraphs administered to employees and contractors working in the Intelligence Community do not contain a single question regarding whether an individual has ever knowingly disregarded the federal regulations (Federal Acquisition Regulation) governing the acquisition of new intelligence capabilities. Nor is there a single question that touches on the matter of honesty in responding to questions posed by Congressional overseers. Frankly, with all the corruption in government that has been uncovered over the years, you’d think someone would have put those questions into the polygraph process.  And no one is asked if they have upheld their oaths as government employees.  In other words, the emphasis in gaining a security clearance is not on honesty as one would surmise, but on loyalty.

I will go straight to the heart of the matter: 9-11 was caused by intelligence professionals, both military and civilian, engaged in a confluence of undesirable behaviors and characteristics, specifically mismanagement, political correctness, corruption, arrogance, and – yes, loyalty – of the worst kind.

Let’s take a look at the events that prompted five former Government employees – four former employees of NSA and one a former senior congressional staff member for the House Permanent Select Subcommittee on Intelligence – to take the step of filing a formal complaint against the National Security Agency with the United States Dept. of Defense Inspector General in September of 2002. 

At the critical time of NSA’s Global Network Intelligence modernization efforts beginning in March 1999 until he left the position of Director NSA, General. Michael V. Hayden (USAF retired), so mismanaged things such that he ended up presiding over the largest loss of intelligence and one of the largest losses of human life in modern U.S. history. Few understand the breadth and depth of the failure. A trusted friend and NSA retiree recently quoted to me words uttered by General Hayden during a planning meeting. Hayden is purported to have said, “I don’t know where we are going, but I sure do like the way we’re going there.” 

During the run-up to 9-11, the United States had only one operational system in the prototype stage of development to deal with the emerging technologies of networked, packet-switched communications; in other words, the Internet.  When given the opportunity, General Hayden and his lieutenants made the decision to ignore the prototype which was already up and running against real data in NSA’s SIGINT Automation Research Center (SARC) headed up by Ed Loomis, a senior computer systems expert.

How do we know that?  The Deputy Director of NSA was Bill Black who personally saw the capability at a briefing that was given specifically for him. At the end of the briefing, he enthusiastically remarked “You’ve made huge break-throughs – why are you being so quiet about them?”  From his perspective he must have believed that such information would normally have found its way to the top of NSA’s management chain.  He should have known that organizational dysfunction and resistance always filter information that should rise to the top of a government organization. It is one of the major obstacles to the smooth and effective functioning of government.  You may wonder why that might be the case. The truth is that good news – as well as bad – typically disrupt the status quo.  They tend to gore certain oxen and no one wants to be embarrassed or have their own power structures threatened. 

Despite the fact that Black found out about the prototype, the Director’s suite at NSA remained silent - we heard nothing more from Mr. Black, and the promising prototype was outright dismantled in the end. You may ask why? You see, the prototype system was an extremely cost-effective and promising solution to NSA’s need. As such, it was a significant threat to the opportunity for big business to obtain lucrative contracts in an attempt to solve one of NSA’s biggest challenges of the Information Age. Ironically, it also threatened General Hayden’s credibility, as he had gone to Congress claiming he needed big bucks to make big changes to NSA’s intelligence-producing capabilities. At a time when outsourcing was the “way ahead,” big budgets meant big contracts for big business in collusion with Government – the military-industrial complex. It was President Eisenhower’s worst fear and something he warned about in his 1960 farewell speech to the American people when he said:

In the councils of government, we must guard against the acquisition of unwarranted influence, whether sought or unsought, by the military-industrial complex. The potential for the disastrous rise of misplaced power exists and will persist.

We must never let the weight of this combination endanger our liberties or democratic processes. We should take nothing for granted. Only an alert and knowledgeable citizenry can compel the proper meshing of the huge industrial and military machinery of defense with our peaceful methods and goals, so that security and liberty may prosper together.

That fear was about to be realized courtesy of General Hayden and a congress all too eager to throw money at any part of the national security infrastructure professing to need it. This version of Eisenhower’s worst fear would have a new twist given the growth of both the Intelligence Community and the World Wide Web or Internet. This version would be called the military-industrial-intelligence complex. That subject alone could be – and has been – the subject of a book: see Tim Shorrock’s Spies for Hire for an in-depth treatment of the subject.  In short, Hayden told congress he needed billions to launch his modernization effort called TRAILBLAZER, when a few hundred million would have been more than enough to enable the fledgling prototype to do the job.

At the time NSA’s fledgling prototype capability called THINTHREAD was being developed, it found an important source of support outside of NSA in the person of Diane Roark, a senior staff member on the House Permanent Select Committee on Intelligence (HPSCI). Diane had the responsibility of providing congressional oversight of NSA’s huge budget. It was her job to find areas of technical excellence within the walls of NSA and do what she could to increase their probability of being used to support NSA’s many missions. She saw the value in the prototype and helped in earmarking small, yet critical amounts of funding to enable its development. Astute and brutally honest, Diane received standing ovations when she spoke as a guest speaker to the NSA workforce, yet to NSA senior management, she was a thorn in the side; she sought and enforced accountability. She was an honest steward of the taxpayers’ money. Not only did she endeavor to positively impact the NSA modernization process by every means at her disposal, only to be abandoned by her management, but she lobbied extensively in all three branches of government to modify or cancel NSA’s infamous warrantless wiretap program. Realizing the odds were stacked against her, she retired in 2002 and moved home to Oregon.

As Director of NSA, General Hayden was the person responsible for ensuring the Agency’s path forward as it sought to remain relevant in a world in which networked communications would know no boundaries. Ignoring the successful THINTHREAD prototype that was operating in the SIGINT Automation Research Center, and armed with the promise of a large infusion of funds from congress, Hayden launched a multi-billion dollar concept called TRAILBLAZERBillions of dollars and some five years later, it was declared a failure.  In recognition of NSA’s inability to successfully manage large programs important to national security, the United States Senate – for the first time in NSA’s history - revoked NSA’s “Milestone Decision Authority” which is Government-speak for the responsibility for planning, acquiring, and implementing major intelligence capabilities (see Senate bill S-1050, the National Defense Authorization Act for Fiscal Year 2004) and delegated the responsibility to the Under Secretary for Acquisition, Technology, and Logistics at the Department of Defense. That act is direct evidence that NSA’s management, especially General Hayden with whom Milestone Decision Authority rested - simply couldn’t be trusted to deliver the key capabilities the country so sorely needed.

Now, one might expect that a failure of such magnitude would impact General Hayden’s career.  Indeed, it did in a most curious way: he earned his fourth star and was promoted to the position of Deputy Director of National Intelligence in 2005.  Not only did Hayden fail to modernize NSA, but he misled the Senate in his confirmation hearings.  When asked about the failure at NSA, Hayden said (paraphrasing), “We overachieved – we tried to do too much,” as if he were some noble warrior who, by virtue of simple over-zealousness, had lost an important battle.  If not a lie, then he was too ignorant of acquisition procedures under the Federal Acquisition Regulation to know that he and the contractors he hired had utterly failed to properly manage the Agency’s efforts to modernize. Yet Congress in examining General Hayden’s performance was no better. It simply blinked, confirmed his nomination, and promoted him to the rank of full General.

What had gone wrong at NSA under Hayden? He mismanaged the entire acquisition process for one of the nation’s most important intelligence agencies. One of his most important obligations was to reduce the risk of failure – it’s called “risk mitigation” in planning circles and is especially important when attempting to make large gains in technical capabilities.  Successful organizations do that by surveying their organization for “good” technical development and knowledge where it exists.  It existed in the THINTHREAD prototype and the innovative minds of those surrounding it.  Yet Hayden ignored it, as did other managers who were championing other development efforts.  They got to Hayden. He listened.  They told him what he wanted to hear:  “We need a lot of money.”  The rest, as they say, is history. After all, a big NSA budget meant lucrative contracts for large businesses with political connections to the Hill.

The ramifications of the NSA failure have not been discussed before now.  What has not been discussed and what is generally not known, is that the costs of Hayden’s failures at NSA were utterly devastating in terms of both lives and intelligence lost. Not only were some 3000 souls lost on 9-11, but NSA wasted billions of taxpayer dollars on Hayden’s TRAILBLAZER boondoggle.  Moreover, while NSA spent some five years going in the wrong direction, the world did not stand still; it continued to change and so did threats and the way they communicate, terrorist organizations and cells came into being and morphed. In short, during those five years it was focused on TRAILBLAZER, NSA was blind to the burgeoning world of networked communications and the continuously emerging threats using them. Five years is roughly equivalent to 8 years in terms of the speed at which technology changes in network-based communications. All that missed knowledge was critical to national security interests, and it could come back to bite the nation - hard. Maybe it already has…

Perhaps more importantly, NSA lost the knowledge of Bill Binney, the visionary behind the Entity Mapping Analysis Business Process – a complex analytical process Bill and I defined a few years ago that enables the complete and rapid analysis of billions of electronic records (commonly referred to as “Big Data”) in order to find potential targets of intelligence interest without violating the Fourth Amendment privacy rights of innocent people. Now – in the wake of the President’s speech on NSA surveillance -  will Congress or the White House be interested in hearing about it?

ABOUT THE AUTHOR: Former NSA senior analyst, J. Kirk Wiebe has extensive training and experience in data analysis and signals intelligence. Wiebe received his undergraduate degree from University of Maryland in College Park and went on to obtain a master’s degree in Russian Studies at Indiana University Bloomington. Serving the NSA for decades, Wiebe helped pioneer a revolutionary information processing system that could have helped detect and prevent the terrorist attacks of 9/11 and accomplished great progress in developing other solutions for high-priority intelligence objectives. While at the NSA, Wiebe won the second highest distinction, the Meritorious Civilian Service Award, along with the Director of CIA’s Meritorious Civilian Service Award, a Letter of Commendation from the secretary of the Air Force, among others.

Disillusioned by the way the government was handling security matters, Wiebe resigned from the NSA and became a whistleblower after reporting the mismanagement and fraud inherent in the NSA’s use of TrailBlazer, the alternate intelligence gathering system chosen in favor of ThinThread, to Congress and informing the public of the government’s misuse of ThinThread technology for domestic surveillance purposes. Subsequently, the FBI raided and rummaged through Wiebe’s house, and Wiebe was summoned to NSA headquarters where his security clearance was suspended. The security clearance suspension along with the NSA’s “blackballing” of Wiebe to other government agencies prevented him from obtaining meaningful work in his area of expertise. Despite all of this, Wiebe continues to participate in whistleblowing activities in attempts to help ensure American citizens’ privacy rights.