Shane Harris of the National Journal continues his first-rate efforts to ferret out the details of the inner workings of the intelligence community in his latest article, headlined “Terrorist Profiling, Version 2.0.” The article focuses on an Air Force procurement solicitation entitled “Tangram,” which seems to be causally linked to the former Total Information Awareness (TIA) program. But more interestingly, the solicitation document describes the challenges that the intelligence community has faced in terror-related data analysis in recent years. Harris writes:
In addition to descriptions of Tangram, the document offers a rare and surprisingly candid analysis of intelligence agencies’ fits and starts — and failures — in other efforts to profile terrorists through data mining: Researchers, for example, haven’t moved beyond “guilt-by-association models” that link suspected terrorists to other, potentially innocent people, and then rank the suspects by level of suspicion.
“To date, the predominant approaches have used a guilt-by-association model to derive suspicion scores,” the Tangram document states. “In the cases where we have knowledge of a seed entity [a known person] in an unknown group, we have been very successful at detecting the entire group. However, in the absence of a known seed entity, how do we score a person if nothing is known about their associates? In such an instance, guilt-by-association fails.”
Intelligence and privacy experts who reviewed the document said that it reaffirms their long-held belief that many computerized terrorist-profiling methods are largely ineffective. It also raises significant privacy concerns, because to distinguish terrorists from innocent people, a system that’s as broad as Tangram purports to be would require access to many databases that contain private information about Americans, the experts said, including credit card transactions, communications records, and even Internet purchases.
I’ve read through the document myself, and it’s true that it’s a candid assessment of the challenges associated with terror-profiling. But after reading it, I don’t come to the conclusion that this program is futile or misguided, as several of the people that Harris talked to seem to do. Instead, I see a research program whose leaders have made progress in some areas, still have meaningful goals in others, and realize that certain objectives are unlikely to be achieved given the practical limitations imposed by human volition and cognition. It makes sense to continue to research and improve these programs, using “synthetic data or foreign-intelligence data already being used by analysts” as the article notes, as one of many facets of our homeland security and counterterrorism efforts.
My main concern about these programs is the lack of oversight, which creates the risk that this type of program will head in a direction that does violate privacy and civil liberties. The application of this type of technology is too important to be left solely to the scientists and technicians. But at the same time, we’ve seen that when these programs are publicly discussed by the executive branch – as was the case with Total Information Awareness – they become vulnerable to demagogic and misguided interpretations and accusations. The debate over the use of data analysis for counterterrorism thus has become something of a shell game over the last few years, which is too bad, but perhaps as good as we can do given the sensitivities involved.