Download the file
Lewis and I have a lot in common: he and I both used to work at the Defense Intelligence Agency, and we’re both geeks at heart who desperately want government to think differently about technology. In this episode, we talk about the tradition of cooperation between government and the technology sector, and whether more data leads to better intelligence analysis. Some quotes:
Anybody from the technology world who is having a conversation with an intelligence agency customer in 2002, 3, 4, 5…very rarely was that conversation (about privacy implications)
(NSA) was trying to meet what it thought were urgent requirements, and therefore kept any discussion of the policy implications and the civil liberty implications completely at bay.
When I would look, on a visit to Baghdad, at what soldiers in the field were doing, analysts were completely hunkered down inside well-protected bases, and staring at a screen the entire time. If you believe that you can have a totality of data…a picture of the Red force that is so accurate that you can predict what the Red force will do at any given time…then you’re gonna stay inside your bunker and never interact with the reality of Iraq…That’s another part of this discussion: What’s the optimality of technology’s contribution in modern warfare and combat?
Items mentioned in this episode:
We didn’t mention it in this episode, but if you want to understand the work of intelligence analysis, you must read Richard Heuer’s The Psychology of Intelligence Analysis. For content specific to this episode, see Chapter 5, “Do you really need more information?”, which is exactly what you think it’s about. If you want even more readings, try Curing Analytic Pathologies, by Jeffrey Cooper, or Analytic Culture in the U.S. Intelligence Community, by Rob Johnston. All three were published by the U.S. Government and are entirely free online.
I reference these works because they all touch on a fundamental problem that is critical to the subject of this podcast: How much information is enough? When do the costs of additional data outweigh its benefits? There are privacy costs, but these works focus on the cognitive costs of data: sooner or later, more information only serves to cloud your thinking. Reforming the analysis process to help analysts reach better decisions with the information they have has been an ongoing struggle for decades.
However, when reading them, keep in mind that today’s analysts track threats of a different nature than those of 30 years ago. Surveilling the Supreme Soviet, whose members are known, to defend against long-term strategic threats is not the same thing as surveilling a network of anonymous actors to defend against subway bombings. Different enough to make it worthwhile to collect metadata on everyone within three hops of a target? I don’t know. But it is different enough to warrant consideration.