Episode 2: Lewis Shepherd

Download the file

Lewis and I have a lot in common: he and I both used to work at the Defense Intelligence Agency, and we’re both geeks at heart who desperately want government to think differently about technology. In this episode, we talk about the tradition of cooperation between government and the technology sector, and whether more data leads to better intelligence analysis. Some quotes:

Anybody from the technology world who is having a conversation with an intelligence agency customer in 2002, 3, 4, 5…very rarely was that conversation (about privacy implications)

(NSA) was trying to meet what it thought were urgent requirements, and therefore kept any discussion of the policy implications and the civil liberty implications completely at bay.

When I would look, on a visit to Baghdad, at what soldiers in the field were doing, analysts were completely hunkered down inside well-protected bases, and staring at a screen the entire time. If you believe that you can have a totality of data…a picture of the Red force that is so accurate that you can predict what the Red force will do at any given time…then you’re gonna stay inside your bunker and never interact with the reality of Iraq…That’s another part of this discussion: What’s the optimality of technology’s contribution in modern warfare and combat?

Items mentioned in this episode:

We didn’t mention it in this episode, but if you want to understand the work of intelligence analysis, you must read Richard Heuer’s The Psychology of Intelligence Analysis. For content specific to this episode, see Chapter 5, “Do you really need more information?”, which is exactly what you think it’s about. If you want even more readings, try Curing Analytic Pathologies, by Jeffrey Cooper, or Analytic Culture in the U.S. Intelligence Community, by Rob Johnston. All three were published by the U.S. Government and are entirely free online.

I reference these works because they all touch on a fundamental problem that is critical to the subject of this podcast: How much information is enough? When do the costs of additional data outweigh its benefits? There are privacy costs, but these works focus on the cognitive costs of data: sooner or later, more information only serves to cloud your thinking. Reforming the analysis process to help analysts reach better decisions with the information they have has been an ongoing struggle for decades.

However, when reading them, keep in mind that today’s analysts track threats of a different nature than those of 30 years ago. Surveilling the Supreme Soviet, whose members are known, to defend against long-term strategic threats is not the same thing as surveilling a network of anonymous actors to defend against subway bombings. Different enough to make it worthwhile to collect metadata on everyone within three hops of a target? I don’t know. But it is different enough to warrant consideration.

Episode 1: Clay Shirky

Download the file

NOTE: There’s some reverb in this episode. I apologize. Audio quality will improve in future episodes.

Clay is the author of several books on Internet society, including “Here Comes Everybody” and “Cognitive Surplus.” I chose him for the first episode because he’s good at reviewing problems as an objective third party. In this discussion, we talk about 9/11 as a criminal act vs an act of war; the militarization of police power in small American towns; and why it’s dangerous for the NSA to have all this data (“It’s not because anybody at the NSA cares what I’m doing.”)

Some quotes:

“As a citizen, I expect that if the citizens don’t approve of what the government’s doing, even if we didn’t know, (the government) won’t do that.”

“The government has simply lost the habit of self-discipline relative to the expectations of American citizens.”

“I think every year we go in which there is no major court case hashing out the 4th Amendment limits of this kind of surveillance is bad for the nation.”

Names and articles mentioned in this episode: