The False Promise of a Crystal Ball

If there’s one theme that could define President Obama’s foreign policy the last six years, it is his tumultuous relationship with the US intelligence community. The IC is Obama’s favorite target when Things Go Wrong: usually because they did not use their crystal ball to correctly predict the future. It is that misperception — that the IC can predict future events — that is at the heart of Obama’s unfair criticism, and its widespread belief is why he can use it so effectively to avoid taking responsibility for his own decisions.

Last night, on 60 Minutes, President Obama answered a question about how ISIS came to be such a dire problem by blaming it on his intelligence officials:

“Our head of the intelligence community, Jim Clapper, has acknowledged that, I think, they underestimated what had been taking place in Syria,” Obama said… Obama’s remarks served as an acknowledgement that the United States in recent years has been largely unaware of the power behind ISIS, which has been characterized as a terrorist group with some qualities of state-backed military forces.

The thing is, this is directly at odds with what the intelligence community had been saying about ISIS for quite some time. If anything, the IC had been sounding the alarm, through leaks and tailored briefings to the White House itself, for longer than ISIS actually had represented a growing and serious threat. In fact, in the middle of the summer they even said so:

The U.S. intelligence community warned about the “growing threat” from Sunni militants in Iraq since the beginning of the year, a senior intelligence official said Tuesday — a claim that challenges assertions by top administration officials that they were caught off guard by the capture of key Iraqi cities.

But look: the idea that the intel community can predict the rise of a single bad actor from a complex ecosystem of bad actors is a pernicious myth. Often, it is a myth perpetuated by those running the agencies.

“As we look at retrospectively on … what we now call the Arab Awakening,” the [DIA] deputy director said, ”what indications should we have picked up that perhaps we didn’t focus on?”

I worked at the DIA right before the Arab Spring, focused on Yemeni politics. And I can tell you: we knew there was discontent bubbling up. We knew that it would affect our mission against al Qaeda in the Arabian Peninsula. But we didn’t know how badly it would affect that mission. Our bosses didn’t care, they just wanted intrigue, order of battle information, and stuff on al Qaeda. Despite being a political analyst, I was never able to get them to care about the politics of Yemen, at least not in a critical way. It simply wasn’t their mission, and because data is never perfect I couldn’t say “in three months you will see this protest movement join a regional explosion of anti-authoritarian protests and eventually topple the regime.”

Frankly, no one could possibly predict such a thing with any accuracy, it is dependent on too many variables.

What intelligence can do, however, is offer probabilities. “Thousands of jobless young men are concentrating in the big cities” is a pretty important indicator that something disruptive is about to happen; but it’s a best guess as to whether it will be a national or regionally significant event. In Yemen, protests and riots are common enough to where it was anyone’s guess whether the protests in 2011 would have historical meaning or not. No one could really say for certain until after they were over and we could see what effect they had.

This was true regionally: for most of the IC, Tunisian politics simply was not a priority for them. Even those focused on North Africa did not see, in late 2010 when the Arab Spring dynamics were building, much of interest happening in Tunisia. It was relatively stable, nearby to other more interesting countries like Algeria. There was no real reason to imagine that something so disruptive as the Arab Spring would take place there.

But such a common sense idea never took hold in the White House. Almost at once, Obama went on the offensive against his own intelligence agencies, blaming them for not magically predicting something for which there were so few indicators immediately beforehand. Even in Tunisia, and in Egypt, people who lived there and lived out the protests daily did not know where they were going — nor could they have predicted them beforehand.

Intelligence is not a crystal ball, and never will be. There is no way to conclusively assemble a vast trove of data on social, political economic, and military factors, filter out what will be important and what will not be in six months’ time, and form a reliable prediction of what will come next. Intuition might allow someone to do those things some of the time, and that is where experience can come into play. But expertise has limited uses as well, and often is no better than chimpanzees throwing darts. In fact, a professional forecaster, Jay Ulfelder, named his blog after such a phenomenon, and his thoughts on prediction are relevant here:

Political scientists shouldn’t get sucked into bickering with their colleagues over small differences in forecast accuracy around single events, because those differences will rarely contain enough information for us to learn much from them. Instead, we should take prediction seriously as a means of testing competing theories by doing two things.

First, we should build forecasting models that clearly represent contrasting sets of beliefs about the causes and precursors of the things we’re trying to predict. Second, we should only really compare the predictive power of those models across multiple events or a longer time span, where we can be more confident that observed differences in accuracy are meaningful. This is basic statistics.

In other words, even with large amounts of data, prediction requires multiple attempts over a long span of time to develop a usable model for prediction. Bickering over individual misses confuses what the data can show and what is possible to derive from them. Moreover, the level of granularity Obama is demanding in his comments — which specific group will emerge on a specific timeline — is just not possible to predict beforehand. Intelligence cannot provide an iron clad prediction of a specific event. It is unreasonable to expect it to. Intelligence can give a probability of an event. But it cannot provide certainty.

Obama has never developed a good understanding of what intelligence does, what it is, and how it can be used most appropriately. His Presidential Daily Brief is filled with maps highlighting the summaries of intelligence findings. It is a benchmark product. Those maps have to come from somewhere, right? Yet when Obama was visiting a Five Guys in 2009 — a local burger joint — he did not know what agency produced all of those maps for him every day.

“So explain to me exactly what this National Geospatial…” Obama said, after the worker mentioned his employer, according to a video of the event.

“We work with, uh, satellite imagery,” the worker, Walter replied.

If there’s one complaint you hear more than any other in the working level of the intelligence community, it is that Obama is completely checked out. When analysts get to go brief their specialty in the Oval Office — a huge honor, one that many cherish as the result of hard work — too many complain that they’re barely even acknowledged. There’s no engagement on the topic. Nothing. Five months into his job, after seeing branded products produced by the National Geospatial Intelligence Agency, Obama did not know what they were or what they did. He never asked what “NGA” means on the corner of the image. He never asked where the color-coded maps of Afghanistan or Iraq came from. He had zero curiosity about it.

(There are countless other examples, including when the IC and DOD clashed over Afghanistan — the IC’s pessimism was right and the DOD’s optimism was wrong but Obama went with the military anyway and then put its star general, David Petraeus, in charge of the CIA. It was a slap in the face.)

So in 2014, five years later, when Obama is still complaining that the intelligence community is still not waving its hands over a crystal ball to tell him about a single actor emerging from a complex threat environment, he is demonstrating that he has never bothered to learn what the IC is or what they do. He has ignored them about ISIS, then he blames them for not getting ISIS “right.”

Yet despite the years Obama has spent belittling and scapegoating the intel community, he still relies on them to build targeting packages against ISIS (and relied on them in Pakistan and Yemen too). So my big question is: if the IC is so bad that it “missed” the Arab Spring, it “missed” ISIS, and so on, why does Obama think they won’t “miss” ISIS?

The answer is that Obama does not think that. He knows they’ll be able to identify targets in Iraq and Syria fairly reliably. It won’t put the civilian casualty count at zero but he has a reasonable confidence they will do a competent job. The outcry he is foisting upon the IC is just the latest round of scapegoating: blaming them for his ignoring their months of warnings. It is to be expected. But it should not be tacitly accepted as truth.

UPDATE: Here is a perfect example of how badly people misunderstand the role of intelligence in policy. In June, the warnings from the intelligence community were so widespread and so dire that in comments to reporters a CIA spokesman went to great pains to say so:

Without directly addressing the CIA’s posture in Iraq, agency spokesman Dean Boyd noted that 40 officers have died in the line of duty since September 2001. He called “offensive” any suggestion that “CIA officers are sitting behind desks, hiding out in green zones, or otherwise taking it easy back at the embassy.”

Boyd said the intelligence community provided plenty of warning to the Obama administration that the insurgent Islamic State in Iraq and Levant, known as ISIL, could move on Iraqi cities.

“Anyone who has had access to and actually read the full extent of CIA intelligence products on ISIL and Iraq should not have been surprised by the current situation,” he said.

The AP went not to note that, despite this prediction, the intelligence community “did not appear to anticipate ISIL’s offensive on June 10 to seize Mosul.” That is because the IC cannot predict such a thing until it is virtually happening. It is a fundamental misunderstanding of what intel is — pretending it is a crystal ball of the future, instead of a considered best guess at future likelihoods.