The False Promise of a Crystal Ball


If there’s one theme that could define Pres­i­dent Oba­ma’s for­eign pol­i­cy the last six years, it is his tumul­tuous rela­tion­ship with the US intel­li­gence com­mu­ni­ty. The IC is Oba­ma’s favorite tar­get when Things Go Wrong: usu­al­ly because they did not use their crys­tal ball to cor­rect­ly pre­dict the future. It is that mis­per­cep­tion — that the IC can pre­dict future events — that is at the heart of Oba­ma’s unfair crit­i­cism, and its wide­spread belief is why he can use it so effec­tive­ly to avoid tak­ing respon­si­bil­i­ty for his own deci­sions.

Last night, on 60 Min­utes, Pres­i­dent Oba­ma answered a ques­tion about how ISIS came to be such a dire prob­lem by blam­ing it on his intel­li­gence offi­cials:

Our head of the intel­li­gence com­mu­ni­ty, Jim Clap­per, has acknowl­edged that, I think, they under­es­ti­mat­ed what had been tak­ing place in Syr­ia,” Oba­ma said… Obama’s remarks served as an acknowl­edge­ment that the Unit­ed States in recent years has been large­ly unaware of the pow­er behind ISIS, which has been char­ac­ter­ized as a ter­ror­ist group with some qual­i­ties of state-backed mil­i­tary forces.

The thing is, this is direct­ly at odds with what the intel­li­gence com­mu­ni­ty had been say­ing about ISIS for quite some time. If any­thing, the IC had been sound­ing the alarm, through leaks and tai­lored brief­in­gs to the White House itself, for longer than ISIS actu­al­ly had rep­re­sent­ed a grow­ing and seri­ous threat. In fact, in the mid­dle of the sum­mer they even said so:

The U.S. intel­li­gence com­mu­ni­ty warned about the “grow­ing threat” from Sun­ni mil­i­tants in Iraq since the begin­ning of the year, a senior intel­li­gence offi­cial said Tues­day — a claim that chal­lenges asser­tions by top admin­is­tra­tion offi­cials that they were caught off guard by the cap­ture of key Iraqi cities.

But look: the idea that the intel com­mu­ni­ty can pre­dict the rise of a sin­gle bad actor from a com­plex ecosys­tem of bad actors is a per­ni­cious myth. Often, it is a myth per­pet­u­at­ed by those run­ning the agen­cies.

As we look at ret­ro­spec­tive­ly on … what we now call the Arab Awak­en­ing,” the [DIA] deputy direc­tor said, ”what indi­ca­tions should we have picked up that per­haps we didn’t focus on?”

I worked at the DIA right before the Arab Spring, focused on Yemeni pol­i­tics. And I can tell you: we knew there was dis­con­tent bub­bling up. We knew that it would affect our mis­sion against al Qae­da in the Ara­bi­an Penin­su­la. But we did­n’t know how bad­ly it would affect that mis­sion. Our boss­es did­n’t care, they just want­ed intrigue, order of bat­tle infor­ma­tion, and stuff on al Qae­da. Despite being a polit­i­cal ana­lyst, I was nev­er able to get them to care about the pol­i­tics of Yemen, at least not in a crit­i­cal way. It sim­ply was­n’t their mis­sion, and because data is nev­er per­fect I could­n’t say “in three months you will see this protest move­ment join a region­al explo­sion of anti-author­i­tar­i­an protests and even­tu­al­ly top­ple the regime.”

Frankly, no one could pos­si­bly pre­dict such a thing with any accu­ra­cy, it is depen­dent on too many vari­ables.

What intel­li­gence can do, how­ev­er, is offer prob­a­bil­i­ties. “Thou­sands of job­less young men are con­cen­trat­ing in the big cities” is a pret­ty impor­tant indi­ca­tor that some­thing dis­rup­tive is about to hap­pen; but it’s a best guess as to whether it will be a nation­al or region­al­ly sig­nif­i­cant event. In Yemen, protests and riots are com­mon enough to where it was any­one’s guess whether the protests in 2011 would have his­tor­i­cal mean­ing or not. No one could real­ly say for cer­tain until after they were over and we could see what effect they had.

This was true region­al­ly: for most of the IC, Tunisian pol­i­tics sim­ply was not a pri­or­i­ty for them. Even those focused on North Africa did not see, in late 2010 when the Arab Spring dynam­ics were build­ing, much of inter­est hap­pen­ing in Tunisia. It was rel­a­tive­ly sta­ble, near­by to oth­er more inter­est­ing coun­tries like Alge­ria. There was no real rea­son to imag­ine that some­thing so dis­rup­tive as the Arab Spring would take place there.

But such a com­mon sense idea nev­er took hold in the White House. Almost at once, Oba­ma went on the offen­sive against his own intel­li­gence agen­cies, blam­ing them for not mag­i­cal­ly pre­dict­ing some­thing for which there were so few indi­ca­tors imme­di­ate­ly before­hand. Even in Tunisia, and in Egypt, peo­ple who lived there and lived out the protests dai­ly did not know where they were going — nor could they have pre­dict­ed them before­hand.

Intel­li­gence is not a crys­tal ball, and nev­er will be. There is no way to con­clu­sive­ly assem­ble a vast trove of data on social, polit­i­cal eco­nom­ic, and mil­i­tary fac­tors, fil­ter out what will be impor­tant and what will not be in six months’ time, and form a reli­able pre­dic­tion of what will come next. Intu­ition might allow some­one to do those things some of the time, and that is where expe­ri­ence can come into play. But exper­tise has lim­it­ed uses as well, and often is no bet­ter than chim­panzees throw­ing darts. In fact, a pro­fes­sion­al fore­cast­er, Jay Ulfelder, named his blog after such a phe­nom­e­non, and his thoughts on pre­dic­tion are rel­e­vant here:

Polit­i­cal sci­en­tists shouldn’t get sucked into bick­er­ing with their col­leagues over small dif­fer­ences in fore­cast accu­ra­cy around sin­gle events, because those dif­fer­ences will rarely con­tain enough infor­ma­tion for us to learn much from them. Instead, we should take pre­dic­tion seri­ous­ly as a means of test­ing com­pet­ing the­o­ries by doing two things.

First, we should build fore­cast­ing mod­els that clear­ly rep­re­sent con­trast­ing sets of beliefs about the caus­es and pre­cur­sors of the things we’re try­ing to pre­dict. Sec­ond, we should only real­ly com­pare the pre­dic­tive pow­er of those mod­els across mul­ti­ple events or a longer time span, where we can be more con­fi­dent that observed dif­fer­ences in accu­ra­cy are mean­ing­ful. This is basic sta­tis­tics.

In oth­er words, even with large amounts of data, pre­dic­tion requires mul­ti­ple attempts over a long span of time to devel­op a usable mod­el for pre­dic­tion. Bick­er­ing over indi­vid­ual miss­es con­fus­es what the data can show and what is pos­si­ble to derive from them. More­over, the lev­el of gran­u­lar­i­ty Oba­ma is demand­ing in his com­ments — which spe­cif­ic group will emerge on a spe­cif­ic time­line — is just not pos­si­ble to pre­dict before­hand. Intel­li­gence can­not pro­vide an iron clad pre­dic­tion of a spe­cif­ic event. It is unrea­son­able to expect it to. Intel­li­gence can give a prob­a­bil­i­ty of an event. But it can­not pro­vide cer­tain­ty.

Oba­ma has nev­er devel­oped a good under­stand­ing of what intel­li­gence does, what it is, and how it can be used most appro­pri­ate­ly. His Pres­i­den­tial Dai­ly Brief is filled with maps high­light­ing the sum­maries of intel­li­gence find­ings. It is a bench­mark prod­uct. Those maps have to come from some­where, right? Yet when Oba­ma was vis­it­ing a Five Guys in 2009 — a local burg­er joint — he did not know what agency pro­duced all of those maps for him every day.

So explain to me exact­ly what this Nation­al Geospa­tial…” Oba­ma said, after the work­er men­tioned his employ­er, accord­ing to a video of the event.

We work with, uh, satel­lite imagery,” the work­er, Wal­ter replied.

If there’s one com­plaint you hear more than any oth­er in the work­ing lev­el of the intel­li­gence com­mu­ni­ty, it is that Oba­ma is com­plete­ly checked out. When ana­lysts get to go brief their spe­cial­ty in the Oval Office — a huge hon­or, one that many cher­ish as the result of hard work — too many com­plain that they’re bare­ly even acknowl­edged. There’s no engage­ment on the top­ic. Noth­ing. Five months into his job, after see­ing brand­ed prod­ucts pro­duced by the Nation­al Geospa­tial Intel­li­gence Agency, Oba­ma did not know what they were or what they did. He nev­er asked what “NGA” means on the cor­ner of the image. He nev­er asked where the col­or-cod­ed maps of Afghanistan or Iraq came from. He had zero curios­i­ty about it.

(There are count­less oth­er exam­ples, includ­ing when the IC and DOD clashed over Afghanistan — the IC’s pes­simism was right and the DOD’s opti­mism was wrong but Oba­ma went with the mil­i­tary any­way and then put its star gen­er­al, David Petraeus, in charge of the CIA. It was a slap in the face.)

So in 2014, five years lat­er, when Oba­ma is still com­plain­ing that the intel­li­gence com­mu­ni­ty is still not wav­ing its hands over a crys­tal ball to tell him about a sin­gle actor emerg­ing from a com­plex threat envi­ron­ment, he is demon­strat­ing that he has nev­er both­ered to learn what the IC is or what they do. He has ignored them about ISIS, then he blames them for not get­ting ISIS “right.”

Yet despite the years Oba­ma has spent belit­tling and scape­goat­ing the intel com­mu­ni­ty, he still relies on them to build tar­get­ing pack­ages against ISIS (and relied on them in Pak­istan and Yemen too). So my big ques­tion is: if the IC is so bad that it “missed” the Arab Spring, it “missed” ISIS, and so on, why does Oba­ma think they won’t “miss” ISIS?

The answer is that Oba­ma does not think that. He knows they’ll be able to iden­ti­fy tar­gets in Iraq and Syr­ia fair­ly reli­ably. It won’t put the civil­ian casu­al­ty count at zero but he has a rea­son­able con­fi­dence they will do a com­pe­tent job. The out­cry he is foist­ing upon the IC is just the lat­est round of scape­goat­ing: blam­ing them for his ignor­ing their months of warn­ings. It is to be expect­ed. But it should not be tac­it­ly accept­ed as truth.

UPDATE: Here is a per­fect exam­ple of how bad­ly peo­ple mis­un­der­stand the role of intel­li­gence in pol­i­cy. In June, the warn­ings from the intel­li­gence com­mu­ni­ty were so wide­spread and so dire that in com­ments to reporters a CIA spokesman went to great pains to say so:

With­out direct­ly address­ing the CIA’s pos­ture in Iraq, agency spokesman Dean Boyd not­ed that 40 offi­cers have died in the line of duty since Sep­tem­ber 2001. He called “offen­sive” any sug­ges­tion that “CIA offi­cers are sit­ting behind desks, hid­ing out in green zones, or oth­er­wise tak­ing it easy back at the embassy.”

Boyd said the intel­li­gence com­mu­ni­ty pro­vid­ed plen­ty of warn­ing to the Oba­ma admin­is­tra­tion that the insur­gent Islam­ic State in Iraq and Lev­ant, known as ISIL, could move on Iraqi cities.

Any­one who has had access to and actu­al­ly read the full extent of CIA intel­li­gence prod­ucts on ISIL and Iraq should not have been sur­prised by the cur­rent sit­u­a­tion,” he said.

The AP went not to note that, despite this pre­dic­tion, the intel­li­gence com­mu­ni­ty “did not appear to antic­i­pate ISIL’s offen­sive on June 10 to seize Mosul.” That is because the IC can­not pre­dict such a thing until it is vir­tu­al­ly hap­pen­ing. It is a fun­da­men­tal mis­un­der­stand­ing of what intel is — pre­tend­ing it is a crys­tal ball of the future, instead of a con­sid­ered best guess at future like­li­hoods.

Joshua Foust used to be a foreign policy maven. Now he helps organizations communicate strategically and build audiences.