Actually, We Don’t Know How Many Civilians Are Dying in Drone Strikes.

Peter Bergen and Katherine Tiedemann at the New America Foundation are keeping one of the most useful datasets on drone strike fatalities that I know of. They’ve been tallying reports of strikes since 2004. They limit their data to those reported by:

“news organizations with deep and aggressive reporting capabilities in Pakistan (the New York Times, Washington Post, and Wall Street Journal), accounts by major news services and networks (the Associated Press, Reuters, Agence France-Presse, CNN, and the BBC), and reports in the leading English-language newspapers in Pakistan (the Daily Times, Dawn, the Express Tribune, and the News), as well as those from Geo TV, the largest independent Pakistani television network.”

This gives them a systematic, if conservative, estimate of total fatalities. They then gather, archive and code the data in a transparent and replicable way – unlike other estimates of drone strikes that don’t provide evidence of how they derive their statistics. Bergen and Tiedemann’s results gives us a descriptive picture of how drone strikes have increased over time and changed by location and impact. Their website includes a set of helpful visualizations:

While I find the effort impressive and have sometimes cited Bergen and Tiedemann’s data as decent mid-range estimates of drone-strike fatalities, I am developing some reservations about the coding methods being used and the inferences being made after looking more closely at their dataset. In particular, Bergen and Tiedemann’s estimates of the ratio between civilian to militant deaths by strikes bears closer examination.

1) It’s important to emphasize that these estimates, most recently outlined in a Foreign Policy article entitled “There Were More Drone Strikes — and Far Fewer Civilians Killed”, do not actually measure of the ratio of civilian to militant deaths. They measure the ratio of reported civilian to reported militant deaths. This is a very important distinction that seems to have been lost on Bergen and Tiedeman, who claim in their recent Foreign Policy piece “even as the number of reported strikes has skyrocketed, the percentage of non-militants killed by the attacks has plummeted.” It is more accurate to say that the percentage of non-militants reported killed by the attacks has plummeted.

Acknowledging that this is data on news reporting more than data on actual deaths puts the data in a different light. For example, the declining trend in ‘civilian deaths’ could mean fewer civilians are in fact being killed. Or it could mean a shift in how reporters are interpreting ‘civilian’ or ‘militant’ over this time period – a period in which the very concept of the “civilian” is being degraded in popular, media and diplomatic discourse both by evolving events and by the notion, among other things, that a person loses their civilian status simply by being suspected of militancy against their government.

2) But let us set aside for a moment the question of whether (and which part of) war law (and therefore the civilian/combatant distinction) really applies to US airpower inside Pakistan. And let’s assume that it is legitimate to treat “suspected militant” as synonymous with “combatant” and “non-suspect” as synonymous with “civilian.” I still worry that Bergen and Tiedemann are overestimating militant deaths in these reports. One of the reasons for this is probably inevitable given their method: they rely on what mainstream reporters say, and reporters rely on information from the governments doing the killing. But another reason is completely within their control: by using “militant” rather than “civilian” as the default code when the actual status of the deceased, according to the reports, is “unknown” or contested.

For example, Bergen and Tiedemann record a December 31, 2009 attack in which CNN reported 2 were killed, 3 injured, and it was unclear whether any of the dead or injured were militants; and in which AFP reported 3 militants were killed and that “the identity of the militants is not known yet”; This event was coded in the Bergen/Teidemann dataset as “Al-Qaeda/Taliban killed: 2-5; Others killed; unknown.”

At a minimum, it would seem to me, this event should have been coded as 2-5 deaths “status unknown” rather than counting as either definitely militants or definitely civilians. In fact, however, it would be more consistent with humanitarian law, from which the civilian/combatant distinction is derived, to record any deaths in which the status of the deceased are unknown as civilians. (Article 50(1) of the 1st Additional Protocol to the Geneva Conventions states that “In case of doubt whether a person is a civilian, that person shall be considered to be a civilian.”)

I would be interested to know how the Bergen/Tiedemann ratio of “civilians”/(non-militant suspects) to “combatants”(militant suspects) would change if their coding were replicated with either of these two minor yet significant changes introduced. (In the case of the Jamestown Foundation study released earlier this year the latter approach would have made an enormous difference in their findings even with males over 13 excluded, jumping the civilian hit rate from 5% to 27%.)

3) All this only goes to show how impoverished our understanding of the civilian impacts of different weapons will remain until some independent verification mechanism is established for tallying and reporting the dead in today’s wars. Important efforts are underway to fill this critical gap in the Geneva regime and should be supported by advocates of human rights and humanitarian accountability.

[cross-posted at Duck of Minerva]