“Sampling 47 clusters and then extrapolating – even if the samplers themselves are likely to use cautious language – is likely to produce massive inflammatory headlines about the effect of a war in a country of 35m, from just ten or twenty thousand interviews”

As Daniel Davies pointed out in the Guardian, there is nothing wrong with the sample size:

http://www.guardian.co.uk/commentisfree/2006/oct/12/howtonotliewithstatistics

[Iraq also wasn't a country of 35m, CIA World Factbook gives 24m in July 2002:

http://www.faqs.org/docs/factbook/geos/iz.html

Davies writing in 2006 in the above article gives 26m]

But the point is that you get a per year mortality rate during the 40 months after the war started of 13.3 per 1000 (95% confidence interval: 10·9–16·1). You then have a pre-war mortality rate of 5.5 per 1000 (4·3–7·1).

The difference between the two figures is the excess mortality, and scaling this up you get 655,000 deaths in those 40 months. The 95% confidence interval is for 400,000 to 950,000 deaths. Roughly 600,000 of the 655,000 figure pertain to “violent” deaths.

The fact is that yes, 600,000 makes a scary headline figure, and you can’t just extrapolate say “It was 600,000″. But you can do the confidence interval, and from that, as Davies points out:

“If you go out and ask 12,000 people whether a family member has died and get reports of 300 deaths from violence, then that is not consistent with there being only 60,000 deaths from violence in a country of 26 million. It is not even nearly consistent.”

60,000 being the figure IBC had at the time. Now they say about 100,000, but that figure is up till NOW. Not July 2006. So you can’t compare the 100,000 figure with the 600,000 one.

“I think that is suspect, and deserves occasional criticism, when there are other bodies = including the Iraqi govt – who think differently. As one of the rivals quoted says: there is somehow a missing half million death certificates.”

Juan Cole says:

“First of all, Iraqi Muslims don’t believe in embalming or open casket funerals days later. They believe that the body should be buried by sunset the day of death, in a plain wooden box. So there is no reason to expect them to take the body to the morgue. Although there are benefits to registering with the government for a death certificate, there are also disadvantages. Many families who have had someone killed believe that the government or the Americans were involved, and will have wanted to avoid drawing further attention to themselves by filling out state forms and giving their address.

Personally, I believe very large numbers of Iraqi families quietly bury their dead without telling the government of all people anything about it. Another large number of those killed is dumped in the Tigris river by their killers. A fisherman on the Tigris looking for lunch recently caught the corpse of a woman. The only remarkable thing about it is that he let it be known to the newspapers. I’m sure the Tigris fishermen throw back unwanted corpses every day.

Not to mention that for substantial periods of time since 2003 it has been dangerous in about half the country just to move around, much less to move around with dead bodies.”

http://www.juancole.com/2006/10/655000-dead-in-iraq-since-bush.html

Put simply, if there was a war going on, would you be surprised that the government doesn’t know about a substantial portion of deaths?

Add to that also the general incompetence of the post-invasion governance by America and Britain, and it doesn’t seem so surprising to me.

“There are also considerable doubts about the pre-war mortality figures – amazingly low, given the high rates of infant mortality, caused to some degree by sanctions.”

See Tim Lambert:

http://scienceblogs.com/deltoid/2004/11/bolt.php

See also Brad Delong:

http://delong.typepad.com/sdj/2006/10/55_deaths_per_y.html

“Take, for example, the statement that 31% of the deaths came from coalition action. Using lower estimates that must be 100,000 people – in just a few months of fighting and using just 1% of the bombs dropped during the Vietnam war (http://wiki.answers.com/Q/How_many_tons_of_bomb_were_dropped_in_Iraq). It certainly seems very out of whack with the much more sustained, brutal War in Vietnam- if a million NV people died in combat, many many of those were killed by SV people.”

There are many things wrong with this paragraph.

1. Not a “few months” of fighting. 40 months. Deaths didn’t stop just because Bush declared Mission Accomplished.

2. As the study points out, the most common cause of death was gunfire. So bombs aren’t anywhere near as important.

3. You cite an unsourced Wiki answers page for your stat.

3. While later you say that bombing is “more targeted”, don’t you think that would make it easy to kill more people? And wouldn’t bombs have gotten more powerful since the 1970s too? Don’t you think that any “insurgents” might have a nasty habit of hanging around civilian areas?

4. You cite a million as the number of NV that died in combat, but what about the number of civilians. What about non-NV? Check out the panel on the right on Wikipedia for example:

http://en.wikipedia.org/wiki/Vietnam_War

Many many more people died in Vietnam than you think.

5. There are also major differences between Iraq and Vietnam. Sectarian violence for one thing. Deliberate attacks on civilians by insurgents for another. Vietnam was bloody, but the violence was of a different kind.

But regardless, that paragraph, and the ones that follow, like on the Blitz (seriously, why cite the Blitz? Why not Stalingrad? I am not saying that Iraq was like Stalingrad, far from it, but why the Blitz? Justify. You can’t just assume the war was like previous ones like the Blitz, or Vietnam – that has to be worked out. The best way to do that is to examine the conditions of the country and do studies e.g. the Lancet one), are just one long argument from incredulity. When you sit down and think about, the mortality rate is quite believable.

Some more discussion of this theme:

http://crookedtimber.org/2006/10/12/statistics-and-the-scale-of-societies/

I should point out that the sources I have linked to here (Delong, Davies, Lambert, Crooked Timber and Cole) are all reasonably reliable sources. Yes they are blog posts and one Cif article, but what I mean is they are people I generally trust normally, they are not random people I just came across on Google, but people I have been reading for a while. Although you already know of Brad Delong of course.

[Interesting, while I was looking for these posts, I came across the name "Tim Worstall" in association with the first Lancet study (2004), which you may find interesting:

http://web.archive.org/web/20041104053528/http://www.techcentralstation.com/102904J.html
http://scienceblogs.com/deltoid/2004/11/tcs2.php
http://crookedtimber.org/2004/11/01/talking-rubbish-about-epidemiology/
http://scienceblogs.com/deltoid/2004/11/tcs3.php

]

Sorry for the length of this, and if it appears snarky or whatever, it was not meant to be. Sorry also, for the lateness of this comment.

Rubbish Jamo.

Name the “other incidents”. You’ve been misled.

I BELIEVE the Lancet found that 90% of so of their interviewees had death certificates. So if that is typical, the nationwide issuance of such certificates ought to help.

Point taken: none of the estimates should be quoted except larded through with caveats.

I don’t mean to imply that the Lancet study is correct, or that the number of casualties must be half-a-million. But the Iraq Body Count is *necessarily* an undercount by the nature of its methodology. The question is, by how much?

So it is strange to quote its estimate as the most accurate, as the media constantly do.

When something seems obvious . …

Err, not true. The Brookings Index stated that it used IBC data for part of its estimates, for part of the period. During the period it used IBC (2003-2005), it used part of IBC data excluding “crime” and then made its own estimates for “crime” to add on top. For 2006 it used UNAMI data and then after that i think it uses the Petraeus figures from the US DoD. On top of this the Brookings figures include estimates for combatants that IBC’s don’t.

Calling Brookings count “the same data” as IBC is false, though there is some overlap.

Likewise with the AP figures. These come from counts of death certificates issued by hospitals and morgues, as compiled by the Iraqi Health Ministry. These cover 2004-2008. The AP used IBC to fill in 2003 and some of 2009, where the death certificate data wasn’t available. IBC uses some official data like this to supplement figures compiled in a different way, but again it is false to say these are “the same data”, though again there is some overlap.

Sean also says: “The Spectator doesn’t even note the criticism that the IBC levels against itself–namely, that it is not really an estimate at all–merely a lower bound; “Gaps in recording and reporting suggest that even our highest totals to date may be missing many civilian deaths from violence.” With as much disorder as was present in Iraq “many” could be an awful lot.”

Fair enough, but then The Spectator should have also quoted that IBC also rejects the notion that this issue explains the difference between its count and the estimate from the Lancet.

Sean says, “[Lancet] used an internationally accepted methodology (random sampling) to estimate actual increased mortality, and that is where the 600 000 figure came from.” Well, there has been much criticism that it did not really use any kind of proper random sampling, and this could explain a lot of the problem:
http://dissident93.wordpress.com/2008/12/15/journal-of-peace-research-award/.

Of course there are many more criticisms of it beyond the sampling, right down to the raw data being fabricated. When the authors of the study won’t even let established survey researchers like AAPOR see something as simple as the _questionnaire_, and stonewall and cover-up on other issues there’s hardly any basis for confidence.

http://en.wikipedia.org/wiki/Lancet_surveys_of_Iraq_War_casualties#Criticisms_and_countercriticisms_2
There are also considerable doubts about the pre-war mortality figures – amazingly low, given the high rates of infant mortality, caused to some degree by sanctions.

Take, for example, the statement that 31% of the deaths came from coalition action. Using lower estimates that must be 100,000 people – in just a few months of fighting and using just 1% of the bombs dropped during the Vietnam war (http://wiki.answers.com/Q/How_many_tons_of_bomb_were_dropped_in_Iraq). It certainly seems very out of whack with the much more sustained, brutal War in Vietnam- if a million NV people died in combat, many many of those were killed by SV people.

Take, also, the Blitz: In 1940-41, the Germans bombed London, a metropolitan area of 8m people, for almost 8 months, continuously and with incredibly crude munitions. A million dwellings were destroyed. About 22,000 Londoners lost their lives. I appreciate that coalition action is not all bombing . . . .but it still seems remarkable that many times more as many might have died during a shorter bombing campaign in a modern more targeted era.

I am clearly an amateur, and my views have little weight. But I find this figure difficult to take in, and whenever I see it repeated, like in other complex matters, it merely makes me reflect upon the writer’s CHOICE of that statistic to use, without qualificatoin, and therefore makes me think more about the writer’s motivations. I do not mean to accuse them of lying. And I do not mean to imply that the difference between two such very high figures can make a difference as to justifying the war.

This is not to say it is correct – it had very wide confidence intervals – but given there’s been no inclination by either government to calculate the impact it deserves more respect than this.