Analysis By Paul Reynolds World affairs correspondent, BBC News website

Many Iraqis have lost relatives to violence

The estimate that about 655,000 people have died in Iraq as a result of the 2003 invasion is such a large figure that it has led to two differing interpretations.

Those who had faith in an earlier report from 2004 - also published in the medical journal The Lancet - are now able to say that this larger survey proves their point that Iraqi deaths have been far greater than publicly reported, and have now reached what the report calls "a humanitarian emergency".

Those who thought that the 2004 survey was exaggerated - it estimated 98,000 additional deaths up until September 2004 - think this one is even more wide of the mark.

Les Roberts, one of the report's authors said: "It may not be extremely precise, but it gets us into the ball park."

Professor Gilbert Burnham, another of the report's authors and an epidemiologist at the Johns Hopkins Bloomberg School of Public Health, said: "We're very confident with the results."

And other epidemiologists supported that view. Ronald Waldman of Columbia University told the Washington Post that the survey used a method that was "tried and true" and that "this is the best estimate of mortality we have."

Update 19 October: there has been a lot of support for the report's methods among the statistical community. For example, stats.org at George Mason University has an online article by Rebecca Goldin who says: "While the Lancet numbers are shocking, the study's methodology is not. The scientific community is in agreement over the statistical methods used to collect the data and the validity of the conclusions drawn by the researchers conducting the study."

However there has been some criticism of the methodology.

An article in the Wall Street Journal by Steven Moore, who worked as a pollster for the coalition authorities in Iraq, attacked the sampling: "The key to the validity of cluster sampling is to use enough cluster points. In their 2006 report... the Johns Hopkins team says it used 47 cluster points for their sample of 1,849 interviews. This is astonishing: I wouldn't survey a junior high school, no less an entire country, using only 47 cluster points."

And on 20 October, Science Magazine reported the queries of researchers at Oxford and Royal Holloway universities. One of them, Sean Gourley of the Physics department at Oxford, said their studies "have found fundamental flaws [in the Lancet report] that lead to an over-estimation of the number of deaths. "

One aspect they questioned was the selection of sample households chosen for interviews. There could be "main street bias", they said, in that households on main streets were more likely to suffer casualties from car bombings. They want an inquiry into the methodology. "It's almost a crime to let it go unchallenged," said Neil Johnson of Oxford.

It may not be extremely precise, but it gets us into the ball park Les Roberts report author

And other groups that track deaths in Iraq dispute the findings. Michael O'Hanlon of the Brookings Institution, which tracks statistics in its Iraq Index, said: "I do not believe the new numbers. I think they're way off." The Brooking Index, relying on the UN (which gets figures from the Iraqi health ministry) and the Iraq Body Count (IBC), estimates the civilian death toll at about 62,000.

The IBC, which counts the number of reported civilian deaths by violence, puts them between 43,850 and 48,693, though it adds that "our maximum refers to reported deaths - which can only be a sample of true deaths unless one assumes that every civilian death has been reported. It is likely that many if not most civilian casualties will go unreported by the media."

(The IBC reaction to the Lancet report has now been published. See below after summary of Lancet report)

One should note here the political nature of this debate, as well as the statistical. Some critics say that the report's authors and the editor of the Lancet Richard Horton have a political agenda in opposing the war in Iraq, and that therefore this should be taken into consideration when discussing this work. In turn of course some of these critics tend to be politically committed themselves.

Report methodology

First, though, to the report itself. Its strength, its authors argue, is in its tried and trusted method.

It took a sample and then extrapolated broad results from that sample. This is a technique used in other areas of conflict, in public opinion polling and in marketing, for example, in assessing television audiences.

In 2004, 33 clusters were chosen across the country with 30 households in each cluster. These households contained 7,868 people. This time, 47 clusters were chosen, with 12,801 people.

Insurgents are now staging daily attacks in Baghdad

The method was to question people about deaths in their household first in the "pre-invasion" period and then in the "post-invasion" period leading up to July 2006.

The difference would constitute what the survey calls "excess deaths".

The report says that there were 82 deaths pre-invasion and 547 post-invasion.

It then multiplied these figures up in relation to the Iraqi population of 27,139,584, and came up with an estimated 654,956 "excess" deaths, 2.5 % of the population.

Some statistical caveats are entered. The lowest estimate of deaths is put at 392,979 and the highest at 942,636. The lowest figure is still much bigger than the other counts.

Of the "excess" deaths, 601,027 were attributed to the violence (mainly from gunfire and mainly among men aged 15-59), the rest coming largely from increased illness and disease.

The report concludes: "Our estimate of excess deaths is far higher than those reported in Iraq through passive surveillance methods. This discrepancy is not unexpected. Data from passive surveillance are rarely complete, even in stable circumstances, and are even less complete during conflict."

IBC response

The Iraqi Body Count response is as follows. Its says the Lancet report implies that:

Of 800,000 wounded people in the past two years, "less than a tenth received any kind of hospital treatment."

Over 7% of the male population has been killed; 10% in central region.

Half a million death certificates were issued to families but not officially recorded.

The Coalition has killed far more people in the last year than in the invasion and Falluja type-operations of earlier years.

The IBC says that such assertions suggested incompetence/fraud on a massive scale by hospitals and ministries, self-destructive behaviour by the wounded, an utter failure by agencies to notice decimation of the male population and an abject media failure to observe the scale of events.

The IBC concludes: "In the light of such extreme and improbable implications, a rational alternative conclusion to be considered is that the authors have drawn conclusions from unrepresentative data. In addition, totals of the magnitude generated by this study are unnecessary to brand the invasion and occupation of Iraq a human and strategic tragedy."

'Missing' dead

One issue that arises is why, to speak crudely, the numbers of bodies being discovered do not match the Lancet figures.

HAVE YOUR SAY How do you know that you are not reporting the same fatality multiple times? Andrew M, London Lancet author answered your questions

If it is assumed that there were 601,000 violent "excess" deaths between March 2003 and July 2006 (about 40 months), that should produce an average of about 500 violent deaths per day.

This is not going to be so all the time, given the spikes of violence, but it is a rough criterion.

The latest figures from the Iraqi health ministry (reported by the Associated Press news agency on 11 October) stated that 2,667 people were killed in Baghdad during September, 400 more than in August.

This gives an average of about 86 per day in the capital.

Baghdad is not the whole country of course, but AP reported the United Nations as saying that in July and August, 6,599 people were killed across the country, of which 5,106 were in Baghdad.

This suggests that Baghdad has by far the highest number of actual and percentage dead.

So, if the current rate in Baghdad is about 86 and the countrywide figure should be about 500 according to the Lancet report, where are the "missing" dead?

Author's reply

I put these points to author Les Roberts who replied: "There have to be ~300 deaths per day from natural cause even if Iraq was the healthiest 26 million people in the world. Where are those bodies? When the MOH [ministry of health] in Iraq is perhaps recording 10% of them, why should they be doing better with politically charged violent deaths. Yes, I think almost nothing is getting reported outside of Baghdad where things are worse."

And he suggested that a way existed of checking his results.

"There has rarely been a scientific report so easily verified or discarded. If someone went to 4 or 6 places picked at random in Iraq, and went to the grave yards for those villages, they could easily see if there are 3 or 4 times more bodies being brought in per week compared to 2002. Or, if someone could go to a couple villages or places, if we are correct, on average ~70 percent of the deaths occurring will be from violence.

"This would take 2 reporters one day to decide if we are basically correct or in error!"

The difficulty of course is that the international media is incapable of getting around safely to do something like that easily. The local media is a source but cannot be relied on by itself.

We are left then with the estimate from this report and the various counts by other groups.

The figures are now even more divergent than they were.