A Baltimore Sun intern asks some hard questions of the US media over a study led by John Hopkins University School of Medicine that was put out in The Lancet.
I saw the story online I don’t recall seeing qualifiers. Reading it online didn’t give me a sense of the story being buried. Comparative media studies can say a lot politically.
(The Lancet study requires registration.)
But let’s move back to the American press. Second case: Some American newspapers actually did report the facts of this story–but then they inserted elements to discredit what they just published. Rather than publishing the news and waiting for new developments or reader response, they create these counterpoints themselves.
Let’s have a look at the title of the story that was published in the Baltimore Sun: “Survey: Iraqi deaths higher,” subtitled “Hopkins-designed study says 100,000 civilians died; Prior estimates, 10,000 to 30,000; Brookings defense expert calls data ‘preposterous’ “
Through this convoluted title information, the paper creates doubt in the reader’s mind about the study’s veracity before the reader has even read the first paragraph. No doubt the reader will keep the “Brookings defense expert’s” point of view in mind while reading.
Seeing the vitrol coming from some regarding the Lancet study, sources that are not equal aside; what ‘number of deaths’ is generally agreed on in the US? And what sources would be acceptable? John Hopkins School of Medicine isn’t, The Lancet isn’t, human rights organizations aren’t. If you wander out to political blogs you see figures from 8 thousand to the Lancet’s 100 thousand.
If The Lancet is seen as unreliable in this study, do you disagree with their other studies ie: drugs, onocology, pediatrics, protocol reviews?
Without arguing justification of war, what source would you accept as reliable for Iraq war dead?
Published 4 years ago
Search
Subscribe 
You are currently browsing the Bene Diction Blogs On weblog archives.
Related Entries
For blog design, Wordpress or MovableType coding or blog consulting, see cre8d design.
It’s a difficult question, this.
Most of us have a tendency to listen to the sources that confirm our own prejudices and I think this one is no exception. My instinct is that a paper which appears in the Lancet deserves to be taken very seriously — you’d have to be a particularly paranoid right-winger to believe that a serious medical journal is part of a liberal media conspiracy.
The problem is not the reliability of the source but the reliability of the methodology. If a source is using a flawed set of data or method of extrapolation then the authority of the source is irrelevant.
Not to pick on Richard but I think he, like many people, is falling for a false dichotomy. The choice isn’t just between whether the source is accurate or intentionally biased. It could just be a matter of having bad data. In fact, the journal’s conclusion could be completely accurate but not supported by their study.
We really shoudln’t be surprised by such errors. A recent study done on medical journals showed a surprisingly high rate of statistical errors that went unnoticed. Too often such problems go unnoticed unless the topic is so politically charged (like this one) that people are motivated to go over the information with a fine toothed comb looking for discrepancies.
As for your final question, I don’t think we can have an accurate means of determining the number. Sometimes we have to humbly admit that in the age of information there are some things that just can’t be known (at least not with a high degree of accuracy).
I think it is reasonable to suppose that a paper published in The Lancet has been subjected to a modicum of scrutiny. My impression is that they don’t just publish any old thing. It doesn’t prove that the conclusions are correct — but I only said that they deserved to be taken seriously. After all, even if they’ve overestimated by 2, that would still be at least double the highest previous estimates of Iraqi dead that I’ve come across.
As you say Joe, some things probably can’t be known for certain, but when there is talk of “a price worth paying”, it is not unreasonable to want to know what that price has been.
A couple of questions.
For lack of a better word, is this an epidemiological study?
I went through Google and the Baltimore Sun article is correct. In the US media the Brookline expert counterpoint was also accepted as accurate.
How did Brookings arrive at their conclusions?
And Joe, I really want to know.
I always read your comments, I tend to think they are a reasonably reliable slice of vocal churched people. What sources would be acceptable to your readers?
I believe that this was approached from an epidemiological perspective.
A couple quick thoughts. The he said, she said model of American journalism means we more and more often substitute balance for truth. Or spin for truth, ie. “this can’t be true because it makes the Bush campaign look bad. ”
Also, Iraqi deaths don’t matter to Americans. We’re more concerned about whether Scott Peterson killed Laci.
“I think it is reasonable to suppose that a paper published in The Lancet has been subjected to a modicum of scrutiny.”
What many people don’t seem to realize is that science as a whole is a very slow-moving conservative field. I recently had the fortune of having a (pretty solid, in my opinion) manuscript rejected by reviewers who wanted to see more data, clearer and more cautious explanations, and more checks. Every last variable has to be accounted for, and it’s tough getting papers through. I don’t know a single researcher who has published something without first having it rejected and modified at least once, if not more times.
Of course, errors still happen, and they have to be addressed with subsequent publications and additional analysis. But there’s never a reason to toss out results that don’t fit expectations.
***And Joe, I really want to know. I always read your comments, I tend to think they are a reasonably reliable slice of vocal churched people. What sources would be acceptable to your readers?***
While I can’t speak for my readers, I would say that the source isn’t as important as the method used to arrive at those numbers. For example, the Lancet used fewer than 1000 homes from which to extrapolate their findings. But is that necessarily the best way to approach the study? What if a similar survey was done on violent death in America using only the residents of Compton in Los Angeles? The conclusion would be that millions of Americans die every year because of gang violence. But, of course, that isn’t the case.
When the numbers can’t be determined with a high degree of certainly I typically look at a number of different sources. For instance, the Iraq Body Count, a database run by a group of academics and peace activists, has put the number of reported civilian deaths at between 14,000-16,000. The Brookings Institute, an American public policy think tank, says between 10-27,000. The UK foreign secretary says fewer than 10,000 while the People’s Kifah claims that it is less than 37,000.
If we combined these sources we would see a range from 10,000 – 37,000. The Lancet, by contrast, has a figure around 100,000. Unless they were using a more reliable methodology than everyone else (and they aren’t) then I think it is fair to discount their article regardless of their reputation as an otherwise reliable source.
(I should also add that since Saddam released over 100,000 violent felons back into the populace prior to the war I would be surprised if there wasn’t more violent deaths than before we arrived.)
A couple of points:
All the statisticians I’ve talked to regarding the article to which Joe referred to (Incongruence between test statistics and P values in medical papers) is that response to the article was overstated. They found errors in journal articles “primarily caused by transcription or typesetting errors, or the incorrect rounding of figures.” The article didn’t focus on statistical methodology. If you look at any publication, you’ll find typographical mistakes
Cluster sampling is a standard sampling method which enables inference about the population. Their results use confidence intervals for the relative risk of death after the US invasion (95% CI 1.6 - 4.2). They adjust the results by removing the Falluja data (as there has been extreme violence there) and the relative risk changes to 1.5 (95% CI 1.1 - 2.3).
What problems do you have with the methodology?
I agree that the Lancet study used about 1 thousand homes in 33 clusters. And I agree that mediagenic toll statistically ranges from 6 thousand to 194 thousand.
I also agree that The Lancet Study included deaths by crime.
I disagree that the methology is similar to surveying gang violence only in a section of Los Angeles. Look at the population proportionally population wise and geographically to the US and Iraq. It is sobering.
Look at what is provided above by a statistican.
A spokesperson for the British government says the study used an extrapolation technique and deferred to the Iraq Body Count.
The Lancet study compared deaths 14 months before March 2003 and 17 months after.
I think The Lancet study is very clear on their methodology - that doesn’t make it faulty.
The Iraq Body Count count deaths immediately attributable to war - using on ground media sources.
I disagree that the prisoners released by Saddam were all ‘violent felons.’ I agree the the estimates of 100 thousand to 125 thousand released are reasonable. These prisoners ranged from innocents to pickpockets, to political dissents to violent felons.
I know you can’t speak for your readers Joe.:^)
They speak for themselves.
Somewhere in this grim swatch of statistics is the truth. It was politically expedient for the UK and US governments to dismiss the Iraq Body Count until recently.
I’m inclined to think no source is acceptable. And as we discuss this on the eve of an assault on Fallujah, I’m sad to say we’d probably agree more civilans are going to be arguable statistics.
I’m meandering…I think the information and question above by Rachel C deserves attention.
I read an extract of the article in The Lancet in a science news feed at the time it came out and was tempted to post on it myself but didn’t because of two main reasons
1. generally, I think it is inappropriate to indulge in body count stakes. The fact that we may have killed more or less civilians than Saddam misses the point that we are killing innocent civilians in Iraq.
2. this is a debate where the presense of this new statistic is unlikely to change anyones mind one way or the other.
Having said that I believe that the study is entirely credible. The methodology and statistics would have been checked by The Lancet which, as Brina points out, is most fussy about these things.
The only other area that I can see that is open for debate is that it is based on self report data and you may argue that the violence in Iraq at the moment is simply more salient (In Saddams time the civilian killings may not have been so obvious). It is a common flaw in the human psyche that we tend to over report things that are more striking. It is why aeroplane travel seems more risky than car travel when, in fact it is safer.
What ever the number, 10,000 50,000 100,000, the real issue is that we are killing civilians large numbers and I think it is crass and inappropriate to indulge in a “body count stakes” with the Hussien regime.