Thursday, February 7, 2008

US health care doesn't measure up

From a Commonwealth Fund summary of a study published in Health Affairs:
Between 1997–98 and 2002–03, amenable mortality fell by an average of 16 percent in all countries except the U.S., where the decline was only 4 percent. In 1997–98, the U.S. ranked 15th out of the 19 countries on this measure—ahead of only Finland, Portugal, the United Kingdom, and Ireland—with a rate of 114.7 deaths per 100,000 people. By 2002–03, the U.S. fell to last place, with 109.7 per 100,000. In the leading countries, mortality rates per 100,000 people were 64.8 in France, 71.2 in Japan, and 71.3 in Australia.
In case you're wondering,
The concept of amenable mortality was developed in the 1970s to assess the quality and performance of health systems and to track changes over time. For this study, the researchers used data from the World Health Organization on deaths from conditions considered amenable to health care, such as treatable cancers, diabetes, and cardiovascular disease.
By the authors' most conservative estimate, 75,000 people died in the US in 2002 who would have lived if we were merely up to the average of the other countries.

Note that this is not a case where our system is actually declining in absolute terms; it's simply improving more slowly than comparable systems. For those of us brought up on the idea that the US was ahead of everyone else and always would be, that's small comfort. But the sooner studies of this sort knock that notion out of our heads, the better.

9 comments:

Paul Botts said...

Well...sort of, kind of, with lots of important caveats, but a little logical examination makes me smell a cooking of the books.

Those authors simply decide to avoid counting types of deaths for which the question of their amenability is "controversial." Indeed the definition of "amenable" turns out, if you read the actual study document, to be not terribly standardized at all. Does it make sense to count, arbitrarily, exactly half of all heart-disease deaths as having been "amenable to health care"? I have no idea and neither, so far as I can tell, do those authors but they chose that exact percentage just for their particular study. They make a whole series of such arbitrary choices of what percentage of count as "amenable" among various types of death. And they also arbitrarily count only deaths under age 75.

So it turns out that the deaths they are declaring to have been "amenable to health care" are only a small fraction of all deaths. And -- wait for it -- yep, the U.S. does a lot better comparatively on the deaths which those authors have arbitrarily decided to not have been amenable (i.e. that they decide would have happened regardless of the performance of the health care system).

See for instance Exhibit 3 in the actual study, percentage decline in mortality among males under 75. The study has arbitrarily defined only 23% of such deaths to have been "amenable", and the U.S. ranks last in percentage improvement for that 23%. But for percentage improvement in the other 77% of types of death, the U.S. ranks right the middle of the 19 countries. For the female category, same answer: the U.S. ranks last in percentage decline of the 32% of deaths which those authors particularly define as "amenable", but middle of the pack in percentage decline of the other 68% of female deaths.

The study does not offer a table combining those results for deaths which those authors are choosing not to count as "amenable". But a little pencil work reveals that if it did, the U.S. would rank actually in the top handful of those 19 countries on percentage improvement in the rates of death which account for almost three-quarters of all deaths.

Sometimes applying the smell test requires doing some detailed smelling, for which third-party summaries really aren't useful.

Jeffrey said...

What Paul said is right on. The other closely related point I wanted to make, is that lots of health care outcomes depend on individual choices and lifestyles (i.e. eat lots of fatty foods, increase your chance of heart disease). So country to country comparisons are very difficult to make without demographic and lifestyle information.

Harold said...

Paul, every blogger should have such a diligent and intelligent commenter. Please don't go and start your own!

Arrgh, apparently Health Affairs is now charging for the full article. However, I can tell that at least one of your charges -- that the authors chose their definition of "amenable" "just for their particular study" isn't the case. In an earlier study at http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=261807
they discuss some of those debatable methodological issues. Their finding then was that rankings of various countries changed quite a bit using "amenable mortality" as the measure; in that (static) comparison the US performance didn't change a lot and couldn't have been their motivation for measuring it that way.

Having been exposed to the US health care system in a new way in the last couple of months, I'm also not content simply to knock down a critical study and walk away (as Jeffrey is more inclined to do) happy that US supremacy remains intact. If this is a bogus measure (I'm unconvinced -- there may be some discussion of it in the literature, I haven't had time to look), then what *is* a good measure? (BTW, I would think that amenable mortality, a concept which attempts to deal with stuff that can be prevented, would at least in part control for some lifestyle choices.)

Harold said...

Hey, why can't you guys come up with great comments like this one, which I did not make up, at
http://content.healthaffairs.org/cgi/eletters/27/1/58#3289
"This Could Be Due To Testosterone"

Paul Botts said...

"However, I can tell that at least one of your charges -- that the authors chose their definition of "amenable" "just for their particular study" isn't the case."

Actually that's not a charge on my part, it's what the authors of this study _state_ that they are doing. Then they lay out the variety of specific death causes for which they made arbitrary choices of what should count as "amenable".


"Having been exposed to the US health care system in a new way in the last couple of months, I'm also not content simply to knock down a critical study and walk away happy that US supremacy remains intact."

Logically, in order to use direct personal experience as a basis for judging one system superior/inferior to another, you would need to have direct experience with both. After all, by definition the experience of interacting with any health care system is often going to be sucky (you're sick/injured/upset). And we know that homo sapiens always remembers more and is influenced more by shitty/unexpected experiences than by uneventful normal ones.

So of course we all, in the wake of directly interacting with the U.S. health care system, tend to think it sucks. And since none of us have the parallel experience of interacting in the same way with the British/French/Canadian/whatever health care system we naturally tend to think that grass is greener. I happen to work with two people who do have that parallel experience, and they are both quite clear that hands down they'd rather deal with the U.S. health care system. But my overall point is simply that on this subject it would be best for conclusions having public-policy implications to be based on some consistency of logic and information sources.

Jeffrey said...

Harold,

Your comment about your experience with American health care is interesting, because my own experience with serious surgery (this guy cracked open my chest to fix a congenital heart defect:

http://www.chicagomag.com/Chicago-Magazine/January-2008/Chicagos-Top-Doctors/A-Change-of-Heart/)

was life-changing (in a totally positive way...with my heart fixed I'm like the "Six-Million Dollar Man"...better, stronger, faster). So I have to disagree with Paul about my experience being "sucky".

As Paul points out, you can read how the authors developed their methodology, which is flawed in many ways. To give just one other related example, I've read other critical reviews of studies that purport to show infant mortality is higher in the U.S. than in other advanced industrial countries. The reviews demonstrate that most of these comparisons ignore the heroic efforts we make in the U.S. to save pre-mature babies. In cross-country comparisons, these babies aren't counted.

Paul Botts said...

Hmm, that seems like such an obvious factor that I'm actually skeptical that cross-country comparisons of infant mortality would ignore it...Jeffrey do you have a link to one of the analyses you're describing?

Meanwhile the current issue of Wired reports on a challenge to one standard part of the argument about how much our health care system costs:
http://news.wired.com/dynamic/stories/O/OBESITY_COST?SITE=WIRE&SECTION=HOME&TEMPLATE=DEFAULT&CTIME=2008-02-05-02-58-08
"Preventing obesity and smoking can save lives, but it doesn't save money, researchers reported Monday. ...In a paper published online Monday in the Public Library of Science Medicine journal, Dutch researchers found that the health costs of thin and healthy people in adulthood are more expensive than those of either fat people or smokers....The researchers found that from age 20 to 56, obese people racked up the most expensive health costs. But because both the smokers and the obese people died sooner than the healthy group, it cost less to treat them in the long run...."

Jeffrey said...

Paul,

Here are some links that discuss the problem of international comparisons:

1) http://www.tcsdaily.com/article.aspx?id=041505B

2) http://econlog.econlib.org/archives/2005/04/international_h_1.html

(also check out the comments)

3) http://econlog.econlib.org/archives/2007/09/another_reason_2.html

None of these posts claim that the U.S. health care system is necessarily "better" than Europe's (or other countries with national health care systems); rather they argue that intra-country comparisons are difficult because it is difficult to control for so many lifestyle differences (which is my original point).

Jeffrey said...

Paul,

Sorry about the links in #s 2 and 3; for some reason they got cut off.

Trying again.

#2) http://econlog.econlib.org/archives
/2005/04/international_h_1.html

#3)
http://econlog.econlib.org/archives
/2007/09/another_reason_2.html