U.S. News's 'Best Hospitals' Clashes With Other Ratings. Is That Bad?

A point I thought you would bring up, Avery, but did not, is the difference between objective ratings (mostly on claims data) and the US News ratings, which include the subjective element of reputation.

So, for instance, Johns Hopkins has consistently been No. 1 in the US News ratings. We now know, thanks to the pioneering work of Dr. Peter Pronovost, that the hospital had serious patient safety issues even while its reputation kept it up on a pedestal. Had there been reliable objective measures of safety at that time, they would not have agreed with the US News rating, which is a composite of different parts. Similar points can be made about the other rating methodologies.

I'm not sure consumers are sophisticated enough yet to understand the different purposes for the different ratings, but I also do not believe the health services research community approaches this properly. For example, the study about a lack of impact within two to three months with regard to some of the measures used a very short time frame.

The good news is there is now a genuine search for fair measures -- I think -- as opposed to the bad old days of simply whining over any and all public measures.

Avery Comarow responds: I appreciate the comments from one of healthcare policy's thought leaders. Yes, reputation is subjective, but we try to minimize it by narrowing the question in three ways. Only board-certified physicians are surveyed; the context is conditions and procedures only in their specialty; and they are asked to name hospitals only where they would send those patients in need of the most sophisticated care. The relative stability of the reputational numbers over the years argues that this approach has merit. If a hospital is in fact not performing well and is resting on its laurels, how long would it take for word to filter through the professional community and the halo to fall off? A couple of years, maybe three or four, but surely not 10 or 20.

To be picky, you're right that Hopkins has been #1 for as long as there has been an Honor Roll, but that doesn't mean it is #1 or even #2 or #3 in all or most of the 16 individual specialty rankings. It just means that compared with other hospitals, Hopkins performed above a specified level across an unusually large range of specialties, and the individual specialty rankings, I hope, are where consumers would go. They would find that Hopkins, while high-ranked, was at the top in just three of the 16 specialties.

As for hospitals with serious safety issues, I question whether any hospital, iconic or otherwise, can claim that it has been tragedy-free. In a post a couple of weeks ago on his blog Wachter's World, hospitalist Bob Wachter had this to say concerning a rash of slipups at one Boston hospital:

"We must educate the media about this fact: if you are not hearing about serious errors from other hospitals, trust me – it is because you’re not hearing about them, not because they’re not happening. This is a case in which the obvious (I just heard about another bad error from Hospital A – it must be less safe than Hospital B) might well be dead wrong."

Michael Millenson of IL 9:03PM November 17, 2008

My introduction is I work for doctors and hospitals and I traveled 200 miles to U of Michigan fifteen times over three years for dental care after four dentists proved themselves to be incompetent and/ or fraudulent. There is a lot of psycho babble and jingoism to rating hospitals. The old saw, opinion is ego and fact is data needs to apply.

First, the reports you apparently favor are self reported by hospitals and are prone to their manipulations. Trust is at a premium, audits are nil. Others are opinions from specialists about their colleagues, and some are produced by firms that make the bulk of their incomes from consulting with US hospitals. This truly makes for confusion, and a number of hospital leaders have called for reform. Your attitude rings of the old Quality number 1 reports from Ford. The OECD and Health Affairs data are more objective and based on interviewing the public.

Second the questions leave out a lot of data. Should a six hundred bed hospital in Ohio promote a stroke program with one radiologist with fellowship in neuroradiology and one or no neurologist with fellowship in stroke? The answer in America is yes, and in fact, they are highly rated because they reported data to make it so. Sadly, there are twenty neuroradiologists in Sacramento thousand of miles away.

Finally, the middle class can't afford what we have, and may forgo necesary care while having to pay for drugs, diagnosis and treatment that are found to be unnecessay or dangerous (e.g. the hospital ER screening "six pack" of CT scanning that exceeds what Nagasacki survivors suffered). The Health Affairs/ OECD data show Americans are really getting short changed on primary care and hospital admissions and length of stay while paying two to three times or more for their overall care compared to Europeans. Just like autos, Americans may discover better alternatives. In a recession/ depression, it may not be a choice.

Don of OH 11:44AM November 16, 2008