Several weeks ago we set out our intent to create new hospital ratings that will accompany the Best Hospitals rankings. They will differ by focusing not on the sickest patients, as Best Hospitals does, but on more typical consumers who need relatively routine procedures such as heart bypass surgery and hip replacement. Since some such procedures frequently involve patients younger than 65, federal Medicare data alone won't take us as far as we'd like to go. We will have to find other resources and, of course, develop a credible methodology. Your thoughts on both are encouraged.
For data, we're taking an especially close look at states that have already blazed a trail in reporting on hospital performance for specific procedures. We'll start with New York's pioneering work on tracking data for heart bypass surgery, valve surgery, and angioplasty. The state health department began publishing three-year death rates for individual hospitals and surgeons two decades ago.
Since the first full report, which covered 1992-1994, the death rate for bypass surgery has dropped some 40 percent at the state's hospitals—and continues to fall. How much of the decline can be credited to public reporting cannot be calculated, but what is known is that with each annual report, surgeons with the worst records tend to retire or stop performing heart surgery.
This table shows the latest data, updated just this week, for valve and combined valve and bypass operations performed in the state during 2008 through 2010. (Skip table and continue reading.)
|Death rates in N.Y. for valve and valve-bypass surgery, 2008-2010|
|Hospital||Cases||Deaths||Observed mortality rate (%)||Expected mortality rate (%)||Risk-adjusted mortality (95% confidence interval)|
|Albany Medical Center||443||18||4.06||4.10||4.55 (2.70-7.20)|
|Arnot Ogden Medical Center||110||5||4.55||2.99||6.97 (2.25-16.27)|
|Bellevue Hospital Center||275||8||2.91||3.62||3.69 (1.59-7.27)|
|Beth Israel Medical Center*||330||25||7.58||4.78||7.27 (4.71-10.74)|
|Buffalo General Hospital||500||18||3.60||3.64||4.54 (2.69-7.17)|
|Champlain Valley Physicians Hospital*||99||9||9.09||3.49||11.98 (5.47-22.74)|
|Ellis Hospital||325||8||2.46||3.70||3.06 (1.32-6.02)|
|Erie County Medical Center||49||2||4.08||4.10||4.57 (0.51-16.50)|
|Good Samaritan Hospital of Suffern||180||7||3.89||3.84||4.65 (1.86-9.57)|
|Lenox Hill Hospital*||653||48||7.35||5.24||6.44 (4.75-8.54)|
|Long Island Jewish Medical Center**||643||16||2.49||5.19||2.20 (1.26-3.58)|
|M.I. Bassett Hospital||120||1||0.83||2.97||1.29 (0.02-7.17)|
|Maimonides Medical Center||428||19||4.44||5.02||4.06 (2.44-6.34)|
|Mercy Hospital||345||18||5.22||3.83||6.25 (3.70-9.88)|
|Millard Fillmore Hospital||279||13||4.66||3.02||7.10 (3.77-12.14)|
|Montefiore Medical Center-Moses Division||457||21||4.60||5.04||4.19 (2.59-6.41)|
|Montefiore Medical Center-Weiler Division||364||24||6.59||5.18||5.85 (3.75-8.71)|
|Mount Sinai Medical Center||1,538||73||4.75||4.27||5.10 (4.00-6.42)|
|New York Hospital-Queens||73||2||2.74||3.45||3.65 (0.41-13.16)|
|New York Methodist Hospital||180||8||4.44||4.65||4.39 (1.89-8.64)|
|N.Y. Presbyterian-Columbia University Medical Center||1,776||80||4.50||4.57||4.52 (3.59-5.63)|
|N.Y. Presbyterian-Weill Cornell Medical Center**||1,212||26||2.15||3.96||2.49 (1.62-3.64)|
|New York University Hospitals Center||1,213||39||3.22||3.42||4.32 (3.07-5.90)|
|North Shore University Hospital||1,398||60||4.29||5.08||3.88 (2.96-4.99)|
|Rochester General Hospital||979||56||5.72||4.94||5.32 (4.02-6.91)|
|SVCMS-St. Vincent's Manhattan||51||4||7.84||2.74||13.15 (3.54-33.67)|
|St. Elizabeth Medical Center*||400||32||8.00||5.34||6.88 (4.71-9.71)|
|St. Francis Hospital||1,963||108||5.50||5.38||4.70 (3.85-5.67)|
|St. Joseph's Hospital||1,061||60||5.66||5.87||4.43 (3.38-5.70)|
|St. Luke's-Roosevelt Hospital Center||258||12||4.65||4.10||5.20 (2.69-9.09)|
|St. Peter's Hospital||861||33||3.83||4.50||3.92 (2.69-5.50)|
|Staten Island University Hospital||303||17||5.61||3.64||7.08 (4.12-11.34)|
|Strong Memorial Hospital||628||31||4.94||3.79||5.98 (4.06-8.48)|
|UHS-Wilson Medical Center||224||12||5.36||3.66||6.73 (3.47-11.75)|
|University Hospital-Brooklyn||142||7||4.93||5.17||4.37 (1.75-9.01)|
|University Hospital-Stony Brook||557||38||6.82||6.06||5.17 (3.66-7.09)|
|University Hospital-Upstate||227||9||3.96||3.66||4.97 (2.27-9.44)|
|Vassar Brothers Medical Center**||513||8||1.56||4.36||1.64 (0.71-3.24)|
|Westchester Medical Center**||503||15||2.98||5.19||2.64 (1.48-4.35)|
|Winthrop University Hospital||573||31||5.41||4.81||5.17 (3.51-7.33)|
|All New York hospitals||22,233||1,021||4.59||N/A||N/A|
*Risk-adjusted rate significantly worse than average
**Risk-adjusted rate significantly better than average
Let's look first at the selection of displayed data, setting aside whether consumers can understand and interpret them. Do they have enough of the right stuff to be a foundation for ratings that patients can use in choosing a hospital for this kind of heart surgery?
An important part of the answer concerns the robustness of the key data point: risk-adjusted mortality rate. It's the one consumers are most likely to focus on, the one that seems to best communicate the likelihood that they will leave the hospital alive and stay that way during the postoperative period. For hospitals at the lower end of the volume scale, however, the 95 percent confidence interval is so broad as to bring into question the value of the risk-adjusted death rates. What message does a range of 1.9 percent to 8.6 percent send to a patient considering this surgery at New York Methodist Hospital? The range even at higher-volume hospitals like Albany Medical Center is wide enough to make patients without some grasp of statistics wonder what to make of the information.
But a good methodology of the kind sought by U.S. News will need to go beyond procedural mortality even if the relevance to consumers and the validity of the numbers are assumed and a clear explanation is included. Should length of stay be factored in? Readmission rate? Patient satisfaction? Selected infection rates?
Probably all of these, and more. Unfortunately, few states go nearly as far as New York does in collecting and analyzing health data and making it available to the public. Hospitals in the state must report data on every bypass, valve replacement, and angioplasty, and on the physician who performs them, in far more detail than is demanded by the federal government for Medicare and Medicaid patients. Each New York patient's risk profile, severity of illness, and outcome, measured by in-hospital and 30-day mortality, must be provided. The state data are drawn from clinical records; Medicare data come from administrative billing codes, which are inherently less precise and more susceptible to tweaking to a hospital's advantage. In New York, health department analysts carry out spot audits, compare hospital-reported information with Medicare data, and check the state's death registry for overlooked or unreported deaths.
The program has real muscle, too. High death rates trigger a state-mandated quality-improvement effort that may include a site visit and inspection by members of the state's cardiac advisory committee. The effort carries great credibility because the approach is so rigorous and the data provides hospitals and doctors with "constant feedback" on where they stand in relation to other providers, says Nicholas Stamato, a Johnson City, N.Y., cardiologist and a member of the advisory committee. "That's why taking a program like New York's national would be so beneficial."
The data collection and quality-assurance steps take time, so each new report presents data that's two years old. As my colleague Avery Comarow likes to say, we're looking for data that's decent and recent, and the New York data is as up to date as any statistics in the public health realm manage to achieve.
Nor is the state standing pat. More than 54,000 angioplasties, which are performed through small tubes and are not surgical procedures, were performed in New York from 2008 through 2010. That is about 2½ times the number of valve and valve-bypass surgeries. The angioplasties resulted in 454 deaths. Mortality ranged from 1 in 200 for elective angioplasty, generally done to relieve chest pain, to 3 in 100 for emergency patients who were treated to clear a blockage that caused a heart attack.
To reduce deaths and cut costs, New York is evaluating a new "appropriateness" measure that evaluates individual patients to determine whether they should even be candidates for the procedure. Some patients with multivessel coronary artery disease fare better with bypass surgery and others can be treated effectively with medication. Studies have shown that 20 percent to 40 percent of the angioplasties done in New York hospitals are inappropriate, says state health commissioner Nirav Shah. Hospital-by-hospital determinations will be released "eventually," Shah has said, but only after the measure has been more extensively tested and hospitals have been given time to reduce their rates of inappropriate procedures.
What should we at U.S. News make of these data, assuming we could pull similar metrics out of every state? What other data should be factored into, or excluded from, an evaluation of hospitals' performance in heart surgeries? If you were advising a friend or family member who wanted help in deciding which local hospital to use for a particular routine procedure, what information would you consider critical?