More About This Year's Physician Survey

Hospitals will not benefit by pressing doctors to respond.

By and + More

We want to respond to questions that have come in from doctors and hospital administrators about how we are conducting this year's survey of physicians for Best Hospitals. The traditional survey, of 3,200 boarded doctors in 16 adult specialties, is currently in the field. As previously announced, we are concurrently surveying a large number of physicians who are members of the physician network Doximity and are not among those contacted for the traditional survey. The results of the two surveys will be separately analyzed and will be incorporated into the 2014-15 rankings, along with physician survey data from prior years and objective measures such as mortality rates and patient safety indicators.

Some administrators' questions reflect a concern that certain hospitals might be disadvantaged if their physicians are underrepresented in Doximity's membership and therefore potentially underrepresented in the Doximity-facilitated survey. That concern is unwarranted for two reasons.

First, we will weigh results of the two surveys differently. The bulk of the reputation score used in the 2014-15 rankings will be derived from the traditional survey. This is appropriate because although its sample size is smaller, it's designed to be representative of a larger proportion of U.S. physicians than the Doximity survey.

Second, the analysis of responses to each survey, which will be led by Best Hospitals data contractor RTI International, will include steps to identify and statistically adjust for potential sources of bias, such as under- or overrepresentation of certain types of physicians among those who respond.

The appeal of the larger sample is fundamental: It will give us more data. All Doximity members who are board certified in one of the 16 Best Hospitals specialties can participate in the Doximity survey, while only a random sample of non-members receive the conventional survey. (Regardless of Doximity membership status, physicians who are not boarded in one of the 16 relevant adult specialties are not eligible for either survey. Survey eligibility criteria, including the required board certifications, are described in detail beginning on Page 29 of the Best Hospitals Methodology Report.) Doximity uses data from state and specialty license boards to determine physicians' certifications, and it verifies each member's identity through a three-step process involving DEA number, challenge questions and offline applicant review.

U.S. News and RTI will use best practices in scientific survey research to ensure the validity and reliability of our analysis. For example, we will compare the responses of the physicians surveyed via Doximity to the results of our traditional survey. We will look at both the type of responses we receive and the characteristics of the responders to determine if there is potential for bias. For example, Doximity survey responders might be likelier to be young or to be affiliated with certain hospitals compared to the overall physician population. Using national benchmarks and statistical adjustments, we will attempt to control for such potential biases. Our goal is to produce results that are representative of all survey-eligible physicians, including those who are not surveyed or who do not respond.

Importantly, a hospital whose physicians are disproportionately represented among Doximity's members cannot expect to enjoy an advantage in the rankings, even if the doctors overwhelmingly respond to nominate that hospital. We are aware that some hospital administrators have urged employed or affiliated physicians to join Doximity for the express purpose of participating in the survey. In some cases, their efforts have taken the form of written communications to their physicians, which we have subsequently obtained.

While we encourage any physician who wishes to join Doximity to do so (it's free), doctors should not be pressured to join. Hearing about attempts to stuff the ballot box is disheartening, if expected. More to the point, any such effort is unlikely to have the desired effect.

The Doximity survey results will be used to determine reputational scores for only the adult Best Hospitals rankings. Pediatric specialists are being surveyed for the Best Children's Hospitals rankings as they have been in previous years.