'Best hospital' lists come under fire

HealthGrades has identified what it calls three major flaws in a recent study published in the Archives of Surgery that says its '50 best hospitals' list "falls short."

[See also: Top 50 hospitals named for 2011.]

The findings of the journal article are fundamentally flawed and could undermine patients' ability to find high-quality hospital care, according to HealthGrades.

The article, "Evaluating Popular Media and Internet-Based Hospital Quality Ratings for Cancer Surgery," compared outcomes for gastrointestinal cancer patients who underwent surgery at hospitals on HealthGrades’ “America’s 50 Best Hospitals” and US News and World Report’s “America’s Best Hospitals” lists. The study asserts that hospitals with the best outcomes for three surgical procedures were often not among those listed as top performers by HealthGrades and US News.

HealthGrades officials say the company's 'America’s 50 Best Hospitals' list is based on patient outcomes across 27 common medical procedures and diagnoses and is intended to reflect a broad spectrum of clinical excellence in patient care.  

“While we’re encouraged by the study authors’ attempts to help patients find accurate quality information, in this case this study may have the opposite effect and might actually discourage patients from getting the information they need to make critically important healthcare decisions,” said  Rick May, HealthGrades' vice president for accelerated clinical excellence. “More than 10 million patients a month visit HealthGrades.com looking for information on hospitals and doctors and they trust us to provide the most accurate information on healthcare quality. We take that responsibility seriously and that’s why, for the last decade, we have conducted the most rigorous statistical analyses of hospital quality in America. We stand firmly behind the accuracy of our hospital ratings, including our list of America’s 50 Best Hospitals.”

HealthGrades cited three flaws in the methodology upon which the Archives of Surgery study findings are based:

• The study authors compared two disparate groups of patients – patients undergoing cancer surgery were compared to non-cancer patients with a variety of other diagnoses, such as patients having knee replacements or being treated for a stroke. HealthGrades said it analyzes outcomes data from 27 different procedures and diagnoses, but not pancreatectomy, esophagectomy and colectomy, which were focused on in the study. HealthGrades hospital ratings for gastrointestinal procedures and diagnoses exclude patients with primary and metastatic cancers because tumor staging and grading information is not available through the Medicare administrative data set.
• The study included only a small fraction of the Medicare patient records analyzed by HealthGrades in its study. According to HealthGrades, the '2008 HealthGrades America’s 50 Best Hospitals' study recognizes hospitals that have demonstrated superior performance spanning eight years of Medicare patient data (1999 to 2006), across common 27 diagnoses and procedures and includes more than 100 million patient records. HealthGrades said the study authors used only one year of data (2005-06) and outcomes for just three procedures, which accounts for 85,000 patient records.
• The study methodology tries to compare two different treatment time frames and different outcome measures. HealthGrades officials said their study uses in-patient mortality and complications, whereas the study authors only looked at 30-day mortality.