Analyst says CMS star ratings program flawed, may create biased results

An independent analysis of hospital star ratings claims the Centers for Medicare and Medicaid Services program is highly flawed and even shows bias against the hospitals that receive these public scores, according to the American Hospital Association.

The AHA commissioned Francis Vella, chair of the department of economics at Georgetown University, to review CMS' Overall Hospital Star Ratings methodology.

"The approach adopted does not consider anything apart from quality outcomes," he said in a letter to the AHA. "For example, it does not adjust for the nature of the patients or the different circumstances hospitals might encounter. I cannot see how one can ignore the implications of these factors on such measures such as mortality etc."

[Also: 168 get 5 stars, 65 get 1 star in latest Hospital Compare patient experience ratings]

Vella specifically targeted the lack of socioeconomic variances in the reporting, an issue that many hospitals and groups like the AHA have said is unfair to institutions since they don't take patient makeup into account.

"Two (or more) identical hospitals could have very different outcomes depending on the type of patient they have, where they are located, the type of health issues they typically face and multiple other factors," he said. "I feel ignoring other determinants of quality outcomes (such as location of hospital and patient composition) potentially biases the results."

The CMS program, which mainly uses patient answers on Hospital Consumer Assessment of Healthcare Providers and Systems surveys to assign an overall star rating that it makes public on its Hospital Compare database, has come under scrutiny. The federal agency has said it is studying how socioeconomics plays into these scores. It also plans to roll out a more comprehensive overall star rating in the near future.

[Also: MedPAC to CMS: Options to factor socioeconomics in star ratings not good enough]

"As currently designed, CMS's star hospital ratings program is not up to the task of providing the public with meaningful and accurate assessments of hospital performance," said Rick Pollack, AHA president and CEO, in a statement. "Patients need reliable information to make important choices regarding their health care. And hospitals and health systems need reliable information so that they can continue to improve the quality of the care delivered. CMS star ratings misses the mark on both accounts."

HCAHPS scores not only fall under public scrutiny, but hospitals can also see their Medicare reimbursements cut if they earn low scores.

[Also: CMS delays hospital star ratings until July after hospital, lawmaker protests]

Vella also said the nature of a 5-star ratings may give the appearance that two hospitals could have wide differences in quality, when in truth they may not.

"Ranking the hospitals by stars is somewhat misleading as it indicates a qualitative jump as one goes from one category to the other and this may be inconsistent with reality and only reflects the scoring algorithm," he said.

Twitter: @HenryPowderly
Contact the author: henry.powderly@himssmedia.com