Why hospital ratings should be looked at with a grain of salt
Five reasons to take hospital ratings with a big grain of salt
In US News and World Report’s 2016-2017 ranking of hospitals, Massachusetts General Hospital ranked No. Three behind the Mayo Clinic and Cleveland Clinic. Steven Senne/AP
t’s hospital ratings season in America, that time of year when marketing executives kick it into high gear to trumpet — and spin — the way their hospitals are graded by outside organizations.
This year, their workload has been especially strenuous. In addition to annual rankings released Tuesday by U.S. News & World Report, the federal Centers for Medicare and Medicaid Services (CMS) just released its star-rating system for hospitals nationwide. While the ratings suggest some valuable information to consumers, here’s why you shouldn’t put too much stock in the results.
No matter how objective the metrics, ratings are inherently subjective
The federal government and private organizations use a vast array of hospital data to break down their spectacle and boil it into easy-to-understand ratings for consumers. U.S. News & World Report formulates its rankings by relying on a mix of physician surveys and data reported to trade groups and the federal government. CMS ratings are based solely on data hospitals are required to report to the agency.
The information itself may be objective, but which data points are used — and how they are weighted — can lead to contradictory conclusions. Consider the following:
Of US News & World Report’s top five hospitals, only one — Mayo Clinic — received a top, five-star rating from CMS. The rest — Cleveland Clinic, Massachusetts General Hospital, UCLA Medical Center, and Johns Hopkins — were not even in CMS’ top 100. (Johns Hopkins did not receive a rating from CMS because of incomplete data).
J.B. Silvers, a professor of health care finance at Case Western Reserve University, said the information, while boiled down into elementary lists and starlets, can be difficult for consumers to unpack, especially when hospitals are working behind the scenes to figure out ways to better their scores every year. “Some hospitals are better at working the numbers than others,” he said. “My guess is the safety net hospitals aren’t as good at it as richer hospitals.”
One bad score, and your rating is shot
Lobbyists and some hospital executives argue that the CMS star-rating system penalizes providers that treat greater numbers of low-income patients. That’s because the CMS system hurts hospitals with higher rates of readmission, a factor strenuously influenced by a patient’s economic status.
Critics have noted that instructing hospitals and safety net hospitals — both of which tend to deal more with sick, low-income patients — fared poorly in CMS ratings.
“On its surface that says, ‘Hey, you’ve got a bias,’” said Silvers. “Big training hospitals are taking care of the most elaborate cases. Hardly any of them are in the five-star category. Most of them are in the one-star category.”
What’s not in the ratings matters, too
Ratings systems capture gigantic amounts of information on everything from whether doctors are washing their arms to mortality rates for heart attack patients. But slew of information is not collected that might help provide a more nuanced — and accurate — assessment of spectacle. One hospital quality specialist noted, for example, that the severity of a cancer patient’s disease is not typically recorded by hospitals in diagnostic information and doesn’t get factored into outcome data and ratings. This can unfairly penalize hospitals that treat large numbers of patients with advanced cancers.
“Stage four cancer patients are more likely to die than stage one cancer patients,” said Harold Miller, chief executive of the Center for Health Care Quality and Payment Reform. “But if you don’t know that information — and you’re measuring mortality rates — then you are going to get a bad reaction.”
Gaming the system
Hospitals routinely launch initiatives to improve scores in particular ratings categories. They can also switch how they parse and report certain information to improve their results. In some cases, those adjustments can mean positive improvements in the care provided to patients. In others, they can skew the results or cause hospitals to avoid treating some patients, altogether.
For example, a latest examine by the American College of Surgeons found that organ transplant centers are removing the sickest patients from their waiting lists because of concerns that a negative outcome could harm their ratings and lead to federal scrutiny.
“You may determine not to take patients who are at risk of bad outcomes if somebody’s going to embark rating you on your outcomes without adjusting decently for that,” Miller said. “Those are the patients that need the hospitals the most, and they are the ones that are going to potentially get disadvantaged by this.”
The marketing agenda
Hospital ratings — particularly the results published by U.S. News & World Report — are often used for hospital advertising campaigns. Most institutions ranked in the top ten will hype their ratings success on billboards, banners, and brochures that get blasted across the regions they serve.
But the glossy mass media messages suggest little valuable information to consumers attempting to figure out where to get treatment for a violated gam or have gall stones eliminated because common procedures are seldom evaluated . “It could be the hospital is bad on some things, and truly good on other things,” Miller noted. “But if you’re the patient, how do you know? You might be going to the hospital for something it’s truly good at, but you can’t always tell that from the ratings.”
About the Author
Leave a Comment Cancel reply
Concerning the reduction in size of Kidney Transplant lists at different major transplant centers, I have been monitoring those numbers over the last six to seven years.
I have a fair amount of practice on these issues:
1. I lost my kidneys, thru no fault of my own, when my kidneys were accidentally demolished by my surgeon during a “Laporascopic Belly Banding procedure;
Two. As a result I was on kidney dialysis for almost five years – a procedure that is universally described by people on dialysis as “Better Than Dying – BUT JUST BEARLY BETTER – and about 10% of dialysis patients determine death is better, and reject further dialysis and die a totally painless death over the course of one to two weeks;
Three. After almost five years on dialysis, I obtained a kidney transplant, and it was an amazing transformation. It was LIKE BEING REBORN. In the very first eight hours, I urinate’d out two liters (Two.Two quarts) of the darkest, smelliest urine you could ever imagine. And within twelve hour I was my old self again – my mind woke up – my sense of humor came roaring back – my coloring returned to normal after almost five years – and I was again Glad TO BE ALIVE – which was often missing during my time on dialysis . I HAD BEEN REBORN .
But I digress…Back to major Transplant Centers knocking approved kidney transplant recipients off their Waiting List in order to IMPROVE THEIR HOSPITAL RATINGS..
For much of that time I have studied Transplsnt Centers, several Transplant Centers had Wait Lists of inbetween Trio,000 and 6,000 patients waiting to get a kidney transplant, albeit they were only transplanting kidneys into one hundred to three hundred recipients per year.
Albeit they would lose some patients to “death” or “transfer to other transplant center,” the number of fresh patients seeking to be added to their Wait Lists, more than made up for the “Losses” the their “Wait Lists.”
Then abruptly the size of all these massive “Wait Lists” all of a sudden dropped to 1,000 to Two,000 potential Recipients on their Wait Lists.
I have been attempting to figure out what force of nature or humankind had cause this unexpected and massive drop in the Wait List size of these major Transplant Centers.
Now I eventually know .
These poor Potential Transplant Patients, who had been waiting for years while on dialysis for years for their chance to eventually obtain a LIFE-SAVING kidney transplant, all of a sudden had their HOPES OF GETTING their fresh, life-saving kidney transplant were abruptly SMASHED, and with total disregard for what it would mean to the hopeful recipient, disqualified” as a Potential Kidney Transplant Recipient so the these major Transplant Centers could “phony up” the number they had to report – by “REMOVING THE SICKEST PATIENTS,” [who had already been fully tested, certified, and approved as “healthy and safe” to receive a kidney transplant by their Transplant Center], FROM THEIR WAITING LIST – JUST SO THAT THESE TRANSPLANT CENTERS COULD HOPE TO UP THEIR “RATING SCORES” TO HELP THEM “BETTER MARKET THEIR HOSPITAL…”
Whatever happened to the Hippocratic Oath.
Eventually, someone is covering this subject. Business has truly fluffed up the pic of some of our countries leading hospitals; if some took closer looks at the malpractice rate or number of litigation cases annually, staff turn over (particularly the improvised/contract hires), or even (compulsory but anonymous) patience opinion polls regarding treatment, I’m sure their throats would drop to the floor.
Greed controls everything. Whoa hahahahahaha!
Do you reminisce how the housing bubble and lack of regulatory oversight triggered a global financial collapse during the Thicket administration? Central to the problem was how RATING AGENCIES like Moody’s and Standard & Poors gave top AAA ratings to even the riskiest loan pools. Relying on untrustworthy ratings is what fated AIG, and the same can happen in healthcare if ratings are not to be trusted.
These days consumers are more likely to use the Internet and Google to search for the procedure they plan to have – “Best knee replacement surgeon in Texas” – and then compare several different rating systems that may even include Yelp and Angie’s List. While not necessarily the best way to search, it’s pretty reliable because of how Google ranks hits. And while it’s still possible to game that system too, it tends to be self-healing, again because of how Google adjusts its ranking algorithms to favor content that people find most valuable.
Why hospital ratings should be looked at with a grain of salt
Five reasons to take hospital ratings with a big grain of salt
In US News and World Report’s 2016-2017 ranking of hospitals, Massachusetts General Hospital ranked No. Three behind the Mayo Clinic and Cleveland Clinic. Steven Senne/AP
t’s hospital ratings season in America, that time of year when marketing executives kick it into high gear to trumpet — and spin — the way their hospitals are graded by outside organizations.
This year, their workload has been especially powerful. In addition to annual rankings released Tuesday by U.S. News & World Report, the federal Centers for Medicare and Medicaid Services (CMS) just released its star-rating system for hospitals nationwide. While the ratings suggest some valuable information to consumers, here’s why you shouldn’t put too much stock in the results.
No matter how objective the metrics, ratings are inherently subjective
The federal government and private organizations use a vast array of hospital data to break down their spectacle and boil it into easy-to-understand ratings for consumers. U.S. News & World Report formulates its rankings by relying on a mix of physician surveys and data reported to trade groups and the federal government. CMS ratings are based solely on data hospitals are required to report to the agency.
The information itself may be objective, but which data points are used — and how they are weighted — can lead to contradictory conclusions. Consider the following:
Of US News & World Report’s top five hospitals, only one — Mayo Clinic — received a top, five-star rating from CMS. The rest — Cleveland Clinic, Massachusetts General Hospital, UCLA Medical Center, and Johns Hopkins — were not even in CMS’ top 100. (Johns Hopkins did not receive a rating from CMS because of incomplete data).
J.B. Silvers, a professor of health care finance at Case Western Reserve University, said the information, while boiled down into ordinary lists and starlets, can be difficult for consumers to unpack, especially when hospitals are working behind the scenes to figure out ways to better their scores every year. “Some hospitals are better at working the numbers than others,” he said. “My guess is the safety net hospitals aren’t as good at it as richer hospitals.”
One bad score, and your rating is shot
Lobbyists and some hospital executives argue that the CMS star-rating system penalizes providers that treat greater numbers of low-income patients. That’s because the CMS system hurts hospitals with higher rates of readmission, a factor intensely influenced by a patient’s economic status.
Critics have noted that instructing hospitals and safety net hospitals — both of which tend to deal more with sick, low-income patients — fared poorly in CMS ratings.
“On its surface that says, ‘Hey, you’ve got a bias,’” said Silvers. “Big instructing hospitals are taking care of the most complicated cases. Hardly any of them are in the five-star category. Most of them are in the one-star category.”
What’s not in the ratings matters, too
Ratings systems capture big amounts of information on everything from whether doctors are washing their palms to mortality rates for heart attack patients. But slew of information is not collected that might help provide a more nuanced — and accurate — assessment of spectacle. One hospital quality specialist noted, for example, that the severity of a cancer patient’s disease is not typically recorded by hospitals in diagnostic information and doesn’t get factored into outcome data and ratings. This can unfairly penalize hospitals that treat large numbers of patients with advanced cancers.
“Stage four cancer patients are more likely to die than stage one cancer patients,” said Harold Miller, chief executive of the Center for Health Care Quality and Payment Reform. “But if you don’t know that information — and you’re measuring mortality rates — then you are going to get a bad response.”
Gaming the system
Hospitals routinely launch initiatives to improve scores in particular ratings categories. They can also switch how they parse and report certain information to improve their results. In some cases, those adjustments can mean positive improvements in the care provided to patients. In others, they can skew the results or cause hospitals to avoid treating some patients, altogether.
For example, a latest examine by the American College of Surgeons found that organ transplant centers are removing the sickest patients from their waiting lists because of concerns that a negative outcome could harm their ratings and lead to federal scrutiny.
“You may determine not to take patients who are at risk of bad outcomes if somebody’s going to embark rating you on your outcomes without adjusting decently for that,” Miller said. “Those are the patients that need the hospitals the most, and they are the ones that are going to potentially get disadvantaged by this.”
The marketing agenda
Hospital ratings — particularly the results published by U.S. News & World Report — are often used for hospital advertising campaigns. Most institutions ranked in the top ten will hype their ratings success on billboards, banners, and brochures that get blasted across the regions they serve.
But the glossy mass media messages suggest little valuable information to consumers attempting to figure out where to get treatment for a violated gam or have gall stones eliminated because common procedures are seldom evaluated . “It could be the hospital is bad on some things, and indeed good on other things,” Miller noted. “But if you’re the patient, how do you know? You might be going to the hospital for something it’s truly good at, but you can’t always tell that from the ratings.”
About the Author
Leave a Comment Cancel reply
Concerning the reduction in size of Kidney Transplant lists at different major transplant centers, I have been monitoring those numbers over the last six to seven years.
I have a fair amount of practice on these issues:
1. I lost my kidneys, thru no fault of my own, when my kidneys were accidentally ruined by my surgeon during a “Laporascopic Tummy Banding procedure;
Two. As a result I was on kidney dialysis for almost five years – a procedure that is universally described by people on dialysis as “Better Than Dying – BUT JUST BEARLY BETTER – and about 10% of dialysis patients determine death is better, and reject further dialysis and die a totally painless death over the course of one to two weeks;
Three. After almost five years on dialysis, I obtained a kidney transplant, and it was an amazing transformation. It was LIKE BEING REBORN. In the very first eight hours, I urinate’d out two liters (Two.Two quarts) of the darkest, smelliest urine you could ever imagine. And within twelve hour I was my old self again – my mind woke up – my sense of humor came roaring back – my coloring returned to normal after almost five years – and I was again Glad TO BE ALIVE – which was often missing during my time on dialysis . I HAD BEEN REBORN .
But I digress…Back to major Transplant Centers knocking approved kidney transplant recipients off their Waiting List in order to IMPROVE THEIR HOSPITAL RATINGS..
For much of that time I have studied Transplsnt Centers, several Transplant Centers had Wait Lists of inbetween Trio,000 and 6,000 patients waiting to get a kidney transplant, albeit they were only transplanting kidneys into one hundred to three hundred recipients per year.
Albeit they would lose some patients to “death” or “transfer to other transplant center,” the number of fresh patients seeking to be added to their Wait Lists, more than made up for the “Losses” the their “Wait Lists.”
Then all of a sudden the size of all these massive “Wait Lists” all of a sudden dropped to 1,000 to Two,000 potential Recipients on their Wait Lists.
I have been attempting to figure out what force of nature or humankind had cause this unexpected and massive drop in the Wait List size of these major Transplant Centers.
Now I eventually know .
These poor Potential Transplant Patients, who had been waiting for years while on dialysis for years for their chance to ultimately obtain a LIFE-SAVING kidney transplant, abruptly had their HOPES OF GETTING their fresh, life-saving kidney transplant were all of a sudden SMASHED, and with total disregard for what it would mean to the hopeful recipient, disqualified” as a Potential Kidney Transplant Recipient so the these major Transplant Centers could “phony up” the number they had to report – by “REMOVING THE SICKEST PATIENTS,” [who had already been fully tested, certified, and approved as “healthy and safe” to receive a kidney transplant by their Transplant Center], FROM THEIR WAITING LIST – JUST SO THAT THESE TRANSPLANT CENTERS COULD HOPE TO UP THEIR “RATING SCORES” TO HELP THEM “BETTER MARKET THEIR HOSPITAL…”
Whatever happened to the Hippocratic Oath.
Ultimately, someone is covering this subject. Business has indeed fluffed up the picture of some of our countries leading hospitals; if some took closer looks at the malpractice rate or number of litigation cases annually, staff turn over (particularly the makeshift/contract hires), or even (compulsory but anonymous) patience opinion polls regarding treatment, I’m sure their throats would drop to the floor.
Greed controls everything. Whoa hahahahahaha!
Do you reminisce how the housing bubble and lack of regulatory oversight triggered a global financial collapse during the Thicket administration? Central to the problem was how RATING AGENCIES like Moody’s and Standard & Poors gave top AAA ratings to even the riskiest loan pools. Relying on untrustworthy ratings is what fated AIG, and the same can happen in healthcare if ratings are not to be trusted.
These days consumers are more likely to use the Internet and Google to search for the procedure they plan to have – “Best knee replacement surgeon in Texas” – and then compare several different rating systems that may even include Yelp and Angie’s List. While not necessarily the best way to search, it’s pretty reliable because of how Google ranks hits. And while it’s still possible to game that system too, it tends to be self-healing, again because of how Google adjusts its ranking algorithms to favor content that people find most valuable.