News media and commentators are giving greater attention to doctor ratings compiled by health plans, online forums, and other groups to aid patients and families in selecting physicians. These rating systems aren’t very helpful, for a slew of reasons:
- As a Washington Post article noted last week, most ratings ignore physician-patient communication–whether the doctor is skilled and committed in listening, explaining, and asking questions about the patient’s concerns.
- Health plan rating systems emphasize performance factors that matter most to the plan, not to the patient. These include factors like (a) keeping costs down by not ordering a lot of tests and procedures, and (b) following the plan’s protocols.
- 3. Online forum rating systems, along with similar listings such as magazine ratings, are not based on any set criteria. Basically, they’re just impressions and lack scientific validity. (We here have been really pleased with a computer technician we found through Angie’s List, but I have no intention of using Angie to identify someone as important to me as a surgeon or cardiologist.)
- All of these systems, to my knowledge, lack reliability, mostly because they are based on very low numbers of raters or events. Thus, a study by RAND found that health plan ratings of how well individual physicians control costs are usually inaccurate.
- Physicians themselves greatly mistrust these rating systems and so are unlikely to cooperate with them. Last month the American Medical Association urged the nation’s health plans to come up with more accurate, reliable, and transparent ratings.
A core problem in rating anything as complicated as a physician’s performance is that, as a practical matter, the rating system has to do one of two things. Either it has to focus on a few, easily counted factors (so that “the quantifiable drives out the significant,” to use a terrific phrase once expressed to me by eminent scientist and engineer Walter Rosenblith). Or, it has to be based on squishy findings with no real validity or reliability.
The lesson is, there’s no easy way to find the physician who’s just right for you. Section 2 (“Finding the Right Doctor”) of our guide offers a lot of tips, and here’s another one: Don’t rely on some third party, who doesn’t know you at all, to make the selection for you.
By the way, similar defects apply to ratings of hospitals and nursing homes, including ratings provided by federal and state government agencies. An added problem is that these ratings usually are based on old information.