Since September 2002, Nielsen has been writing occasional commentaries on issues related to the development of the standard national patient survey -- otherwise known as HCAHPS -- that has now been folded into the federal government's Healthcare Quality Initiative.
It goes without saying that Gallup supports the Healthcare Quality Initiative and the notion that consumers should have the best information when making healthcare decisions. After all, the idea underlying our healthcare research is that consumers have a right to both objective and subjective assessments of quality, including a hospital’s level of clinical experience, incidence of medical errors, incidence of mortality and morbidity, incidence of infections and pneumonia -- as well as the opinions and perceptions of past patients.
So naturally, we’ve been following the development of a national patient satisfaction survey closely and commenting on the process along the way. On Dec. 22, 2003, the Agency for Healthcare Research and Quality (AHRQ), together with the U.S. Centers for Medicare and Medicaid Services (CMS), issued the results of the HCAHPS three-state pilot study that used the proposed hospital patient satisfaction questionnaire. Several interested parties, including Gallup, have provided feedback on the results.
Does It Measure the Right Things?
Perhaps the most basic area of concern has been that, despite its length, the questionnaire may miss the mark when it comes to providing the type of information most useful to consumers.
For example, while the "experience" model on which the survey is based is useful in determining the incidence of pneumonia, medical errors, and other clinical issues, it is less appropriate for assessing patients’ subjective perceptions. That a patient reports that a nurse appeared when he or she pressed the call button isn’t necessarily as helpful from the consumer’s standpoint as is the quality of his or her interaction with the nurse after the nurse appeared.
Another criticism has been that the "domains of care" framework used to develop the questionnaire was not empirically derived. Despite admonitions by the research community that the domains were unsound, AHRQ went ahead with the model -- only to determine from the pilot data that the criticism was justified. According to the HCAHPS pilot results analysis: "While some items correlated strongly with their hypothesized composite, it was clear that the general hypothesized structure of the instrument was inconsistent with the observed data." This is extremely important to the effectiveness of the instrument; starting with faulty assumptions implies that all the resulting conclusions may also be faulty.
Is It Too Long?
The length of the survey still also generates misgivings. Two weeks ago, the American Hospital Association (AHA) issued a press release announcing that it supports a 10-question survey. Additionally, AHA Executive Vice President Rick Pollack stated that while standardization of patient satisfaction information will help consumers, the "collection of this information must not impede the quality improvement work underway in America’s hospitals, nor should it be too costly, duplicative or cumbersome to implement."
The pilot results have helped somewhat in this regard -- the number of items on the survey was reduced from 66 to 32. But it’s still too long: 32 items would add seven to nine minutes to most existing telephone patient satisfaction surveys, dramatically increasing costs and respondent burden, with little added insight or benefit. The consensus at Gallup and among some of our industry competitors is that no more that six items are necessary to establish a scientific, reportable national patient satisfaction standard.
Bottom Line
For many onlookers, the HCAHPS pilot study has done little to allay concerns that the survey is overly burdensome and may not provide consumers with valuable, tangible information that could affect their choice of hospitals. There is no evidence in the HCAHPS study analysis of links to important indicators of quality care -- such as average length of stay, incidence of medical errors, risk indexed beds, and employee engagement.
The Gallup Organization and some of our competitors have committed to pooling our scientific resources to assist CMS in developing a more supportable system for reporting patient perceptions of the quality of care they receive from hospitals. We believe that, if properly developed, this system will prove to be of great advantage to future hospital patients and to the hospital industry. As currently proposed, however, the HCAHPS survey offers little benefit to either hospitals or consumers.