Letters
Editor’s Note: Minnesota Medicine received the following letter regarding MN Community Measurement’s quality-reporting requirements and their effect on groups that participate in federal comparative effectiveness research. We asked MN Community Measurement to respond. Here is the exchange between Barbara Yawn, M.D., from Olmsted Medical Center and officials from MN Community Measurement.
A Problem of Rigidity
Measuring and reporting quality metrics has become a goal for many payers, employers, and public-interest groups.1,2 The idea being that if we publicly report some type of quality indicator, all health care groups will strive for higher scores and, therefore, achieve better outcomes.3 MN Community Measurement was one of the first organizations to convince large and small health care groups to report outpatient quality metrics. Their work has been held up as an example to be emulated on a national level.4
The metrics MN Community Measurement uses are designed by a group of practitioners in the state who review and approve them with limited expert support on evidence-based reviews. MN Community Measurement puts the metrics in a standardized format. Clinics then collect data and send that information to MN Community Measurement for analysis and publication. Because of the connection between publicly reported metrics and pay-for-performance programs, the metrics drive the standard of care in our community for several chronic conditions.
Recently, the pressure to use these metrics and the unwillingness of MN Community Measurement to waver from them has led to a significant problem in carrying out federally funded comparative effectiveness research. Two examples illustrate this point.
One case is a study of decision aids for patients with diabetes funded by AHRQ under the R34 mechanism (R34 – 5R34DK084009). The decision aids provide patients with information regarding the risks and benefits of daily aspirin therapy. The risk-to-benefit ratio presented was based on the latest evidence from the medical literature and led several patients without diagnosed cardiac disease to refuse daily aspirin therapy. MN Community Measurement does not take patient preference into account. The publicly reported metric is simply the number of patients with diabetes who appear to have been prescribed daily aspirin according to medication records review.5
Before participating in this study, physicians had compliance levels of 60 percent to 80 percent. When using the patient decision aids, they had levels of less than 40 percent. For some who were participating in pay-for-performance programs, these scores resulted in a salary decrease. In addition, during the year of the study, evidence became available that aspirin indeed did provide more risk than benefit for people with diabetes who did not have known cardiovascular disease. However, the metric did not change and, thus, rewarded care that was not evidence-based and did not meet the recommendations of several national groups including the American Heart Association.6,7
In the second case, the Agency for Healthcare Quality and Research funded a five-year $2.5 million study (1R011HS018431) on the comparative effectiveness of older asthma control tools listed in the 2007 NAEPP guidelines (eg, ACT) with newer tools shown to improve outcomes in primary care settings. Sites randomized to the newer tools would fail the asthma metric set by MN Community Measurement since they only approve of tools mentioned in the 2007 guidelines. Several practices in Minnesota and one in Wisconsin felt they could not participate in the federal study because of the poor scores they would receive on the publicly reported asthma metrics and the resulting decrease in reimbursement they would receive if they were randomized to the group using the new tools. A formal request was made to MN Community Measurement to provide a waiver to those participating in the study. The request was refused with the comment “We do not want to block research, but we will only accept the asthma control measures approved by the guidelines.”
There are two unintended consequences of the rigid MN Community Measurement process. It is blocking research, and, in the case of the diabetes study, it is continuing to support inappropriate care. The asthma and diabetes studies will go forward, but the work will have to be done in other states. Is this what we want for quality improvement in Minnesota?
The potential solutions seem clear. First, MN Community Measurement needs to provide a waiver to practices that participate in large federally funded comparative effectiveness studies, especially when they compare the tools used in older approved metrics with newer ones. And second, when evidence becomes available that a metric may be supporting harmful interventions, it needs to review the data and if they appear to warrant closer evaluation immediately state that the metric in question will not be assessed or reported that year. And it needs to allow Minnesota practices to do what this quality program professes to do: support the provision of evidence-based high-quality care and allow practices to help determine comparative effectiveness.
To encourage MN Community Measurement to move forward, I urge researchers, quality advocates, and payers to make this a public issue. Since the organization was founded on the belief that public reporting will move quality forward, they should also believe in publicly discussing their quality-assessment processes. Quality metric reporting and pay for performance should not block federally funded quality research or innovations in care.
Barbara P. Yawn, M.D., M.Sc.
Director of Research
Olmsted Medical Center
References
1. U.S. Department of Health and Human Services Agency for Healthcare Research. Selecting Quality and Resource Use Measures: A Decision Guide for Community Quality Collaboratives. Available at: www.ahrq.gov/qual/perfmeasguide/perfmeasappb.htm. Accessed November 30, 2010.
2. U.S. Department of Health and Human Services Agency for Healthcare Research. Best Practices in Public Reporting No. 1. How To Effectively Present Health Care Performance Data To Consumers. Available at: www.ahrq.gov/qual/pubrptguide1.htm. Accessed November 30, 2010.
3. Rand Health. Public Reporting of Health Care Quality Information. Available at: www.rand.org/health/surveys_tools/public_reporting.html. Accessed November 30, 2010.
4. Robert Wood Johnson Foundation. Quality/Equality. Available at: www.rwjf.org/qualityequality/af4q/communities/minnesota.jsp. Accessed November 30, 2010.
5. The D5. The Five Goals for Living Well with Diabetes. Available at: www.thed5.org. Accessed November 30, 2010.
6. Reinberg S. Experts Advise At-Risk Diabetics to Begin Daily Aspirin Therapy. Available at: www.medicinenet.com/script/main/art.asp?articlekey=116680. Accessed November 30, 2010.
7. Daily Aspirin Therapy: Understand the Benefits and Risks. Available at: www.mayoclinic.com/health/daily-aspirin-therapy/HB00073. Accessed November 30, 2010.
MN Community Measurement Responds
Dr. Yawn is certainly correct that Minnesota has been a leader in using quality metrics to support improvement in care for people in our community. We have seen significant improvement in the results for many of our measures over the last several years. Medical groups across the state have worked to both standardize processes and redesign care to improve results. To support quality improvement, we believe that our measures should be based on the latest evidence, evaluate patient outcomes and not just processes of care, be as inclusive as possible of all patient populations, be standardized and aligned across the community in order to have the greatest impact, and be developed and updated using a multi-stakeholder process supported by clinical experts.
Dr. Yawn’s example of changes in the science of diabetes care actually shows how these principles can work to ensure that measures are up to date. In 2010, the Institute for Clinical Systems Improvement and the American Diabetes Association changed their recommendations for use of aspirin therapy in diabetes care based on new evidence. MN Community Measurement’s technical advisory committee, which included experts in endocrinology, family medicine, and internal medicine, recommended that the measure be changed to require aspirin use only in those patients with ischemic vascular disease. This change will go into effect for 2011. Since the measure is based on 2010 dates of service, we are able to keep the measure aligned with the timing of the change in the guideline.
The asthma care study referenced by Dr. Yawn presents another interesting issue: How should we address patients involved in clinical trials? In 2009, MN Community Measurement formed a technical advisory committee to develop a new asthma care measure that would go beyond our former HEDIS measure on appropriate use of medications. The new measure includes assessment of asthma control using standardized tools, assessment of asthma risk using patient-reported emergency room visits and hospitalizations, and use of a written asthma action/management plan.
To assess asthma control, the committee approved the use of three standardized tools that have had extensive validation. One of the gaps in care that the committee identified was the lack of consistent use of these standardized assessment tools in our community. Dr. Yawn recommended to the committee that another assessment tool, the Asthma APGAR, which she proposes to use in the federally funded research project, should also be included. After much consideration, the committee felt the APGAR tool did not have enough validation to be included in a standardized measure for public reporting, but it could be reconsidered once additional evidence was obtained. This recommendation was also reviewed and accepted by MN Community Measurement’s Measurement and Reporting Committee and Board of Directors, submitted for public comment, and then accepted as part of the Minnesota Statewide Quality Reporting Rule submitted for 2011.
We believe our first priority is to encourage greater use of tools and processes such as those in the asthma measure that have already been documented to improve care. Clinicians participating in clinical trials should consider incorporating validated instruments for testing their experimental tools and processes. The standardized tools are easy to use and will not hinder new studies of clinical efficacy. One of the benefits of MN Community Measurement is that it allows health care sites to be compared using measures that can be applied to all eligible practices. That way, we can identify processes that can be widely adopted to improve patient care. Participation in clinical research, while commendable, should not constitute grounds for exemption from measuring performance using standardized and validated instruments. If this were to be a rule, all clinical sites that participate in clinical or academic research could claim exemption.
Jim Chase
President
MN Community Measurement
Beth Averbeck, M.D. and Linda Walling, M.D.
Co-chairs
Measurement and Reporting Committee
Kaiser Lim, M.D.
Asthma Technical Advisory Committee
MN Community Measurement