Cover Story
Test Anxiety
As specialty boards have raised the bar for recertification, many wonder, Does passing a test indicate competence?
By Howard Bell
Mention board recertification and the first thought most doctors have is about the test they must pass in order to continue practicing medicine. Physicians with time-limited board certifications have to take an exam every seven to 10 years. But now that maintaining board certification also requires completing learning modules, computer-based simulations, and practice quality assessments, physicians are questioning whether an exam is necessary, much less whether it assures competence.
How to judge whether a physician is capable is a hot topic these days. Government payers and private insurers are calling for physicians to routinely demonstrate that their medical knowledge and skills are up to date. State boards of medical practice are asking whether their systems for maintaining licensure are adequate for ensuring that doctors know and do what they’re supposed to. The public wants guarantees that the doctor who’s treating their family member is up to snuff.
Although physicians don’t take issue with the importance of keeping their knowledge current and improving their practice, they are concerned about having to take a test that, if they fail, could affect their livelihood. Is a high-stakes exam a good measure of a physician’s ability?
Differing Opinions
The American Board of Medical Specialties (ABMS) says yes. So did the Federation of State Medical Boards until this spring, when the organization that oversees state medical boards changed its mind somewhat and said tests were just one of the ways physicians could demonstrate their competence.
Minnesota physicians are divided about the value of taking a recertification test. “It demonstrates knowledge, and knowledge is the basis for decision-making,” says Mark Eggen, M.D., a Twin Cities anesthesiologist with Midwest Anesthesiology who is in his second year on the Minnesota Board of Medical Practice (BMP).
Failing Grade
What happens if a physician who is participating in Maintenance of Certification (MOC) fails a recertification exam? According to Twin Cities otolaryngologist Jon Thomas, M.D., who was a member of the Minnesota Board of Medical Practice’s task force on assessing physician competency, any physician who fails an exam can retake it, although the number of times a test can be retaken is subject to the rules of the specialty board.
“However, they are not certified or recertified until successful completion,” he says. Physicians with time-limited certification who lose their certification, “will likely find it difficult to get staff privileges at most Minnesota hospitals and may lose their position at large health systems,” says Linda Van Etta, M.D., an infectious disease specialist at St. Luke’s Hospital in Duluth who chaired the task force when it was established in 2006.
What happens if a lifetime-certified physician fails repeatedly also varies among specialties; but according to Thomas, lifetime certificate holders are not at risk for losing their certification. “Legally, the certificates are protected and cannot be taken away regardless of [MOC] participation or exam failure,” he says.—H.B.
|
Eggen has lifetime board certification, but he is participating in the American Board of Anesthesiology’s Maintenance of Certification (MOC) program and took the exam even though he didn’t have to. “It’s a professionalism thing,” he says. “When I took the exam, every question was something I should know. It was very relevant, and there’s nothing like the pressure of an exam to make you focus and absorb the material.” But a BMP task force (see “No New Requirements for Licensure—for Now,” p. 27) that has studied the issue of assessing physician competency did not reach a consensus about the value of recertification tests.
Linda Van Etta, M.D., an infectious disease specialist at St. Luke’s Hospital in Duluth who initially chaired the task force, is one of several members who think exams should no longer be required for maintaining board certification. Recertification tests measure factual recall, not diagnostic reasoning, she argues. The recall must be done without access to outside references, which contradicts what physicians are encouraged to do in the real world—rely on good sources of information instead of on memory. The tests also don’t assess important competencies such as professionalism and procedural skills, and preparing for them takes time away from seeing patients, which in this era of production-based medicine largely determines a physician’s compensation, she says. In addition, they might contain esoteric questions about diagnoses a physician might rarely, if ever, see. “Who do you want your pathologist to be,” she asks, “the one who can answer a test question about a rare tumor or the one who can find the lung cancer in a stack of blinded specimens?”
Van Etta also questions whether there is evidence that taking these tests makes you a better physician. “The ABMS keeps saying they’ve got studies and robust data showing recertification tests improve patient outcomes, and we keep asking them to show us their robust data. They can’t do it. It’s not there.” She thinks MOC should be more meaningful and not something that burdens busy physicians. “There are better ways to improve patient outcomes than by cramming for a test that may or may not be relevant to your practice. Adult learners don’t learn this way. The test needs to disappear.”
The Certification Story
It used to be that doctors were finished with test-taking when they left medical school. But that changed with the rise of specialization. In the early 1900s, when doctors began to specialize, boards were formed to define the qualifications of a specialist and to assure the public that a physician who claimed to be a specialist was indeed qualified. The Board of Ophthalmology was the first board, formed in 1917. Others soon followed. In 1933, 15 specialty boards joined together to form the ABMS, the organization that oversees the board-certification process. Today, 24 specialties comprise the ABMS.
Until 2002, each specialty board had its own certification standards. Some offered time-limited certification, while others went to lifetime certification. For some, practice assessment was part of the certification process, while others just required the test. Over time, the ABMS began to recommend time-limited certification, the thinking being that the study and preparation necessary to pass a test would keep a doctor current with ever-changing information. The American Board of Family Medicine was the first board to adopt time-limited certification back in 1969. Many boards switched in the late 1980s and ’90s. Since 2006, all of the ABMS’s 24 specialties have been granting time-limited board certification, requiring both a test and practice assessment.
In 2000, the boards that make up the ABMS began using an MOC process that emphasizes lifelong learning and attempts to provide a more accurate measure of competency. Now to maintain board certification, physicians with time-limited certification must participate in MOC. They must hold a valid, unrestricted medical license in at least one state, participate in lifelong learning and self-assessment using their specialty board’s approved CME modules, and participate in practice quality improvement (PQI) initiatives by comparing their evidence-based practices with those of their peers and national benchmarks. Although the process is more comprehensive, having to pass a test is still part of the mix.
Making MOC Work
No New Requirements for Licensure - For Now
In 2006, a Minnesota Board of Medical Practice (BMP) task force set out to explore how an initiative being developed by the Federation of State Medical Boards (FSMB), the organization that oversees state medical boards across the country, might affect Minnesota physicians.
The FSMB was in the process of creating its own framework for continuous professional development called Maintenance of Licensure (MOL), and it appeared to be similar to the Maintenance of Certification (MOC) framework required by specialty boards. Because all physicians with time-limited certification in the state are required to participate in MOC, some questioned whether licensure was even a concern, as the requirements for recertification are much more stringent than those for licensure in Minnesota.
The task force found that 60 percent of board-certified physicians in Minnesota have lifetime certification and 40 percent have time-limited certification. So more than half of board-certified physicians in the state did not have to participate in MOC activities. In addition, 25 percent of Minnesota physicians were not board-certified, meaning that one out of four was not eligible to participate in MOC as it’s currently structured.
Those findings prompted concern that requiring physicians to pass a recertification test in order to be licensed might lead to physician shortages in some parts of the state, as older physicians might opt to not take the exam and retire early. “We don’t want to discourage physicians from practicing in Minnesota,” says Linda Van Etta, M.D., a Duluth infectious disease physician who chaired the task force.
The task force decided not to recommend changes in Minnesota’s licensing requirements. So for now, Minnesota physicians still need only to pay a fee and attest to completing 25 CME credits a year to renew their license.
Task force members presented their findings and concerns at the October 2008 annual meeting of the FSMB. They made four recommendations related to MOL:
Have all physicians—board-certified or not—complete educational modules on a regular basis that are practice-relevant and include quality-improvement content;
Require health care systems to provide feedback to physicians about how well they’re doing on patient outcomes for such things as diabetes care, treating hypertension, and managing heart failure;
Provide physicians with patient identifiers so they can contact patients to enhance compliance and ultimately improve outcomes; and
Use electronic health record technology to help physicians improve episodes of care, for example, using pop-up screens that alert the physician that a patient’s blood pressure is outside the recommended guidelines.
As a result of testimony from the Minnesota group and others, the FSMB opted to make taking a recertification test an optional part of MOL.
In May, the FSMB made MOL its official policy and formed a group to help states wanting to implement it. Several have expressed interest, according to Jon Thomas, M.D., a Twin Cities otolaryngologist who was on the BMP’s task force and is the FSMB’s representative to the American Board of Medical Specialties. He notes that even for those states that jump on the MOL bandwagon, change won’t come quickly. He expects it to be a five- to 10-year process.—H.B.
|
Because of this new emphasis on MOC, specialty boards have been busy creating and refining their learning modules, PQI initiatives, and recertification tests to make them more accessible and pertinent to busy physicians. The requirements for anesthesiology, for example, have physicians completing interactive online learning modules and computerized case simulations, and providing a letter from a partner attesting to their good character, according to Eggen.
Such changes are having an effect on CME. In 2006, the Accreditation Council for CME released new standards for accrediting CME offerings: They must now focus on improving physician performance, clinical competence, and patient outcomes. “Traditional CME that is not tied to MOC is becoming less important for measuring physician competency,” Eggen says.
Many specialty boards now require physicians to complete online self-assessment modules (SAMs), which help
them identify the areas they most need to study. The American Board of Family Medicine requires its members to complete one self-assessment module per year on topics such as chronic pain management, hypertension, diabetes, and well-child care. “You pick the ones you want to do so they’re practice-relevant,” says Keith Stelter, M.D., associate director of the family medicine residency program at Mayo Health System-Mankato. Stelter served on the BMP’s task force on physician competency and is a member of the American Academy of Family Physicians’ Continuing Professional Development Committee.
One module takes three to 15 hours to complete and is worth 15 CME credits, according to Stelter. It costs about $225. Physicians must complete a multiple choice test and correctly answer at least 80 percent of the questions in their area. The module shows where to find the correct answers to questions and provides a synopsis of literature with sources for further reading. “Family practice is one of those specialties where completing MOC is fairly straightforward,” Stelter says. “Do one SAM per year, do a PQI twice every 10 years, complete 300 hours of additional approved CME, then take the recertification test every 10 years and you’ve largely completed MOC,” he says.
Starting this January, the academy is adding “translation into practice assessment” to its CME offerings, according to Stelter. Physicians get extra credit when they answer questions such as: Did you do additional reading on this topic? How did you enhance patient care by using what you learned? and What challenges have you had in implementing the knowledge in clinical practice?
The American Board of Family Medicine has authorized Mayo to develop its own “Group SAMs,” according to Stelter, which Mayo can then incorporate into its traditional symposiums and conferences. “A SAM then becomes a group learning experience,” Stelter says, “which enriches the learning because of the give-and-take among colleagues.”
Practice quality improvement is the newest component of MOC, and it’s the requirement “that has physicians most fidgety” because PQI must result in a documented improvement in a physician’s clinical practice, says Aaron Friedman, M.D., a professor of pediatrics at the University of Minnesota who serves on the American Board of Pediatrics’ MOC Committee. “These programs are complex to design,” he says, “and they’re expensive and time-consuming, especially for small group practices. They’re also harder to customize to a physician’s practice because PQI often requires participation from the rest of the clinic.”
Many specialty boards still aren’t clear about how to implement PQI programs, according to Friedman. Templates make it easier for physicians to complete such projects. For example, the American Society of Neuroradiology is working on a templated PQI module for reducing the radiation dose during CT scanning that can be used by all neuroradiologists. In addition, a clinic’s chronic disease registry ties in nicely with PQI activities, according to Stelter.
According to Friedman, the family medicine, internal medicine, and pediatrics boards authorized Mayo to develop its own PQI programs because Mayo has demonstrated a strong institutional commitment to quality improvement. In addition, the American Board of Pediatrics has incorporated PQI into its MOC process. It requires that 40 percent of the work done during an MOC cycle be lifetime learning activities, 40 percent PQI activities, and the remaining 20 percent either lifetime learning, PQI, or a mixture of both.
Although PQI is the most challenging part of the MOC process, it is also likely to produce the most measurable improvements in patient care, according to Van Etta—the kind of systems improvements the BMP task force concluded are essential to maintaining competence. “It offers a much better chance of improving patient outcomes,” she says. “Certainly better than taking a test.”
Growing Acceptance
Thus far, MOC is getting mixed reviews from physicians. Voluntary participation among lifetime-certified physicians varies greatly by specialty partly because some are further along in MOC implementation and partly because in some specialties, physicians are more motivated to participate because they want to keep up to date with changes in technology and procedures. Eggen says about half of lifetime board-certified anesthesiologists nationwide are participating in MOC and even take the test although they don’t have to.
A significant number of non-time-limited board-certified pediatricians are participating in MOC, according to Friedman. “Some pediatricians feel it’s more than they should have to do,” he says. “But they also realize we’re not going back to the way it was and that there is value to them and their patients in staying up-to-date.” Another incentive for doing MOC: Pediatricians take the recertification test every 10 years instead of every seven years if they participate.
Most doctors don’t expect the tests required for MOC to go away anytime soon, if ever. And all specialty boards are retooling their exams to be as fair, clear, and relevant to practice as possible. Even Van Etta acknowledges that some specialties, among them family medicine, have “a very practice-relevant exam,” in which physicians can choose which test modules to take.
It’s true tests don’t measure some key competencies, but their defenders point out that you can’t expect any type of assessment to do it all. Although most believe other components of MOC will render tests less important as time goes on, for now they’re here to stay. MM
Howard Bell is a medical writer in Onalaska, Wisconsin, and a frequent contributor to Minnesota Medicine.