Rebecca L. Johnson, MD
Gary Procop, MD
August 2016—The American Board of Pathology has been selected to participate in the American Board of Medical Specialties’ Maintenance of Certification Assessment Initiative pilot. Currently, MOC Part III assesses a diplomate’s knowledge, judgment, and skills with a secure examination. The aim of the new initiative is to develop and test a new model for assessment that will be formative and summative.
The ABMS will facilitate and support the participating member boards in developing, delivering, and evaluating the alternative assessment models. The goal is to improve our diplomates’ MOC Part III experience while retaining a reliable and valid assessment that allows for credible summative decisions about medical knowledge, judgment, and skills for continuing certification.
In response to feedback from our diplomates who participate in MOC, the ABP continues to modify and improve the MOC program to be more relevant and valuable, and this initiative is likely to be a significant and well-received addition.
There is a growing emphasis in adult education and learning on the use of frequent, longitudinal assessments with retesting of key content as a means of improving knowledge retention. Longitudinal assessment models represent an evolution from traditional, point-in-time secure exams. In addition to being an assessment of learning, this will also be assessment for learning. While the program specifics are still being developed, the plan is for the ABP to email to participants every three months 25 to 30 multiple-choice questions that can be answered at any time during that quarter. Once the question is answered, diplomates will be asked how relevant that question is to the general practice of pathology, how relevant the question is to their practice, and how sure they were of their answer, from very sure to just guessing. Diplomates then receive immediate feedback with the correct answer, a short narrative about the topic, and references. They then have the opportunity to answer the question a second time. This new assessment will incorporate many of the features of the American Board of Anesthesiology’s MOCA Minute. (See www.theaba.org/MOCA/MOCA-Minute for information on the ABA program.)
The ABP plans to launch the Assessment Initiative pilot in 2017 with volunteer diplomates. Volunteers will agree to answer a minimum number of questions each year, commit to the duration of the pilot, and provide feedback to the ABP on the initiative. During the initial phase of the pilot, the ABP will not assess a volunteer diplomate’s performance. Participation in the MOC assessment pilot will allow our diplomates to have a voice and active role in creating the next generation of MOC assessment, which may be an alternative to the MOC Part III exam in three to five years. Their participation and feedback will help inform and assess the pilot’s development and offer valuable insight into the user experience. During the pilot, diplomates will still be required to take and pass the MOC exam before the end of their first 10 years after initial time-limited certification, but this pilot could help diplomates prepare for the MOC exam. The ABP is also considering offering participation in the pilot to pathology residents as a learning tool.
The greatest challenge in making this assessment relevant for all ABP diplomates will be developing a sufficiently large and diverse bank of items (questions) to be able to offer specialty- and practice-relevant assessment. This will take several years, so we anticipate that the pilot will last for three to five years. The ABP plans to add more volunteers to the pilot as sufficient subspecialty items are developed. We will solicit volunteers who are interested in being item writers or reviewers or both.
CertLink is the ABMS shared technology platform that will support pilot longitudinal assessment models. CertLink will present multiple-choice questions to participating diplomates and provide immediate feedback, an essential element in learning and retention. Diplomates will have a personal dashboard that summarizes their performance and areas of strength and weakness.
In the coming months, the ABP will share more information and pilot details as they become available, request volunteers to participate in the Assessment Initiative pilot, and crowdsource for item development and request volunteers to be item writers and reviewers. The ABP will provide instructions on good item development and a template and website for item submission.
If you have questions, or if you would like to volunteer for the pilot initiative or volunteer to be an item writer or reviewer, please send an email to the ABP at PathCertLink@abpath.org.
Dr. Johnson is chief executive officer of the American Board of Pathology. Dr. Procop is ABP president.