A Conversation about Discontinuing USMLE Step 2 Clinical Skills

In this episode of USMLE® Connection, Dave Johnson, the Senior Vice President at Federation of State Medical Boards, and Chris Feddock, Executive Director for the Clinical Skills Evaluation Collaboration, discuss the discontinuation of the Step 2 CS exam, the reasons behind the decision, and next steps. 

Transcript

A Conversation about Discontinuing USMLE Step 2 Clinical Skills 

Suzanne McEllhenney: Hello, and welcome to USMLE Connection. A podcast designed to keep you in the know about all things USMLE. I'm Suzanne McEllhenney, Director of Program Management for USMLE. In today's episode, we'll hear about the decision to discontinue efforts to create a modified Step 2 Clinical Skills exam, or Step 2 CS, which was temporarily suspended in the spring due to the COVID-19 outbreak and the need to follow public health guidelines. We're here today with David Johnson, Chief Assessment Officer at the Federation of State Medical Boards, and Dr. Chris Feddock, Executive Director for the Clinical Skills Evaluation Collaboration or CSEC, which administered Step 2 CS. Let's get started. Chris and Dave, thank you so much for joining me today.  

Chris Feddock: Pleasure to be here.

David Johnson: Glad to be joining you, Suzanne.

Suzanne McEllhenney: In late January, the USMLE program announced that it was discontinuing the revitalization work underway for Step 2 CS. Can you help us understand what happened?  

Chris Feddock: Sure. Suzanne, let me start by providing some background information and then I'll turn it over to Dave to follow up with some updates since last May. You know, to begin with. I think it's important to keep in mind that the USMLE program has been working on transforming the assessment of clinical skills for the past several years. Well, before the pandemic even started. In fact, beginning in 2018, we have been exploring both new content to assess clinical skills and new formats, different ways of assessing clinical skills. Then of course, in March 2020, COVID-19 really threw a wrench into all of that initial thinking. We found ourselves in a very challenging situation. Given the close contact that was required for the exam and our concerns about the health and safety, not only of our staff, but the examinees, we had to make the decision to immediately suspend all exam administrations.  

David Johnson: That's right, Chris. And then in May of last year, when it was clear to us that COVID-19 was going to be a multi-month issue, we extended the suspension of the exam for 12 to 18 months, you know, taking us to this winter. And our hope was that we could stand up a modified exam on a virtual platform that removed the risk of exposure. You know, again, as you mentioned for examinees and staff protection, and to do that in this 12 to 18 month timeframe. We also set a goal to make the exam appreciably better than the prior assessment within that timetable that we'd announced. And accordingly, we had regular check-ins with leadership, both the Federation and NBME to track progress toward a modified CS. When we got to this winter's check-in to review status, we came to the conclusion that we were not going to be able to stand up a modified and appreciably better exam on the timeline that we had envisioned.  

Chris Feddock: That's an important point, Dave. You know, I agree, the bottom line was that the work we had completed really showed that any short-term exam modifications that we could accomplish in that timeline ld would not meet our expectations for improvement and innovation.  

Suzanne McEllhenney: Thanks, Dave and Chris. Could you also elaborate on some of the reasons for ending the approach to relaunch the exam as well as some of the lessons that you learned?  

David Johnson: Sure, Suzanne, I'm happy to start us off. You know, I think it's safe to say we've learned a lot since May of 2020, and one of those things is that the medical education and healthcare environments have continued to change at a very rapid pace. Now we anticipated that this pace of change was going to continue and by the time our modified exam would have been launched, ongoing innovations in assessment would have surpassed some of the methods we could develop for Step 2 CS in the short term.  

Chris Feddock: That's right, Dave. I'd also add that, this was really an extensive effort with teams that really had staff from all three organizations, the Federation of State Medical Boards, FSMB, the Educational Commission on Foreign Medical Graduates or ECFMG and NBME. And although those teams made significant progress, you know, many of the aspects of the exam design would really be unclear for a number of months. You know, extensive piloting really had to occur in order for us to ensure that the future exam met the rigorous standards of USMLE. And in addition to redesigning the content, we also strived to reduce or remove some of those challenges that examinees, you know, have informed us about over the past several years. Top of that list really included, you know, the fees for the Step 2 CS, as well as the need to travel to one of the six cities to take the exam. And again, over the past 10 months, you know, we, we learned a lot of lessons, particularly the technology demands and complexity, that really is required to administer a virtual exam. We closely followed and learned from the other assessment organizations who, you know, over this time period, tried to expand exam access broadly and fairly. I think some of our takeaway points were the potential connectivity issues, proctoring logistics, and then honestly, difficulties in providing equal access to all examinees.

David Johnson: You know, and Chris, if I could just add - I think in the end there was no one single factor, be it COVID or the cost associated with the exam or perceptions of value. There was no one single factor that actually determined the decision to discontinue Step 2 Clinical Skills, but all of those factors contributed to a holistic review and led to the decision to end a short-term relaunch of the exam itself.  

Suzanne McEllhenney: From the way you've both have described it, it sounds like COVID-19 caused administrative obstacles and interrupted plans that had really been proceeding for some time. Could you talk about the announcement itself and why it felt surprising to so many of us?  

David Johnson: Sure, Suzanne. I think once the organization, so the Federation and the NBME, reached a decision to discontinue the exam, we wanted to respect everyone's need to be informed as quickly as possible. And from regulators to examinees there were many stakeholders that we needed to reach out and get this word to. And I have to say, or admit, that the change is a lot to absorb. It was a lot of information to share with others and for them to take in. And I recognize we still have to provide additional details regarding our plans moving forward.  

Suzanne McEllhenney: Thanks, Dave. Can you explain why USMLE remains committed to assessing clinical skills, and what are your thoughts on the path forward for clinical skills assessment?  

David Johnson: Well, state medical boards are deeply committed to ensuring that physicians provide safe and effective patient care. And USMLE continues to provide important assessment data to help inform a medical board decisions about licensure. Right now, we're creating a plan to engage with both colleagues at state medical boards and those in the medical education community to help develop some innovative ways that we can assess those competencies associated with clinical skills. The work done across our respective organizations over the past 10 months will help inform that path as we move forward. And we are thinking of clinical skills in a broader way for licensure.  

Chris Feddock: You know, Dave, I agree. And I think that's an excellent point. You know, at this point in time, we really have no plans to develop another standalone, full-day exam. You know, we're taking our time to develop an approach that transforms clinical skills assessment for the future. You know, given the resources that we had dedicated to the short-term work, you know, this decision actually enables us to not only focus on, but actually accelerate work to innovate the assessment of physician skills.  

Suzanne McEllhenney: So what are the next steps?  

David Johnson: I'm happy to start with that, Chris. In terms of next steps, I think future work on assessment of clinical skills will be guided by our engagement with the state medical boards and the medical education community. In addition, the composite committee is going to be reviewing some policy questions that naturally arise from the discontinuation of Step 2 Clinical Skills. And we'll be issuing some updates as decisions are made and more information becomes available.  

Chris Feddock: I'll also add that international medical graduates, or IMGs, you know, previously took Step 2 CS as a requirement for ECFMG certification, which will enable them to begin residency training. You know, since Step 2 CS was suspended last spring, the ECFMG has offered pathways to allow IMGs to meet those clinical and communication skills requirements allowing them to participate in the residency match. You know, in fact, the ECFMG has recently announced expanded options for the 2022 match. Listeners interested in those pathways should visit the ECFMG.org website for more information.  

Suzanne McEllhenney: Dave and Chris, thank you for giving our listeners a glimpse into the Clinical Skills decision.  

David Johnson: Oh, you're welcome, Suzanne. I'm glad to be here.  

Chris Feddock: Yes. Thank you, Suzanne. Glad to participate.