Page 1/14

Title: A Comparison of Orthopaedic Surgery andInternal Medicine Perceptions of USMLE Step 1Pass/Fail Scoring: A National Program DirectorSurveyFrederick Mun  ( frederickmun96@gmail.com )

Penn State Milton S. Hershey Medical CenterAlyssa R. Scott 

Penn State Milton S. Hershey Medical CenterDavid Cui 

Penn State Milton S. Hershey Medical CenterErik B. Lehman 

Penn State Milton S. Hershey Medical CenterSeong Ho Jeong 

Yale New Haven HospitalAlia Chisty 

Penn State Milton S. Hershey Medical CenterPaul J. Juliano 

Penn State Milton S. Hershey Medical CenterWilliam L. Hennrikus 

Penn State Milton S. Hershey Medical CenterEileen F. Hennrikus 

Penn State Milton S. Hershey Medical Center

Research Article

Keywords: USMLE, Step 1, pass/fail scoring, orthopaedic surgery, internal medicine

Posted Date: February 25th, 2021

DOI: https://doi.org/10.21203/rs.3.rs-237028/v1

License: This work is licensed under a Creative Commons Attribution 4.0 International License.  Read Full License

Page 2/14

Abstract

BackgroundUnited States Medical Licensing Examination Step 1 will transition from numeric grading to pass/fail,sometime after January 2022. The aim of this study was to compare how program directors inorthopaedics and internal medicine perceive a pass/fail Step 1 will impact the residency applicationprocess.

MethodsA 27-item, validated survey was distributed through REDCap to 197 U.S. orthopaedic residency programdirectors and 554 U.S. internal medicine residency program directors. Program director emails wereobtained from the American Medical Association’s Fellowship and Residency Electronic InteractiveDatabase.

ResultsWe received 58 (36.0%) orthopaedic and 125 (22.8%) internal medicine program director responses. Themajority of both groups disagree with the change to pass/fail, and felt that the decision was nottransparent. Both groups believe that the Step 2 Clinical Knowledge exam and clerkship grades will takeon more importance. Compared to internal medicine, orthopaedic PDs have greater odds of emphasizingresearch (OR 6.43, CI 3.18-13.00, p < 0.001), letters of recommendation from known faculty (OR 3.22, CI1.77–5.87, p < 0.001)), Alpha Omega Alpha membership (OR 4.96, CI 2.65–9.25, p < 0.001),leadership/extracurricular activities (OR 3.69, CI 1.96–6.95, p < 0.001), audition elective rotations (OR6.20, CI 3.21–11.97, p < 0.001) and personal knowledge of the applicant (OR 2.93, CI 1.61–5.34, p < 0.001). Both groups believe that allopathic students from less prestigious medical schools, osteopathicstudents, and international medical graduates will be disadvantaged. Orthopaedic and internal medicineprogram directors agree that medical schools should adopt a graded pre-clinical curriculum, and thatthere should be a cap on the number of residency applications a student can submit.

ConclusionOrthopaedic and internal medicine program directors disagree with the change of Step 1 to pass/fail.They also believe that this transition will make the match process more di�cult, and disadvantagestudents from less highly-regarded medical schools. Both groups will rely more heavily on the Step 2clinical knowledge exam score, but orthopaedics will place more importance on research, letters ofrecommendation, Alpha Omega Alpha membership, leadership/extracurricular activities, personalknowledge of the applicant, and audition electives.

Page 3/14

BackgroundThe United States Medical Licensing Examinations (USMLE) currently consists of three numericallyscored knowledge based exams and a pass/fail clinical skills exam. On February 12, 2020, the Federationof State Medical Boards (FSMB) and the National Board of Medical Examiners (NBME) announced thatthe �rst of these three exams, Step 1, would change from the three-digit numeric score to reporting apass/fail outcome -- starting sometime after January 2022.1 This change occurred after extensivenational discussion on potential ways to optimize the transition from undergraduate to graduate medicaleducation.1,2

The unintended consequences of a scored Step 1 on medical students and residency programs werewidely reported in 2008 by the Committee to Evaluate the USMLE Program (CEUP). CEUP recognized thediverse stakeholders affected by Step 1 score reporting and suggested a need to redesign USMLE.2

Although a numerically scored Step 1 provided an objective measure for residency programs to assessapplicants from different backgrounds and medical schools, the perceived overemphasis on this test wascontroversial. In March 2019, the Invitational Conference on USMLE Scoring (InCUS), sponsored by theAssociation of American Medical Colleges (AAMC), American Medical Association (AMA), NBME, FSMB,and Educational Commission for Foreign Medical Graduates (ECFMG), convened to review the USMLE’spractice of numeric score reporting. InCUS concluded that licensing examinations for undergraduatemedical students must be reevaluated. FSMB and NBME took into consideration the recommendationsfrom InCUS, and stated that the change to a pass/fail Step 1 would help residency programs to refocuson the main purpose of Step 1, to assess medical licensure eligibility.1,2

Step 1 is one of the most important factors used to screen applicants by internal medicine (IM) andorthopaedic surgery residency programs.1–3 Some residency programs also place a high emphasis onother factors, such as grades in required clerkships, research experiences, extracurricular activities, lettersof recommendation, and audition rotations. With the absence of a scored Step 1, all residency programswill need to reconsider the factors used to determine which applicants are selected. The aim of this studywas to compare the perceptions among program directors (PDs) in orthopaedics and IM on the change ofStep 1 from a graded to a pass/fail exam, and its impact on the residency application process.

MethodsThis study was exempted by the College of Medicine’s Institutional Review Board. We obtained publiclyavailable contact information for PDs at all active Accreditation Council for Graduate Medical Education(ACGME) orthopaedic surgery and IM residency programs through the American Medical Association’sFellowship and Residency Electronic Interactive Database (FREIDA Online). We identi�ed and contacted197 orthopaedic surgery PDs and 554 internal medicine PDs. A survey invitation email with theappropriate informed consent information was sent to each PD. Submission of the survey indicated therespondents’ consent to participate. All responses were collected anonymously.

Page 4/14

This multi-center, cross-sectional research survey was developed, housed, and distributed throughREDCap Data Capture. The survey was developed using criteria analyzed in the 2018 National ResidentMatching Program (NRMP) Program Director survey, which investigated the factors involved in selectingapplicants to interview.3 Our survey questions used single-answer multiple-choice, multiple-answermultiple-choice, and a Likert scale ranging from 1 (strongly disagree or least important) to 5 (stronglyagree or most important) formats. Prior to distribution, the questions were pretested and tested withsubsets of respondents resulting in a 27-item validated survey. No funding was utilized.

The orthopaedic electronic survey was distributed on April 17, 2020. A follow-up was emailed 2 weeksafter the initial submission. The survey was closed on May 5, 2020. The IM electronic survey wasdistributed on July 8, 2020. A follow-up was emailed 2 weeks after the initial submission. The survey wasclosed on August 5, 2020.

Orthopaedic and IM PD responses were summarized with descriptive statistics and percentages prior toany analysis to check their distributions. Given that the outcome variables were ordinal variables on aLikert scale of agreement or importance ranging from 1 to 5, we used ordinal logistic regression tocompare the survey responses of orthopaedic and IM PDs. Odds ratios were estimated from this analysisas the effect size to assess the direction and magnitude of any signi�cant associations, and these oddsratios were interpreted both in terms of increasing importance or agreement, or in terms of decreasingimportance or agreement depending on the outcome variable. Statistical signi�cance was set at p < 0.05,and all analyses were performed using SAS version 9.4 (SAS Institute, Cary, NC).

ResultsOf the 197 orthopaedic programs contacted, 161 were included in this study due to invalid andoverlapping email accounts. The orthopaedic PD survey had 58 respondents, for a response rate of36.0%. Of the 554 IM programs contacted, 548 programs were included in this study due to invalid andoverlapping email accounts. The IM PD survey had 125 respondents, for a response rate of 22.8%. Thedata, including the number of respondents for each survey question, odds ratios, and p-values are shownin Table 1.

How do internal medicine and orthopaedic program directors feel about the pass/fail Step 1? 

The majority of orthopaedic (82.8%) and IM (74.4%) PDs disagree with the change to pass/fail Step 1.Additionally, 86.2% of orthopaedic and 79.2% of IM PDs disagree that the decision to transition topass/fail Step 1 was transparent. Only 43.1% of orthopaedic and 39.2% of IM PDs agree that a gradedStep 1 measures the ability of applicants to succeed in their respective �elds. Orthopaedic and IM PDsboth equally disagree with the change to pass/fail Step 1 (OR 1.36, CI 0.75-2.46, p=0.315) and agree thata graded Step 1 does not adequately measure the ability of an applicant to succeed in residency (OR1.18, CI 0.67-2.06, p=0.570). 

How will a pass/fail Step 1 impact internal medicine and orthopaedic residency programs?

Page 5/14

Few orthopaedic (10.4%) and IM (20.7%) PDs agree that a pass/fail Step 1 will help to create better futurephysicians. Orthopaedic PDs have lower odds of saying that the change to pass/fail Step 1 will make thematch process less fair and meritocratic (OR 0.53, CI 0.30-0.93, p=0.026). 

How will the pass/fail change impact the importance of the factors reviewed by internal medicine andorthopaedic residency programs when assessing applicants?

The majority of orthopaedic (89.7%) and IM (69.6%) PDs believe that Step 1 will become less importantwhen it transitions to pass/fail, and 96.9% of orthopaedic and 88.0% of IM PDs agree that Step 2 ClinicalKnowledge exam (CK) score will be more important. Additionally, 77.6% of orthopaedic and 67.2% of IMPDs say grades in required clerkships will be more important. The majority of both orthopaedic and IMPDs believe that personal knowledge of the applicant and an audition elective will be more important. 

Compared to IM PDs, orthopaedic PDs have signi�cantly greater odds of saying the followingcomponents will be of greater importance after the change to pass/fail Step 1: research experience (OR6.43, CI 3.18-13.00, p<0.001); letters of recommendation from faculty they know (OR 3.22, CI 1.77-5.87,p<0.001); Alpha Omega Alpha (AOA) membership (OR 4.96, CI 2.65-9.25, p<0.001);leadership/extracurricular activities (OR 3.69, CI 1.96-6.95, p<0.001); personal knowledge of the applicant(OR 2.93, CI 1.61-5.34, p<0.001) and audition electives (OR 6.20, CI 3.21-11.97, p<0.001). 

How will various applicant groups be affected by the change to a pass/fail Step 1?  

The majority of orthopaedic and IM PDs believe that allopathic (MD) students not attending highly-regarded medical schools, osteopathic (DO) students, and international medical graduates (IMGs) will bedisadvantaged by the change of Step 1 to pass/fail. Compared to IM PDs, orthopaedic PDs have greaterodds of believing that MD students who do not attend highly-regarded medical schools (OR 2.14, CI 1.19-3.88, p=0.012) and DO students (OR 2.59, CI 1.45-4.61, p=0.001) will be disadvantaged in the matchprocesses due to the Step 1 pass/fail transition. 

How will changing to a pass/fail Step 1 affect medical students interested in internal medicine andorthopaedics?

Orthopaedic and IM PDs both have low expectations (<30%) that the change to a pass/fail Step 1 willallow students to focus more on learning medicine rather than studying for the Step exam, to seek outmore leadership and extracurricular activities, or to pursue more hobbies and self-development.Orthopaedic PDs have 6.19 times greater odds (CI 2.80-13.69, p<0.001) of believing a pass/fail Step 1will encourage applicants to pursue more research experiences. Less than 10% of IM PDs thought thepass/fail change would encourage more research experience. Compared to IM PDs, orthopaedic PDshave 3.50 times greater odds (CI 1.83-6.71, p<0.001) of thinking that the Step 1 pass/fail change wouldencourage students to attend more audition electives.

What are the future implications on residency applications and medical education?

Page 6/14

With the change to pass/fail Step 1, a majority of orthopaedic and IM PDs agree that medical schoolsshould adopt a graded pre-clinical curriculum, and that there should be a cap on the number of residencyapplications an applicant can submit. 

DiscussionThe present study shows that the majority of orthopaedic and IM PDs do not support the Step 1 pass/failchange. Both orthopaedic and IM PDs believe Step 2 CK will be signi�cantly more important. After thepass/fail change, orthopaedic PDs are signi�cantly more likely than IM PDs to assign greater value toresearch experience, letters of recommendation from faculty they know, AOA membership,leadership/extracurricular activities, personal knowledge of the applicant, and audition electives. Bothorthopaedic and IM PDs believe students who attend highly-regarded medical schools will be advantagedby the move to make Step 1 pass/fail. However, more orthopaedic PDs believe DO students and MDstudents attending less prestigious medical schools will be disadvantaged, while more IM PDs believeIMGs will be disadvantaged. 

Currently, standardized examinations play a major role in evaluating residency applicants acrossspecialties, with Step 1 often being considered the most important test.1-3 In the 2020 NRMP Survey, PDsacross all specialties were asked to rank the factors used to interview and rate applicants on a scale of 1(not at all important) to 5 (very important). When selecting which applicants to interview, orthopaedic andIM PDs provided high ratings, over 4/5, for Step 1. Furthermore, over 90% of orthopaedic and IM PDs citedStep 1 as an important factor for screening applicants for an interview.3 

There is minimal evidence supporting a scored Step 1’s ability to screen for successful residents.4-8 Evenin our survey, the majority of both orthopaedic and IM PDs felt that a graded Step 1 did not adequatelymeasure the ability of an applicant to succeed in residency. Despite the limited predictive value ofstandardized tests for future success, both orthopaedics and IM have used Step 1 as a screeningmeasure for applicants due to the lack of additional objective criteria.3 Historically, both orthopaedics andIM have emphasized standardized test scores. According to the NRMP, U.S. MD seniors applying toorthopaedics had an average Step 1 score of 248 and a Step 2 CK score of 255. U.S. MD seniors applyingto IM had an average Step 1 score of 235 and a Step 2 CK score of 248.9 In our study, the majority oforthopaedic and IM PDs report that Step 2 CK scores will be more important.

The results of our study also suggest that these specialties may place a greater emphasis on factorsoutside of standardized exam scores, as both orthopaedic and IM PDs believe grades in requiredclerkships, personal knowledge of the applicant, and audition electives will now be more important.Additionally, orthopaedic PDs are more likely than IM PDs to say research experience, AOA membership,letters of recommendation from faculty they know, leadership/extracurricular activities, personalknowledge of the applicant, and audition electives will be more important. Based on our analysis,orthopaedic PDs are also 6.2 times more likely (p<0.0001) to believe a pass/fail Step 1 will encourageapplicants to pursue more research experiences and 3.5 times more likely (p=0.0002) to say it will

Page 7/14

encourage students to attend more audition electives. Recently, many orthopaedic programs have beenpublicizing holistic changes in their residency selection criteria on social media. For instance, theUniversity of Missouri - Kansas City Orthopaedic Surgery Residency program announced on Twitter thatthey would no longer be considering Step 1 and Step 2 CK scores. They plan on considering other factorslike personal statement, recommendation letters, dean’s letter, research, audition rotations, grades inrequired clerkships, and interest in their program.10 This is one example of an orthopaedic programmoving toward a more holistic application review. A possible positive consequence of this move towardsa more holistic process is that students from nontraditional, underrepresented, or underservedbackgrounds may have more opportunities to be selected for interviews.11

Interestingly, in the 2020 Match, U.S. MD seniors applying to orthopaedics had an average of 14.3abstracts/presentations/publications, 8 volunteer experiences, and 40.3% were part of AOA; thoseapplying to IM had an average of 6.2 abstracts/presentations/publications, 7.3 volunteer experiences,and 17.4% were part of AOA.9 This difference may be due to the more competitive match process inorthopaedics. In 2020, U.S. MD seniors applying to IM had a match rate 97.1%, while orthopaedics had amatch rate of 79.7%.9

This work found that 83.2% of IM PDs believe IMGs will be disadvantaged by the change to pass/failStep 1, while only 58.6% of orthopaedic PDs believe IMGs will be disadvantaged. This difference may berelated to the lack of IMGs in orthopaedics, and the prevalence of IMGs in IM residencies. In 2019, only1.5% of orthopaedic residents were IMGs, while 38.8% of IM residents were IMGs.12 IMGs frequently relyon Step 1 scores to help them stand out when applying to US residency programs. Many residencyprograms often use Step 1 scores to quickly sift through the large volume of IMG applicants.13-15 IM PDswill have to consider new criteria by which to evaluate IMGs, while orthopaedic PDs will likely not facethis challenge to the same extent, as IMGs make up such a small portion of those entering the �eld oforthopaedic surgery.

The present study demonstrated that orthopaedic PDs are more likely to say DOs will be disadvantagedby the change to a pass/fail Step 1. When applying for residency, DO applicants may face additionalchallenges compared to MD students.15,16 This may be due to a variety of factors, including limitedresearch opportunities for students at DO schools compared to students at MD schools, as most USosteopathic medical schools are typically not associated with large academic institutions.16

A limitation of this study was that we were unable to obtain responses from all orthopaedic and IMresidency PDs. However, we believe our response rates (36.0% for orthopaedics and 22.4% for IM) arereasonable and representative. Another limitation of this study is that some contact information providedby FREIDA Online did not belong to the PD, but instead the program’s coordinator or administrativeassistant. Our email requested program coordinators and administrative assistants to send the survey toPD. 

Page 8/14

ConclusionsIn conclusion, the majority of orthopaedic and IM PDs do not support the Step 1 pass/fail change. Bothgroups will rely more heavily on Step 2 CK. Orthopaedic PDs are signi�cantly more likely to value researchexperience, letters of recommendation from faculty they know, AOA membership,leadership/extracurricular activity, personal knowledge of the applicant, and audition electives.

Abbreviations-USMLE: United States Medical Licensing Examinations 

-FSMB: Federation of State Medical Boards 

-NBME: National Board of Medical Examiners

-CEUP: Committee to Evaluate the USMLE Program 

-InCUS: Invitational Conference on USMLE Scoring 

-AAMC: Association of American Medical Colleges 

-AMA: American Medical Association

-ECFMG: Educational Commission for Foreign Medical Graduates 

-ACGME: Accreditation Council for Graduate Medical Education

-IM: Internal Medicine

-PD: Program Director

-FREIDA Online: Fellowship and Residency Electronic Interactive Database 

-NRMP: National Resident Matching Program 

-CK: Clinical Knowledge exam

-AOA: Alpha Omega Alpha

-IMG: International Medical Graduates

-COMPLEX: Comprehensive Osteopathic Medical Licensing Examination

Declarations

Page 9/14

Ethics approval and consent to participate: The study was exempted by the Penn State College ofMedicine’s IRB.

Ethical declarations: Informed consent was obtained from all participants. There were no participantsunder 18. All methods were carried out in accordance to relevant guidelines and regulations.

Consent for publication: Our survey invitation emails with the appropriate informed consent informationwere sent to each PD. Submission of the survey indicated the respondents’ consent to participate. Allresponses were collected anonymously.

Availability of data and materials: The datasets generated and/or analyzed during the current study arenot publicly available because the survey was conducted independently at our institution, but areavailable from the corresponding author on reasonable request.

Competing interests: The authors declare that they have no competing interests.

Funding: No funding was utilized.

Authors' contributions: All authors made substantial contributions to the conception and design of theproject. FM, ARS, DC, SJ, and EBL acquired, analyzed, and interpreted the data.

FM, AS, WLH, and EFH were major contributors in writing the manuscript. All authors read and approvedthe �nal manuscript.

Acknowledgements: Not applicable

Authors' information: Eileen Hennrikus is a professor of internal medicine at Penn State College ofMedicine. William Hennrikus is professor of orthopaedics, and associate dean of continuing education atPenn State College of Medicine. Paul Juliano is the program director of Penn State’s orthopaedic surgeryresidency program. Alia Chisty is the program director of Penn State’s internal medicine residencyprogram.

References1. USMLE: Change to Pass/Fail Score Reporting for Step 1. https://www.usmle.org/incus (2020).

Accessed 1 Mar 2020.

2. Committee to Evaluate the USMLE Program (CEUP): Comprehensive Review of USMLE: Summary ofthe Final Report and Recommendations. https://www.usmle.org/pdfs/cru/CEUP-Summary-Report-June2008.pdf (2008). Accessed 1 Mar 2020.

3. National Resident Matching Program, Data Release and Research Committee: Results of the 2018NRMP Program Director Survey. https://www.nrmp.org/wp-content/uploads/2018/07/NRMP-2018-Program-Director-Survey-for-WWW.pdf (2018). Accessed 1 Mar 2020.

Page 10/14

4. Kay C, Jackson JL, Frank M. The relationship between internal medicine residency graduateperformance on the ABIM certifying examination, yearly in-service training examinations, and theUSMLE Step 1 examination. Acad Med. 2015;90(1):100-104. doi:10.1097/ACM.0000000000000500

5. Kenny S, McInnes M, Singh V. Associations between residency selection strategies and doctorperformance: a meta-analysis. Med Educ. 2013;47(8):790-800. doi:10.1111/medu.12234

�. Lee AG, Oetting TA, Blomquist PH, et al. A multicenter analysis of the ophthalmic knowledgeassessment program and American Board of Ophthalmology written qualifying examinationperformance. Ophthalmology. 2012;119(10):1949-1953. doi:10.1016/j.ophtha.2012.06.010

7. McCaskill QE, Kirk JJ, Barata DM, Wludyka PS, Zenni EA, Chiu TT. USMLE step 1 scores as asigni�cant predictor of future board passage in pediatrics. Ambul Pediatr. 2007;7(2):192-195.doi:10.1016/j.ambp.2007.01.002

�. Angus SV, Williams CM, Stewart EA, Sweet M, Kisielewski M, Willett LL. Internal Medicine ResidencyProgram Directors' Screening Practices and Perceptions About Recruitment Challenges. Acad Med.2020;95(4):582-589. doi:10.1097/ACM.0000000000003086

9. National Resident Matching Program, Results and Data: 2020 Main Residency Match.https://www.nrmp.org/main-residency-match-data/ (2020). Accessed 10 Oct 2020.

10. Twitter: UMKC Orthopaedic Surgery Residency.https://twitter.com/umkcortho/status/1296124456897785857?lang=en (2020). Accessed 22 Aug2020.

11. Aibana O, Swails JL, Flores RJ, Love L. Bridging the Gap: Holistic Review to Increase Diversity inGraduate Medical Education. Acad Med. 2019;94(8):1137-1141.doi:10.1097/ACM.0000000000002779

12. ACGME Department of Information Services/Applications and Data Analysis: ACGME Data ResourceBook 2019-2020. https://www.acgme.org/About-Us/Publications-and-Resources/Graduate-Medical-Education-Data-Resource-Book (2019). Accessed 1 Mar 2020.

13. Boulet JR, Pinsky WW. Reporting a Pass/Fail Outcome for USMLE Step 1: Consequences andChallenges for International Medical Graduates. Acad Med. 2020;95(9):1322-1324.doi:10.1097/ACM.0000000000003534

14. Desai A, Hegde A, Das D. Change in Reporting of USMLE Step 1 Scores and Potential Implications forInternational Medical Graduates. JAMA. 2020;323(20):2015-2016. doi:10.1001/jama.2020.2956

15. Aziz F, Bechara CF. USMLE Step 1 Scoring System Change to Pass/Fail-Implications for InternationalMedical Graduates [published online ahead of print, 2020 Sep 2]. JAMA Surg.2020;10.1001/jamasurg.2020.2850. doi:10.1001/jamasurg.2020.2850

1�. Salari S, Deng F. A Stepping Stone Toward Necessary Change: How the New USMLE Step 1 ScoringSystem Could Affect the Residency Application Process. Acad Med. 2020;95(9):1312-1314.doi:10.1097/ACM.0000000000003501

17. Aiyer A, Sankar V, Summers S, et al. Unifying the Orthopaedic Surgery Residency Application ProcessUnder a Single Accreditation System: A Primer. J Am Acad Orthop Surg. 2020;28(7):263-267.

Page 11/14

doi:10.5435/JAAOS-D-19-00755

TableTable 1. A Comparison of Orthopaedic and Internal Medicine Program Directors’Perceptions of the USMLE Step 1 Pass/Fail Transition

Page 12/14

OrthopaedicsN (%)

InternalMedicineN (%)

Oddsratio(95%CI)

p-value

How do orthopaedic and internal medicine program directors feel about the pass/failStep 1? Do not agree with the change to pass/failStep 1

48 (82.8) 93(74.4)

1.36(0.75-2.46)

0.315

The decision to transition to pass/fail Step 1was not transparent and did not adequatelyinvolve all stakeholders

50 (86.2) 99(79.2)

1.26(0.71-2.26)

0.430

A graded Step 1 adequately measured theability of an applicant to succeed 

25 (43.1) 49(39.2)

1.18(0.67-2.06)

0.570

How will a pass/fail Step 1 impact orthopaedic and internal medicine residencyprograms?A pass/fail Step 1 will not make the matchprocess fair and meritocratic

25 (43.1)  76(60.8)

*0.53(0.30-0.93)

0.026

A pass/fail Step 1 will help to create betterfuture physicians

12 (20.7)  13(10.4)

1.46(0.83-2.56)

0.190

How will the pass/fail change impact the importance of the factors reviewed byorthopaedic and internal medicine residency programs when assessing applicants?Step 1 exam result will be less important  52 (89.7) 87

(69.6)*3.74(2.00-7.01)

<0.001

Step 2 CK will be more important 56 (96.6) 110(88.0)

1.56(0.83-2.91)

0.167

Step 2 CS will be more important 23 (39.7) 54(43.2)

0.78(0.43-1.41)

0.406

Grades in required clerkships will be moreimportant

45 (77.6) 84(67.2)

1.76(0.98-3.18)

0.059

Research experience will be more important 29 (50.0)  17(13.6) 

*6.43(3.18-13.00)

<0.001

Letters of recommendation fromorthopaedic/internal medicine faculty thatprogram directors know will be moreimportant

44 (75.9)  71(56.8) 

*3.22(1.77-5.87)

<0.001

Letters of recommendation fromorthopaedic/internal medicine faculty thatprogram directors do not know will be moreimportant

30 (51.7)  53(42.4) 

1.65(0.91-2.99)

0.098

Letters of recommendation from faculty notwithin specialty will be more important

9 (15.5)  15(12.0) 

0.71(0.35-1.43)

0.334

Personal statement will be more important 14 (24.1)  26(20.8) 

1.17(0.57-2.42)

0.664

Medical student performance evaluations(MSPE)/Dean’s letter will be more important

25 (43.1)  66(52.8) 

0.62(0.34-1.13)

0.116

Alpha Omega Alpha (AOA) membership will 39 (67.2)  36 *4.96 <0.001

Page 13/14

be more important (28.8)  (2.65-9.25)

Gold Humanism Society membership will bemore important

22 (37.9)  33(26.4) 

1.83(0.96-3.50)

0.068

Leadership/extracurriculars will be moreimportant

34 (58.6)  33(26.4) 

*3.69(1.96-6.95)

<0.001

Personal knowledge of applicant will be moreimportant

47 (81.0)  82(65.6)

*2.93(1.61-5.34)

<0.001

Audition electives within department will bemore important

50 (86.2)  79(63.2)

*6.20(3.21-11.97)

<0.001

How will various applicant groups be affected by the change to a pass/fail Step 1? All MD students will be advantaged 10 (17.2)  29

(23.2) 0.61(0.35-1.09)

0.094

MD students who attend a highly-regardedmedical schools will be advantaged

35 (60.3)  69(55.2) 

1.17(0.66-2.05)

0.593

MD students who do not attend a highly-regarded medical school will bedisadvantaged

44 (75.9) 79(63.2)

*2.14(1.19-3.88)

0.012

DO students will be disadvantaged 37 (63.8)  65(52.0)

*2.59(1.45-4.61)

0.001

International medical graduates (IMGs) willbe disadvantaged

34 (58.6)  104(83.2)

0.631(0.35-1.13)

0.118

How will changing to a pass/fail Step 1 affect medical students interested in orthopaedicsand internal medicine?Allow students to focus more on learningmedicine rather than studying for Step 1

13 (22.4) 32(25.6)

0.84(0.41-1.75)

0.642

Encourage more research experiences  23 (39.7) 12 (9.6) *6.19(2.80-13.69)

<0.001

Encourage more leadership/extracurriculars 16 (27.6) 23(18.4)

1.69(0.81-3.51)

0.160

Allow students to pursue more hobbies/self-development

6 (10.3) 27(21.6)

0.42(0.16-1.08)

0.072

Encourage students to attend more auditionelectives

34 (58.6) 36(28.8)

*3.50(1.83-6.71)

<0.001

Encourage applicants to apply to moreresidency programs

39 (67.2) 71(56.8)

1.56(0.81-3.00)

0.181

Encourage applicants to apply to otherspecialties in addition to their primaryspecialty of interest

27 (46.6) 49(39.2)

1.32(0.72-2.53)

0.348

What are the future implications on residency applications and medical education?With the change to pass/fail Step 1, shouldmedical schools adopt a graded pre-clinicalcurriculum?

37 (63.8) 

65(52.0)

1.10(0.51-2.35)

0.810

With the change to pass/fail Step 1, should 42 (72.4)  69 1.70 0.201

Page 14/14

there be a cap on the number of residencyapplications a medical student can submit?

   

(55.2)    

(0.75-3.86)

 Table 1 shows the distribution of responses for our survey sent to all orthopaedics andinternal medicine residency program directors in the United States. The bolded anditalicized text in the table are the main questions asked in our survey. The odds ratioscompare orthopaedics and internal medicine program director responses. The p-valuereports the significance of the odds ratios. An asterisk (*) indicates statistically significance(p<0.05).