Panel interview for internal medicine residency selection: pros and cons

Introduction

Selection of residents is an extremely important process for a residency program as it determines the quality of patient care as well as the academic and research progress of the program. Most studies have focused on the relationship between the national board exam scores, such as the United States Medical Licensing Examination (USMLE) and the subsequent clinical performance of the selected residents. However, a low to moderate correlation has been found between board scores and residency performance.¹  

Additionally other studies have assessed the residency program directors’ opinions about the relative importance of various selection strategies. They found the top three common selection elements were clerkship grades, reference letters, and the USMLE Step 1 scores.²

There is no consensus as to which selection strategy is the most useful in choosing the candidates. Thus, the existing selection process is not ideal and has many shortcomings. For instance, it may not give an accurate representation of the interviewee as a medical resident.³ Likewise, there may be a major reliance on one parameter, such as the USMLE scores. This may come at the expense of other parameters, such as the candidate’s communication skills, character, and professionalism.⁴ Also, there is inter- and intra-observer variability in rating the residency candidates. These shortcomings make the selection process subjective and unreliable, which should be avoided when making assessments.⁵

Adopting a new interview style

During the 2015-2016 interview season, the University of Toledo Internal Medicine Residency Program introduced a panel-style interview. Previously, the residency program relied on one-on-one interviews with various faculty members to evaluate candidates. Changing the interview style allowed an opportunity for multiple interviewers (the panel) to evaluate a residency candidate. Our panel members consisted of the program director/associate program director, a faculty member, and the chief resident. Also, under the one-on-one interview format, it allowed the interviewer greater access to beforehand knowledge of the candidate, which was changed under the panel interview style. Instead, the interviewers were blinded to the objective criteria, such as USMLE scores. After completion of the interview, the panel members individually scored the residency candidate using various subjective elements, such as communication skills, teaching potential, and professionalism. Subsequently, these scores were used by the residency program to calculate the final rankings of all the candidates during the interview season.

Lessons learned

The panel-style interview approach was designed to emphasize a balanced assessment tool that takes into account both the objective and subjective aspects of the residency candidate. We found that the quality of medical school was the strongest objective factor in the final ranking of the candidates. We question whether this was due to our arbitrary scoring system that might give too much advantage to US programs over international schools. Likewise, evaluators might show some bias toward residency applicants from our medical school. Another strong factor was the subjective component of the scoring system, which came ahead of the USMLE scores. We considered the fact that since panel members were blinded to objective factors, they were less likely to anchor too heavily on USMLE scores and dismiss the other qualities revealed during the interview when it was time to score the candidate. Another pertinent finding was that the program director/associate program director, faculty members, and the chief resident scored the candidates similarly when assessing the subjective elements. This suggests the panel members were consistent in how they viewed the candidate’s professionalism, teaching potential, and communication skills.

Conclusions

The residency interview process varies across different academic programs around the US and in other countries. Some residency programs may give more weight to the subjectively assessed potentials of the candidates.

Conversely, some residency programs may rely more on the objective parameters, such as USMLE scores, quality of medical school, academic performance, and scholarly activity. In either case, interviewing candidates is a complex process for residency programs and should include a multitude of assessment tools. We believe our panel style interview approach to evaluate internal medicine residency candidates is the first step in the development of both a standardized and balanced assessment tool at our institution.

Conflicts of Interest

The authors declare that they have no conflict of interest.

References

1. Smilen SW, Funai EF and Bianco AT. Residency selection: should interviewers be given applicants' board scores? Am J Obstet Gynecol. 2001; 184: 508-513.
   Full Text PubMed
2. Green M, Jones P and Thomas JX. Selection criteria for residency: results of a national program directors survey. Acad Med. 2009; 84: 362-367.
   Full Text PubMed
3. Doherty EM and Nugent E. Personality factors and medical training: a review of the literature. Med Educ. 2011; 45: 132-140.
   Full Text PubMed
4. Brustman LE, Williams FL, Carroll K, Lurie H, Ganz E and Langer O. The effect of blinded versus nonblinded interviews in the resident selection process. J Grad Med Educ. 2010; 2: 349-353.
   Full Text PubMed
5. Leinster S. Assessment in medical training. Lancet. 2003; 362: 1770.
   Full Text PubMed