The Mount Sinai Journal of Medicine - Mount Sinai School of Medicine

Volume 72 Number 1 January 2005	back to contents

Correlation between Housestaff Performance on the United States Medical Licensing Examination and Standardized Patient Encounters	47-49
William D. Rifkin, M.D.1, and Arthur Rifkin, M.D.2

From the 1Department of Medicine, Yale University School of Medicine, New Haven, CT, and 2Department of Psychiatry, Albert Einstein School of Medicine, Bronx, NY and the Zucker Hillside Hospital, Glen Oaks, NY.

The work was done at the Department of Medicine, Maimonides Medical Center, Brooklyn, NY.

Address all correspondence to William Rifkin, M.D., Associate Director, Yale Primary Care Residency Program, Waterbury Hospital, 64 Robbins Street, Waterbury, CT 06721; e-mail: wrifkin@wtbyhosp.chime.org

Grant support: none. Previously presented as a poster at the Society of General Internal Medicine national meeting, April 30 – May 3, 2003, Vancouver, British Columbia.

Accepted for publication July 2004.

Abstract

Background: There is interest in the use of “standardized patients” to assist in evaluating medical trainees’ clinical skills, which may be difficult to evaluate with written exams alone. Previous studies of the validity of observed structured clinical exams have found low correlation with various written exams as well as with faculty evaluations. Since the United States Medical Licensing Examination (USMLE) results are often used by training programs in the selection of applicants, we assessed the correlation between performance on an observed structured clinical exam and the USMLE, steps 1 and 2, for internal medicine housestaff.

Methods: We collected scores on the USMLE, steps 1 and 2, and the overall score from a required standardized patient encounter for all PGY-1 trainees, in a single urban teaching hospital. Pearson coefficients were used to compare the USMLE and observed structured clinical exam performance.

Results: The two steps of the USMLE correlated with each other to a large extent (r=0.65, df=30, p=0.0001). However, both steps of the USMLE correlated poorly with the observed structured clinical exam (step 1 r=0.2, df=32, p=0.27; step 2 r=0.09, df=30, p=0.61).

Conclusions: The low correlation between the USMLE and performance on a structured clinical exam suggests that either the written exam is a poor predictor of actual clinical performance, the small window of clinical skills measured by the structured clinical exam is inadequate, or the two methods evaluate different skill sets entirely. Our findings are consistent with previous work finding low correlations between structured clinical exams and accepted common means of evaluation, such as faculty evaluations, other written exams and program director assessments. The medical education community needs to develop an objective, valid method of measuring important, yet subjective, skill-sets such as interpersonal communication, empathy and efficient data collection.

KEYWORDS

Standardized patients, observed structured clinical examination, housestaff, United States Medical Licensing Examination (USMLE), performance.

Back Issues | Indexes | Search |