Examination Preparation Methods
- What Works?
Are there correlates for test preparation methods and high stakes examination scores? A study by Thadani, et al at the NBME using surveys of first-time step 1 test takers indicated that examinees showed the greatest benefit from:
However, the most common preparation approach was the use of commercially prepared study guides by 98% of the survey respondents, a greater number than had used traditional textbooks. There was little or no evidence of achievement of higher scores as a consequence of using commercially prepared material, after controlling for pre-matriculation characteristics and other study methods used. The self-selected students taking preparation courses disproportionately included students with lower entering MCAT scores and ultimately lower step 1 scores, indicating a selection bias for such a study method. It is possible that time-intensive preparation courses may compete with other preparation methods that are more effective or time-efficient.
A PBL curriculum can result in better USMLE step scores and better prepare graduates with knowledge and skills needed to practice within a complex health care system. Outcomes support the investment of financial and human resources in a PBL curriculum.
The self-assessment products produced by the NBME, including Comprehensive Basic Science Self-Assessment (CBSSA) and NBME Comprehensive Clinical Science Self-Assessment (CCSSA) can be used to predict performance on USMLE Step 1 and Step 2 examinations respectively. Use of the NBME's Comprehensive Basic Science Examination (CBSE) predicts future performance on step 1, particularly a second CBSE.
- References:
Thadani RA, Swanson DB, Galbraith RM. A preliminary analysis of different approaches to preparing for the USMLE step 1. Acad Med. 2000 Oct;75(10 Suppl):S40-42.
Hoffman K, Hosokawa M, Blake R Jr, Headrick L, Johnson G. Problem-based learning outcomes: ten years of experience at the University of Missouri-Columbia School of Medicine. Acad Med. 2006 Jul;81(7):617-25.
Sawhill A, Butler A, Ripkey D, Swanson DB, Subhiyah R, Thelman J, Walsh W, Holtzman KZ, Angelucci K. Using the NBME self-assessments to project performance on USMLE Step 1 and Step2: impact of test administration conditions. Acad Med. 2004 Oct;79(10 Suppl):S55-7.
Glew RH, Ripkey DR, Swanson DB. Relationship between students' performances on the NBME Comprehensive Basic Science Examination and the USMLE Step 1: a longitudinal investigation at one school. Acad Med. 1997 Dec;72(12):1097-102.
- What Doesn't Work?
Is there a role for commercial offerings such as review books and courses? For 30 years the medical literature has documented that there is no demonstrable effect for "test prep" materials, and that anxiety on the part of students coupled with aggressive marketing tactics by commercial providers of the products are the most likely cause for widespread usage of such products.
Students taking commercial step 1 preparatory courses tend to have lower step 1 scores and lower medical school grades, compared to students who do not take such courses, and performance on step 1 shows correlation with the medical school grades, not the preparation course.
Another study of students taking a commercial review course with coaching for step 1 showed that they scored no better than students who studied on their own. When step 1 scores of students who took the review course were compared to the scores they received on the NBME Comprehensive Basic Science Examination (CBSE), neither the students who performed above average nor those who performed below average on the CBSE improved their performance on Step 1 as a result of the coaching courses.
There is a placebo effect that can be effective in some persons. However, this effect has not been well-studied in educational settings.
- References:
McGaghie WC, Downing SM, Kubilius R. What is the impact of commercial test preparation courses on medical examination performance? Teach Learn Med. 2004 Spring;16(2):202-11.
Zhang C, Rauchwarger A, Toth C, O'Connell M. Student USMLE step 1 preparation and performance. Adv Health Sci Educ Theory Pract. 2004;9(4):291-7.
Werner LS, Bull BS. The effect of three commercial coaching courses on Step One USMLE performance. Med Educ. 2003 Jun;37(6):527-31.
Finniss DG, Kaptchuk TJ, Miller F, Benedetti F. Biological, clinical, and ethical advances of placebo effects. Lancet. 2010;375:686-95.
- Timing is Everything...Isn't It?
Does spending more time in exam preparation, beyond the regular curriculum, make a difference? What if I give myself another month to study? Reality check: mean test scores will not change. A large general knowledge exam measures knowledge gained over a long period of time in an organized manner, and there is little that will alter outcomes in the short run. In fact, students who are out of the environment of a rigorous and well-organized, ongoing curriculum may go backwards academically, losing ground in knowledge retention and application. Hence, students can take high-stakes examinations early in a window of opportunity for completion without any diminution in score. If anxiety is an issue, then early is better than later.
- References:
Pohl CA, Robeson MR, Hojat M, Veloski JJ. Sooner or later? USMLE step 1 performance and test administration date at the end of the second year. Acad Med. 2002;77(10 Suppl):S17-9.
Pohl CA, Robeson MR, Veloski J. USMLE Step 2 performance and test administration date in the fourth year of medical school. Acad Med. 2004;79(10 Suppl):S49-51.
|
|
Learning menu.
Next topic