TY - JOUR

T1 - Ensuring Competency of Novice Laparoscopic Surgeons

T2 - Exploring Standard Setting Methods and their Consequences

AU - Thinggaard, Ebbe

AU - Bjerrum, Flemming

AU - Strandbygaard, Jeanett

AU - Gögenur, Ismail

AU - Konge, Lars

N1 - Copyright © 2016 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

PY - 2016

Y1 - 2016

N2 - OBJECTIVE: Simulation-based assessment tools have been developed to allow for proficiency-based simulator training in laparoscopy. However, few studies have examined the consequences of different standard setting methods or examined what level of proficiency is considered adequate for trainees. The objectives of the present study were to explore the consequences of different standard setting methods and to examine the proficiency level that surgical trainees are expected to reach, before performing supervised surgery on patients.DESIGN: Study participants undertook the Training and Assessment of Basic Laparoscopic Techniques test. The tests were video-recorded and rated using a simple scoring system based on number of errors and time. Participants were then asked to assess how high a score a novice should reach before performing supervised surgery on a patient. We then compared 3 methods of standard setting: expert performance level, contrasting groups method, and a modified Angoff method.SETTING: The study was conducted at the Copenhagen Academy for Medical Education and Simulation. The academy provides surgical simulation training in laparoscopy for trainees at the hospitals in the Capital Region and the Zealand Region of Denmark.PARTICIPANTS: Participants were recruited among surgical trainees in their first year of specialty training from surgery, gynecology, and urology departments. A total of 40 participants were included and completed the trial.RESULTS: The different standard setting methods resulted in different pass/fail levels. At the expert performance level, the pass/fail level was 474 points-the contrasting groups method resulted in 358 points and the modified Angoff method resulted in 311 points among experienced surgeons, and 386 points among trainees. The different proficiency levels resulted in a failure rate of 0% to 50% of experienced surgeons and a pass rate of 0% to 25% of novices. Novice laparoscopic surgeons set a higher pass/fail level than experienced surgeons did (p = 0.008).CONCLUSION: Required proficiency levels varies depending on the standard setting method used, which highlights the importance of using an established standard setting method to set the pass/fail level.

AB - OBJECTIVE: Simulation-based assessment tools have been developed to allow for proficiency-based simulator training in laparoscopy. However, few studies have examined the consequences of different standard setting methods or examined what level of proficiency is considered adequate for trainees. The objectives of the present study were to explore the consequences of different standard setting methods and to examine the proficiency level that surgical trainees are expected to reach, before performing supervised surgery on patients.DESIGN: Study participants undertook the Training and Assessment of Basic Laparoscopic Techniques test. The tests were video-recorded and rated using a simple scoring system based on number of errors and time. Participants were then asked to assess how high a score a novice should reach before performing supervised surgery on a patient. We then compared 3 methods of standard setting: expert performance level, contrasting groups method, and a modified Angoff method.SETTING: The study was conducted at the Copenhagen Academy for Medical Education and Simulation. The academy provides surgical simulation training in laparoscopy for trainees at the hospitals in the Capital Region and the Zealand Region of Denmark.PARTICIPANTS: Participants were recruited among surgical trainees in their first year of specialty training from surgery, gynecology, and urology departments. A total of 40 participants were included and completed the trial.RESULTS: The different standard setting methods resulted in different pass/fail levels. At the expert performance level, the pass/fail level was 474 points-the contrasting groups method resulted in 358 points and the modified Angoff method resulted in 311 points among experienced surgeons, and 386 points among trainees. The different proficiency levels resulted in a failure rate of 0% to 50% of experienced surgeons and a pass rate of 0% to 25% of novices. Novice laparoscopic surgeons set a higher pass/fail level than experienced surgeons did (p = 0.008).CONCLUSION: Required proficiency levels varies depending on the standard setting method used, which highlights the importance of using an established standard setting method to set the pass/fail level.

U2 - 10.1016/j.jsurg.2016.05.008

DO - 10.1016/j.jsurg.2016.05.008

M3 - Journal article

C2 - 27324697

VL - 73

SP - 986

EP - 991

JO - Journal of Surgical Education

JF - Journal of Surgical Education

SN - 1931-7204

IS - 6

ER -