TY - JOUR

T1 - Development of knowledge tests for multi-disciplinary emergency training

T2 - a review and an example

AU - Sorensen, J. L.

AU - Thellesen, L.

AU - Strandbygaard, J.

AU - Svendsen, Kira Dynnes

AU - Christensen, Karl Bang

AU - Johansen, M

AU - Langhoff-Roos, P.

AU - Ekelund, K.

AU - Ottesen, B.

AU - Van der Vleuten, C.

PY - 2015/1

Y1 - 2015/1

N2 - Background: The literature is sparse on written test developmentin a post-graduate multi-disciplinary setting. Developing and evalu-ating knowledge tests for use in multi-disciplinary post-graduatetraining is challenging. The objective of this study was to describethe process of developing and evaluating a multiple-choice question(MCQ) test for use in a multi-disciplinary training program inobstetric-anesthesia emergencies.Methods: A multi-disciplinary working committee with 12members representing six professional healthcare groups andanother 28 participants were involved. Recurrent revisions of theMCQ items were undertaken followed by a statistical analysis. TheMCQ items were developed stepwise, including decisions on aimsand content, followed by testing for face and content validity, con-struct validity, item–total correlation, and reliability.Results: To obtain acceptable content validity, 40 out of originally50 items were included in the final MCQ test. The MCQ test wasable to distinguish between levels of competence, and good con-struct validity was indicated by a significant difference in the meanscore between consultants and first-year trainees, as well as betweenfirst-year trainees and medical and midwifery students. Evaluationof the item–total correlation analysis in the 40 items set revealed that11 items needed re-evaluation, four of which addressed contentissues in local clinical guidelines. A Cronbach’s alpha of 0.83 forreliability was found, which is acceptable.Conclusion: Content and construct validity and reliability wereacceptable. The presented template for the development of thisMCQ test could be useful to others when developing knowledgetests and may enhance the overall quality of test development.

AB - Background: The literature is sparse on written test developmentin a post-graduate multi-disciplinary setting. Developing and evalu-ating knowledge tests for use in multi-disciplinary post-graduatetraining is challenging. The objective of this study was to describethe process of developing and evaluating a multiple-choice question(MCQ) test for use in a multi-disciplinary training program inobstetric-anesthesia emergencies.Methods: A multi-disciplinary working committee with 12members representing six professional healthcare groups andanother 28 participants were involved. Recurrent revisions of theMCQ items were undertaken followed by a statistical analysis. TheMCQ items were developed stepwise, including decisions on aimsand content, followed by testing for face and content validity, con-struct validity, item–total correlation, and reliability.Results: To obtain acceptable content validity, 40 out of originally50 items were included in the final MCQ test. The MCQ test wasable to distinguish between levels of competence, and good con-struct validity was indicated by a significant difference in the meanscore between consultants and first-year trainees, as well as betweenfirst-year trainees and medical and midwifery students. Evaluationof the item–total correlation analysis in the 40 items set revealed that11 items needed re-evaluation, four of which addressed contentissues in local clinical guidelines. A Cronbach’s alpha of 0.83 forreliability was found, which is acceptable.Conclusion: Content and construct validity and reliability wereacceptable. The presented template for the development of thisMCQ test could be useful to others when developing knowledgetests and may enhance the overall quality of test development.

U2 - 10.1111/aas.12428

DO - 10.1111/aas.12428

M3 - Journal article

C2 - 25363488

VL - 59

SP - 123

EP - 133

JO - Acta Anaesthesiologica Scandinavica

JF - Acta Anaesthesiologica Scandinavica

SN - 0001-5172

IS - 1

ER -