Background: The literature is sparse on written test developmentin a post-graduate multi-disciplinary setting. Developing and evalu-ating knowledge tests for use in multi-disciplinary post-graduatetraining is challenging. The objective of this study was to describethe process of developing and evaluating a multiple-choice question(MCQ) test for use in a multi-disciplinary training program inobstetric-anesthesia emergencies.
Methods: A multi-disciplinary working committee with 12members representing six professional healthcare groups andanother 28 participants were involved. Recurrent revisions of theMCQ items were undertaken followed by a statistical analysis. TheMCQ items were developed stepwise, including decisions on aimsand content, followed by testing for face and content validity, con-struct validity, item–total correlation, and reliability.
Results: To obtain acceptable content validity, 40 out of originally50 items were included in the final MCQ test. The MCQ test wasable to distinguish between levels of competence, and good con-struct validity was indicated by a significant difference in the meanscore between consultants and first-year trainees, as well as betweenfirst-year trainees and medical and midwifery students. Evaluationof the item–total correlation analysis in the 40 items set revealed that11 items needed re-evaluation, four of which addressed contentissues in local clinical guidelines. A Cronbach’s alpha of 0.83 forreliability was found, which is acceptable.
Conclusion: Content and construct validity and reliability wereacceptable. The presented template for the development of thisMCQ test could be useful to others when developing knowledgetests and may enhance the overall quality of test development.