Development and Validation of a Multiple-Choice Question Paper in Basic Colonoscopy

Background and Study Aims: Factual knowledge underpins competence in all clinical skills. A multiple-choice question paper (MCQ) was designed as part of an accelerated colonoscopy training week (ACTW). In the United Kingdom, there are no validated methods of assessing core knowledge in colonoscopy. The aims of this study were to develop an MCQ paper, to demonstrate its construct validity, and to establish whether the ACTW improved core knowledge.
Materials and Methods: Two 30-question MCQ papers (A and B) were designed. Delegates attending the British Society of Gastroenterology (BSG) 2004 meeting responded either to MCQ paper A or B. Demographic data were collected on their experience in colonoscopy. Doctors attending the ACTW completed either MCQ paper A or B before the course, and the other paper after the course.
Results: Seventy-eight delegates at the BSG meeting completed the MCQ. There was a significant difference in the scores between those who had carried out less than 200 colonoscopies (mean 32.5 %) and those who had performed more than 200 (mean 58.4 %; P < 0.0001). Seventeen doctors attending the ACTW completed the MCQ. The mean score for papers A and B increased significantly following the course - 57.2 % before the ACTW, 67 % after it (P = 0.003).
Conclusion: This MCQ is capable of differentiating between endoscopists with different levels of experience in colonoscopy. Doctors attending an ACTW significantly improve their knowledge in colonoscopy. A validated MCQ such as this could be used as part of the assessment process to ascertain competence in colonoscopy.

References

• 1 Bloom B S, Committee of College and University Examiners. Taxonomy of educational objectives: the classification of educational goals. London; Longman 1956
• 2 Bell G D, Neale G, Wilkinson M. et al .Guidelines for informed consent for endoscopic procedures. Report for the BSG. www.bsg.org.uk [1999]. 
• 3 Cairns S, Scholefield J H. Guidelines for colorectal cancer screening in high risk groups.  Gut. 2002;  51 (Suppl 5) V1-V2
• 4 Cotton P, Williams C B. Practical gastrointestinal endoscopy, 5th ed. Oxford; Blackwell Scientific 2003
• 5 Eisen G M, Baron T H, Dominitz J A. et al . Guideline on the management of anticoagulation and antiplatelet therapy for endoscopic procedures.  Gastrointest Endosc. 2002;  55 775-779
• 6 Teague R. Safety and sedation during endoscopic procedures. Report for the BSG. www.bsg.org.uk [2003]. 
• 7 Case S M, Swanson D B. Constructing written test questions for the basic and clinical sciences. Report for the NBME. www.nbme.org/about/itemwriting.asp [2003]. 
• 8 Fleming P R. The administration of a multiple choice question bank.  Med Educ. 1984;  18 372-376
• 9 Harden R M. Constructing multiple choice questions of the multiple true/false type.  Med Educ. 1979;  13 305-312
• 10 Holsgrove G, Elzubeir M. Imprecise terms in UK medical multiple-choice questions: what examiners think they mean.  Med Educ. 1998;  32 343-350
• 11 Lowe D. Set a multiple choice question (MCQ) examination.  BMJ. 1991;  302 780-782
• 12 Gledhill R F. Formative (in-course) graduate assessment: design of an accurate multiple-choice question examination and the training merits thereof.  S Afr Med J. 1983;  64 476-477
• 13 Joint Advisory Group on Gastrointestinal Endoscopy. Guidelines for the training, appraisal and assessment of trainees in gastrointestinal endoscopy. Report for the BSG. www.bsg.org.uk [2004]. 
• 14 Lennox B. Marking multiple-choice examinations.  Br J Med Educ. 1967;  1 203-211
• 15 Schuwirth L W, van der Vleuten C P. ABC of learning and teaching in medicine: written assessment.  BMJ. 2003;  326 643-645
• 16 Clayden G, Chaput de Saintonge M, Fleck A. et al .Assessment of medical students: report of the working group to the Medical Executive Committee, University of London. London; University of London 1995
• 17 Harden R McG, Brown R A, Biran L A. et al . Multiple choice questions: to guess or not to guess.  Med Educ. 1976;  10 27-32