Abstract
Objectives: We designed our study with the hypothesis that open ended Short Answer type Questions (SAQs), no matter how carefully framed, cannot be as objective as Multiple Choice type Questions (MCQs).
Methods: The study was conducted on 1st year MBBS students (n=99) studying at AIIMS, Jodhpur. A written test on 'Blood & Immunity' was conducted containing same questions in two formats; twelve MCQs (type E) in section A and 12 SAQs in section B. Maximum marks for all questions in both sections were equal. All the answers of section B were evaluated separately by two different examiners to reduce the subjectivity and a model answer sheet for both the sections was prepared and provided to both the examiners.
Results: The difference in the scores in Section B SAQs that were evaluated by two different examiners was not statistically significant. Mean of the marks awarded by the two examiners was taken as the final score of each student in section B. The difference in the scores by the students in the two sections was also non-significant (p=0.14). A significant correlation (r=0.99, p<0.0001) was found in SAQ and MCQ scores. Bland- Altman analysis also showed no proportion of bias and the two methods of scoring were in agreement with each other.
Conclusion: The results suggest that meticulously-framed open-ended short answer type questions can be as objective as multiple choice type questions.
Keywords
Multiple choice questions - medical education - assessment - open-ended questions
