Download PDF
CC BY-NC-ND 4.0 · Ann Natl Acad Med Sci 2016; 52(03): 173-182
DOI: 10.1055/s-0040-1712619
Original Article

Short Answer Open-Ended versus Multiple-Choice Questions: A Comparison of Objectivity

Bharati Mehta
1   Department of Physiology, All India Institute of Medical Sciences, Jodhpur
,
Bharti Bhandari
1   Department of Physiology, All India Institute of Medical Sciences, Jodhpur
,
Parul Sharma
2   RAK College of Medical Sciences,UAE
,
Rimplejeet Kaur
3   Dr. S. N. Medical College, Jodhpur.
› Author Affiliations
› Further Information
Also available at
  PDF Download Permissions and Reprints

Abstract

Objectives: We designed our study with the hypothesis that open ended Short Answer type Questions (SAQs), no matter how carefully framed, cannot be as objective as Multiple Choice type Questions (MCQs).

Methods: The study was conducted on 1st year MBBS students (n=99) studying at AIIMS, Jodhpur. A written test on 'Blood & Immunity' was conducted containing same questions in two formats; twelve MCQs (type E) in section A and 12 SAQs in section B. Maximum marks for all questions in both sections were equal. All the answers of section B were evaluated separately by two different examiners to reduce the subjectivity and a model answer sheet for both the sections was prepared and provided to both the examiners.

Results: The difference in the scores in Section B SAQs that were evaluated by two different examiners was not statistically significant. Mean of the marks awarded by the two examiners was taken as the final score of each student in section B. The difference in the scores by the students in the two sections was also non-significant (p=0.14). A significant correlation (r=0.99, p<0.0001) was found in SAQ and MCQ scores. Bland- Altman analysis also showed no proportion of bias and the two methods of scoring were in agreement with each other.

Conclusion: The results suggest that meticulously-framed open-ended short answer type questions can be as objective as multiple choice type questions.

Keywords

Multiple choice questions - medical education - assessment - open-ended questions


Publication History

Article published online:
09 May 2020

© .

Thieme Medical and Scientific Publishers Private Ltd.
A-12, Second Floor, Sector -2, NOIDA -201301, India

 

  • References

  • 1 Schuwirth LWT, van der Vleuten CPM (2003). ABC of learning and teaching in medicine: Written assessment. BMJ 326(7390): 643–645.
  • 2 Bhagat O, Bhandari B, Mehta B, Sir car S (2014) . Objective Structured Practical Examination and Conventional Practical Examination: a Comparison of S c o r e s . Med Sci Educ 24(4):395–399.
  • 3 Schuwirth LWT, van der Vleuten CPM (2004). Different written assessment methods: what can be said about their strengths and weaknesses ? Med Educ 38(9):974–979.
  • 4 Scoring rubric development: validity and reliability. Moskal, Barbara M & Jon A. Leydens [Internet]. [cited 2016 May 9]. Available from : http://pareonline.net/getvn.asp?v=7 &n=10
  • 5 Pinckard RN, McMahan CA, Prihoda TJ, Littlefield JH, Jones AC (2009). Short-answer examinations improve student performance in an oral and maxillofacial pathology course. J Dent Educ 73(8):950–961.
  • 6 Rabinowitz HK, Hojat M (1989). A comparison of the modified essay question and multiple choice question formats: their relationship to clinical performance. Fam Med 21(5):364–367.
  • 7 Anatol T, Hariharan S (2009). Reliability of the evaluation of students' answers to essay-type questions. West Indian Med J 58(1):13–16.
  • 8 Wakeford RE, Roberts S (1979). A pilot experiment on the interexaminer reliability of short essay questions . Med Educ 13(5):342–344.
  • 9 Bland J , Altman D ( 1 9 8 6 ) . Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1(8476):307–310..
  • 10 Pepple DJ, Young LE, Carroll RG (2010). A comparison of student performance in multiple-choice and long essay questions in the MBBS stage I physiology examination at the University of the West Indies (Mona Campus). Adv Physiol Educ 34(2):86–89.
  • 11 Mujeeb AM , Pardeshi ML , Ghongane BB (2010). Comparative assessment of multiple choice questions versus short essay questions in pharmacology examinations. Indian J Med Sci 64(3):118–124.
  • 12 Rabinowitz HK (1986). The modified essay question: effect of author location on student performance . Med Educ 20(4):318–320.
  • 13 McCloskey DI, Holland RA (1976). A comparison of student performances in answering essaytype and multiple-choice questions. Med Educ 10(5):382–385.
  • 14 Palmer EJ, Devitt PG (2007). Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions ? Research paper. BMC Med Educ 7:49.
  • 15 Moeen-Uz-Zafar, Badr-Aljarallah (2011). Evaluation of mini-essay questions (MEQ) and multiple choice questions (MCQ) as a tool for assessing the cognitive skills of undergraduate students at the Department of Medicine. Int J Health Sci 5(2 Suppl 1):43–44.
  • 16 Hift RJ (2014). Should essays and other “open-ended”-type questions retain a place in written summative assessment in clinical medicine? BMC Med Educ 14:249.
  • 17 Forsdyke DR (1978). A comparison of short and multiple choice questions in the evaluation of students of biochemistry. Med Educ 12(5):351–356.
  • 18 Hettiaratchi ES (1978) . A comparison of student performance in two parallel physiology tests in multiple choice and short answer forms. Med Educ 12(4):290–296.
  • 19 Adewoye EO , Oyebola OD , Bamgboye EA (2000) . A comparative study of students' performance in pre clinical physiology assessed by short and long essays. Afr J Med Med Sci 29(2):155–159.
  • 20 Oyebola DD , Adewoye OE , Iyaniwura JO , Alada AR , Fasanmade AA, Raji Y (2000). A comparative study of students' performance in pre clinical physiology assessed by multiple choice and short essay questions. Afr J Med Med Sci 29( 3 - 4):201–205.
  • 21 Anyika EN, Anyika DI (2008). Essay, multiple-choice (MCQ) and combined (essay with MCQ) type examinations: the pharmacy students' perspective. Niger Q J Hosp Med 18(1):12–15.