Skip to main content

Research Repository

Advanced Search

Evaluating professionalism in medical undergraduates using selected response questions: findings from an item response modelling study

Tiffin, P.; Finn, G.; McLachlan, J.C.

Evaluating professionalism in medical undergraduates using selected response questions: findings from an item response modelling study Thumbnail


Authors

P. Tiffin

G. Finn

J.C. McLachlan



Abstract

Background: Professionalism is a difficult construct to define in medical students but aspects of this concept may be important in predicting the risk of postgraduate misconduct. For this reason attempts are being made to evaluate medical students' professionalism. This study investigated the psychometric properties of Selected Response Questions (SRQs) relating to the theme of professional conduct and ethics comparing them with two sets of control items: those testing pure knowledge of anatomy, and; items evaluating the ability to integrate and apply knowledge ("skills"). The performance of students on the SRQs was also compared with two external measures estimating aspects of professionalism in students; peer ratings of professionalism and their Conscientiousness Index, an objective measure of behaviours at medical school. Methods: Item Response Theory (IRT) was used to analyse both question and student performance for SRQs relating to knowledge of professionalism, pure anatomy and skills. The relative difficulties, discrimination and 'guessabilities' of each theme of question were compared with each other using Analysis of Variance (ANOVA). Student performance on each topic was compared with the measures of conscientiousness and professionalism using parametric and non-parametric tests as appropriate. A post-hoc analysis of power for the IRT modelling was conducted using a Monte Carlo simulation. Results: Professionalism items were less difficult compared to the anatomy and skills SRQs, poorer at discriminating between candidates and more erratically answered when compared to anatomy questions. Moreover professionalism item performance was uncorrelated with the standardised Conscientiousness Index scores (rho = 0.009, p = 0.90). In contrast there were modest but significant correlations between standardised Conscientiousness Index scores and performance at anatomy items (rho = 0.20, p = 0.006) though not skills (rho = .11, p = .1). Likewise, students with high peer ratings for professionalism had superior performance on anatomy SRQs but not professionalism themed questions. A trend of borderline significance (p = .07) was observed for performance on skills SRQs and professionalism nomination status. Conclusions: SRQs related to professionalism are likely to have relatively poor psychometric properties and lack associations with other constructs associated with undergraduate professional behaviour. The findings suggest that such questions should not be included in undergraduate examinations and may raise issues with the introduction of Situational Judgement Tests into Foundation Years selection.

Citation

Tiffin, P., Finn, G., & McLachlan, J. (2011). Evaluating professionalism in medical undergraduates using selected response questions: findings from an item response modelling study. BMC Medical Education, 11, Article 43. https://doi.org/10.1186/1472-6920-11-43

Journal Article Type Article
Publication Date Jun 1, 2011
Deposit Date Jul 12, 2011
Publicly Available Date Nov 6, 2012
Journal BMC Medical Education
Electronic ISSN 1472-6920
Publisher BioMed Central
Peer Reviewed Peer Reviewed
Volume 11
Article Number 43
DOI https://doi.org/10.1186/1472-6920-11-43
Public URL https://durham-repository.worktribe.com/output/1538988

Files

Published Journal Article (231 Kb)
PDF

Copyright Statement
© 2011 Tiffin et al. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.




You might also like



Downloadable Citations