Evaluation of a postgraduate examination for primary care: perceptions and performance of general practitioner trainers in the multiple choice paper of the … (original) (raw)
Related papers
International Journal of Medical Education, 2014
To investigate if the psychometric qualities of an OSCE consisting of more complex simulated patient encounters remain valid and reliable in the assessment of postgraduate trainees in general practice. Methods: In this intervention study without control group, the traditional OSCE was formally replaced by the new, complex version. The study population was composed by all postgraduate trainees (second and third phase) in general practice during the ongoing academic year. Data were handled and collected as part of the formal assessment program. Univariate analyses, the variance of scores and multivariate analyses were performed to assess the test qualities. Results: A total of 340 students participated. Average final scores were slightly higher for third-phase students (t-test, p =0.05). Overall test scores were equally distributed on station level, circuit level and phase level. A multiple regression analysis revealed that test scores were dependent on the stations and circuits, but not on the master phase. Conclusions: In a changing learning environment, assessment and evaluation strategies require reorientation. The reliability and validity of the OSCE remain subject to discussion. In particular, when it comes to content and design, the traditional OSCE might underestimate the performance level of postgraduate trainees in general practice. A reshaping of this OSCE to a more sophisticated design with more complex patient encounters appears to restore the validity of the test results.
Selecting general practice specialty trainees: where next?
British Journal of General Practice, 2010
Selection into GP speciality training is based on results of a multi-method job analysis study. Six key competency domains were identified as priorities to assess through the current national selection process, including empathy, communication, integrity, clinical expertise, problemsolving, and resilience. Each applicant is assessed using clinical problem-solving and situational judgement machine marked tests, followed by high fidelity exercises at regional selection centres. These show good internal reliability and predictive validity, with high correlations with subsequent job performance and outcomes in the MRCGP examinations. Candidate feedback is generally positive, where candidates prefer multiple opportunities to demonstrate their aptitude. When comparing selection methods, candidates perceive high fidelity assessments (for example, a consultation exercise with a simulated patient) as the most job relevant and fair. Emerging data provides opportunities to both streamline and re-engineer the process, so as to optimise efficiency (including costeffectiveness) while further developing its robustness. Logistic considerations favour delivery of multiple equated versions of machine marked tests in invigilated test centres. As with other assessments in medical education, concerns remain about differential performance between UK and international medical graduates. There is an urgent need to review the job analysis and selection criteria given profound changes in UK general practice taking place over the last decade.
BMC Family Practice, 2012
Background: In Europe, a comparable scope of training in GP can be observed especially in the field of knowledge. This feasibility study determines if a knowledge test is suitable in the context of a neighboring country. Methods: A Dutch knowledge multiple choice test was used after validation of its content in Flanders (Belgium) in the academic year 2010-2011. Satisfaction with the test format was assessed. The test was taken by general practice trainees and trainers. Group scores of trainees in year 1, 2 and 3 and their trainers were compared to Dutch participants as a control group.