Current Milestones Assessment Practices, Needs and Challenges of Program Directors: A Case Study in a Pediatric Hospital Setting (original) (raw)
Related papers
Cureus, 2021
Introduction Accreditation Council for Graduate Medical Education's (ACGME's) Milestones assessment requirement has placed new demands on Program Directors (PDs), especially those with limited knowledge of assessment and evaluation activities. There is a lack of clarity on how Program Director (PDs)/Associate PDs (APDs) are effectively implementing milestones assessment and evaluation practices in the Graduate Medical Education programs. The purpose of this study was to investigate current assessment practices, needs, and challenges of PDs in implementing milestones assessment within their residency and fellowship programs in a pediatric hospital setting. Methods This study used a collective case study approach to obtain information from PDs, APDs, and Clinical Competency Committee (CCC) Chairs in 19 graduate programs at a pediatric hospital. We used structured meetings with planned agendas and a pre-formatted template to itemize program needs/difficulties/challenges in the ...
Assessing competencies using milestones along the way
Medical teacher, 2015
This paper presents perspectives and controversies surrounding the use of milestones to assess competency in outcomes-based medical education. Global perspectives (Canada, Europe, and the United States) and developments supporting their rationales are discussed. In Canada, there is a significant movement away from conceptualizing competency based on time, and a move toward demonstration of specific competencies. The success of this movement may require complex (rather than reductionist) milestones that reflect students' progression through complexity and context and a method to narrate their journey. European countries (United Kingdom, France, and Germany) have stressed the complexity associated with time and milestones for medical students to truly achieve competence. To meet the changing demands of medicine, they view time as actually providing students with knowledge and exposure to achieve various milestones. In the United States, milestones are based on sampling throughout ...
An Assessment System for the Pediatrics Milestone Project
Open Journal of Social Sciences, 2014
Board of Pediatrics (ABP) to assess resident physicians, is presented in this paper. Using the cutting-edge mobile technology, such as iPad, as the system platform, simple and easy to use interface for teachers to specify the competency categories and provide evaluation of the residents and immediate feedback on their progress of learning after daily rounds, this system enables the implementation of the medical education assessment strategy.
Journal of Graduate Medical Education, 2015
Background Undergraduate medical education (UME) follows the lead of graduate medical education (GME) in moving to competency-based assessment. The means for and the timing of competency-based assessments in UME are unclear. Objective We explored the feasibility of using the Accreditation Council for Graduate Medical Education Transitional Year (TY) Milestones to assess student performance during a mandatory, fourth-year capstone course. Methods Our single institution, observational study involved 99 medical students who completed the course in the spring of 2014. Students' skills were assessed by self, peer, and faculty assessment for 6 existing course activities using the TY Milestones. Evaluation completion rates and mean scores were calculated. Results Students' mean milestone levels ranged between 2.2 and 3.6 (on a 5-level scoring rubric). Level 3 is the performance expected at the completion of a TY. Students performed highest in breaking bad news and developing a quality improvement project, and lowest in developing a learning plan, working in interdisciplinary teams, and stabilizing acutely ill patients. Evaluation completion rates were low for some evaluations, and precluded use of the data for assessing student performance in the capstone course. Students were less likely to complete separate online evaluations. Faculty were less likely to complete evaluations when activities did not include dedicated time for evaluations. Conclusions Assessment of student competence on 9 TY Milestones during a capstone course was useful, but achieving acceptable evaluation completion rates was challenging. Modifications are necessary if milestone scores from a capstone are intended to be used as a handoff between UME and GME.
Programmatic Assessment of Level 1 Milestones in Incoming Interns
Academic Emergency Medicine, 2014
Objectives: With the Accreditation Council for Graduate Medical Education (ACGME) Next Accreditation System, emergency medicine (EM) residency programs will be required to report residents' progress through the EM milestones. The milestones include five progressively advancing skill levels, with Level 1 defining the skill set of a medical school graduate and Level 5, that of an attending physician. The ACGME stresses that multiple forms of assessment should be used to ensure capture of the multifaceted competencies. The objective of this study was to determine the feasibility and results of programmatic assessment of Level 1 milestones using multisource assessments for incoming EM interns in July. Methods: The study population was interns starting in 2012 and 2013. Interns' Level 1 milestone assessment was done with four distinct methods: 1) the postgraduate orientation assessment (POA) by the Graduate Medical Education Office for all incoming interns (this multistation examination covers nine of the EM milestones and includes standardized patient cases, task completion, and computer-based stations); 2) direct observation of patient encounters by core faculty using a milestones-based clinical skills competency checklist; 3) the global monthly assessment at the end of the intern orientation month that was updated to reflect the EM milestones; and 4) faculty assessment during procedural labs. These occurred during the July orientation month that included the POA, clinical shifts, didactic sessions, and procedure labs.
Journal of Graduate Medical Education, 2015
Background The Pediatrics Milestone Project uses behavioral anchors, narrative descriptions of observable behaviors, to describe learner progression through the Accreditation Council for Graduate Medical Education competencies. Starting June 2014, pediatrics programs were required to submit milestone reports for their trainees semiannually. Likert-type scale assessment tools were not designed to inform milestone reporting, creating a challenge for Clinical Competency Committees. Objective To determine if milestone-based assessments better stratify trainees by training level compared to Likert-type assessments. Methods We compared assessment results for 3 subcompetencies after changing from a 5-point Likert scale to milestone-based behavioral anchors in July 2013. Program leadership evaluated the new system by (1) comparing PGY-1 mean scores on Likert-type versus milestone-based assessments; and (2) comparing mean scores on the Likert-type versus milestone-based assessments across PG...
Medical Teacher, 2012
Background: In the USA, the Accreditation Council of Graduate Medical Education, Educational Innovations Project is a partner in reshaping residency training to meet increasingly complex systems of health care delivery. Aim: We describe the creation and implementation of milestones as a vehicle for translating educational theory into practice in preparing residents to provide safe, autonomous patient care. Method: Six program faculty leaders, all with advanced medical education training, met in an iterative process of developing, implementing, and modifying milestones until a final set were vetted. Results: We first formed the profile of a Master Internist. We then translated it into milestone language and implemented its integration across the program. Thirty-seven milestones were applied in all settings and rotations to reach explicit educational outcomes. We created three types of milestones: Progressive, build one on top of the other to mastery; additive, adding multiple behaviors together to culminate in mastery; and descriptive, using a proscribe set of complex, predetermined steps toward mastery. Conclusions: Using milestones, our program has enhanced an educational model into explicit, end of training goals. Milestone implementation has yielded positive results toward competency-based training and others may adapt our strategies in a similar effort.
International Journal of Medical Education, 2016
Objectives: To pilot test if Orthopaedic Surgery residents could self-assess their performance using newly created milestones, as defined by the Accreditation Council on Graduate Medical Education. Methods: In June 2012, an email was sent to Program Directors and administrative coordinators of the154 accredited Orthopaedic Surgery Programs, asking them to send their residents a link to an online survey. The survey was adapted from the Orthopaedic Surgery Milestone Project. Completed surveys were aggregated in an anonymous, confidential database. SAS 9.3 was used to perform the analyses. Results: Responses from 71 residents were analyzed. First and second year residents indicated through self-assessment that they had substantially achieved Level 1 and Level 2 milestones. Third year residents reported they had substantially achieved 30/41, and fourth year residents, all Level 3 milestones. Fifth year, graduating residents, reported they had substantially achieved 17 Level 4 milestones, and were extremely close on another 15. No milestone was rated at Level 5, the maximum possible. Earlier in training, Patient Care and Medical Knowledge milestones were rated lower than the milestones reflecting the other four competencies of Practice Based Learning and Improvement, Systems Based Practice, Professionalism, and Interpersonal Communication. The gap was closed by the fourth year. Conclusions: Residents were able to successfully self-assess using the 41 Orthopaedic Surgery milestones. Respondents' rate improved proficiency over time. Graduating residents report they have substantially, or close to substantially, achieved all Level 4 milestones. Milestone self-assessment may be a useful tool as one component of a program's overall performance assessment strategy.