Debra Darosa - Academia.edu (original) (raw)
Papers by Debra Darosa
CONTEXT The Association of American Medical Colleges' Institute for Improving Medical Education's... more CONTEXT The Association of American Medical Colleges' Institute for Improving Medical Education's report entitled 'Effective Use of Educational Technology' called on researchers to study the effectiveness of multimedia design principles. These principles were empirically shown to result in superior learning when used with college students in laboratory studies, but have not been studied with undergraduate medical students as participants. METHODS A pre-test ⁄ post-test control group design was used, in which the traditional-learning group received a lecture on shock using traditionally designed slides and the modified-design group received the same lecture using slides modified in accord with Mayer's principles of multimedia design. Participants included Year 3 medical students at a private, midwestern medical school progressing through their surgery clerkship during the academic year 2009–2010. The medical school divides students into four groups; each group attends the surgery clerkship during one of the four quarters of the academic year. Students in the second and third quarters served as the modified-design group (n = 91) and students in the fourth-quarter clerkship served as the traditional-design group (n = 39). RESULTS Both student cohorts had similar levels of pre-lecture knowledge. Both groups showed significant improvements in retention (p < 0.0001), transfer (p < 0.05) and total scores (p < 0.0001) between the pre-and post-tests. Repeated-measures ANOVA analysis showed statistically significant greater improvements in retention (F = 10.2, p = 0.0016) and total scores (F = 7.13, p = 0.0081) for those students instructed using principles of multimedia design compared with those instructed using the traditional design.
Journal of surgical education
To develop operative independence with essential procedures by the end of their training, residen... more To develop operative independence with essential procedures by the end of their training, residents need graded autonomy as they progress through training. This study compares autonomy expectations, as defined by faculty and residents, with autonomy measured in the operating room. Operative procedures performed by general surgery residents between November 2012 and June 2013 were each assigned an autonomy score by the operating attending physician using a previously described rating scale (Zwisch). Scores range from minimum autonomy, "show and tell," to maximum autonomy, "supervision only." Autonomy expectations were defined by a survey asking faculty and residents what autonomy-level residents should achieve during each year of training for each of the 10 most commonly performed procedures. Faculty expectations, resident expectations, and actual operating room autonomy data were compared using analysis of variance with post hoc analysis by Tukey honestly signifi...
Journal of surgical education
The existing methods for evaluating resident operative performance interrupt the workflow of the ... more The existing methods for evaluating resident operative performance interrupt the workflow of the attending physician, are resource intensive, and are often completed well after the end of the procedure in question. These limitations lead to low faculty compliance and potential significant recall bias. In this study, we deployed a smartphone-based system, the Procedural Autonomy and Supervisions System, to facilitate assessment of resident performance according to the Zwisch scale with minimal workflow disruption. We aimed to demonstrate that this is a reliable, valid, and feasible method of measuring resident operative autonomy. Before implementation, general surgery residents and faculty underwent frame-of-reference training to the Zwisch scale. Immediately after any operation in which a resident participated, the system automatically sent a text message prompting the attending physician to rate the resident's level of operative autonomy according to the 4-level Zwisch scale. O...
The American Journal of Surgery, 2014
Rural surgeons have unique learning needs not easily met by traditional continuing medical educat... more Rural surgeons have unique learning needs not easily met by traditional continuing medical education courses. A multidisciplinary team developed and implemented a skills curriculum focused on leadership and communication, advanced endoscopy, emergency urology, emergency gynecology, facial plastic surgery, ultrasound, and management of fingertip amputations. Twenty-five of 30 (89%) rural surgeons who completed a follow-up course evaluation reported that the knowledge acquired during the course had improved their practice and/or the quality of patient care, particularly by refining commonly used skills and expanding the care options they could offer to their patients. The surgeons reported incorporating changes in their communication and interaction with colleagues. This course was successful, from participants&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39; perspectives, in providing hands-on mentored training for a variety of skills that reflect the broad scope of practice of surgeons in rural areas. Attendees felt that their participation resulted in important behavior and practice changes.
The American Journal of Surgery, 2003
Students consistently identified inadequate feedback as a deficiency in our third-year clerkship.... more Students consistently identified inadequate feedback as a deficiency in our third-year clerkship. We asked students to solicit one faculty and one resident every 2 weeks for written feedback on a &amp;amp;amp;amp;amp;amp;amp;quot;feedback prescription pad.&amp;amp;amp;amp;amp;amp;amp;quot; Each prescription requested four comments: two things the student did well and two things the student needs to improve. Students rated feedback using a five-point scale. A three-point categorization scheme was employed to assess the quality of feedback. Students&amp;amp;amp;amp;amp;amp;amp;#39; rating of feedback improved significantly compared with a previous time period (3.5 +/- 1.2 versus 2.6 +/- 1.2, P &amp;amp;amp;amp;amp;amp;amp;lt;0.01). Interrater reliability of our categorization scheme was high (kappa &amp;amp;amp;amp;amp;amp;amp;gt; or =0.75, P &amp;amp;amp;amp;amp;amp;amp;lt;0.01) and demonstrated that only 10% of comments were specific enough to qualify as effective feedback. Feedback prescription pads were a simple method to facilitate feedback. Although students appreciated feedback, most feedback was inadequate. Faculty development programs to enhance student feedback should be a priority of clinical medical education.
World Journal of Surgery, 1994
Emerging changes in health care delivery will have a significant impact on the structure of surgi... more Emerging changes in health care delivery will have a significant impact on the structure of surgical education in academic departments of surgery. Based on some assumptions as to the probable nature of the final product of this reform, this article encourages a proactive stance by surgical educators to anticipate changes and move toward restructuring in areas of curricular content, the teaching process, performance evaluation strategies, and faculty infrastructure of the academic department. Curriculum changes must bridge the gap between public health and medicine and continue the aggressive trend toward teaching in the outpatient setting. Surgical educators must adapt to evolving computer and instructional technology that will make multimedia presentations, distance education, teleconferencing, hypermedia, and virtual reality commonplace in the teaching setting. Increased emphasis on accountability and accreditation will require stringent criteria in performance and program evaluation methodology. The academic infrastructure will need to adapt to the changing goal of training more general surgeons and fewer specialists and yet maintain the fundamental responsibility of an academic surgeon for mentoring the medical student and surgical resident.
The American Journal of Surgery, 2007
Research on performance evaluation highlights the importance of using multiple measures to develo... more Research on performance evaluation highlights the importance of using multiple measures to develop an accurate profile of students, yet we found no literature describing the use of a standard-setting method for determining a pass/fail cutoff for a clerkship based on multiple assessment methods. Steps in setting an absolute standard for a pass/fail grade are described. The new cut-off score was used to compare what decisions would have been made had it been applied in previous clerkships. We successfully applied the Hofstee method to ascertain a new standard pass/fail cutoff for our total surgery clerkship score. Had this absolute score been used in 4 prior clerkships, 150 instead of 152 would have passed the clerkship, and 10 instead of 8 would have failed the clerkship. A standard-setting method can be applied to a final clinical clerkship grade even when multiple performance measures are used.
The American Journal of Surgery, 2004
The purpose of the study is to describe the academic preparation, scope of duties, and scholarly ... more The purpose of the study is to describe the academic preparation, scope of duties, and scholarly activity of professional educators in surgery departments. Educators with doctoral degrees employed as full-time faculty in surgery departments were surveyed to determine terms of employment, academic preparation, scope of duties, and job satisfaction. Twelve of 13 educators responded and participated in the study. Educators spent, on average, 22% of their time on research activities, 33% on administrative responsibilities, 13% on teaching, 13% counseling students and residents, and 7% writing grants. They spent approximately 34% of their time with surgical faculty, 19% with residents/fellows, and 14% with medical students. Educators&amp;amp;amp;amp;amp;amp;amp;amp;amp;#39; contributions to surgery departments included improvements in assessment and evaluation, educational conferences, recruitment, and research productivity. Professional educators provide support needed to meet the growing demands and requirements of surgical education. Study findings may inform those interested in recruiting a professional educator to their faculty.
The American Journal of Surgery, 1995
In August 1993 the American College of Surgeons sponsored a course entitled "Surgeons as... more In August 1993 the American College of Surgeons sponsored a course entitled "Surgeons as Educators" (SAE) aimed at equipping academic surgeons with the knowledge and skills necessary to enhance surgical education administration, curriculum, teaching, and evaluation. The instructional design model used to construct the course called for a formal needs assessment to determine the importance, current skill level, and priority of what needed to be learned to be an effective educator. The needs assessment was accomplished using a job analysis and questionnaire approach. The 68-item questionnaire was mailed to 320 academic surgeons representing eight medical schools. A 62% response rate was achieved. Results indicated the education-related tasks or activities that faculty felt were important to their careers, as well as their perceived level of development in each area. Descriptive statistics were used to summarize the responses that were critical to the SAE faculty in helping prioritize, sequence, and time ration course content. Collective results became the foundation for developing the SAE curriculum by the course's five faculty members. A well-done needs assessment does not necessarily guarantee course success; however, it is the first and critical step to planning an educationally sound faculty development course or program designed for adult learners.
The American Journal of Surgery, 1996
Residents are more likely to learn from educational experiences when properly prepared from the s... more Residents are more likely to learn from educational experiences when properly prepared from the start of instruction and expectations are clear and consistent. The purpose of this study was to examine what first year residents need to know and be able to perform before starting their rotation in the intensive care unit (ICU). The sample included surgical faculty, first through fifth year general surgery residents, and ICU nurses. A survey was developed that listed 110 knowledge and skill items felt to be possible prerequisite learning objectives. Respondents were asked to evaluate the required level of knowledge or skill on a 5-point Likert-type scale. Items were divided into knowledge or skill categories and then ranked in descending order by their mean ratings. Four knowledge and 10 skill items were ranked highly by all 3 groups of respondents. Kruskall-Wallis analyses found 58 items with statistically significant differences in mean ratings between groups of respondents. With this information a self assessment tool was developed to determine the level of performance on these identified learning objectives. A course of instruction can now be developed to assist residents in meeting these objectives before they assume patient care responsibilities in the ICU.
The American Journal of Surgery, 2010
The purpose of this study was to compare the intraoperative learning needs and educational resour... more The purpose of this study was to compare the intraoperative learning needs and educational resource use of junior and senior residents. Our goal was to gain a better understanding of the progression of learning needs in surgical training. Residents (n = 125) completed a previously validated, 27-item survey indicating the following: (1) the extent to which traditional learning resources are used when preparing for cases in the operating room, and (2) which intraoperative management topics in which they believed they were deficient despite preoperative preparation. On a scale of 1 to 5, with 5 indicating frequent use, postgraduate year (PGY)-5 residents (n = 39) indicated surgical atlases (4.15; SD, .90) and surgical texts (4.15; SD, .90) were their most frequently used resources when preparing for a case in the operating room. In contrast, PGY-1 residents (n = 32) indicated anatomy atlases (3.97; SD, .93) and advice from colleagues (3.64; SD, .90) were their most frequently used resources when preparing for a case in the operating room. Despite the differences in how the PGY-5 group and the PGY-1 group prepared for a case, of 12 intraoperative management topics both groups believed they were the least prepared for instrument use/selection and suture selection. Today&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;s residents represent a heterogeneous group of individuals with different learning needs based on level of experience, knowledge, and learning style. Our study highlights unexpected but critical learning needs for senior-level residents that can and should be readily addressed.
The American Journal of Surgery, 2008
The Patient Assessment and Management Examination (PAME) is a standardized patient examination de... more The Patient Assessment and Management Examination (PAME) is a standardized patient examination designed to assess management skills of senior residents. This study explored the relationship between faculty and resident self-evaluation by using PAME. Methods: Nine postgraduate year (PGY) 4 and PGY5 residents were examined with a 5 case PAME. Faculty rated interactions between residents and standardized patients and residents rated themselves based on review of audio-video recordings of their interactions. We examined correlations between faculty and resident self-assessments. Results: Faculty and resident ratings of physical examination skills was the only competency that correlated significantly. Correlations were not significant for the other 15 competencies (Pearson r, to .262). Correlation was no better when examined within each case. Conclusions: Although PAME may be a useful tool, this study suggests that even senior residents do not assess their performance as clinicians similarly to faculty. Further research is needed to better understand the source of these disagreements.
The American Journal of Surgery, 2002
This study was designed to evaluate the impact of changes made to our morbidity and mortality (M&... more This study was designed to evaluate the impact of changes made to our morbidity and mortality (M&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;M) conference. A 23-item survey using corresponding Likert-type scales was created. Faculty and residents were asked to anonymously complete the surveys in June 1999. Based on this information, specific modifications were made to the conference. The same survey was administered to faculty and residents in the Fall of 2000. Analysis was performed using Student t tests. Postsurvey findings showed residents felt eight components improved significantly (P &amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;0.05). Faculty noted nonsignificant improvement in nine survey items and decline in nine items (five unchanged). Changes in content and structure made to enhance our M&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;M conference&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;s educational value resulted in significant improvements as perceived by the surgical residents. Interestingly, these changes had only minimal impact on faculty perceptions.
The American Journal of Surgery, 2000
Surgical education peer-reviewed publications have markedly increased over the last decade. The p... more Surgical education peer-reviewed publications have markedly increased over the last decade. The purpose of this study was to review the surgical education literature published over the last 10 years and address the following questions: What subjects in surgical education tend to be studied? What are the most to least commonly employed research designs and statistics? Has there been a change in how research data are collected? Where are these studies published? A literature search encompassing surgical education papers published between January 1988 and August 1998 was performed. Four investigators coded qualifying abstracts on journal type, subject of research, data collection methods, research design, and statistics. Each investigator was asked to code 10 articles at the start of the study to assess interrater reliability. A total of 420 abstracts were evaluated. Interrater reliability yielded percent agreements ranging from 82% to 96%. Curriculum and teaching were the most frequent topics studied (40%), followed by assessment (23%) and program evaluation (18%). Most research designs used were descriptive (41%). Experimental design has progressively increased from 2% in 1988-89 to 16% in 1998. A total of 551 statistical methods were accounted for in the 420 abstracts. The most common statistical analyses used were descriptive statistics (32%). The predominant mode of data collection was through testing or direct observations (34%). Survey instruments followed closely as a popular data collection method at 27%. The majority of papers were published in peer-reviewed surgical journals (64%),followed by medical education journals (22%) and "other" journals (14%). An analysis of the surgical education literature demonstrates the growing emphasis on the use of educational research to explore relevant issues and problems. Descriptive research is most popular, with an increasing trend in experimental research. Publication of educational research in peer-reviewed surgical journals is becoming more popular. This study informs those interested in the surgical education research literature of current trends, and what they need to know for a more critical appraisal of this body of literature.
The American Journal of Surgery, 1995
Research has shown that medical students are generally accepted by patients, but specific details... more Research has shown that medical students are generally accepted by patients, but specific details that elucidate patient perceptions of the advantages and disadvantages to student involvement have not been documented. This study was designed to determine variables that influence patient satisfaction with students. Patients were interviewed by one of two faculty members using a questionnaire-style format that covered 12 variables regarding patient care. Patients were asked to rate the extent to which medical students helped or hindered their hospital stay, with regard to the 12 variables. Patients' attitudes were favorable regardless of the students' extent of clinical experience or clinical abilities or the patients' age or length of hospital stay. Patients reported that students spent time with them and answered their questions. Most patients stated that they would allow students to participate in their future hospital care. Positive patient-student interactions can have important effects on patients' expectations and their acceptance of future encounters with students. This fact is becoming increasingly important due to the changes in health care and the decreasing incidence of inpatient surgical encounters.
The American Journal of Surgery, 2011
in 2008, the Surgical Council on Resident Education selected 33 residency programs to pilot its G... more in 2008, the Surgical Council on Resident Education selected 33 residency programs to pilot its General Surgery Resident Curriculum Website Portal. The portal aims to reduce program variability in curricula, align teaching and learning with essential content, and improve resident study and performance. two online surveys were sent to all program directors and their residents before releasing the portal. Data from 32 programs and 899 residents (84%) were analyzed to determine the extent to which preimplementation characteristics supported the portal&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;s rationale and illuminated barriers to its use and impact on learning. the need for curriculum content and access to online texts varied markedly across programs. Residents had easy onsite access to the Internet and used it heavily for immediate purposes. Fewer residents used the Web for planned activities and proactive study. On average, residents reported studying an hour or less a day. the portal appears to serve curricular resource needs and may better direct resident study. Programs are advised to consciously integrate the SCORE curriculum and portal into residency training and faculty development.
Teaching and Learning in Medicine, 2004
Residency programs are required to teach and evaluate trainees in the area of professionalism and... more Residency programs are required to teach and evaluate trainees in the area of professionalism and medical ethics. Prior to developing a curriculum in this area, residents and fellows were surveyed to assess learning needs. A case-based survey was developed based on published curricula. Residents and fellows were asked to describe their comfort level in 11 clinical scenarios on a Likert-type scale ranging from 1 (not at all comfortable) to 10 (extremely comfortable). 151 surveys were returned for an overall response rate of 73%. Comfort levels ranged from a low of 3.1 to a high of 8.5 on the 10-point scale. Despite additional years of clinical training, fellows only reported an increased comfort level in 1 case. Learning needs exist in residents and fellows in the area of medical ethics. Use of a needs assessment was instrumental in planning and designing an ethics curriculum.
Teaching and Learning in Medicine, 2001
A comprehensive and up-to-date curriculum requires periodic formal review to ensure it continues ... more A comprehensive and up-to-date curriculum requires periodic formal review to ensure it continues to meet learners' needs. This study describes a model for evaluating a surgery clerkship curriculum designed to determine the appropriateness of its learning objectives to the general professional education of a physician. A survey was mailed to graduates who pursued generalist residencies. Respondents estimated the number of patients encountered annually with specified presenting complaints or disease entities and the percentage of time these were referred to surgeons. For 23 technical procedures, respondents estimated the frequency done annually and whether remaining proficient in the skill was considered important. The majority of graduates reported the need to remain proficient in 19 technical procedures. Numerous patient problems were identified as requiring careful instruction so that learners know when and when not to refer for surgical intervention. The clerkship was modified to include skills and topics not previously included or appropriately emphasized.
CONTEXT The Association of American Medical Colleges' Institute for Improving Medical Education's... more CONTEXT The Association of American Medical Colleges' Institute for Improving Medical Education's report entitled 'Effective Use of Educational Technology' called on researchers to study the effectiveness of multimedia design principles. These principles were empirically shown to result in superior learning when used with college students in laboratory studies, but have not been studied with undergraduate medical students as participants. METHODS A pre-test ⁄ post-test control group design was used, in which the traditional-learning group received a lecture on shock using traditionally designed slides and the modified-design group received the same lecture using slides modified in accord with Mayer's principles of multimedia design. Participants included Year 3 medical students at a private, midwestern medical school progressing through their surgery clerkship during the academic year 2009–2010. The medical school divides students into four groups; each group attends the surgery clerkship during one of the four quarters of the academic year. Students in the second and third quarters served as the modified-design group (n = 91) and students in the fourth-quarter clerkship served as the traditional-design group (n = 39). RESULTS Both student cohorts had similar levels of pre-lecture knowledge. Both groups showed significant improvements in retention (p < 0.0001), transfer (p < 0.05) and total scores (p < 0.0001) between the pre-and post-tests. Repeated-measures ANOVA analysis showed statistically significant greater improvements in retention (F = 10.2, p = 0.0016) and total scores (F = 7.13, p = 0.0081) for those students instructed using principles of multimedia design compared with those instructed using the traditional design.
Journal of surgical education
To develop operative independence with essential procedures by the end of their training, residen... more To develop operative independence with essential procedures by the end of their training, residents need graded autonomy as they progress through training. This study compares autonomy expectations, as defined by faculty and residents, with autonomy measured in the operating room. Operative procedures performed by general surgery residents between November 2012 and June 2013 were each assigned an autonomy score by the operating attending physician using a previously described rating scale (Zwisch). Scores range from minimum autonomy, "show and tell," to maximum autonomy, "supervision only." Autonomy expectations were defined by a survey asking faculty and residents what autonomy-level residents should achieve during each year of training for each of the 10 most commonly performed procedures. Faculty expectations, resident expectations, and actual operating room autonomy data were compared using analysis of variance with post hoc analysis by Tukey honestly signifi...
Journal of surgical education
The existing methods for evaluating resident operative performance interrupt the workflow of the ... more The existing methods for evaluating resident operative performance interrupt the workflow of the attending physician, are resource intensive, and are often completed well after the end of the procedure in question. These limitations lead to low faculty compliance and potential significant recall bias. In this study, we deployed a smartphone-based system, the Procedural Autonomy and Supervisions System, to facilitate assessment of resident performance according to the Zwisch scale with minimal workflow disruption. We aimed to demonstrate that this is a reliable, valid, and feasible method of measuring resident operative autonomy. Before implementation, general surgery residents and faculty underwent frame-of-reference training to the Zwisch scale. Immediately after any operation in which a resident participated, the system automatically sent a text message prompting the attending physician to rate the resident's level of operative autonomy according to the 4-level Zwisch scale. O...
The American Journal of Surgery, 2014
Rural surgeons have unique learning needs not easily met by traditional continuing medical educat... more Rural surgeons have unique learning needs not easily met by traditional continuing medical education courses. A multidisciplinary team developed and implemented a skills curriculum focused on leadership and communication, advanced endoscopy, emergency urology, emergency gynecology, facial plastic surgery, ultrasound, and management of fingertip amputations. Twenty-five of 30 (89%) rural surgeons who completed a follow-up course evaluation reported that the knowledge acquired during the course had improved their practice and/or the quality of patient care, particularly by refining commonly used skills and expanding the care options they could offer to their patients. The surgeons reported incorporating changes in their communication and interaction with colleagues. This course was successful, from participants&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39; perspectives, in providing hands-on mentored training for a variety of skills that reflect the broad scope of practice of surgeons in rural areas. Attendees felt that their participation resulted in important behavior and practice changes.
The American Journal of Surgery, 2003
Students consistently identified inadequate feedback as a deficiency in our third-year clerkship.... more Students consistently identified inadequate feedback as a deficiency in our third-year clerkship. We asked students to solicit one faculty and one resident every 2 weeks for written feedback on a &amp;amp;amp;amp;amp;amp;amp;quot;feedback prescription pad.&amp;amp;amp;amp;amp;amp;amp;quot; Each prescription requested four comments: two things the student did well and two things the student needs to improve. Students rated feedback using a five-point scale. A three-point categorization scheme was employed to assess the quality of feedback. Students&amp;amp;amp;amp;amp;amp;amp;#39; rating of feedback improved significantly compared with a previous time period (3.5 +/- 1.2 versus 2.6 +/- 1.2, P &amp;amp;amp;amp;amp;amp;amp;lt;0.01). Interrater reliability of our categorization scheme was high (kappa &amp;amp;amp;amp;amp;amp;amp;gt; or =0.75, P &amp;amp;amp;amp;amp;amp;amp;lt;0.01) and demonstrated that only 10% of comments were specific enough to qualify as effective feedback. Feedback prescription pads were a simple method to facilitate feedback. Although students appreciated feedback, most feedback was inadequate. Faculty development programs to enhance student feedback should be a priority of clinical medical education.
World Journal of Surgery, 1994
Emerging changes in health care delivery will have a significant impact on the structure of surgi... more Emerging changes in health care delivery will have a significant impact on the structure of surgical education in academic departments of surgery. Based on some assumptions as to the probable nature of the final product of this reform, this article encourages a proactive stance by surgical educators to anticipate changes and move toward restructuring in areas of curricular content, the teaching process, performance evaluation strategies, and faculty infrastructure of the academic department. Curriculum changes must bridge the gap between public health and medicine and continue the aggressive trend toward teaching in the outpatient setting. Surgical educators must adapt to evolving computer and instructional technology that will make multimedia presentations, distance education, teleconferencing, hypermedia, and virtual reality commonplace in the teaching setting. Increased emphasis on accountability and accreditation will require stringent criteria in performance and program evaluation methodology. The academic infrastructure will need to adapt to the changing goal of training more general surgeons and fewer specialists and yet maintain the fundamental responsibility of an academic surgeon for mentoring the medical student and surgical resident.
The American Journal of Surgery, 2007
Research on performance evaluation highlights the importance of using multiple measures to develo... more Research on performance evaluation highlights the importance of using multiple measures to develop an accurate profile of students, yet we found no literature describing the use of a standard-setting method for determining a pass/fail cutoff for a clerkship based on multiple assessment methods. Steps in setting an absolute standard for a pass/fail grade are described. The new cut-off score was used to compare what decisions would have been made had it been applied in previous clerkships. We successfully applied the Hofstee method to ascertain a new standard pass/fail cutoff for our total surgery clerkship score. Had this absolute score been used in 4 prior clerkships, 150 instead of 152 would have passed the clerkship, and 10 instead of 8 would have failed the clerkship. A standard-setting method can be applied to a final clinical clerkship grade even when multiple performance measures are used.
The American Journal of Surgery, 2004
The purpose of the study is to describe the academic preparation, scope of duties, and scholarly ... more The purpose of the study is to describe the academic preparation, scope of duties, and scholarly activity of professional educators in surgery departments. Educators with doctoral degrees employed as full-time faculty in surgery departments were surveyed to determine terms of employment, academic preparation, scope of duties, and job satisfaction. Twelve of 13 educators responded and participated in the study. Educators spent, on average, 22% of their time on research activities, 33% on administrative responsibilities, 13% on teaching, 13% counseling students and residents, and 7% writing grants. They spent approximately 34% of their time with surgical faculty, 19% with residents/fellows, and 14% with medical students. Educators&amp;amp;amp;amp;amp;amp;amp;amp;amp;#39; contributions to surgery departments included improvements in assessment and evaluation, educational conferences, recruitment, and research productivity. Professional educators provide support needed to meet the growing demands and requirements of surgical education. Study findings may inform those interested in recruiting a professional educator to their faculty.
The American Journal of Surgery, 1995
In August 1993 the American College of Surgeons sponsored a course entitled "Surgeons as... more In August 1993 the American College of Surgeons sponsored a course entitled "Surgeons as Educators" (SAE) aimed at equipping academic surgeons with the knowledge and skills necessary to enhance surgical education administration, curriculum, teaching, and evaluation. The instructional design model used to construct the course called for a formal needs assessment to determine the importance, current skill level, and priority of what needed to be learned to be an effective educator. The needs assessment was accomplished using a job analysis and questionnaire approach. The 68-item questionnaire was mailed to 320 academic surgeons representing eight medical schools. A 62% response rate was achieved. Results indicated the education-related tasks or activities that faculty felt were important to their careers, as well as their perceived level of development in each area. Descriptive statistics were used to summarize the responses that were critical to the SAE faculty in helping prioritize, sequence, and time ration course content. Collective results became the foundation for developing the SAE curriculum by the course's five faculty members. A well-done needs assessment does not necessarily guarantee course success; however, it is the first and critical step to planning an educationally sound faculty development course or program designed for adult learners.
The American Journal of Surgery, 1996
Residents are more likely to learn from educational experiences when properly prepared from the s... more Residents are more likely to learn from educational experiences when properly prepared from the start of instruction and expectations are clear and consistent. The purpose of this study was to examine what first year residents need to know and be able to perform before starting their rotation in the intensive care unit (ICU). The sample included surgical faculty, first through fifth year general surgery residents, and ICU nurses. A survey was developed that listed 110 knowledge and skill items felt to be possible prerequisite learning objectives. Respondents were asked to evaluate the required level of knowledge or skill on a 5-point Likert-type scale. Items were divided into knowledge or skill categories and then ranked in descending order by their mean ratings. Four knowledge and 10 skill items were ranked highly by all 3 groups of respondents. Kruskall-Wallis analyses found 58 items with statistically significant differences in mean ratings between groups of respondents. With this information a self assessment tool was developed to determine the level of performance on these identified learning objectives. A course of instruction can now be developed to assist residents in meeting these objectives before they assume patient care responsibilities in the ICU.
The American Journal of Surgery, 2010
The purpose of this study was to compare the intraoperative learning needs and educational resour... more The purpose of this study was to compare the intraoperative learning needs and educational resource use of junior and senior residents. Our goal was to gain a better understanding of the progression of learning needs in surgical training. Residents (n = 125) completed a previously validated, 27-item survey indicating the following: (1) the extent to which traditional learning resources are used when preparing for cases in the operating room, and (2) which intraoperative management topics in which they believed they were deficient despite preoperative preparation. On a scale of 1 to 5, with 5 indicating frequent use, postgraduate year (PGY)-5 residents (n = 39) indicated surgical atlases (4.15; SD, .90) and surgical texts (4.15; SD, .90) were their most frequently used resources when preparing for a case in the operating room. In contrast, PGY-1 residents (n = 32) indicated anatomy atlases (3.97; SD, .93) and advice from colleagues (3.64; SD, .90) were their most frequently used resources when preparing for a case in the operating room. Despite the differences in how the PGY-5 group and the PGY-1 group prepared for a case, of 12 intraoperative management topics both groups believed they were the least prepared for instrument use/selection and suture selection. Today&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;s residents represent a heterogeneous group of individuals with different learning needs based on level of experience, knowledge, and learning style. Our study highlights unexpected but critical learning needs for senior-level residents that can and should be readily addressed.
The American Journal of Surgery, 2008
The Patient Assessment and Management Examination (PAME) is a standardized patient examination de... more The Patient Assessment and Management Examination (PAME) is a standardized patient examination designed to assess management skills of senior residents. This study explored the relationship between faculty and resident self-evaluation by using PAME. Methods: Nine postgraduate year (PGY) 4 and PGY5 residents were examined with a 5 case PAME. Faculty rated interactions between residents and standardized patients and residents rated themselves based on review of audio-video recordings of their interactions. We examined correlations between faculty and resident self-assessments. Results: Faculty and resident ratings of physical examination skills was the only competency that correlated significantly. Correlations were not significant for the other 15 competencies (Pearson r, to .262). Correlation was no better when examined within each case. Conclusions: Although PAME may be a useful tool, this study suggests that even senior residents do not assess their performance as clinicians similarly to faculty. Further research is needed to better understand the source of these disagreements.
The American Journal of Surgery, 2002
This study was designed to evaluate the impact of changes made to our morbidity and mortality (M&... more This study was designed to evaluate the impact of changes made to our morbidity and mortality (M&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;M) conference. A 23-item survey using corresponding Likert-type scales was created. Faculty and residents were asked to anonymously complete the surveys in June 1999. Based on this information, specific modifications were made to the conference. The same survey was administered to faculty and residents in the Fall of 2000. Analysis was performed using Student t tests. Postsurvey findings showed residents felt eight components improved significantly (P &amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;0.05). Faculty noted nonsignificant improvement in nine survey items and decline in nine items (five unchanged). Changes in content and structure made to enhance our M&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;M conference&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;s educational value resulted in significant improvements as perceived by the surgical residents. Interestingly, these changes had only minimal impact on faculty perceptions.
The American Journal of Surgery, 2000
Surgical education peer-reviewed publications have markedly increased over the last decade. The p... more Surgical education peer-reviewed publications have markedly increased over the last decade. The purpose of this study was to review the surgical education literature published over the last 10 years and address the following questions: What subjects in surgical education tend to be studied? What are the most to least commonly employed research designs and statistics? Has there been a change in how research data are collected? Where are these studies published? A literature search encompassing surgical education papers published between January 1988 and August 1998 was performed. Four investigators coded qualifying abstracts on journal type, subject of research, data collection methods, research design, and statistics. Each investigator was asked to code 10 articles at the start of the study to assess interrater reliability. A total of 420 abstracts were evaluated. Interrater reliability yielded percent agreements ranging from 82% to 96%. Curriculum and teaching were the most frequent topics studied (40%), followed by assessment (23%) and program evaluation (18%). Most research designs used were descriptive (41%). Experimental design has progressively increased from 2% in 1988-89 to 16% in 1998. A total of 551 statistical methods were accounted for in the 420 abstracts. The most common statistical analyses used were descriptive statistics (32%). The predominant mode of data collection was through testing or direct observations (34%). Survey instruments followed closely as a popular data collection method at 27%. The majority of papers were published in peer-reviewed surgical journals (64%),followed by medical education journals (22%) and "other" journals (14%). An analysis of the surgical education literature demonstrates the growing emphasis on the use of educational research to explore relevant issues and problems. Descriptive research is most popular, with an increasing trend in experimental research. Publication of educational research in peer-reviewed surgical journals is becoming more popular. This study informs those interested in the surgical education research literature of current trends, and what they need to know for a more critical appraisal of this body of literature.
The American Journal of Surgery, 1995
Research has shown that medical students are generally accepted by patients, but specific details... more Research has shown that medical students are generally accepted by patients, but specific details that elucidate patient perceptions of the advantages and disadvantages to student involvement have not been documented. This study was designed to determine variables that influence patient satisfaction with students. Patients were interviewed by one of two faculty members using a questionnaire-style format that covered 12 variables regarding patient care. Patients were asked to rate the extent to which medical students helped or hindered their hospital stay, with regard to the 12 variables. Patients' attitudes were favorable regardless of the students' extent of clinical experience or clinical abilities or the patients' age or length of hospital stay. Patients reported that students spent time with them and answered their questions. Most patients stated that they would allow students to participate in their future hospital care. Positive patient-student interactions can have important effects on patients' expectations and their acceptance of future encounters with students. This fact is becoming increasingly important due to the changes in health care and the decreasing incidence of inpatient surgical encounters.
The American Journal of Surgery, 2011
in 2008, the Surgical Council on Resident Education selected 33 residency programs to pilot its G... more in 2008, the Surgical Council on Resident Education selected 33 residency programs to pilot its General Surgery Resident Curriculum Website Portal. The portal aims to reduce program variability in curricula, align teaching and learning with essential content, and improve resident study and performance. two online surveys were sent to all program directors and their residents before releasing the portal. Data from 32 programs and 899 residents (84%) were analyzed to determine the extent to which preimplementation characteristics supported the portal&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;s rationale and illuminated barriers to its use and impact on learning. the need for curriculum content and access to online texts varied markedly across programs. Residents had easy onsite access to the Internet and used it heavily for immediate purposes. Fewer residents used the Web for planned activities and proactive study. On average, residents reported studying an hour or less a day. the portal appears to serve curricular resource needs and may better direct resident study. Programs are advised to consciously integrate the SCORE curriculum and portal into residency training and faculty development.
Teaching and Learning in Medicine, 2004
Residency programs are required to teach and evaluate trainees in the area of professionalism and... more Residency programs are required to teach and evaluate trainees in the area of professionalism and medical ethics. Prior to developing a curriculum in this area, residents and fellows were surveyed to assess learning needs. A case-based survey was developed based on published curricula. Residents and fellows were asked to describe their comfort level in 11 clinical scenarios on a Likert-type scale ranging from 1 (not at all comfortable) to 10 (extremely comfortable). 151 surveys were returned for an overall response rate of 73%. Comfort levels ranged from a low of 3.1 to a high of 8.5 on the 10-point scale. Despite additional years of clinical training, fellows only reported an increased comfort level in 1 case. Learning needs exist in residents and fellows in the area of medical ethics. Use of a needs assessment was instrumental in planning and designing an ethics curriculum.
Teaching and Learning in Medicine, 2001
A comprehensive and up-to-date curriculum requires periodic formal review to ensure it continues ... more A comprehensive and up-to-date curriculum requires periodic formal review to ensure it continues to meet learners' needs. This study describes a model for evaluating a surgery clerkship curriculum designed to determine the appropriateness of its learning objectives to the general professional education of a physician. A survey was mailed to graduates who pursued generalist residencies. Respondents estimated the number of patients encountered annually with specified presenting complaints or disease entities and the percentage of time these were referred to surgeons. For 23 technical procedures, respondents estimated the frequency done annually and whether remaining proficient in the skill was considered important. The majority of graduates reported the need to remain proficient in 19 technical procedures. Numerous patient problems were identified as requiring careful instruction so that learners know when and when not to refer for surgical intervention. The clerkship was modified to include skills and topics not previously included or appropriately emphasized.