Gordon Schiff - Academia.edu (original) (raw)

Papers by Gordon Schiff

Research paper thumbnail of Systems engineering analysis of diagnostic referral closed-loop processes

BMJ Open Quality, 2021

BackgroundClosing loops to complete diagnostic referrals remains a significant patient safety pro... more BackgroundClosing loops to complete diagnostic referrals remains a significant patient safety problem in most health systems, with 65%–73% failure rates and significant delays common despite years of improvement efforts, suggesting new approaches may be useful. Systems engineering (SE) methods increasingly are advocated in healthcare for their value in studying and redesigning complex processes.ObjectiveConduct a formative SE analysis of process logic, variation, reliability and failures for completing diagnostic referrals originating in two primary care practices serving different demographics, using dermatology as an illustrating use case.MethodsAn interdisciplinary team of clinicians, systems engineers, quality improvement specialists, and patient representatives collaborated to understand processes of initiating and completing diagnostic referrals. Cross-functional process maps were developed through iterative group interviews with an urban community-based health centre and a te...

Research paper thumbnail of Table 4, Change ideas for preventing and minimizing diagnostic error

Research paper thumbnail of Health Information Technology Center for Education and Research on Therapeutics (HIT-CERT) - Final Report

Purpose: The purpose of the BWH HIT-CERT was to leverage HIT for pharmacosurveillance, medication... more Purpose: The purpose of the BWH HIT-CERT was to leverage HIT for pharmacosurveillance, medication-related clinical decision support, and to identify new ways to utilize information coming from medication-related decision support. Moreover, we strove to advance our understanding of how providers are responding to medication-related decision support. Scope: Within the three BWH HIT-CERT subprojects, our team examined questions relating to appropriateness, safety, and efficacy across diverse patient populations. The projects covered multiple settings including outpatients, inpatients and recently discharged inpatients. Methods: We completed three research projects over a five-year period, each of which addressed one or more of the programmatic interest areas including patient safety, development and enhancement of tools, health care system interventions, and translation into practice or policy. The three subprojects involved: 1) leveraging new technologies to improve pharmacosurveillan...

Research paper thumbnail of Figure 1, Relationships between diagnostic process errors, misdiagnosis, and adverse events

Research paper thumbnail of Association of Display of Patient Photographs in the Electronic Health Record With Wrong-Patient Order Entry Errors

Research paper thumbnail of Table 1, General medical error studies that reported errors in diagnosis

Research paper thumbnail of Table 2, Illustrative disease-specific studies of diagnosis errors

Research paper thumbnail of Personal Formularies of Primary Care Physicians Across 4 Health Care Systems

JAMA Network Open, 2021

IMPORTANCE More conservative prescribing has the potential to reduce adverse drug events and pati... more IMPORTANCE More conservative prescribing has the potential to reduce adverse drug events and patient harm and cost; however, no method exists defining the extent to which individual clinicians prescribe conservatively. One potential domain is prescribing a more limited number of drugs. Personal formularies-defined as the number and mix of unique, newly initiated drugs prescribed by a physician-may enable comparisons among clinicians, practices, and institutions. OBJECTIVES To develop a method of defining primary care physicians' personal formularies and examine how they differ among primary care physicians at 4 institutions; evaluate associations between personal formularies and patient, physician, and practice site characteristics; and empirically derive and examine the variability of the top 200 core drugs prescribed at the 4 sites. DESIGN, SETTING, AND PARTICIPANTS This retrospective cohort study was conducted at 4 US health care systems among 4655 internal and family medicine physicians and 4 930 707 patients who had at least 1 visit to these physicians between January 1, 2017, and December 31, 2018. EXPOSURES Personal formulary size was defined as the number of unique, newly initiated drugs. MAIN OUTCOMES AND MEASURES Personal formulary size and drugs used, physician and patient characteristics, core drugs, and analysis of selected drug classes. RESULTS The study population included 4655 primary care physicians (2274 women [48.9%]; mean [SD] age, 48.5 [4.4] years) and 4 930 707 patients (16.5% women; mean [SD] age, 51.9 [8.3] years). There were 41 378 903 outpatient prescriptions written, of which 9 496 766 (23.0%) were new starts. Institution median personal formulary size ranged from 150 (interquartile range, 82.0-212.0) to 296 (interquartile range, 230.0-347.0) drugs. In multivariable modeling, personal formulary size was significantly associated with panel size (total number of unique patients with faceto-face encounters during the study period; 1.2 medications per 100 patients), physician's total number of encounters (5.7 drugs per 10% increase), and physician's sex (−6.2 drugs per 100 patients for female physicians). There were 1527 unique, newly prescribed drugs across the 4 sites. Fewer than half the drugs (626 [41.0%]) were used at every site. Physicians' prescribing of drugs from a pooled core list varied from 0% to 100% of their prescriptions. CONCLUSIONS AND RELEVANCE Personal formularies, measured at the level of individual physicians and institutions, reveal variability in size and mix of drugs. Similarly, defining a list of commonly prescribed core drugs in primary care revealed interphysician and interinstitutional differences. Personal formularies and core medication lists enable comparisons and may identify outliers and opportunities for safer and more appropriate prescribing.

Research paper thumbnail of Comparison of a Prototype for Indications-Based Prescribing With 2 Commercial Prescribing Systems

Research paper thumbnail of A Prescription For Enhancing Electronic Prescribing Safety

Health Affairs, 2018

While electronic prescribing has been shown to reduce medication errors and improve prescribing s... more While electronic prescribing has been shown to reduce medication errors and improve prescribing safety, it is vulnerable to error-prone processes. We review six intersecting areas in which changes to electronic prescribing systems, particularly in the outpatient setting, could transform medication ordering quality and safety. We recommend incorporating medication indications into electronic prescribing, establishing a single shared online medication list, implementing the transmission of electronic cancellation orders to pharmacies (CancelRx) to ensure that drugs are safely and reliably discontinued, implementing standardized structured and codified prescription instructions, reengineering clinical decision support, and redesigning electronic prescribing to facilitate the ordering of nondrug alternatives.

Research paper thumbnail of What do Australian consumers, pharmacists and prescribers think about documenting indications on prescriptions and dispensed medicines labels?: A qualitative study

BMC health services research, Jan 15, 2017

Documenting the indication on prescriptions and dispensed medicines labels is not standard practi... more Documenting the indication on prescriptions and dispensed medicines labels is not standard practice in Australia. However, previous studies that have focused on the content and design of dispensed medicines labels, have suggested including the indication as a safety measure. The aim of this study was to investigate the perspectives of Australian consumers, pharmacists and prescribers on documenting the indication on prescriptions and dispensed medicines labels. Semi-structured interviews were conducted and mock-up of dispensed medicines labels were designed for participants. Consumers (n = 19) and pharmacists (n = 7) were recruited by convenience sample at community pharmacies in Sydney (Australia) and prescribers (n = 8), including two medical students, were recruited through snowballing. Thirty-four participants were interviewed. Most participants agreed that documenting the indication would be beneficial especially for patients who are forgetful or take multiple medications. Part...

Research paper thumbnail of Using drug knowledgebase information to distinguish between look-alike-sound-alike drugs

Journal of the American Medical Informatics Association : JAMIA, Jan 22, 2018

To extract drug indications from a commercial drug knowledgebase and determine to what extent dru... more To extract drug indications from a commercial drug knowledgebase and determine to what extent drug indications can discriminate between look-alike-sound-alike (LASA) drugs. We extracted drug indications disease concepts from the MedKnowledge Indications module from First Databank Inc. (South San Francisco, CA) and associated them with drugs on the Institute for Safe Medication Practices (ISMP) list of commonly confused drug names. We used high-level concepts (rather than granular concepts) to represent the general indications for each drug. Two pharmacists reviewed each drug's association with its high-level indications concepts for accuracy and clinical relevance. We compared the high-level indications for each commonly confused drug pair and categorized each pair as having a complete overlap, partial overlap or no overlap in high-level indications. Of 278 LASA drug pairs, 165 (59%) had no overlap and 58 (21%) had partial overlap in high-level indications. Fifty-five pairs (20%...

Research paper thumbnail of Reduced Effectiveness of Interruptive Drug-Drug Interaction Alerts after Conversion to a Commercial Electronic Health Record

Journal of general internal medicine, Jan 15, 2018

Drug-drug interaction (DDI) alerts in electronic health records (EHRs) can help prevent adverse d... more Drug-drug interaction (DDI) alerts in electronic health records (EHRs) can help prevent adverse drug events, but such alerts are frequently overridden, raising concerns about their clinical usefulness and contribution to alert fatigue. To study the effect of conversion to a commercial EHR on DDI alert and acceptance rates. Two before-and-after studies. 3277 clinicians who received a DDI alert in the outpatient setting. Introduction of a new, commercial EHR and subsequent adjustment of DDI alerting criteria. Alert burden and proportion of alerts accepted. Overall interruptive DDI alert burden increased by a factor of 6 from the legacy EHR to the commercial EHR. The acceptance rate for the most severe alerts fell from 100 to 8.4%, and from 29.3 to 7.5% for medium severity alerts (P < 0.001). After disabling the least severe alerts, total DDI alert burden fell by 50.5%, and acceptance of Tier 1 alerts rose from 9.1 to 12.7% (P < 0.01). Changing from a highly tailored DDI alerting...

Research paper thumbnail of Reasons for computerised provider order entry (CPOE)-based inpatient medication ordering errors: an observational study of voided orders

BMJ Quality & Safety, 2017

ObjectiveMedication voiding is a computerised provider order entry (CPOE)-based discontinuation m... more ObjectiveMedication voiding is a computerised provider order entry (CPOE)-based discontinuation mechanism that allows clinicians to identify erroneous medication orders. We investigated the accuracy of voiding as an indicator of clinician identification and interception of a medication ordering error, and investigated reasons and root contributors for medication ordering errors.MethodUsing voided orders identified with a void alert, we conducted interviews with ordering and voiding clinicians, followed by patient chart reviews. A structured coding framework was used to qualitatively analyse the reasons for medication ordering errors. We also compared clinician-CPOE-selected (at time of voiding), clinician-reported (interview) and chart review-based reasons for voiding.ResultsWe conducted follow-up interviews on 101 voided orders. The positive predictive value (PPV) of voided orders that were medication ordering errors was 93.1% (95% CI 88.1% to 98.1%, n=94). Using chart review-based...

Research paper thumbnail of Methodologies for evaluating strategies to reduce diagnostic error: report from the research summit at the 7th International Diagnostic Error in Medicine Conference

Diagnosis, 2016

In this article we review current evidence on strategies to evaluate diagnostic error solutions, ... more In this article we review current evidence on strategies to evaluate diagnostic error solutions, discuss the methodological challenges that exist in investigating the value of these strategies in patient care, and provide recommendations for methods that can be applied in investigating potential solutions to diagnostic errors. These recommendations were developed iteratively by the authors based upon initial discussions held during the Research Summit of the 7th Annual Diagnostic Error in Medicine Conference in September 2014. The recommendations include the following elements for designing studies of diagnostic research solutions: (1) Select direct and indirect outcomes measures of importance to patients, while also practical for the particular solution; (2) Develop a clearly-stated logic model for the solution to be tested; (3) Use rapid, iterative prototyping in the early phases of solution testing; (4) Use cluster-randomized clinical trials where feasible; (5) Avoid simple pre-p...

Research paper thumbnail of Exploring the potential for using drug indications to prevent look-alike and sound-alike drug errors

Expert Opinion on Drug Safety, 2017

Background. Look-alike, sound-alike (LASA) drug names are a cause of medication errors with resul... more Background. Look-alike, sound-alike (LASA) drug names are a cause of medication errors with resulting patient harm and healthcare costs. This study assessed to which extent the use of the generic drug name, therapeutic class, health problem, and the U.S. Food and Drug Administration (FDA)-approved indications might be used to differentiate LASA drug pairs. Research design and methods. We collected information about LASA drug pairs reported by the FDA to have look-alike sound-alike similarities. To assess potential for differentiating LASA drug pairs, we compared the following drug characteristics: generic name, therapeutic class, health problem, and FDA-approved indication. Results. For the 33 FDA reported LASA drug pairs we identified a total of 432 FDA-approved indications. Using the generic name, therapeutic class, health problem and drug indication we were able to differentiate 8 (24.2%), 24 (72.7%), 25 (75.8%) and 26 (78.8%), respectively of the 33 LASA drug pairs. Using the generic name, therapeutic class, and health problem we were able to distinguish 31 (7.2%), 212 (49.1%), and 269 (62.3%), respectively of the 432 FDA-approved indications for the LASA drug pairs. Conclusions. Including the FDA-approved indication in the drug prescription may be used to differentiate LASA drug pairs and thus, prevent wrong drug medication errors.

Research paper thumbnail of Learning from errors: analysis of medication order voiding in CPOE systems

Journal of the American Medical Informatics Association, 2017

Objective: Medication order voiding allows clinicians to indicate that an existing order was plac... more Objective: Medication order voiding allows clinicians to indicate that an existing order was placed in error. We explored whether the order voiding function could be used to record and study medication ordering errors. Materials and Methods: We examined medication orders from an academic medical center for a 6-year period (2006–2011; n = 5 804 150). We categorized orders based on status (void, not void) and clinician-provided reasons for voiding. We used multivariable logistic regression to investigate the association between order voiding and clinician, patient, and order characteristics. We conducted chart reviews on a random sample of voided orders (n = 198) to investigate the rate of medication ordering errors among voided orders, and the accuracy of clinician-provided reasons for voiding. Results: We found that 0.49% of all orders were voided. Order voiding was associated with clinician type (physician, pharmacist, nurse, student, other) and order type (inpatient, prescription,...

Research paper thumbnail of Incorporating Indications into Medication Ordering--Time to Enter the Age of Reason

The New England journal of medicine, Jan 28, 2016

Research paper thumbnail of Screening for medication errors using an outlier detection system

Journal of the American Medical Informatics Association : JAMIA, Mar 19, 2017

The study objective was to evaluate the accuracy, validity, and clinical usefulness of medication... more The study objective was to evaluate the accuracy, validity, and clinical usefulness of medication error alerts generated by an alerting system using outlier detection screening. Five years of clinical data were extracted from an electronic health record system for 747 985 patients who had at least one visit during 2012-2013 at practices affiliated with 2 academic medical centers. Data were screened using the system to detect outliers suggestive of potential medication errors. A sample of 300 charts was selected for review from the 15 693 alerts generated. A coding system was developed and codes assigned based on chart review to reflect the accuracy, validity, and clinical value of the alerts. Three-quarters of the chart-reviewed alerts generated by the screening system were found to be valid in which potential medication errors were identified. Of these valid alerts, the majority (75.0%) were found to be clinically useful in flagging potential medication errors or issues. A clinical...

Research paper thumbnail of Addressing Ambulatory Safety and Malpractice: The Massachusetts PROMISES Project

Health Services Research, 2016

More than half of malpractice claims involve care in the ambulatory setting (Zuccotti and Sato 20... more More than half of malpractice claims involve care in the ambulatory setting (Zuccotti and Sato 2011; Saber Tehrani et al. 2013). As the sole project of the seven AHRQ-funded Patient Safety and Medical Liability Demonstration Program grants to focus on outpatient malpractice safety and risk, the Massachusetts PROMISES (Proactive Reduction of Outpatient Malpractice: Improving Safety, Efficiency, and Satisfaction) project sought to identify and improve safety in what we referred to as "3 + 1" areas of known risk in the outpatient setting: test result, referral, and medication management plus overarching communication issues (Agency for Healthcare Research and Quality 2014). We assembled a coalition of key Massachusetts safety, regulatory, malpractice, and academic groups and recruited 25 small-and medium-sized primary care practices of which 16 were randomly selected to receive a multifaceted improvement intervention. While the quantitative outcomes and evaluation fell short of demonstrating some of the significant improvements we had hoped to show, here we describe how we developed and fielded our intervention, describing some of the lessons learned in the course of this project and implications for future efforts in this field. We chose to address the "3 + 1" outpatient safety risk areas based on their demonstrated role as both areas of vulnerability as well as potential for improvement. First, failed follow-up of abnormal tests has consistently been found to be a malpractice safety risk (Schiff 2006; Callen et al. 2012; McDonald et al. 2013). Second, based on earlier work, and data collected during the PROMISES project, problems in referral management also clearly represent problematic processes with "dropped balls" that pose malpractice risks. Baseline surveys of the practices consistently revealed that this was an area where

Research paper thumbnail of Systems engineering analysis of diagnostic referral closed-loop processes

BMJ Open Quality, 2021

BackgroundClosing loops to complete diagnostic referrals remains a significant patient safety pro... more BackgroundClosing loops to complete diagnostic referrals remains a significant patient safety problem in most health systems, with 65%–73% failure rates and significant delays common despite years of improvement efforts, suggesting new approaches may be useful. Systems engineering (SE) methods increasingly are advocated in healthcare for their value in studying and redesigning complex processes.ObjectiveConduct a formative SE analysis of process logic, variation, reliability and failures for completing diagnostic referrals originating in two primary care practices serving different demographics, using dermatology as an illustrating use case.MethodsAn interdisciplinary team of clinicians, systems engineers, quality improvement specialists, and patient representatives collaborated to understand processes of initiating and completing diagnostic referrals. Cross-functional process maps were developed through iterative group interviews with an urban community-based health centre and a te...

Research paper thumbnail of Table 4, Change ideas for preventing and minimizing diagnostic error

Research paper thumbnail of Health Information Technology Center for Education and Research on Therapeutics (HIT-CERT) - Final Report

Purpose: The purpose of the BWH HIT-CERT was to leverage HIT for pharmacosurveillance, medication... more Purpose: The purpose of the BWH HIT-CERT was to leverage HIT for pharmacosurveillance, medication-related clinical decision support, and to identify new ways to utilize information coming from medication-related decision support. Moreover, we strove to advance our understanding of how providers are responding to medication-related decision support. Scope: Within the three BWH HIT-CERT subprojects, our team examined questions relating to appropriateness, safety, and efficacy across diverse patient populations. The projects covered multiple settings including outpatients, inpatients and recently discharged inpatients. Methods: We completed three research projects over a five-year period, each of which addressed one or more of the programmatic interest areas including patient safety, development and enhancement of tools, health care system interventions, and translation into practice or policy. The three subprojects involved: 1) leveraging new technologies to improve pharmacosurveillan...

Research paper thumbnail of Figure 1, Relationships between diagnostic process errors, misdiagnosis, and adverse events

Research paper thumbnail of Association of Display of Patient Photographs in the Electronic Health Record With Wrong-Patient Order Entry Errors

Research paper thumbnail of Table 1, General medical error studies that reported errors in diagnosis

Research paper thumbnail of Table 2, Illustrative disease-specific studies of diagnosis errors

Research paper thumbnail of Personal Formularies of Primary Care Physicians Across 4 Health Care Systems

JAMA Network Open, 2021

IMPORTANCE More conservative prescribing has the potential to reduce adverse drug events and pati... more IMPORTANCE More conservative prescribing has the potential to reduce adverse drug events and patient harm and cost; however, no method exists defining the extent to which individual clinicians prescribe conservatively. One potential domain is prescribing a more limited number of drugs. Personal formularies-defined as the number and mix of unique, newly initiated drugs prescribed by a physician-may enable comparisons among clinicians, practices, and institutions. OBJECTIVES To develop a method of defining primary care physicians' personal formularies and examine how they differ among primary care physicians at 4 institutions; evaluate associations between personal formularies and patient, physician, and practice site characteristics; and empirically derive and examine the variability of the top 200 core drugs prescribed at the 4 sites. DESIGN, SETTING, AND PARTICIPANTS This retrospective cohort study was conducted at 4 US health care systems among 4655 internal and family medicine physicians and 4 930 707 patients who had at least 1 visit to these physicians between January 1, 2017, and December 31, 2018. EXPOSURES Personal formulary size was defined as the number of unique, newly initiated drugs. MAIN OUTCOMES AND MEASURES Personal formulary size and drugs used, physician and patient characteristics, core drugs, and analysis of selected drug classes. RESULTS The study population included 4655 primary care physicians (2274 women [48.9%]; mean [SD] age, 48.5 [4.4] years) and 4 930 707 patients (16.5% women; mean [SD] age, 51.9 [8.3] years). There were 41 378 903 outpatient prescriptions written, of which 9 496 766 (23.0%) were new starts. Institution median personal formulary size ranged from 150 (interquartile range, 82.0-212.0) to 296 (interquartile range, 230.0-347.0) drugs. In multivariable modeling, personal formulary size was significantly associated with panel size (total number of unique patients with faceto-face encounters during the study period; 1.2 medications per 100 patients), physician's total number of encounters (5.7 drugs per 10% increase), and physician's sex (−6.2 drugs per 100 patients for female physicians). There were 1527 unique, newly prescribed drugs across the 4 sites. Fewer than half the drugs (626 [41.0%]) were used at every site. Physicians' prescribing of drugs from a pooled core list varied from 0% to 100% of their prescriptions. CONCLUSIONS AND RELEVANCE Personal formularies, measured at the level of individual physicians and institutions, reveal variability in size and mix of drugs. Similarly, defining a list of commonly prescribed core drugs in primary care revealed interphysician and interinstitutional differences. Personal formularies and core medication lists enable comparisons and may identify outliers and opportunities for safer and more appropriate prescribing.

Research paper thumbnail of Comparison of a Prototype for Indications-Based Prescribing With 2 Commercial Prescribing Systems

Research paper thumbnail of A Prescription For Enhancing Electronic Prescribing Safety

Health Affairs, 2018

While electronic prescribing has been shown to reduce medication errors and improve prescribing s... more While electronic prescribing has been shown to reduce medication errors and improve prescribing safety, it is vulnerable to error-prone processes. We review six intersecting areas in which changes to electronic prescribing systems, particularly in the outpatient setting, could transform medication ordering quality and safety. We recommend incorporating medication indications into electronic prescribing, establishing a single shared online medication list, implementing the transmission of electronic cancellation orders to pharmacies (CancelRx) to ensure that drugs are safely and reliably discontinued, implementing standardized structured and codified prescription instructions, reengineering clinical decision support, and redesigning electronic prescribing to facilitate the ordering of nondrug alternatives.

Research paper thumbnail of What do Australian consumers, pharmacists and prescribers think about documenting indications on prescriptions and dispensed medicines labels?: A qualitative study

BMC health services research, Jan 15, 2017

Documenting the indication on prescriptions and dispensed medicines labels is not standard practi... more Documenting the indication on prescriptions and dispensed medicines labels is not standard practice in Australia. However, previous studies that have focused on the content and design of dispensed medicines labels, have suggested including the indication as a safety measure. The aim of this study was to investigate the perspectives of Australian consumers, pharmacists and prescribers on documenting the indication on prescriptions and dispensed medicines labels. Semi-structured interviews were conducted and mock-up of dispensed medicines labels were designed for participants. Consumers (n = 19) and pharmacists (n = 7) were recruited by convenience sample at community pharmacies in Sydney (Australia) and prescribers (n = 8), including two medical students, were recruited through snowballing. Thirty-four participants were interviewed. Most participants agreed that documenting the indication would be beneficial especially for patients who are forgetful or take multiple medications. Part...

Research paper thumbnail of Using drug knowledgebase information to distinguish between look-alike-sound-alike drugs

Journal of the American Medical Informatics Association : JAMIA, Jan 22, 2018

To extract drug indications from a commercial drug knowledgebase and determine to what extent dru... more To extract drug indications from a commercial drug knowledgebase and determine to what extent drug indications can discriminate between look-alike-sound-alike (LASA) drugs. We extracted drug indications disease concepts from the MedKnowledge Indications module from First Databank Inc. (South San Francisco, CA) and associated them with drugs on the Institute for Safe Medication Practices (ISMP) list of commonly confused drug names. We used high-level concepts (rather than granular concepts) to represent the general indications for each drug. Two pharmacists reviewed each drug's association with its high-level indications concepts for accuracy and clinical relevance. We compared the high-level indications for each commonly confused drug pair and categorized each pair as having a complete overlap, partial overlap or no overlap in high-level indications. Of 278 LASA drug pairs, 165 (59%) had no overlap and 58 (21%) had partial overlap in high-level indications. Fifty-five pairs (20%...

Research paper thumbnail of Reduced Effectiveness of Interruptive Drug-Drug Interaction Alerts after Conversion to a Commercial Electronic Health Record

Journal of general internal medicine, Jan 15, 2018

Drug-drug interaction (DDI) alerts in electronic health records (EHRs) can help prevent adverse d... more Drug-drug interaction (DDI) alerts in electronic health records (EHRs) can help prevent adverse drug events, but such alerts are frequently overridden, raising concerns about their clinical usefulness and contribution to alert fatigue. To study the effect of conversion to a commercial EHR on DDI alert and acceptance rates. Two before-and-after studies. 3277 clinicians who received a DDI alert in the outpatient setting. Introduction of a new, commercial EHR and subsequent adjustment of DDI alerting criteria. Alert burden and proportion of alerts accepted. Overall interruptive DDI alert burden increased by a factor of 6 from the legacy EHR to the commercial EHR. The acceptance rate for the most severe alerts fell from 100 to 8.4%, and from 29.3 to 7.5% for medium severity alerts (P < 0.001). After disabling the least severe alerts, total DDI alert burden fell by 50.5%, and acceptance of Tier 1 alerts rose from 9.1 to 12.7% (P < 0.01). Changing from a highly tailored DDI alerting...

Research paper thumbnail of Reasons for computerised provider order entry (CPOE)-based inpatient medication ordering errors: an observational study of voided orders

BMJ Quality & Safety, 2017

ObjectiveMedication voiding is a computerised provider order entry (CPOE)-based discontinuation m... more ObjectiveMedication voiding is a computerised provider order entry (CPOE)-based discontinuation mechanism that allows clinicians to identify erroneous medication orders. We investigated the accuracy of voiding as an indicator of clinician identification and interception of a medication ordering error, and investigated reasons and root contributors for medication ordering errors.MethodUsing voided orders identified with a void alert, we conducted interviews with ordering and voiding clinicians, followed by patient chart reviews. A structured coding framework was used to qualitatively analyse the reasons for medication ordering errors. We also compared clinician-CPOE-selected (at time of voiding), clinician-reported (interview) and chart review-based reasons for voiding.ResultsWe conducted follow-up interviews on 101 voided orders. The positive predictive value (PPV) of voided orders that were medication ordering errors was 93.1% (95% CI 88.1% to 98.1%, n=94). Using chart review-based...

Research paper thumbnail of Methodologies for evaluating strategies to reduce diagnostic error: report from the research summit at the 7th International Diagnostic Error in Medicine Conference

Diagnosis, 2016

In this article we review current evidence on strategies to evaluate diagnostic error solutions, ... more In this article we review current evidence on strategies to evaluate diagnostic error solutions, discuss the methodological challenges that exist in investigating the value of these strategies in patient care, and provide recommendations for methods that can be applied in investigating potential solutions to diagnostic errors. These recommendations were developed iteratively by the authors based upon initial discussions held during the Research Summit of the 7th Annual Diagnostic Error in Medicine Conference in September 2014. The recommendations include the following elements for designing studies of diagnostic research solutions: (1) Select direct and indirect outcomes measures of importance to patients, while also practical for the particular solution; (2) Develop a clearly-stated logic model for the solution to be tested; (3) Use rapid, iterative prototyping in the early phases of solution testing; (4) Use cluster-randomized clinical trials where feasible; (5) Avoid simple pre-p...

Research paper thumbnail of Exploring the potential for using drug indications to prevent look-alike and sound-alike drug errors

Expert Opinion on Drug Safety, 2017

Background. Look-alike, sound-alike (LASA) drug names are a cause of medication errors with resul... more Background. Look-alike, sound-alike (LASA) drug names are a cause of medication errors with resulting patient harm and healthcare costs. This study assessed to which extent the use of the generic drug name, therapeutic class, health problem, and the U.S. Food and Drug Administration (FDA)-approved indications might be used to differentiate LASA drug pairs. Research design and methods. We collected information about LASA drug pairs reported by the FDA to have look-alike sound-alike similarities. To assess potential for differentiating LASA drug pairs, we compared the following drug characteristics: generic name, therapeutic class, health problem, and FDA-approved indication. Results. For the 33 FDA reported LASA drug pairs we identified a total of 432 FDA-approved indications. Using the generic name, therapeutic class, health problem and drug indication we were able to differentiate 8 (24.2%), 24 (72.7%), 25 (75.8%) and 26 (78.8%), respectively of the 33 LASA drug pairs. Using the generic name, therapeutic class, and health problem we were able to distinguish 31 (7.2%), 212 (49.1%), and 269 (62.3%), respectively of the 432 FDA-approved indications for the LASA drug pairs. Conclusions. Including the FDA-approved indication in the drug prescription may be used to differentiate LASA drug pairs and thus, prevent wrong drug medication errors.

Research paper thumbnail of Learning from errors: analysis of medication order voiding in CPOE systems

Journal of the American Medical Informatics Association, 2017

Objective: Medication order voiding allows clinicians to indicate that an existing order was plac... more Objective: Medication order voiding allows clinicians to indicate that an existing order was placed in error. We explored whether the order voiding function could be used to record and study medication ordering errors. Materials and Methods: We examined medication orders from an academic medical center for a 6-year period (2006–2011; n = 5 804 150). We categorized orders based on status (void, not void) and clinician-provided reasons for voiding. We used multivariable logistic regression to investigate the association between order voiding and clinician, patient, and order characteristics. We conducted chart reviews on a random sample of voided orders (n = 198) to investigate the rate of medication ordering errors among voided orders, and the accuracy of clinician-provided reasons for voiding. Results: We found that 0.49% of all orders were voided. Order voiding was associated with clinician type (physician, pharmacist, nurse, student, other) and order type (inpatient, prescription,...

Research paper thumbnail of Incorporating Indications into Medication Ordering--Time to Enter the Age of Reason

The New England journal of medicine, Jan 28, 2016

Research paper thumbnail of Screening for medication errors using an outlier detection system

Journal of the American Medical Informatics Association : JAMIA, Mar 19, 2017

The study objective was to evaluate the accuracy, validity, and clinical usefulness of medication... more The study objective was to evaluate the accuracy, validity, and clinical usefulness of medication error alerts generated by an alerting system using outlier detection screening. Five years of clinical data were extracted from an electronic health record system for 747 985 patients who had at least one visit during 2012-2013 at practices affiliated with 2 academic medical centers. Data were screened using the system to detect outliers suggestive of potential medication errors. A sample of 300 charts was selected for review from the 15 693 alerts generated. A coding system was developed and codes assigned based on chart review to reflect the accuracy, validity, and clinical value of the alerts. Three-quarters of the chart-reviewed alerts generated by the screening system were found to be valid in which potential medication errors were identified. Of these valid alerts, the majority (75.0%) were found to be clinically useful in flagging potential medication errors or issues. A clinical...

Research paper thumbnail of Addressing Ambulatory Safety and Malpractice: The Massachusetts PROMISES Project

Health Services Research, 2016

More than half of malpractice claims involve care in the ambulatory setting (Zuccotti and Sato 20... more More than half of malpractice claims involve care in the ambulatory setting (Zuccotti and Sato 2011; Saber Tehrani et al. 2013). As the sole project of the seven AHRQ-funded Patient Safety and Medical Liability Demonstration Program grants to focus on outpatient malpractice safety and risk, the Massachusetts PROMISES (Proactive Reduction of Outpatient Malpractice: Improving Safety, Efficiency, and Satisfaction) project sought to identify and improve safety in what we referred to as "3 + 1" areas of known risk in the outpatient setting: test result, referral, and medication management plus overarching communication issues (Agency for Healthcare Research and Quality 2014). We assembled a coalition of key Massachusetts safety, regulatory, malpractice, and academic groups and recruited 25 small-and medium-sized primary care practices of which 16 were randomly selected to receive a multifaceted improvement intervention. While the quantitative outcomes and evaluation fell short of demonstrating some of the significant improvements we had hoped to show, here we describe how we developed and fielded our intervention, describing some of the lessons learned in the course of this project and implications for future efforts in this field. We chose to address the "3 + 1" outpatient safety risk areas based on their demonstrated role as both areas of vulnerability as well as potential for improvement. First, failed follow-up of abnormal tests has consistently been found to be a malpractice safety risk (Schiff 2006; Callen et al. 2012; McDonald et al. 2013). Second, based on earlier work, and data collected during the PROMISES project, problems in referral management also clearly represent problematic processes with "dropped balls" that pose malpractice risks. Baseline surveys of the practices consistently revealed that this was an area where