An overview of clinical decision support systems: benefits, risks, and strategies for success (original) (raw)

Abstract

Computerized clinical decision support systems, or CDSS, represent a paradigm shift in healthcare today. CDSS are used to augment clinicians in their complex decision-making processes. Since their first use in the 1980s, CDSS have seen a rapid evolution. They are now commonly administered through electronic medical records and other computerized clinical workflows, which has been facilitated by increasing global adoption of electronic medical records with advanced capabilities. Despite these advances, there remain unknowns regarding the effect CDSS have on the providers who use them, patient outcomes, and costs. There have been numerous published examples in the past decade(s) of CDSS success stories, but notable setbacks have also shown us that CDSS are not without risks. In this paper, we provide a state-of-the-art overview on the use of clinical decision support systems in medicine, including the different types, current use cases with proven efficacy, common pitfalls, and potential harms. We conclude with evidence-based recommendations for minimizing risk in CDSS design, implementation, evaluation, and maintenance.

Similar content being viewed by others

Introduction: What is a clinical decision support system?

A clinical decision support system (CDSS) is intended to improve healthcare delivery by enhancing medical decisions with targeted clinical knowledge, patient information, and other health information.1 A traditional CDSS is comprised of software designed to be a direct aid to clinical-decision making, in which the characteristics of an individual patient are matched to a computerized clinical knowledge base and patient-specific assessments or recommendations are then presented to the clinician for a decision.2 CDSSs today are primarily used at the point-of-care, for the clinician to combine their knowledge with information or suggestions provided by the CDSS. Increasingly however, there are CDSS being developed with the capability to leverage data and observations otherwise unobtainable or uninterpretable by humans.

Computer-based CDSSs can be traced to the 1970s. At the time, they had poor system integration, were time intensive and often limited to academic pursuits.3,4 There were also ethical and legal issues raised around the use of computers in medicine, physician autonomy, and who would be at fault when using the recommendation of a system with imperfect ‘explainability’.5 Presently, CDSS often make use of web-applications or integration with electronic health records (EHR) and computerized provider order entry (CPOE) systems. They can be administered through desktop, tablet, smartphone, but also other devices such as biometric monitoring and wearable health technology. These devices may or may not produce outputs directly on the device or be linked into EHR databases.[6](/articles/s41746-020-0221-y#ref-CR6 "Dias, D. Wearable health devices—vital sign monitoring, systems and technologies. https://doi.org/10.3390/s18082414

             (2018).")

CDSSs have been classified and subdivided into various categories and types, including intervention timing, and whether they have active or passive delivery.7,8 CDSS are frequently classified as knowledge-based or non-knowledge based. In knowledge-based systems, rules (IF-THEN statements) are created, with the system retrieving data to evaluate the rule, and producing an action or output7; Rules can be made using literature-based, practice-based, or patient-directed evidence.2 CDSS that are non-knowledge based still require a data source, but the decision leverages artificial intelligence (AI), machine learning (ML), or statistical pattern recognition, rather than being programmed to follow expert medical knowledge.7 Non-knowledge based CDSS, although a rapidly growing use case for AI in medicine, are rife with challenges including problems understanding the logic that AI uses to produce recommendations (black boxes), and problems with data availability.9 They have yet to reach widespread implementation. Both types of CDSS have common components with subtle differences, illustrated in Fig. 1.

Fig. 1: Diagram of key interactions in knowledge-based and non-knowledge based CDSS.

They are composed of (1) base: the rules that are programmed into the system (knowledge-based), the algorithm used to model the decision (non-knowledge based), as well as the data available, (2) inference engine: takes the programmed or AI-determined rules, and data structures, and applies them to the patient’s clinical data to generate an output or action, which is presented to the end user (eg. physician) through the (3) communication mechanism: the website, application, or EHR frontend interface, with which the end user interacts with the system9.

Full size image

CDSS have been endorsed by the US Government’s Health and Medicare acts, financially incentivizing CDS implementation into EHRs.[10](/articles/s41746-020-0221-y#ref-CR10 "HITECH Act Enforcement Interim Final Rule. https://www.hhs.gov/hipaa/for-professionals/special-topics/HITECH-act-enforcement-interim-final-rule/index.html

            .") In 2013, an estimated 41% of U.S. hospitals with an EHR, also had a CDSS, and in 2017, 40.2% of US hospitals had advanced CDS capability (HIMSS Stage 6).[11](/articles/s41746-020-0221-y#ref-CR11 "Electronic medical record adoption model requirements. 
              https://www.himssanalytics.org/emram
              
            , Accessed 25 Aug 2019 (2017).") Elsewhere, adoption rates of EMRs have been promising, with approximately 62% of practitioners in Canada in 2013.[12](/articles/s41746-020-0221-y#ref-CR12 "Chang, F. & Gupta, N. Progress in electronic medical record adoption in Canada Recherche Les progrès dans l’ adoption du dossier médical électronique au Canada. Canadian Family Physician. 61, 1076–1084 (2015).") Canada has had significant endorsement from the government level, as well as Infoway, a not-for-profit corporation.[13](/articles/s41746-020-0221-y#ref-CR13 "Healthcare Information and Management Systems Society (HIMSS). Electronic Health Records: A Global Perspective, 2nd edn. 
              https://s3.amazonaws.com/rdcms-himss/files/production/public/HIMSSorg/Content/files/Globalpt1-edited%20final.pdf
              
             (2010).") England has also been a world leader in healthcare IT investment, with up to 20 billion euros invested back in 2010.[13](/articles/s41746-020-0221-y#ref-CR13 "Healthcare Information and Management Systems Society (HIMSS). Electronic Health Records: A Global Perspective, 2nd edn. 
              https://s3.amazonaws.com/rdcms-himss/files/production/public/HIMSSorg/Content/files/Globalpt1-edited%20final.pdf
              
             (2010).") Several countries have also managed to implement national health records, at least for patient-facing data, including Denmark, Estonia, Australia, and others.[14](/articles/s41746-020-0221-y#ref-CR14 "Nøhr, C. et al. Nationwide citizen access to their health data: analysing and comparing experiences in Denmark, Estonia and Australia. 1–11. 
              https://doi.org/10.1186/s12913-017-2482-y
              
             (2017).")

The scope of functions provided by CDSS is vast, including diagnostics, alarm systems, disease management, prescription (Rx), drug control, and much more.15 They can manifest as computerized alerts and reminders, computerized guidelines, order sets, patient data reports, documentation templates, and clinical workflow tools.16 Each CDSS function will be discussed in detail throughout this review, with the potential and realized benefits of these functions, as well as unintended negative consequences, and strategies to avoid harm from CDSS. Methodology used to inform the review is shown in Box 1.

Functions and advantages of CDSS

Patient safety

Strategies to reduce medication errors commonly make use of CDSS (Table 1). Errors involving drug-drug interactions (DDI) are cited as common and preventable, with up to 65% of inpatients being exposed to one or more potentially harmful combinations.17 CPOE systems are now designed with drug safety software that has safeguards for dosing, duplication of therapies, and DDI checking.18 The types of alerts generated by these systems are among the most disseminated kind of decision support.19 However, studies have found a high level of variability between how alerts for DDIs are displayed (e.g., passive or active/disruptive), which are prioritized,20,21 and in the algorithms used to identify DDIs.18,22 Systems often have varying degrees of irrelevant alerts presented, and there is no standard for how best to implement which alerts to providers. The US Office of the National Coordinator for Health Information Technology has developed a list of ‘high-priority’ list of DDIs for CDS, which has reached various levels of endorsement and deployment in CDSS’ of other countries including the U.K., Belgium, and Korea.20,21,23

Table 1 Benefits of clinical decision support systems (CDSS), possible harms, and evidence-based mitigation strategies.

Full size table

Other systems targeting patient safety include electronic drug dispensing systems (EDDS), and bar-code point-of-care (BPOC) medication administration systems.24 These are often implemented together to create a ‘closed loop’, where each step of the process (prescribing, transcribing, dispensing, administering) is computerized and occurs within a connected system. At administration, the medication is automatically identified through radio-frequency identification (RFID) or barcodes and crosschecked with patient information and prescriptions. This presents another target for CDSS and the potential benefit is the prevention of medication administration errors occurring at the ‘bedside’ (opposed to further upstream). Adoption is relatively low, partly due to high technology requirements and costs.25 However; studies show good efficacy for these systems in reducing errors.26 Mohoney et al. showed that many of these systems can be combined with CPOE and CDSS simultaneously, with reduced prescribing error rates for drug allergy detection, excessive dosing, and incomplete or unclear ordering.24 As with most CDSS, errors can still be made if providers omit or deliberately work around the technology.27

CDSS also improve patient safety through reminder systems for other medical events, and not just those that are medication related. Among numerous examples, a CDSS for blood glucose measurement in the ICU was able to decrease the number of hypoglycemia events.28 This CDSS automatically prompted nurses to take a glucose measurement according to a local glucose monitoring protocol, which specified how often measurements should be done according to specific patient demographics and previous glucose levels/trends.28

Overall, CDSS targeting patient safety through CPOE and other systems have generally been successful in reducing prescribing and dosing errors, contraindications through automated warnings, drug-event monitoring and more.29 Patient safety can be considered a secondary objective (or requirement) of almost all types of CDSS, no matter the primary purpose for their implementation.

Clinical management

Studies have shown CDSS can increase adherence to clinical guidelines.30 This is significant because traditional clinical guidelines and care pathways have been shown to be difficult to implement in practice with low clinician adherance.31,32 The assumption that practitioners will read, internalize, and implement new guidelines has not held true.[33](/articles/s41746-020-0221-y#ref-CR33 "Shortliffe, T. Medical thinking: what should we do? http://www.openclinical.org/medicalThinking2006Summary2.html

             (2006).") However, the rules implicitly encoded in guidelines can be literally encoded into CDSS. Such CDSS can take a variety of forms, from standardized order sets for a targeted case, alerts to a specific protocol for the patients it pertains to, reminders for testing, etc. Furthermore, CDSS can assist with managing patients on research/treatment protocols,[34](/articles/s41746-020-0221-y#ref-CR34 "Lipton, J. A. et al. Impact of an alerting clinical decision support system for glucose control on protocol compliance and glycemic control in the intensive cardiac care unit. Diabetes Technol. Ther. 13, 343–349 (2011).") tracking and placing orders, follow-up for referrals, as well as ensuring preventative care.[35](/articles/s41746-020-0221-y#ref-CR35 "Salem, H. et al. A multicentre integration of a computer led follow up in surgical oncology is valid and safe. BJU Int. 
              https://doi.org/10.1111/bju.14157
              
             (2018).")

CDSS can also alert clinicians to reach out to patients who have not followed management plans, or are due for follow-up, and help identify patients eligible for research based on specific criteria.[36](/articles/s41746-020-0221-y#ref-CR36 "Health Information Technology Foundations Module 28: Clinical Decision Support Basics. Carnegie Mellon University Open Learning Initiative. https://oli.cmu.edu/jcourse/workbook/activity/page?context=e6f7c0b180020ca600c0f4e5957d6f8c

            .") A CDSS designed and implemented at Cleveland Clinic provides a point-of-care alert to physicians when a patient’s record matches clinical trial criteria.[37](/articles/s41746-020-0221-y#ref-CR37 "Embi, P. J., Jain, A., Clark, J. & Harris, C. M. Development of an electronic health record-based Clinical Trial Alert system to enhance recruitment at the point of care. AMIA Annu. Symp. Proc. 2005, 231–235 (2005).") The alert prompts the user to complete a form which establishes eligibility and consent-to-contact, forwards the patient’s chart to the study coordinator, and prints a clinical trial patient information sheet.

Cost containment

CDSS can be cost-effective for health systems, through clinical interventions,38 decreasing inpatient length-of-stay, CPOE-integrated systems suggesting cheaper medication alternatives,39 or reducing test duplication. A CPOE-rule was implemented in a pediatric cardiovascular intensive care unit (ICU) that limited the scheduling of blood count, chemistry and coagulation panels to a 24-h interval.40 This reduced laboratory resource utilization with a projected cost savings of $717,538 per year, without increasing length of stay (LOS), or mortality.

CDSS can notify the user of cheaper alternatives to drugs, or conditions that insurance companies will cover. In Germany, many inpatients are switched to drugs on hospital drug formularies. After finding that 1 in 5 substitutions were incorrect, Heidelberg hospital developed a drug-switch algorithm and integrated it into their existing CPOE system.41 The CDSS could switch 91.6% of 202 medication consultations automatically, with no errors, increasing safety, reducing workload and reducing cost for providers.

Administrative functions

CDSS provide support for clinical and diagnostic coding, ordering of procedures and tests, and patient triage. Designed algorithms can suggest a refined list of diagnostics codes to aid physicians in selecting the most suitable one(s). A CDSS was conceived to address inaccuracy of ICD-9 emergency department(ED) admission coding (ICD is International Statistical Classification of Diseases, standardized codes used to represent diseases and diagnoses).42 The tool used an anatomographical interface (visual, interactive representation of the human body) linked to ICD codes to help ED physicians accurately find diagnostic admission codes faster.

CDSS can directly improve quality of clinical documentation. An obstetric CDSS featured an enhanced prompting system, significantly improving documentation of indications for labor induction and estimated fetal weight, compared to control hospital.43 Documentation accuracy is important because it can directly aid clinical protocols. For example, a CDSS was implemented to ensure patients were properly vaccinated following splenectomy, to combat the increased risk of infections (including pneumococcal, Haemophilus influenzae, meningococcal, etc.) that comes with spleen removal. However, the authors found that 71% of patients with the term ‘splenectomy’ in their EHR did not have it documented on their problem list (which was what triggers the CDSS alert).44 A supplemental CDSS was then developed to enhance problem list documentation of splenectomy,[45](/articles/s41746-020-0221-y#ref-CR45 "McEvoy, D., Gandhi, T. K., Turchin, A. & Wright, A. Enhancing problem list documentation in electronic health records using two methods: the example of prior splenectomy. BMJ Qual Saf. https://doi.org/10.1136/bmjqs-2017-006707

             (2017).") and improve the utility of the original vaccination CDSS.

Diagnostics support

CDSS for clinical diagnosis are known as diagnostic decision support systems (DDSS). These systems have traditionally provided a computerized ‘consultation’ or filtering step, whereby they might be provided data/user selections, and then output a list of possible or probable diagnoses.[46](/articles/s41746-020-0221-y#ref-CR46 "Berner E. Clinical Decision Support Systems: Theory and Practice 3rd edn. https://doi.org/10.1007/978-0-387-38319-4

             (2016).") Unfortunately, DDSS have not had as much influence as other types of CDSS (yet) for reasons including negative physician perceptions and biases, poor accuracy (often due to gaps in data availability), and poor system integration requiring manual data entry.[47](/articles/s41746-020-0221-y#ref-CR47 "Berner, E. S. Diagnostic decision support systems: why aren’t they used more and what can we do about it? AMIA Annu. Symp. Proc. 2006, 1167–1168 (2006)."),[48](/articles/s41746-020-0221-y#ref-CR48 "Segal, M. M. et al. Experience with integrating diagnostic decision support software with electronic health records: benefits versus risks of information sharing. EGEMs Gener. Evid. Methods Improv. Patient Outcomes 5, 23 (2017).") The latter is improving with better EHR-integration and standardized vocabulary like Snomed Clinical Terms.

A good example of an effective DDSS is one which was created by Kunhimangalam et al.49 for diagnosis of peripheral neuropathy using fuzzy logic. Through 24 input fields which include symptoms and diagnostic test outputs, they achieved 93% accuracy compared to experts at identifying motor, sensory, mixed neuropathies, or normal cases. While this has great utility, especially in countries with less access to established clinical experts, there is also a desire for systems that can supplement specialist diagnostics. DXplain is an electronic reference based DDSS that provides probable diagnosis based on clinical manifestations.50 In a randomized control trial involving 87 family medicine residents, those randomized to use the system showed significantly higher accuracy (84% vs. 74%) on a validated diagnosis test involving 30 clinical cases.50

Given the known incidence of diagnostic errors, particularly in primary care,51 there is a lot of hope for CDSS and IT solutions to bring improvements to diagnosis.52 We are now seeing diagnostic systems being developed with non-knowledge-based techniques like machine learning, which may pave the way for more accurate diagnosis. The Babylon AI powered Triage and Diagnostic System in the U.K. is a good example of the potential, but also of the work that still has to be done before these systems are ready for primetime.53,54

Diagnostics support: imaging

Knowledge-based imaging CDSS are typically used for image ordering, where CDSS can aid radiologists in selecting the most appropriate test to run, providing reminders of best practice guidelines, or alerting contraindications to contrast, for example.55 An interventional CDS for image ordering at Virginia Mason Medical Center was shown to substantially decrease the utilization rate of lumbar MRI for low back pain, head MRI for headache, and sinus CT for sinusitis.56 The CDS required a series of questions to be answered by providers prior to image ordering (POC), to verify appropriateness. Importantly, if an image was denied, an alternative was suggested by the system. Another commercialized example is RadWise®, which guides clinicians to the most relevant imaging order by analyzing patient symptoms and matching them with a large database of diagnoses, while also providing appropriate use recommendations at the point of care.[57](/articles/s41746-020-0221-y#ref-CR57 "DSS Inc. Radiology Decision Support (RadWise®). https://www.dssinc.com/products/integrated-clinical-products/radwise-radiology-decision-support/

            .")

There is great interest in non-knowledge based CDS for enhanced imaging and precision radiology (‘radiomics’).58,[59](/articles/s41746-020-0221-y#ref-CR59 "Oakden-rayner, L. et al. Precision Radiology: predicting longevity using feature engineering and deep learning methods in a radiomics framework. Sci. Rep. https://doi.org/10.1038/s41598-017-01931-w

             (2017).") With images accounting for increasing amounts of medical data, but requiring extensive manual interpretation, providers need technologies to aid them in extracting, visualizing, and interpreting.[60](/articles/s41746-020-0221-y#ref-CR60 "From Invisible to Visible: IBM Demos AI to Radiologists. 
              https://www-03.ibm.com/press/us/en/pressrelease/51146.ws
              
            . Accessed Aug 2019 (2016).") AI technologies are proving capable of providing insights into data beyond what humans can.[61](/articles/s41746-020-0221-y#ref-CR61 "Greenspan, H., Ginneken van, B. & Summers, R. M. Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans. Med. Imaging 35, 1153–1159 (2016).") To do so, these technologies make use of advanced pixel recognition and image classification algorithms, most prominently: deep learning (DL).[62](/articles/s41746-020-0221-y#ref-CR62 "Suzuki, K. & Chen, Y. Artificial intelligence in decision support systems for diagnosis in medical imaging. 
              https://doi.org/10.1007/978-3-319-68843-5
              
             (2018).") IBM Watson Health, DeepMind, Google, and other companies are at the forefront, developing products for use in tumor detection,[63](/articles/s41746-020-0221-y#ref-CR63 "IBM Watson Health - IBM Watson for Oncology. 
              https://www.ibm.com/watson/health/oncology-and-genomics/oncology/
              
            . Accessed 25 Aug 2019 (2018).") medical imaging interpretation,[64](/articles/s41746-020-0221-y#ref-CR64 "Lunit Inc. 
              https://lunit.io/en/
              
            . Accessed Aug 2019 (2018).") diabetic retinopathy diagnosis,[65](/articles/s41746-020-0221-y#ref-CR65 "Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA - J. Am. Med. Assoc. 316, 2402–2410 (2016).") Alzheimer’s diagnosis through multimodal feature learning,[62](/articles/s41746-020-0221-y#ref-CR62 "Suzuki, K. & Chen, Y. Artificial intelligence in decision support systems for diagnosis in medical imaging. 
              https://doi.org/10.1007/978-3-319-68843-5
              
             (2018).") and countless more. IBM Watson’s ‘Eyes of Watson’, has been able to combine image recognition of a brain scan with text recognition of case descriptions to provide comprehensive decision support (or what IBM describes as a ‘cognitive assistant’).[60](/articles/s41746-020-0221-y#ref-CR60 "From Invisible to Visible: IBM Demos AI to Radiologists. 
              https://www-03.ibm.com/press/us/en/pressrelease/51146.ws
              
            . Accessed Aug 2019 (2016).")

Several projects have been able to demonstrate performance that is disputably ‘on par’ with human experts.65,66,67,[68](/articles/s41746-020-0221-y#ref-CR68 "Hannun, A. Y. et al. FOCUS | Letters FOCUS | Letters Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network letters | FOCUS letters | FOCUS. Nat. Med. https://doi.org/10.1038/s41591-018-0268-3

             (2019).") For example, Google’s team trained a deep convolutional neural network (CNN) to detect diabetic retinopathy (blood vessel damage in the eye) from a dataset of 130,000 retinal images with a very high sensitivity and specificity.[65](/articles/s41746-020-0221-y#ref-CR65 "Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA - J. Am. Med. Assoc. 316, 2402–2410 (2016).") The algorithms performance was on par with US board certified ophthalmologists. Another study just recently published by the Stanford group demonstrated a CNN for detecting arrhythmias on electrocardiogram that exceeded the accuracy (F1 and sensitivity with matched specificity) of the average cardiologist on all rhythm classes.[68](/articles/s41746-020-0221-y#ref-CR68 "Hannun, A. Y. et al. FOCUS | Letters FOCUS | Letters Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network letters | FOCUS letters | FOCUS. Nat. Med. 
              https://doi.org/10.1038/s41591-018-0268-3
              
             (2019).") With the current rate of progress, some experts controversially speculate that in 15–20 years, the majority of diagnostic imaging interpretation will be done (or at least pre-processed) by computers.[69](/articles/s41746-020-0221-y#ref-CR69 "Erickson, B. J. Machine Intelligence in Medical Imaging (Society for Imaging Informatics, SIIM, 2016).") For the time being however, we should think of these early systems as an addition or augmentation to a clinician’s available toolset.

Diagnostics support: laboratory and pathology

Another subset of diagnostics where CDSS can be useful is laboratory testing and interpretation. Alerts and reminders for abnormal lab results are simple and ubiquitous in EHR systems. CDSS can also extend the utility of lab-based tests for the purpose of avoiding riskier or more invasive diagnostics. In Hepatitis B and C testing, liver biopsies are considered the gold standard for diagnosis, while non-invasive lab tests are not accurate enough to be accepted. However; AI models are being developed that combine multiple tests (serum markers, imaging, and gene tests) to produce much greater accuracy.[70](/articles/s41746-020-0221-y#ref-CR70 "Keltch, B., Lin, Y. & Bayrak, C. Comparison of AI techniques for prediction of liver fibrosis in hepatitis patients patient facing systems. J. Med. Syst. https://doi.org/10.1007/s10916-014-0060-y

             (2014).") There is also application for CDSS as an interpretation tool where a test’s reference ranges are highly personalized, for example age, sex, or disease subtypes.[71](/articles/s41746-020-0221-y#ref-CR71 "Mørkrid, L. et al. Continuous age- and sex-adjusted reference intervals of urinary markers for cerebral creatine deficiency syndromes: a novel approach to the definition of reference intervals. Clin. Chem. 61, 760–768 (2015).")

Pathology reports are crucial as decision points for many other medical specialties. Some CDSS can be used for automated tumor grading. This was done for urinary bladder tumor grading and estimating recurrence, with up to 93% accuracy.72 The same has been done for brain tumor classification and grading.73 There are many other examples including computerized ECG analysis, automated arterial blood gas interpretation, protein electrophoresis reports, and CDSS for blood cell counting.[46](/articles/s41746-020-0221-y#ref-CR46 "Berner E. Clinical Decision Support Systems: Theory and Practice 3rd edn. https://doi.org/10.1007/978-0-387-38319-4

             (2016).")

Patient-facing decision support

With the advent of the ‘Personal Health Record’ (PHR), we are seeing CDS functionality integrated, similar to EHRs, with the patient as the end user or ‘manager’ of the data. This is a great step towards patient-focused care, and CDS-supported PHRs are the ideal tool to implement shared decision-making between patient and provider, specifically because CDSS can remove a ‘lack of information’ as a barrier to a patient’s participation in their own care.74 PHRs are frequently designed as an extension of commercial EHR software, or as standalone web-based or mobile-based applications.75 When connected to EHRs, PHRs can have a two way relationship, whereby information entered directly by the patient can be available to their providers, and also information in the EHR can be transmitted to the PHR for patients to view.76

One of the earliest PHRs, the “Patient Gateway”, was simply a dashboard for patients to view medications and labs, and communicate with their physicians.77 This has expanded and some systems now allow patients to modify their own record of care, effecting the EHR data as well.78 Another example is Vanderbilt University’s MyHealthAtVanderbilt, a PHR fully integrated into the institutional EHR. In addition to disease-targeted delivery of patient educational materials, they incorporated a Flu Tool for patients with flu-like symptoms to decide the level of care they need and then help them seek treatment.79 Symptom tracking is a useful and common feature of PHRs, but the variety of collected data is virtually limitless, from allergies to insurance coverage to prescription and medication information.[80](/articles/s41746-020-0221-y#ref-CR80 "Roehrs, A., Da Costa, C. A., Da Rosa Righi, R. & De Oliveira, K. S. F. Personal health records: A systematic literature review. J. Med. Internet Res. https://doi.org/10.2196/jmir.5876

             (2017).") Furthermore, PHRs and other patient monitoring applications can be designed to collect information from health devices and other wearables, to create actionable insights for providers. An excellent example exists in diabetes care. Many systems are already in use,[81](/articles/s41746-020-0221-y#ref-CR81 "Benhamou, P. Y. Improving diabetes management with electronic health records and patients’ health records. Diabetes Metab. 37(Suppl. 4), S53–S56 (2011).") but one in particular pioneered by the Stanford School of Medicine uses a wearable glucose monitor which transmits data to an Apple device (HealthKit).[82](/articles/s41746-020-0221-y#ref-CR82 "Kumar, R. B., Goren, N. D., Stark, D. E., Wall, D. P. & Longhurst, C. A. Automated integration of continuous glucose monitor data in the electronic health record using consumer technology. J. Am. Med. Inf. Assoc. 23, 532–537 (2016).") Apple has made HealthKit interoperable with the Epic EHR and Epic PHR, “MyChart”. This successfully allows providers to monitor glucose trends in their patients in between visits, and contact them through MyChart for follow up or urgent recommendations. The pilot study demonstrated improved provider workflow, communication with patients, and ultimately quality of care.[82](/articles/s41746-020-0221-y#ref-CR82 "Kumar, R. B., Goren, N. D., Stark, D. E., Wall, D. P. & Longhurst, C. A. Automated integration of continuous glucose monitor data in the electronic health record using consumer technology. J. Am. Med. Inf. Assoc. 23, 532–537 (2016).") Various other medical fields are deploying similar systems for monitoring that combines PHR/EHR, wearable technologies, and CDSS, including but not limited to heart failure (cardiology), hypertension, sleep apnea, palliative/elder care, and more.

It is worth noting that as PHRs have become more advanced with CDSS capabilities, there has also been increasing emphasis on the design of these systems to serve shared decision making between patient and provider, and to be interactive tools to make patients more knowledgeable/involved in their own care. PHRs that only serve as a repository for health information are now seen as missing the mark, particularly by patients themselves.75

Pitfalls of CDSS

Fragmented workflows

CDSS can disrupt clinician workflow, especially in the case of stand-alone systems. Many early CDSS were designed as systems that required the provider to document or source information outside their typical workspace. CDSS also disrupt workflow if designed without human information processing and behaviors in mind. In response, CDSS have been designed using the ‘think-aloud’ method to model practitioners’ workflow and create a system with better usability.83

Disrupted workflow can lead to increased cognitive effort, more time required to complete tasks, and less time face-to-face with patients. Even when CDSS are well integrated within existing information systems, there can be disconnect between face-to-face interactions and interaction with a computer workstation. Studies have found that practitioners with more experiential knowledge are less likely to use, and more likely to override CDSS.84

Alert fatigue and inappropriate alerts

Studies have found up to 95% of CDSS alerts are inconsequential, and often times physicians disagree with or distrust alerts.85 Other times they just do not read them. If physicians are presented with excessive/unimportant alerts, they can suffer from alert fatigue.86

Disruptive alerts should be limited to more life-threatening or consequential contraindications, such as serious allergies. However; even allergy alerts can be incorrect, and clinicians will often verify themselves, especially if the source is another site/hospital/practitioner.85,[87](/articles/s41746-020-0221-y#ref-CR87 "Van Laere, S. et al. Clinical decision support systems for drug allergy checking: systematic review. https://doi.org/10.2196/jmir.8206

             (2018).") Medication alerts can also be specialty specific, but irrelevant when taken out of context. For example, an alert against using broad-spectrum antibiotics such as vancomycin may be inappropriate in ICU.[85](/articles/s41746-020-0221-y#ref-CR85 "Ash, J. S., Sittig, D. F., Campbell, E. M., Guappone, K. P. & Dykstra, R. H. Some unintended consequences of clinical decision support systems. AMIA Annu Symp. Proc. AMIA Symp. AMIA Symp. 2007, 26–30 (2007).") An alert against duplicate medications may be inappropriate in inflammatory bowel disease clinics, where the same class of drug can be applied through different administration routes for increased effect.

Impact on user skill

Prior to CPOE and CDSS, healthcare providers, pharmacists, and nurses were relied upon exclusively to double-check orders. CDSS can create the impression that verifying the accuracy of an order is unnecessary or automatic.85 This is an important myth to dispel.

It is also important to consider the potential long-term effect of a CDSS on users. Over time a CDSS can exert a training effect, so that the CDSS itself may no longer be required. Coined the “carry-over effect”, it is most likely with CDSS that are educational in nature.[88](/articles/s41746-020-0221-y#ref-CR88 "Wyatt, J. & Spiegelhalter, D. Field trials of medical decision-aids: potential problems and solutions. American Medical Informatics Association. 3–7. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2247484/

             (1991).") Conversely, providers may develop too much reliance or trust on a CDSS for a specific task.[89](/articles/s41746-020-0221-y#ref-CR89 "Goddard, K., Roudsari, A. & Wyatt, J. Automation bias - A hidden issue for clinical decision support system use. Stud. Health Technol. Inform. 164, 17–22 (2011).") This could be compared to using a calculator for mathematical operations over a long period of time, and then having poorer mental math skills. It is potentially problematic as the user has less independence and will be less equipped for that task should they switch to an environment without the CDSS.

CDSS may be dependent on computer literacy

Lack of technological proficiency can be hindering when engaging with a CDSS.[90](/articles/s41746-020-0221-y#ref-CR90 "Devaraj, S., Sharma, S. K., Fausto, D. J., Viernes, S. & Kharrazi, H. Barriers and facilitators to clinical decision support systems adoption: a systematic review. J. Bus Adm. Res. https://doi.org/10.5430/jbar.v3n2p36

             (2014)."),[91](/articles/s41746-020-0221-y#ref-CR91 "Leslie, S. J. et al. Clinical decision support software for management of chronic heart failure: Development and evaluation. Comput Biol. Med. 36, 495–506 (2006).") This can vary by the design details of the CDSS, but some have been found to be overly complex, relying too much on user skill.[90](/articles/s41746-020-0221-y#ref-CR90 "Devaraj, S., Sharma, S. K., Fausto, D. J., Viernes, S. & Kharrazi, H. Barriers and facilitators to clinical decision support systems adoption: a systematic review. J. Bus Adm. Res. 
              https://doi.org/10.5430/jbar.v3n2p36
              
             (2014)."),[92](/articles/s41746-020-0221-y#ref-CR92 "Murray, E. et al. Why is it difficult to implement e-health initiatives? A qualitative study. Implement Sci. 6, 6 (2011).") Systems should aim to stay as close to the core functionality of the pre-existing system as possible. Regardless, all new systems have a learning period, and so baseline evaluations of users’ technological competence may be appropriate. Further training can then be provided to facilitate full use of CDSS capabilities,[93](/articles/s41746-020-0221-y#ref-CR93 "Lai, F., Macmillan, J., Daudelin, D. H. & Kent, D. M. The potential of training to increase acceptance and use of computerized decision support systems for medical diagnosis. Hum. Factors J. Hum. Factors Erg. Soc. 48, 95–108 (2006).") or more explicit guidance incorporated into the CDSS’ recommendations themselves.[94](/articles/s41746-020-0221-y#ref-CR94 "Ojeleye, L. Ensuring effective computerised clinical decision support. Prescriber 27, 54–56 (2016).") This information could be implemented as info buttons to be non-disruptive.[95](/articles/s41746-020-0221-y#ref-CR95 "Cook, D. A., Teixeira, M. T., Heale, B. S. E., Cimino, J. J. & Del Fiol, G. Context-sensitive decision support (infobuttons) in electronic health records: a systematic review. J. Am. Med Inf. Assoc. 24, 460–468 (2017).")

System and content maintenance

Maintenance of CDSS is an important but often neglected part of the CDSS life-cycle. This includes technical maintenance of systems, applications and databases that power the CDSS. Another challenge is the maintenance of knowledge-base and its rules, which must keep apace with the fast-changing nature of medical practice and clinical guidelines. Even the most advanced healthcare institutions report difficulty keeping their systems up to date as knowledge inevitably changes.85 Order sets and the algorithmic rules behind the CDSS have been identified as particularly difficult.85

Operational impact of poor data quality and incorrect content

EHRs and CDSSs rely on data from external, dynamic systems and this can create novel deficiencies. As an example, some CDSS modules might encourage ordering even when the hospital lacks adequate supplies. In a study by Ash et al.85, a number of experts indicated that at their hospital, Hemoccult tests or pneumococcal vaccine inventories run out quickly, but this is not communicated to the CDSS.

Medication and problem lists can be problematic, if not updated or used appropriately. At one site, the medication list might be a list of dispensations, which means patients may or may not be taking them(and thus must still be asked in person).85 Other medication lists are generated from CPOE orders only, thus still requiring manual confirmation that patients are taking the medication. Systems that make it easy to distinguish these are ideal. It is also a major area where PHRs could create a solution, by collecting medication adherence data directly from patients.

In poorly designed systems, users may develop workarounds that compromise data, such as entering generic or incorrect data.85 The knowledge base of CDSS is dependent on a centralized, large clinical data repository. Quality of data can affect quality of decision support. If data collection or input into the system is unstandardized, the data is effectively corrupted. You may design a system for use at the point-of-care, but when applied to real world environments and data, will not be utilized properly. The importance of using informational standards such as ICD, SNOMED, and others, cannot be understated.

Lack of transportability and interoperability

Despite ongoing development for the better part of three decades, CDSS (and even EHRs in general) suffer from interoperability issues. Many CDSS exist as cumbersome stand-alone systems, or exist in a system that cannot communicate effectively with other systems.

What makes transportability so difficult to achieve? Beyond programming complexities that can make integration difficult, the diversity of clinical data sources is a challenge.96 There is a reluctance or perceived risk associated with transporting sensitive patient information. Positively, interoperability standards are continuously being developed and improved, such as Health Level 7 (HL7) and Fast Healthcare Interoperability Resources (FHIR). These are already being utilized in commercial EHR vendors.[97](/articles/s41746-020-0221-y#ref-CR97 "Index - FHIR v3.0.1. (2018). https://www.hl7.org/fhir/index.html

            . Accessed July 2019.") Several government agencies, medical organizations and informatics bodies are actively supporting and some even mandating the use of these interoperability standards in health systems.[98](#ref-CR98 "Katehakis, D. G. Towards the Development of a National eHealth Interoperability Framework to Address Public Health Challenges in Greece. Proceedings of the First International Workshop on Semantic Web Technologies for Health Data Management, SWH@ISWC. 2164, 1–9 (2018)."),[99](#ref-CR99 "EHRIntelligence. 5 Ways States Mandate Health Information Exchange. 
              https://ehrintelligence.com/news/5-ways-states-mandate-health-information-exchange
              
            . Accessed Aug 2019 (2015)."),[100](/articles/s41746-020-0221-y#ref-CR100 "European Commission Report. Commission Recommendation on a European Electronic Health Record Exchange Format (C(2019)800) of 6 February 2019. 
              https://ec.europa.eu/digital-single-market/en/news/recommendation-european-electronic-health-record-exchange-form
              
             (2019).")

The cloud also offers a potential solution to interoperability (and other EHR ailments such as data sync, software updating, etc.[101](/articles/s41746-020-0221-y#ref-CR101 "Bresnick, J. HealthITAnalytics. Interoperability, Low Costs Make Cloud-Based EHRs a Favorite. https://healthitanalytics.com/news/interoperability-low-costs-make-cloud-based-ehrs-a-favorite

            . Accessed July 2019 (2015).")). Cloud EHRs have open architecture, newer standards, and more flexible connectivity to other systems.[102](/articles/s41746-020-0221-y#ref-CR102 "Fernández-Cardeñosa, G., De La Torre-Díez, I., López-Coronado, M. & Rodrigues, J. J. P. C. Analysis of cloud-based solutions on EHRs systems in different scenarios. J. Med. Syst. 36, 3777–3782 (2012).") It is also a common misconception that data stored on a cloud is more vulnerable. This is not necessarily true. Web-based EHRs are required to store data in high-level storage centers with advanced encryption and other safeguards. They must comply with national data security standards including the Health Insurance Portability and Accountability Act (HIPAA) in the USA, Personal Information Protection and Electronic Documents Act (PIPEDA) in Canada, or the Data Protection Directive and General Data Protection Regulation (GDPR) in Europe, to name a few.[103](/articles/s41746-020-0221-y#ref-CR103 "Rodrigues, J. J. P. C., de la Torre, I., Fernández, G. & López-Coronado, M. Analysis of the security and privacy requirements of cloud-based electronic health records systems. J. Med. Internet Res. 15, e186 (2013).") They can be just as safe (or just as vulnerable) as traditional, server-based architecture.[103](/articles/s41746-020-0221-y#ref-CR103 "Rodrigues, J. J. P. C., de la Torre, I., Fernández, G. & López-Coronado, M. Analysis of the security and privacy requirements of cloud-based electronic health records systems. J. Med. Internet Res. 15, e186 (2013).") In fact, there are often fewer people who have access to unencrypted data in cloud storage centers vs. server-based records.[103](/articles/s41746-020-0221-y#ref-CR103 "Rodrigues, J. J. P. C., de la Torre, I., Fernández, G. & López-Coronado, M. Analysis of the security and privacy requirements of cloud-based electronic health records systems. J. Med. Internet Res. 15, e186 (2013).")

Financial challenges

Up to 74% of those with a CDSS said that financial viability remains a struggle.104 Outset costs to set up and integrate new systems can be substantial. Ongoing costs can continue to be an issue indefinitely as new staff need to be trained to use the system, and system updates are required to keep pace with current knowledge.

Results from cost analyses of CDSS implementations are mixed, controversial, and sparse.105,106,107,108 Whether an intervention is cost-effective depends on a wide range of factors, including those specific to the environment, both political and technological.105 Cost benefit assessment in itself can be limited, with challenges such as a lack of standardized metrics.107 This is an emerging research area and much work needs to be done to advance our understanding of the financial effects of CDSS.

Conclusion

CDSS have been shown to augment healthcare providers in a variety of decisions and patient care tasks, and today they actively and ubiquitously support delivery of quality care. Some applications of CDSS have more evidence behind them, especially those based on CPOE. Support for CDSS continues to mount in the age of the electronic medical record, and there are still more advances to be made including interoperability, speed and ease of deployment, and affordability. At the same time, we must stay vigilant for potential downfalls of CDSS, which range from simply not working and wasting resources, to fatiguing providers and compromising quality of patient care. Extra precautions and conscientious design must be taken when building, implementing, and maintaining CDSS. A portion of these considerations were covered in this review, but further review will be required in practice, especially as CDSS continue to evolve in complexity through advances in AI, interoperability, and new sources of data.

References

1. Osheroff, J. et al. Improving Outcomes with Clinical Decision Support: An Implementer’s Guide. (HIMSS Publishing, 2012).
2. Sim, I. et al. Clinical decision support systems for the practice of evidence-based medicine. J. Am. Med Inf. Assoc. Jamia. 8, 527–534 (2001).
   Article CAS Google Scholar
3. De Dombal, F. Computers, diagnoses and patients with acute abdominal pain. Arch. Emerg. Med. 9, 267–270 (1992).
4. Shortliffe, E. H. & Buchanan, B. G. A model of inexact resoning in medicine. Math. Biosci. 379, 233–262 (1975).
   Google Scholar
5. Middleton, B., Sittig, D. F. & Wright, A. Clinical decision support: a 25 year retrospective and a 25 year vision. Yearb. Med. Inform. 25(S 01), S103–S116 (2016).
   Article Google Scholar
6. Dias, D. Wearable health devices—vital sign monitoring, systems and technologies. https://doi.org/10.3390/s18082414 (2018).
7. Berner, E. S. (Ed.). Clinical Decision Support Systems (Springer, New York, NY, 2007).
8. Osheroff, J., Pifer, E., Teigh, J., Sittig, D. & Jenders, R. Improving Outcomes with Clinical Decision Support: An Implementer’s Guide. (HIMS, 2005).
9. Deo, R. C. Machine learning in medicine. Circulation 132, 1920–1930 (2015).
   Article PubMed PubMed Central Google Scholar
10. HITECH Act Enforcement Interim Final Rule. https://www.hhs.gov/hipaa/for-professionals/special-topics/HITECH-act-enforcement-interim-final-rule/index.html.
11. Electronic medical record adoption model requirements. https://www.himssanalytics.org/emram, Accessed 25 Aug 2019 (2017).
12. Chang, F. & Gupta, N. Progress in electronic medical record adoption in Canada Recherche Les progrès dans l’ adoption du dossier médical électronique au Canada. Canadian Family Physician. 61, 1076–1084 (2015).
13. Healthcare Information and Management Systems Society (HIMSS). Electronic Health Records: A Global Perspective, 2nd edn. https://s3.amazonaws.com/rdcms-himss/files/production/public/HIMSSorg/Content/files/Globalpt1-edited%20final.pdf (2010).
14. Nøhr, C. et al. Nationwide citizen access to their health data: analysing and comparing experiences in Denmark, Estonia and Australia. 1–11. https://doi.org/10.1186/s12913-017-2482-y (2017).
15. Omididan, Z. & Hadianfar, A. The role of clinical decision support systems in healthcare (1980-2010): a systematic review study. Jentashapir Sci.-Res Q. 2, 125–134 (2011).
    Google Scholar
16. Kabane, S. M. Healthcare and the Effect of Technology: Developments, Challenges and Advancements: Developments, Challenges and Advancements. Medical Information Science Reference (2010).
17. Vonbach, P., Dubied, A., Krähenbühl, S. & Beer, J. H. Prevalence of drug-drug interactions at hospital entry and during hospital stay of patients in internal medicine. Eur. J. Intern. Med. 19, 413–420 (2008).
    Article PubMed Google Scholar
18. Helmons, P. J., Suijkerbuijk, B. O., Nannan Panday, P. V. & Kosterink, J. G. W. Drug-drug interaction checking assisted by clinical decision support: a return on investment analysis. J. Am. Med. Inf. Assoc. Jamia. 22, 764–772 (2015).
    Article Google Scholar
19. Koutkias, V. & Bouaud, J. Contributions from the 2017 Literature on Clinical Decision Support. Yearb. Med. Inf. 27, 122–128 (2018).
    Article CAS Google Scholar
20. Phansalkar, S. et al. High-priority drug – drug interactions for use in electronic health records. J. Am. Med. Inform. Assoc. 19, 735–743 (2012).
21. Cornu, P., Phansalkar, S., Seger, D. L., Cho, I. & Pontefract, S. International Journal of Medical Informatics High-priority and low-priority drug – drug interactions in di ff erent international electronic health record systems: a comparative study. Int. J. Med. Inf. 111, 165–171 (2018).
    Article Google Scholar
22. McEvoy, D. S. et al. Variation in high-priority drug-drug interaction alerts across institutions and electronic health records. J. Am. Med. Inf. Assoc. 24, 331–338 (2017).
    Article Google Scholar
23. Cho, I., Lee, J., Choi, J., Hwang, H. & Bates, D. W. National rules for drug-drug interactions: are they appropriate for tertiary hospitals?. J. Korean Med. Sci. 31, 1887–1896 (2016).
    Article PubMed PubMed Central Google Scholar
24. Mahoney, C. D., Berard-Collins, C. M., Coleman, R., Amaral, J. F. & Cotter, C. M. Effects of an integrated clinical information system on medication safety in a multi-hospital setting. Am. J. Health Syst. Pharm. 64, 1969–1977 (2007).
    Article PubMed Google Scholar
25. Peris-Lopez, P., Orfila, A., Mitrokotsa, A. & van der Lubbe, J. C. A. A comprehensive RFID solution to enhance inpatient medication safety. Int. J. Med. Inf. 80, 13–24 (2011).
    Article Google Scholar
26. Levtzion-korach, O. et al. Effect of bar-code technology on the safety of medication administration. N. Engl. J. Med. 362, 1698–1707 (2010).
    Article PubMed Google Scholar
27. van der Veen, W. et al. Association between workarounds and medication administration errors in bar-code-assisted medication administration in hospitals. J. Am. Med. Inf. Assoc. 25, 385–392 (2018).
    Article Google Scholar
28. Eslami, S. et al. Effects of two different levels of computerized decision support on blood glucose regulation in critically ill patients. Int J. Med. Inf. 81, 53–60 (2012).
    Article Google Scholar
29. Jia, P., Zhang, L., Chen, J., Zhao, P. & Zhang, M. The effects of clinical decision support systems on medication safety: an overview. PLoS ONE 11, 1–17 (2016).
    Google Scholar
30. Kwok, R., Dinh, M., Dinh, D. & Chu, M. Improving adherence to asthma clinical guidelines and discharge documentation from emergency departments: Implementation of a dynamic and integrated electronic decision support system. Emerg. Med. Australas. 21, 31–37 (2009).
    PubMed Google Scholar
31. Davis, D. A. & Taylor-Vaisey, A. Translating guidelines into practice: a systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. Can. Med. Assoc. J. 157, 408–416 (1997).
    CAS Google Scholar
32. Michael, C., Rand, C. S., Powe, N. R., Wu, A. W. & Wilson, M. H. Why don’ t physicians follow clinical practice guidelines? a framework for improvement. Jama,. 282, 1458–1465 (1999).
    Article Google Scholar
33. Shortliffe, T. Medical thinking: what should we do? http://www.openclinical.org/medicalThinking2006Summary2.html (2006).
34. Lipton, J. A. et al. Impact of an alerting clinical decision support system for glucose control on protocol compliance and glycemic control in the intensive cardiac care unit. Diabetes Technol. Ther. 13, 343–349 (2011).
    Article PubMed Google Scholar
35. Salem, H. et al. A multicentre integration of a computer led follow up in surgical oncology is valid and safe. BJU Int. https://doi.org/10.1111/bju.14157 (2018).
36. Health Information Technology Foundations Module 28: Clinical Decision Support Basics. Carnegie Mellon University Open Learning Initiative. https://oli.cmu.edu/jcourse/workbook/activity/page?context=e6f7c0b180020ca600c0f4e5957d6f8c.
37. Embi, P. J., Jain, A., Clark, J. & Harris, C. M. Development of an electronic health record-based Clinical Trial Alert system to enhance recruitment at the point of care. AMIA Annu. Symp. Proc. 2005, 231–235 (2005).
38. Calloway, S., Akilo, H. & Bierman, K. Impact of a clinical decision support system on pharmacy clinical interventions, documentation efforts, and costs. Hosp. Pharm. 48, 744–752 (2013).
    Article PubMed PubMed Central Google Scholar
39. McMullin, S. T. et al. Impact of an evidence-based computerized decision support system on primary care prescription costs. Ann. Fam. Med. 2, 494–498 (2004).
    Article PubMed PubMed Central Google Scholar
40. Algaze, C. A. et al. Use of a checklist and clinical decision support tool reduces laboratory use and improves cost. Pediatrics 137, e20143019 (2016).
    Article Google Scholar
41. Pruszydlo, M. G., Walk-Fritz, S. U., Hoppe-Tichy, T., Kaltschmidt, J. & Haefeli, W. E. Development and evaluation of a computerised clinical decision support system for switching drugs at the interface between primary and tertiary care. BMC Med. Inf. Decis. Mak. 12, 1 (2012).
    Article Google Scholar
42. Bell, C. M., Jalali, A. & Mensah, E. A decision support tool for using an ICD-10 anatomographer to address admission coding inaccuracies: a commentary. Online J. Public Health Inform. 5, 222 (2013).
    Article PubMed PubMed Central Google Scholar
43. Haberman, S. et al. Effect of clinical-decision support on documentation compliance in an electronic medical record. Obstet. Gynecol. 114, 311–317 (2009).
    Article PubMed Google Scholar
44. Turchin, A., Shubina, M. & Gandhi, T. NLP for patient safety: splenectomy and pneumovax. In Proc. AMIA 2010 Annual Symposium (2010).
45. McEvoy, D., Gandhi, T. K., Turchin, A. & Wright, A. Enhancing problem list documentation in electronic health records using two methods: the example of prior splenectomy. BMJ Qual Saf. https://doi.org/10.1136/bmjqs-2017-006707 (2017).
46. Berner E. Clinical Decision Support Systems: Theory and Practice 3rd edn. https://doi.org/10.1007/978-0-387-38319-4 (2016).
47. Berner, E. S. Diagnostic decision support systems: why aren’t they used more and what can we do about it? AMIA Annu. Symp. Proc. 2006, 1167–1168 (2006).
48. Segal, M. M. et al. Experience with integrating diagnostic decision support software with electronic health records: benefits versus risks of information sharing. EGEMs Gener. Evid. Methods Improv. Patient Outcomes 5, 23 (2017).
    Article Google Scholar
49. Kunhimangalam, R., Ovallath, S. & Joseph, P. K. A clinical decision support system with an integrated EMR for diagnosis of peripheral neuropathy. J. Med. Syst. 38, 38 (2014).
    Article PubMed Google Scholar
50. Martinez-Franco, A. I. et al. Diagnostic accuracy in Family Medicine residents using a clinical decision support system (DXplain): a randomized-controlled trial. Diagn. Berl. Ger. 5, 71–76 (2018).
    Article Google Scholar
51. Singh, H., Schiff, G. D., Graber, M. L., Onakpoya, I. & Thompson, M. J. The global burden of diagnostic errors in primary care. BMJ Qual. Saf. 26, 484–494 (2017).
    Article PubMed Google Scholar
52. Singh, H., Meyer, A. N. D. & Thomas, E. J. The frequency of diagnostic errors in outpatient care: Estimations from three large observational studies involving US adult populations. BMJ Qual. Saf. 23, 727–731 (2014).
    Article PubMed PubMed Central Google Scholar
53. Razzaki, S. et al. A comparative study of artificial intelligence and human doctors for the purpose of triage and diagnosis. arXiv preprint arXiv:1806.10698 (2018).
54. Fraser, H., Coiera, E. & Wong, D. Safety of patient-facing digital symptom checkers. Lancet 392, 2263–2264 (2018).
    Article PubMed Google Scholar
55. Georgiou, A., Prgomet, M., Markewycz, A., Adams, E. & Westbrook, J. I. The impact of computerized provider order entry systems on medical-imaging services: a systematic review. J. Am. Med. Inf. Assoc. 18, 335–340 (2011).
    Article Google Scholar
56. Blackmore, C. C., Mecklenburg, R. S. & Kaplan, G. S. Effectiveness of clinical decision support in controlling inappropriate imaging. JACR 8, 19–25 (2019).
    Google Scholar
57. DSS Inc. Radiology Decision Support (RadWise®). https://www.dssinc.com/products/integrated-clinical-products/radwise-radiology-decision-support/.
58. Giardino, A. et al. Role of imaging in the era of precision medicine. Acad. Radiol. 24, 639–649 (2017).
    Article PubMed Google Scholar
59. Oakden-rayner, L. et al. Precision Radiology: predicting longevity using feature engineering and deep learning methods in a radiomics framework. Sci. Rep. https://doi.org/10.1038/s41598-017-01931-w (2017).
60. From Invisible to Visible: IBM Demos AI to Radiologists. https://www-03.ibm.com/press/us/en/pressrelease/51146.ws. Accessed Aug 2019 (2016).
61. Greenspan, H., Ginneken van, B. & Summers, R. M. Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans. Med. Imaging 35, 1153–1159 (2016).
    Article Google Scholar
62. Suzuki, K. & Chen, Y. Artificial intelligence in decision support systems for diagnosis in medical imaging. https://doi.org/10.1007/978-3-319-68843-5 (2018).
63. IBM Watson Health - IBM Watson for Oncology. https://www.ibm.com/watson/health/oncology-and-genomics/oncology/. Accessed 25 Aug 2019 (2018).
64. Lunit Inc. https://lunit.io/en/. Accessed Aug 2019 (2018).
65. Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA - J. Am. Med. Assoc. 316, 2402–2410 (2016).
    Article Google Scholar
66. Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115 (2017).
    Article CAS PubMed Google Scholar
67. Rajpurkar, P. et al. Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. _arXiv preprint ar_Xiv:1711.05225 (2017).
68. Hannun, A. Y. et al. FOCUS | Letters FOCUS | Letters Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network letters | FOCUS letters | FOCUS. Nat. Med. https://doi.org/10.1038/s41591-018-0268-3 (2019).
69. Erickson, B. J. Machine Intelligence in Medical Imaging (Society for Imaging Informatics, SIIM, 2016).
70. Keltch, B., Lin, Y. & Bayrak, C. Comparison of AI techniques for prediction of liver fibrosis in hepatitis patients patient facing systems. J. Med. Syst. https://doi.org/10.1007/s10916-014-0060-y (2014).
71. Mørkrid, L. et al. Continuous age- and sex-adjusted reference intervals of urinary markers for cerebral creatine deficiency syndromes: a novel approach to the definition of reference intervals. Clin. Chem. 61, 760–768 (2015).
    Article PubMed CAS Google Scholar
72. Spyridonos, P., Cavouras, D., Ravazoula, P. & Nikiforidis, G. A computer-based diagnostic and prognostic system for assessing urinary bladder tumour grade and predicting cancer recurrence. Med. Inf. Internet Med. 27, 111–122 (2002).
    Article CAS Google Scholar
73. Tsolaki, E. et al. Fast spectroscopic multiple analysis (FASMA) for brain tumor classification: a clinical decision support system utilizing multi-parametric 3T MR data. Int. J. Comput. Assist Radio. Surg. 10, 1149–1166 (2015).
    Article Google Scholar
74. Davis, S., Roudsari, A., Raworth, R., Courtney, K. L. & Mackay, L. Shared decision-making using personal health record technology: a scoping review at the crossroads. J. Am. Med. Inf. Assoc. 24, 857–866 (2017).
    Article Google Scholar
75. Fuji, K. T. et al. Standalone personal health records in the United States: meeting patient desires. Health Technol. 2, 197–205 (2012).
    Article Google Scholar
76. Tang, P. C., Ash, J. S., Bates, D. W., Overhage, J. M. & Sands, D. Z. Personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. J. Am. Med. Inf. Assoc. 13, 121–126 (2006).
    Article CAS Google Scholar
77. Wald, J. S. et al. A patient-controlled journal for an electronic medical record: issues and challenges. Stud. Health Technol. Inform. 107(Pt 2), 1166–1170 (2004).
    PubMed Google Scholar
78. Hanauer, D. A., Preib, R., Zheng, K. & Choi, S. W. Patient-initiated electronic health record amendment requests. J. Am. Med. Inf. Assoc. 21, 992–1000 (2014).
    Article Google Scholar
79. Rosenbloom, S. T. et al. Triaging patients at risk of influenza using a patient portal. J. Am. Med. Inf. Assoc. 19, 549–554 (2012).
    Article Google Scholar
80. Roehrs, A., Da Costa, C. A., Da Rosa Righi, R. & De Oliveira, K. S. F. Personal health records: A systematic literature review. J. Med. Internet Res. https://doi.org/10.2196/jmir.5876 (2017).
81. Benhamou, P. Y. Improving diabetes management with electronic health records and patients’ health records. Diabetes Metab. 37(Suppl. 4), S53–S56 (2011).
    Article PubMed Google Scholar
82. Kumar, R. B., Goren, N. D., Stark, D. E., Wall, D. P. & Longhurst, C. A. Automated integration of continuous glucose monitor data in the electronic health record using consumer technology. J. Am. Med. Inf. Assoc. 23, 532–537 (2016).
    Article Google Scholar
83. Kilsdonk, E., Peute, L. W., Riezebos, R. J., Kremer, L. C. & Jaspers, M. W. M. Uncovering healthcare practitioners’ information processing using the think-aloud method: From paper-based guideline to clinical decision support system. Int. J. Med. Inf. 86, 10–19 (2016).
    Article CAS Google Scholar
84. Dowding, D. et al. Nurses’ use of computerised clinical decision support systems: A case site analysis. J. Clin. Nurs. 18, 1159–1167 (2009).
    Article PubMed Google Scholar
85. Ash, J. S., Sittig, D. F., Campbell, E. M., Guappone, K. P. & Dykstra, R. H. Some unintended consequences of clinical decision support systems. AMIA Annu Symp. Proc. AMIA Symp. AMIA Symp. 2007, 26–30 (2007).
    Google Scholar
86. Khalifa, M. & Zabani, I. Improving utilization of clinical decision support systems by reducing alert fatigue: Strategies and recommendations. Stud. Health Technol. Inform. 226, 51–54 (2016).
    PubMed Google Scholar
87. Van Laere, S. et al. Clinical decision support systems for drug allergy checking: systematic review. https://doi.org/10.2196/jmir.8206 (2018).
88. Wyatt, J. & Spiegelhalter, D. Field trials of medical decision-aids: potential problems and solutions. American Medical Informatics Association. 3–7. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2247484/ (1991).
89. Goddard, K., Roudsari, A. & Wyatt, J. Automation bias - A hidden issue for clinical decision support system use. Stud. Health Technol. Inform. 164, 17–22 (2011).
    PubMed Google Scholar
90. Devaraj, S., Sharma, S. K., Fausto, D. J., Viernes, S. & Kharrazi, H. Barriers and facilitators to clinical decision support systems adoption: a systematic review. J. Bus Adm. Res. https://doi.org/10.5430/jbar.v3n2p36 (2014).
91. Leslie, S. J. et al. Clinical decision support software for management of chronic heart failure: Development and evaluation. Comput Biol. Med. 36, 495–506 (2006).
    Article PubMed Google Scholar
92. Murray, E. et al. Why is it difficult to implement e-health initiatives? A qualitative study. Implement Sci. 6, 6 (2011).
    Article PubMed PubMed Central Google Scholar
93. Lai, F., Macmillan, J., Daudelin, D. H. & Kent, D. M. The potential of training to increase acceptance and use of computerized decision support systems for medical diagnosis. Hum. Factors J. Hum. Factors Erg. Soc. 48, 95–108 (2006).
    Article Google Scholar
94. Ojeleye, L. Ensuring effective computerised clinical decision support. Prescriber 27, 54–56 (2016).
    Article Google Scholar
95. Cook, D. A., Teixeira, M. T., Heale, B. S. E., Cimino, J. J. & Del Fiol, G. Context-sensitive decision support (infobuttons) in electronic health records: a systematic review. J. Am. Med Inf. Assoc. 24, 460–468 (2017).
    Article Google Scholar
96. Sujansky, W. Heterogeneous database integration in biomedicine. J. Biomed. Inform. 34, 285–298 (2001).
    Article CAS PubMed Google Scholar
97. Index - FHIR v3.0.1. (2018). https://www.hl7.org/fhir/index.html. Accessed July 2019.
98. Katehakis, D. G. Towards the Development of a National eHealth Interoperability Framework to Address Public Health Challenges in Greece. Proceedings of the First International Workshop on Semantic Web Technologies for Health Data Management, SWH@ISWC. 2164, 1–9 (2018).
99. EHRIntelligence. 5 Ways States Mandate Health Information Exchange. https://ehrintelligence.com/news/5-ways-states-mandate-health-information-exchange. Accessed Aug 2019 (2015).
100. European Commission Report. Commission Recommendation on a European Electronic Health Record Exchange Format (C(2019)800) of 6 February 2019. https://ec.europa.eu/digital-single-market/en/news/recommendation-european-electronic-health-record-exchange-form (2019).
101. Bresnick, J. HealthITAnalytics. Interoperability, Low Costs Make Cloud-Based EHRs a Favorite. https://healthitanalytics.com/news/interoperability-low-costs-make-cloud-based-ehrs-a-favorite. Accessed July 2019 (2015).
102. Fernández-Cardeñosa, G., De La Torre-Díez, I., López-Coronado, M. & Rodrigues, J. J. P. C. Analysis of cloud-based solutions on EHRs systems in different scenarios. J. Med. Syst. 36, 3777–3782 (2012).
     Article PubMed Google Scholar
103. Rodrigues, J. J. P. C., de la Torre, I., Fernández, G. & López-Coronado, M. Analysis of the security and privacy requirements of cloud-based electronic health records systems. J. Med. Internet Res. 15, e186 (2013).
104. Kabachinski, J. A look at clinical decision support systems. Biomed. Instrum. Technol. 47, 432–434 (2013).
     Article PubMed Google Scholar
105. O’Reilly, D., Tarride, J.-E., Goeree, R., Lokker, C. & McKibbon, K. A. The economics of health information technology in medication management: a systematic review of economic evaluations. J. Am. Med. Inf. Assoc. 19, 423–438 (2012).
     Article Google Scholar
106. Bright, T. J. et al. Effect of clinical decision-support systems: a systematic review. Ann. Intern Med. 157, 29–43 (2012).
     Article PubMed Google Scholar
107. Jacob, V. et al. Cost and economic benefit of clinical decision support systems (CDSS) for cardiovascular disease prevention: a community guide systematic review. J. Am. Med. Inf. Assoc. Jamia. 24, 669–676 (2017).
     Article Google Scholar
108. Main, C. et al. Computerised decision support systems in order communication for diagnostic, screening or monitoring test ordering: systematic reviews of the effects and cost-effectiveness of systems. Health Technol. Assess. 14, 1–227 (2010).
     CAS PubMed Google Scholar
109. Scheepers-Hoeks, A. M., Grouls, R. J., Neef, C. & Korsten, H. H. Strategy for implementation and first results of advanced clinical decision support in hospital pharmacy practice. Stud. Health Technol. Inform. 148, 142–148 (2009).
110. Edlin, R., McCabe, C., Hulme, C., Hall, P. & Wright, J. Cost effectiveness modelling for health technology assessment. https://doi.org/10.1007/978-3-319-15744-3 (2015).
111. Vermeulen, K. M. et al. Cost-effectiveness of an electronic medication ordering system (CPOE/CDSS) in hospitalized patients. Int J. Med, Inf. 83, 572–580 (2014).
     Article CAS Google Scholar
112. Okumura, L. M., Veroneze, I., Burgardt, C. I. & Fragoso, M. F. Effects of a computerized provider order entry and a clinical decision support system to improve cefazolin use in surgical prophylaxis: a cost saving analysis. Pharm. Pract. 14, 1–7 (2016).
     Google Scholar
113. Osheroff, J. A. et al. A roadmap for national action on clinical decision support. J. Am. Med. Inf. Assoc. 14, 141–145 (2007).
     Article Google Scholar
114. Greenes, R. A. Clinical Decision Support 2nd edn. The Road to Broad Adoption https://doi.org/10.1016/B978-0-12-398476-0.00035-X (2014).
115. Bonney, W. Impacts and risks of adopting clinical decision support systems. In: Efficient Decision Support Systems - Practice and Challenges in Biomedical Related Domain. IntechOpen https://doi.org/10.5772/711 (2011).
116. Bates, D. W. et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J. Am. Med. Inf. Assoc. 10, 523–530 (2003).
     Article Google Scholar
117. Kawamoto, K. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. Bmj 330, 765–0 (2005).
     Article PubMed PubMed Central Google Scholar
118. Health Level Seven International - Homepage. http://www.hl7.org/. Accessed 29 Aug 2019 (2018).
119. IHTSDO. History of SNOMED CT. Ihtsdo. http://www.ihtsdo.org/snomed-ct/what-is-snomed-ct/history-of-snomed-ct (2015).
120. Marco-Ruiz, L. & Bellika, J. G. Semantic Interoperability in Clinical Decision Support Systems: A Systematic Review. Stud. Health Technol. Inf. 216, 958 (2015).
     Google Scholar
121. Angraal, S., Krumholz, H. M. & Schulz, W. L. Blockchain technology: applications in health care. Circ. Cardiovasc. Qual. Outcomes https://doi.org/10.1161/CIRCOUTCOMES.117.003800 (2017).
122. Ivan, D. Moving toward a blockchain-based method for the secure storage of patient records. ONC/NIST Use of Blockchain for Healthcare and Research Workshop. Gaithersburg, Maryland, United States: ONC/NIST (2016).
123. Eichner, J. & Das, M. Challenges and Barriers to Clinical Decision Support (CDS) Design and Implementation Experienced in the Agency for Healthcare Research and Quality CDS Demonstrations. Agency Healthc Res. Qual. Website. 29. https://healthit.ahrq.gov/sites/default/files/docs/page/CDS_challenges_and_barriers.pdf (2010).
124. Khalifa, M. Clinical decision support: strategies for success. Procedia Comput. Sci. 37, 422–427 (2014).
     Article Google Scholar
125. Sittig, D. F. Electronic Health Records: Challenges in Design and Implementation. (CRC Press, 2014).
126. Harper, B. D. & Norman, K. L. Improving User Satisfaction: The Questionnaire for User Interaction Satisfaction Version 5.5. Proceedings of the 1st Annual Mid-Atlantic Human Factors Conference, (pp. 224–228), Virginia Beach, VA. (1993).
127. Lewis, J. R. The system usability scale: past, present, and future. Int J. Hum.-Comput. Interact. 34, 577–590 (2018).
     Article Google Scholar
128. Lewis, J. R. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J. Hum.-Comput. Interact. 7, 57–78 (1995).
     Article Google Scholar
129. Lewis, J. R. Measuring perceived usability: the CSUQ, SUS, and UMUX. Int J. Hum.-Comput Interact. 34, 1148–1156 (2018).
     Article Google Scholar

Download references

Acknowledgements

The author’s (R.T.S.) work was supported by a Canada Institute of Health (CIHR) Research Graduate Scholarship (CGS-M). Thank you to Nathan Stern for discussion and initial review of the manuscript.

Author information

Authors and Affiliations

1. Department of Medicine, Division of Gastroenterology, University of Alberta, Edmonton, Canada
   Reed T. Sutton, Daniel C. Baumgart, Daniel C. Sadowski, Richard N. Fedorak & Karen I. Kroeker
2. Chief Medical Information Office, Alberta Health Services, Edmonton, Canada
   David Pincock

Authors

1. Reed T. Sutton
   You can also search for this author inPubMed Google Scholar
2. David Pincock
   You can also search for this author inPubMed Google Scholar
3. Daniel C. Baumgart
   You can also search for this author inPubMed Google Scholar
4. Daniel C. Sadowski
   You can also search for this author inPubMed Google Scholar
5. Richard N. Fedorak
   You can also search for this author inPubMed Google Scholar
6. Karen I. Kroeker
   You can also search for this author inPubMed Google Scholar

Contributions

R.T.S. reviewed the literature, drafted and revised the manuscript. D.P., D.C.S., D.C.B., K.I.K., and R.N.F. provided critical input, discussion, and revision of the manuscript. All authors approved the final submitted version of the manuscript.

Corresponding author

Correspondence toKaren I. Kroeker.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Cite this article

Sutton, R.T., Pincock, D., Baumgart, D.C. et al. An overview of clinical decision support systems: benefits, risks, and strategies for success.npj Digit. Med. 3, 17 (2020). https://doi.org/10.1038/s41746-020-0221-y

Download citation

• Received: 21 February 2019
• Accepted: 19 December 2019
• Published: 06 February 2020
• DOI: https://doi.org/10.1038/s41746-020-0221-y

This article is cited by