Outpatient Electronic Health Records and the Clinical Care and Outcomes of Patients With Diabetes Mellitus (original) (raw)

. Author manuscript; available in PMC: 2013 Apr 2.

Abstract

Background

Physicians can receive federal payments for meaningful use of complete certified electronic health records (EHRs). Evidence is limited on how EHR use affects clinical care and outcomes.

Objective

To examine the association between use of a commercially available certified EHR and clinical care processes and disease control in patients with diabetes.

Design

Quasi-experimental design with outpatient EHR implementation sequentially across 17 medical centers. Multivariate analyses adjusted for patient characteristics, medical center, time trends, and patient-level clustering.

Setting

Kaiser Permanente Northern California, an integrated delivery system.

Patients

169 711 patients with diabetes mellitus.

Intervention

Use of a commercially available certified EHR.

Measurements

Drug treatment intensification and hemoglobin A1c (HbA1c) and low-density lipoprotein cholesterol (LDL-C) testing and values.

Results

Use of an EHR was associated with statistically significant improvements in treatment intensification after HbA1c values of 9% or greater (odds ratio, 1.10 [95% CI, 1.06 to 1.14]) or LDL-C values of 2.6 to 3.3 mmol/L (100 to 129 mg/dL) (odds ratio, 1.06 [CI, 1.03 to 1.09]); increases in 365-day retesting for HbA1c and LDL-C levels among all patients, with the most dramatic change among patients with the worst disease control (HbA1c ≥9% or LDL-C ≥3.4 mmol/L [ ≥130 mg/dL]); and decreased 90-day retesting among patients with HbA1c level less than 7% or LDL-C level less than 2.6 mmol/L (<100 mg/dL). The EHR was also associated with statistically significant reductions in HbA1c and LDL-C levels, with the largest reductions among patients with the worst control (0.06-mmol/L [2.19-mg/dL] reduction among patients with baseline LDL-C ≥3.4 mmol/L [ ≥130 mg/dL]; P < 0.001).

Limitation

The EHR was implemented in a setting with strong baseline performance on cardiovascular care quality measures.

Conclusion

Use of a commercially available, certified EHR was associated with improved drug treatment intensification, monitoring, and physiologic control among patients with diabetes, with greater improvements among patients with worse control and less testing in patients already meeting guideline-recommended glycemic and lipid targets.

Primary Funding Source

National Institute of Diabetes and Digestive and Kidney Diseases.

The use of electronic clinical information holds promise for improving the quality and efficiency of medical care. Federal incentives for meaningful use of certified electronic health records (EHRs), which total 29billion,canbeasmuchas29 billion, can be as much as 29billion,canbeasmuchas44 000 per physician, and financial penalties for lack of certified EHR use will begin in 2015 (1). Before these incentives came into effect, adoption of EHRs in the United States had been slow (2).

In theory, an EHR increases the availability of clinical information and facilitates decision making at the point of care. However, the actual effects on clinical care are controversial, with recent studies and reviews finding mixed effects of outpatient EHR implementation (3-8). Previous studies have primarily focused on individual health information technology (IT) functions or partial EHR systems. Rigorously designed, large-scale studies on the longitudinal effects of a fully integrated, commercially available complete outpatient EHR system on chronic disease care and clinical outcomes are lacking.

We studied the association between implementing a commercially available outpatient EHR and clinical care pathways and outcomes in patients with diabetes mellitus. Our study took advantage of a natural experiment created by the staggered implementation of a certified EHR system over 17 medical centers in a large integrated delivery system (IDS). We hypothesized that the EHR would improve rates of outpatient monitoring and drug treatment intensification based on evidence-based guidelines, and we examined the effect of the EHR on disease control outcomes linked with these care processes.

Methods

Setting

This study was conducted at Kaiser Permanente Northern California, a large, prepaid IDS providing comprehensive medical care for more than 3 million members, including outpatient, inpatient, emergency department, pharmacy, and laboratory services (9).

Between 2005 and 2008, Kaiser Permanente Northern California implemented a commercially available outpatient certified EHR. The implementation was staggered across 17 medical centers with 45 facilities (Figure), providing both a quasi-experimental setting to examine the EHR effects as well as concurrent controls to adjust for secular trends in diabetes care practice unrelated to the EHR. Although rollout of the certified EHR was not random, the sequence was not systematically designed according to the ability of the medical centers to implement the EHR and did not coincide with any other large systematic organizational changes. We confirmed that there was no statistically significant association between the order of EHR implementation and the mean hemoglobin A1c (HbA1c) level of patients at each medical center before implementation (P = 0.61) (Appendix Figure, available at www.annals.org).

Figure. Staggered EHR implementation by medical center: quasi-experimental study with concurrent controls.

This figure shows the schedule of staggered outpatient EHR implementation across all study medical centers during the study period (2004–2009; dark shade) and the number of study patients at each medical center. After implementation, the EHR completely replaced the paper medical chart and a limited patchwork of preexisting nonintegrated health information technology tools. Use of those early health information technology tools was limited because paper-based alternatives were still in use. EHR = electronic health record.

The outpatient EHR completely replaced the paper-based medical record and a limited patchwork of preexisting nonintegrated health IT tools. Use of those early health IT tools was limited because paper-based alternatives were still in use. The EHR is an EpicCare-based (Epic Systems, Verona, Wisconsin) integrated health IT system that increased the amount of information available at the point of care, presenting integrated clinical information in an electronic medical record with comprehensive computer-based provider order entry, diabetes-specific decision support for laboratory testing and treatment intensification, and secure messaging between providers and with patients. This system has been certified as a complete EHR, thereby qualifying for federal “meaningful use” payments.

Study Population and Data

Our study population included all IDS members (aged >1 year) who were in the diabetes clinical registry of the health plan as of the last quarter of 2003. Members left the study cohort when they first disenrolled from the IDS (average, 4.9% per year) or died (average, 2.6% per year). We used sensitivity analyses among continuously enrolled patients to confirm that attrition did not bias our results (Appendix Tables 1 to 4, available at www.annals.org). All study data were identified by using IDS laboratory test, outpatient pharmacy, and administrative databases.

HbA1c and Low-Density Lipoprotein Cholesterol Tests and Treatment Intensification

We focused on 2 clinical measures: glycemic control (as measured by HbA1c level) and low-density lipoprotein cholesterol (LDL-C) level. Using all tests from 2004 to 2009, for each patient we defined the time between any given test (“index test”) and the subsequent test (retest interval).

For all HbA1c or LDL-C testing intervals, we defined whether each interval’s index test was followed by a treatment intensification by using criteria that have been described previously (10) and are summarized in the next paragraph. Because adjustments in insulin were difficult to identify in pharmacy data, we excluded HbA1c tests for patients receiving insulin within 1 year before the test when examining treatment intensification. For LDL-C tests, we excluded patients from analyses of treatment intensification once they received a high-dose, high-potency statin (for example, simvastatin, 80 mg; atorvastatin, 80 mg; or rosuvastatin, 40 mg) because there is uncertainty about incremental benefit of further lipid-lowering treatment intensification (11).

To define treatment intensifications, we compared antidiabetic or lipid-lowering drugs dispensed within 180 days before a given index test with those dispensed within 60 days after the test (or until the next test if the interval was <60 days). We defined patients as having a treatment change if any of the following occurred in their diabetes or cholesterol medication treatments between the pretest and posttest periods: an increase in the number of drug classes, an increase in daily dosage of an ongoing drug, a switch to a drug in the same class with increase in bioequivalent dose category, a switch to another drug from a distinct class without decrease in number of classes, or the addition of insulin (for HbA1c analyses).

HbA1c or LDL-C Value

To determine the effect of the EHR on HbA1c and LDL-C values, we examined all HbA1c and LDL-C values for the patients in our study cohort. To examine differences in the effect of the EHR across different levels of control, we categorized patients by their baseline value before the beginning of our study (last value in 2003). Only patients with a baseline value were included in this analysis of test values (88% had a baseline HbA1c value, and 85% had a baseline LDL-C value).

Statistical Analysis

We linked each patient in the study population to the medical facility where they sought care and defined each patient’s tests according to whether the EHR was in use at their facility at the time of the test. We defined a facility as using the EHR once it was used for at least 80% of outpatient visits in a given calendar month. All analyses were implemented using Stata, release 10 (StataCorp, College Station, Texas) (12).

We used multivariate logistic regression to examine the association between EHR and treatment intensification, with interaction terms for EHR status and test value to allow for potentially different EHR effects by index level. We used a 2-level model-based analysis in which intensification opportunities were clustered within facilities using a marginal (population-averaged) approach, and we corrected the SEs to account for multiple tests per facility (Stata logistic command, cluster option; see the Supplement, available at www.annals.org, for additional detail).

We examined the association between EHR use and retesting intervals of 90 days and 1 year by using multivariate logistic regression with interaction terms to allow for potentially different EHR effects by index level and corrected all SEs to account for multiple tests per facility (Stata logistic command, cluster option) (13). We chose these time intervals because the American Diabetes Association recommends quarterly retesting of patients who do not meet treatment goals (14) and because Healthcare Effectiveness Data and Information Set quality measures include annual HbA1c and LDL-C tests. We also used a Cox proportional hazards survival analysis stratified by the index test level to examine overall time to retest.

All models were adjusted for patient age, sex, race or ethnicity, neighborhood socioeconomic status (low socioeconomic status was defined as 20% of residents having household incomes below the federal poverty level or 25% of residents aged 25 years or older having less than a high school education, based on U.S. census block group data from the 2000 census), comorbid conditions (based on IDS clinical registries for asthma, coronary artery disease, hypertension, and congestive heart failure), medical center, and background temporal trends (indicators for year and month). Models examining treatment intensification were also adjusted for drug adherence history (having enough medication to cover ≥80% of the year before the test).

To examine the association between EHR use and follow-up HbA1c and LDL-C values, we used linear regression models with fixed effects at the patient level, an approach commonly used in the econometrics literature (15) (see the Supplement for additional detail), while adjusting for calendar quarter and calendar year (Stata xtreg command, fe option). We examined the effect of the EHR separately for patients with different levels of baseline HbA1c or LDL-C control. For each patient, we classified the first test after EHR implementation as having been done during the process of transition to the EHR because it probably captured effects of treatment decisions that were based on the previous test value obtained before implementation of the EHR. We defined each patient’s second and subsequent values after EHR implementation as being post-EHR follow-up values. This allowed for the patient to be fully exposed to the EHR and its potential effect on treatment and follow-up pathways (for example, index test; potential treatment intensification, if needed; and follow-up test).

The Kaiser Foundation Research Institute Institutional Review Board reviewed and approved the study protocol. Waiver of informed consent was obtained because of the nature of the study.

Results

Our study included the 169 711 patients in the health plan’s clinical diabetes registry at the end of 2003 (Table 1). During the study period (2004–2009), these patients had a total of 1 372 735 HbA1c and 1 268 086 LDL-C tests (Appendix Table 5, available at www.annals.org); 47.3% of HbA1c and 45.3% of LDL-C tests were done after implementation of the certified EHR.

Table 1.

Baseline Patient Characteristics

Treatment Intensification

There were 972 115 HbA1c tests among 129 433 patients not receiving insulin and 1 095 991 LDL-C tests among 151 838 patients not already receiving a high-dose, high-potency statin. Before EHR implementation, 24.0% of HbA1c tests with results between 7% and 8.9% and 42.9% of HbA1c tests with results of 9% or greater were followed by treatment intensification within 60 days, whereas 20.2% of LDL-C tests with results of 2.6 to 3.3 mmol/L (100 to 129 mg/dL) and 29.5% of LDL-C tests with results of 3.4 mmol/L or greater (≥130 mg/dL) were followed by treatment intensification within 60 days (Appendix Table 6, available at www.annals.org).

Table 2 shows the predicted probability of treatment intensification within 60 days by index HbA1c or LDL-C level, both before and after the EHR was implemented. In multivariate analyses, the EHR was associated with a statistically significant (P < 0.001) increase in treatment intensification for HbA1c test results of 9% or greater (odds ratio [OR], 1.10 [95% CI, 1.05 to 1.15]) and results of 7% to 8.9% (OR, 1.12 [CI, 1.06 to 1.18]). The EHR did not statistically significantly affect treatment intensification likelihood for HbA1c values less than 7% (OR, 0.98 [CI, 0.94 to 1.02]; P = 0.29). Similarly, the EHR was associated with a statistically significant increase in treatment intensification for LDL-C test values of 2.6 to 3.3 mmol/L (100 to 129 mg/dL) (OR, 1.06 [CI, 1.00 to 1.12]; P = 0.036), but there was no statistically significant change in treatment intensification for values of 3.4 mmol/L or greater (≥130 mg/dL) (OR, 0.97 [CI, 0.91 to 1.04]; P = 0.46). With the EHR, the likelihood of treatment intensification after LDL-C values less than 2.6 mmol/L (<100 mg/dL) decreased significantly (OR, 0.88 [CI, 0.82 to 0.94]; P < 0.001).

Table 2.

Association Between EHR and Treatment Intensification Within 60 days, by Index HbA1c or LDL-C Value*

Time to HbA1c or LDL-C Retest

Before EHR implementation, 20.7% of HbA1c tests were followed by another test within 90 days and 87.9% were followed by another test within 1 year, whereas 21.0% of LDL-C tests were followed by another test within 90 days and 86.3% were followed by another test within 1 year (Appendix Table 6).

In multivariate analyses, the EHR was associated with a statistically significantly increased probability of having a follow-up test within 1 year across all levels of HbA1c and LDL-C (P < 0.05) (Table 3). In analyses specifically examining retesting within 90 days (Table 3), the EHR was associated with a statistically significantly decreased likelihood of retesting within 90 days if the index result already showed good control (P < 0.005). Also, the EHR was associated with a statistically significantly overall faster rate of retesting after elevated HbA1c or LDL-C levels (P < 0.001), with larger increases in the rate of testing with higher index test values (Table 3).

Table 3.

Association Between EHR and Time to HbA1c and LDL-C Follow-up Retesting, by HbA1c and LDL-C Level

LDL-C and HbA1c Values

Table 4 shows the association between EHR use and follow-up HbA1c and LDL-C values. Across all baseline HbA1c and LDL-C levels, the EHR was associated with statistically significantly reduced follow-up values (P < 0.001), with greater reductions among patients with higher baseline values. Among patients with a baseline HbA1c value of 9% or greater, the EHR was associated with a decrease of 0.14% in the HbA1c value (CI, 0.11% to 0.18%). Reductions were 0.08% (CI, 0.07% to 0.09%) for patients with baseline HbA1c values of 7% to 8.9% and 0.05% (CI, 0.04% to 0.05%) for those with baseline values less than 7%.

Table 4.

Association Between EHR and Follow-up HbA1c and LDL-C Values

Among patients with an LDL-C value of 3.4 mmol/L or greater (≥130 mg/dL), use of the EHR was associated with a decrease of 0.06 mmol/L (2.19 mg/dL) (CI, 0.04 to 0.07 mmol/L [1.64 to 2.74 mg/dL]). Among those with an LDL-C value of 2.6 to 3.3 mmol/L (100 to 129 mg/dL), there was a reduction of 0.04 mmol/L (1.44 mg/dL) (CI, 0.03 to 0.05 mmol/L [1.10 to 1.77 mg/dL]), and for those with an LDL-C value less than 2.6 mmol/L (<100 mg/dL), there was a reduction of 0.02 mmol/L (0.72 mg/dL) (CI, 0.01 to 0.03 mmol/L [0.46 to 0.99 mg/dL]).

Discussion

In a study of the natural experiment involving the staggered implementation of an outpatient certified EHR, we found that EHR use was associated with improved rates of medication treatment intensification, follow-up monitoring, and glycemic and lipid control in patients with diabetes. We found that the EHR-associated improvements were greater among patients with worse disease control than among those already in good control, which is consistent with thoughtful clinical decision making. For example, patients with diabetes who were in poorer control received follow-up HbA1c and LDL-C tests faster, and these values showed a statistically significant improvement. For patients already meeting recommended glycemic and lipid control targets, EHR use was associated with lower rates of testing, as defined by reductions in repeated testing within 90 days.

The EHR helped alignment with quality measures and clinical guidelines for treatment. Increases in information availability, decision support, and order entry functionality helped clinicians to better target retesting. When we examined yearly retesting, the threshold measured by the Healthcare Effectiveness Data and Information Set (13) for all patients with diabetes, we found that the EHR was associated with increased testing across all baseline HbA1c and LDL-C levels. The EHR was associated with a decrease in 90-day retesting among patients already under control, which may represent a decrease in potential overtesting. A study in the same IDS showed that treatment intensification is an important process measure of care quality (13). Our study shows that the EHR was associated with treatment intensification among patients with elevated HbA1c levels and LDL-C levels of 2.6 to 3.3 mmol/L (100 to 129 mg/dL) and that their laboratory values improved accordingly. Interestingly, however, although the EHR did not change treatment intensification rates among patients with the highest LDL-C levels, it was statistically significantly associated with reductions in follow-up LDL-C values. It is possible that the EHR had other effects, including helping to improve patient adherence to existing medications.

Our findings, which are consistent across many steps in the care pathway and are proportional to clinical risk levels, suggest actual improvements in the clinical care of patients with diabetes. These early effects on linked care processes and patient outcomes also suggest the potential for future downstream improvements in major clinical event rates and health. The lack of any measurable unintended harm in the outcomes for this study is also important because implementation of an EHR could worsen as well as improve care (3). Further monitoring is important to identify other potential unintended consequences of any EHR implementation. The introduction of federal financial penalties in 2015 underscores the importance of demonstrating whether the initial EHR effects are positive.

Our findings provide an important contribution to previous evidence by examining targeted, clinically relevant process measures associated with the implementation of a complete certified EHR in a large patient population by taking advantage of a rigorous quasi-experimental design. Many previous studies involving health IT interventions and diabetes management were limited by small sample sizes or use of only specific health IT features rather than evaluation of a complete EHR system (as required by definitions of “meaningful use”) or were cross-sectional and did not adequately adjust for secular trends in diabetes care (3, 5, 8, 11, 16-19). Results from these studies were mixed, with some showing improvements in LDL-C and HbA1c screening and outcomes (8) and others showing mixed or even negative results (16-21). To our knowledge, no studies have examined the effect of EHR use on treatment intensification or the targeted effect by disease severity (3, 5, 8, 11, 16-19). This is a critical policy area that needs more and better evidence.

Although our finding of the targeted effect of the EHR among patients with the greatest clinical need is important, the magnitude of the improvement itself may seem modest. Our analysis focused specifically on the incremental benefit associated with the direct clinical use of the complete outpatient EHR system and was designed to exclude the secular trend of any other ongoing programs or improvements in diabetes management. Although the IDS in this study already had high levels of baseline diabetes care quality and there was an overall trend toward improvement in quality during our study period, we isolated only those improvements specifically associated with EHR use.

There are several limitations to the generalizability of our findings. In studies of the effect of EHRs on clinical care, the baseline level of care quality and clinical information availability are important. Because this study was conducted within a large IDS with a clinical diabetes registry and sophisticated and systematic disease management programs at baseline, our findings may not necessarily generalize directly to EHR use in other health care settings. The overall EHR effects were favorable and statistically significant, but as expected, the magnitude of the effects was not identical across individual medical centers. Still, a recent review did not find differences between the effects of EHRs among single-institution studies at health IT leaders and EHRs implemented in other settings (3). It is likely that EHR implementation could bring more dramatic improvements in other settings, where baseline rates of control are lower or disease management capabilities are more limited. Providers already had access to a patchwork of nonintegrated health IT applications at baseline, with separate logins and lack of information sharing between applications. These applications would not qualify for EHR certification. Again, it is possible that in another care setting without baseline availability of limited health IT, an even greater improvement in diabetes care quality might follow implementation of an integrated EHR. Also, although we used a rigorous quasi-experimental study design with concurrent controls, we cannot rule out unmeasured confounding because this is an observational study. Finally, although the outcomes examined in this study are widely used population measures of diabetes quality, individualized goals of patient treatment may vary. Future studies should continue to examine the effect of EHRs on downstream clinical events.

A certified complete EHR system increases the amount and timeliness of clinical information available at the point of care with embedded decision support and order entry. Even with federal incentive payments to offset the costs, implementing a complete EHR system requires a large up-front investment of money and time, with careful coordination across stakeholders and end users. We found that EHR use in an IDS was associated with improved care quality and clinical outcomes in patients with diabetes. Of note, we found that the effect of the EHR varied across specific patient subgroups, resulting in increased testing, treatment, and physiologic improvement for those with the greatest needs and appropriately decreased testing and treatment intensification for individuals already achieving guideline-recommended glycemic and lipid targets. Overall, our study suggests that the EHR may be a powerful tool to help clinicians deliver well-targeted, high-quality chronic disease care and improve patient outcomes.

Supplementary Material

Appendix

Acknowledgments

Role of the Funding Source

This study was funded by the National Institute of Diabetes and Digestive and Kidney Diseases. The funders had no role in the design, conduct, or reporting of this analysis or in the decision to submit the manuscript for publication.

Footnotes

Publisher's Disclaimer: Disclaimer: This is the prepublication, author-produced version of a manuscript accepted for publication in Annals of Internal Medicine. This version does not include post-acceptance editing and formatting. The American College of Physicians, the publisher of Annals of Internal Medicine, is not responsible for the content or presentation of the author-produced accepted version of the manuscript or any version that a third party derives from it. Readers who wish to access the definitive published version of this manuscript and any ancillary material related to this manuscript (e.g., correspondence, corrections, editorials, linked articles) should go to www.annals.org or to the print issue in which the article appears. Those who cite this manuscript should cite the published version, as it is the official version of record.

References

• 1.Jha AK. Meaningful use of electronic health records: the road ahead. JAMA. 2010;304:1709–10. doi: 10.1001/jama.2010.1497. [DOI] [PubMed] [Google Scholar]
• 2.DesRoches CM, Campbell EG, Rao SR, Donelan K, Ferris TG, Jha A, et al. Electronic health records in ambulatory care---a national survey of physicians. N Engl J Med. 2008;359:50–60. doi: 10.1056/NEJMsa0802005. [DOI] [PubMed] [Google Scholar]
• 3.Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff (Millwood) 2011;30:464–71. doi: 10.1377/hlthaff.2011.0178. [DOI] [PubMed] [Google Scholar]
• 4.Romano MJ, Stafford RS. Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Arch Intern Med. 2011;171:897–903. doi: 10.1001/archinternmed.2010.527. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 5.Costa BM, Fitzgerald KJ, Jones KM, Dunning Am T. Effectiveness of IT-based diabetes management interventions: a review of the literature. BMC Fam Pract. 2009;10:72. doi: 10.1186/1471-2296-10-72. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 6.Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144:742–52. doi: 10.7326/0003-4819-144-10-200605160-00125. [DOI] [PubMed] [Google Scholar]
• 7.Blumenthal D. Guiding the health information technology agenda. Interviewed by David J Brailer. Health Aff (Millwood) 2010;29:586–95. doi: 10.1377/hlthaff.2010.0274. [DOI] [PubMed] [Google Scholar]
• 8.Cebul RD, Love TE, Jain AK, Hebert CJ. Electronic health records and quality of diabetes care. N Engl J Med. 2011;365:825–33. doi: 10.1056/NEJMsa1102519. [DOI] [PubMed] [Google Scholar]
• 9.Krieger N. Overcoming the absence of socioeconomic data in medical records: validation and application of a census-based methodology. Am J Public Health. 1992;82:703–10. doi: 10.2105/ajph.82.5.703. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 10.Selby JV, Uratsu CS, Fireman B, Schmittdiel JA, Peng T, Rodondi N, et al. Treatment intensification and risk factor control: toward more clinically relevant quality measures. Med Care. 2009;47:395–402. doi: 10.1097/mlr.0b013e31818d775c. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 11.Sharma M, Ansari MT, Abou-Setta AM, Soares-Weiser K, Ooi TC, Sears M, et al. Systematic review: comparative effectiveness and harms of combination therapy and monotherapy for dyslipidemia. Ann Intern Med. 2009;151:622–30. doi: 10.7326/0003-4819-151-9-200911030-00144. [DOI] [PubMed] [Google Scholar]
• 12.StataCorp. Stata Base Reference Manual, Release 10. College Station, TX: StataCorp; 2007. [Google Scholar]
• 13.Cameron AC, Miller DL. Robust inference with clustered data. In: Ullah A, Giles DEA, editors. Handbook of Empirical Economics and Finance. Boca Raton, FL: Taylor and Francis Group; 2010. pp. 1–25. [Google Scholar]
• 14.American Diabetes Association. Standards of medical care in diabetes---2011. Diabetes Care. 2011;34(Suppl 1):S11–61. doi: 10.2337/dc11-S011. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 15.Wooldridge JM. Econometric Analysis of Cross Section and Panel Data. Cambridge, MA: MIT Press; 2002. [Google Scholar]
• 16.Herrin J, da Graca B, Nicewander D, Fullerton C, Aponte P, Stanek G, et al. The effectiveness of implementing an electronic health record on diabetes care and outcomes. Health Serv Res. 2012;47:1522–40. doi: 10.1111/j.1475-6773.2011.01370.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 17.O’Connor PJ, Crain AL, Rush WA, Sperl-Hillen JM, Gutenkauf JJ, Duncan JE. Impact of an electronic medical record on diabetes quality of care. Ann Fam Med. 2005;3:300–6. doi: 10.1370/afm.327. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 18.Sequist TD, Gandhi TK, Karson AS, Fiskio JM, Bugbee D, Sperling M, et al. A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. J Am Med Inform Assoc. 2005;12:431–7. doi: 10.1197/jamia.M1788. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 19.Zhou L, Soran CS, Jenter CA, Volk LA, Orav EJ, Bates DW, et al. The relationship between electronic health record use and quality of care over time. J Am Med Inform Assoc. 2009;16:457–64. doi: 10.1197/jamia.M3128. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 20.Mangione CM, Gerzoff RB, Williamson DF, Steers WN, Kerr EA, Brown AF, et al. TRIAD Study Group. The association between quality of care and the intensity of diabetes disease management programs. Ann Intern Med. 2006;145:107–16. doi: 10.7326/0003-4819-145-2-200607180-00008. [DOI] [PubMed] [Google Scholar]
• 21.Shojania KG, Ranji SR, McDonald KM, Grimshaw JM, Sundaram V, Rushakoff RJ, et al. Effects of quality improvement strategies for type 2 diabetes on glycemic control: a meta-regression analysis. JAMA. 2006;296:427–40. doi: 10.1001/jama.296.4.427. [DOI] [PubMed] [Google Scholar]
• 22.Therneau TM, Grambsch PM. Modeling Survival Data: Extending the Cox Model. New York: Springer-Verlag; 2000. Multiple Events per Subject; pp. 169–229. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix