Problem of immortal time bias in cohort studies: example using statins for preventing progression of diabetes (original) (raw)

1. Research
2. Problem of immortal...
3. Problem of immortal time bias in cohort studies: example using statins for preventing progression of diabetes

Research Methods & Reporting BMJ 2010;340 doi: https://doi.org/10.1136/bmj.b5087 (Published 12 March 2010) Cite this as: BMJ 2010;340:b5087

1. Linda E Lévesque, assistant professor of epidemiology123,
2. James A Hanley, professor of biostatistics14,
3. Abbas Kezouh, biostatistician4,
4. Samy Suissa, professor of epidemiology and biostatistics14
5. 1Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montréal, Canada
6. 2Department of Community Health and Epidemiology, Queen’s University, Kingston, Canada
7. 3Kingston, Frontenac, Lennox and Addington Public Health, Kingston, Canada
8. 4Center for Clinical Epidemiology, Jewish General Hospital, McGill University, Montréal, Canada
9. Correspondence to: L Lévesque linda.levesque{at}queensu.ca

• Accepted 25 August 2009

Immortal time in observational studies can bias the results in favour of the treatment group, but it is not difficult to identify and avoid

Well designed observational studies have made important contributions to our understanding of the risks and benefits of drug treatment. Such studies are often the first to identify or confirm important adverse health events associated with drugs, as seen recently with the cardiac effects of ergot derived dopamine agonists1 and cyclo-oxygenase 2 inhibitors.2 3 Observational studies can also assess aspects of drug safety, such as the time varying nature of risk, which cannot be readily appraised using an experimental design.4

Cohort studies are often preferred to case-control studies because they are less susceptible to certain biases.5 6 However, the inappropriate accounting of follow-up time and treatment status in the design and analysis of such studies can introduce immortal time bias.7

What is immortal time bias?

Immortal time refers to a period of follow-up during which, by design, death or the study outcome cannot occur.8 In pharmacoepidemiology studies, immortal time typically arises when the determination of an individual’s treatment status involves a delay or wait period during which follow-up time is accrued—for example, waiting for a prescription to be dispensed after discharge from hospital when the discharge date represents the start of follow-up (box 1).9 10 11 12 13 14 This wait period is considered immortal because individuals who end up in the treated or exposed group have to survive (be alive and event free) until the treatment definition is fulfilled. If they have an event before taking up treatment they are in the untreated or unexposed group. Bias is introduced when this period of “immortality” is either misclassified with regards to treatment status or excluded from the analysis (fig 1⇓).7 Immortal time bias is particularly problematic because it necessarily biases the results in favour of the treatment under study by conferring a spurious survival advantage to the treated group.

Box 1: Common manifestations of immortal time

• Treatment defined as at least one prescription dispensed after hospital discharge, when the discharge date represents the start of follow-up (cohort entry)—for example, dispensation of an inhaled corticosteroid after a hospital stay for chronic obstructive pulmonary disease9
• Treatment groups defined in terms of when after hospital discharge (start of follow-up) a prescription is dispensed—for example, cardiac drugs dispensed within 7 days of discharge for acute myocardial infarction versus later10 or early versus delayed dispensation of clopidogrel post percutaneous coronary intervention11
• Treatment defined as at least one prescription dispensed after a diagnosis, when the date of diagnosis represents the start of follow-up—for example, starting interferon beta after diagnosis of multiple sclerosis12
• Treatment status determined over the duration of follow-up—for example, determining an individual’s immunisation status at the end of each influenza season13 or use of β blockers any time during follow-up14

Fig 1 Immortal time bias is introduced in cohort studies when the period of immortal time is either incorrectly attributed to the treated group through a time fixed analysis (top) or excluded from the analysis because the start of follow-up for the treated group is defined by the start of treatment and is, by design, later than that for the untreated group (bottom)

• Download figure
• Open in new tab
• Download powerpoint

Immortal time bias is increasingly common in cohort studies of drug effects.7 A recent example is a study of 3-hydroxy-3-methylglutaryl coenzyme A reductase inhibitors (statins) that reported a 26% reduction in the risk of diabetes progression with one year or more of treatment (adjusted hazard ratio 0.74, 95% confidence interval 0.56 to 0.97).15 This is a surprising finding given that this association would be expected to be subject to confounding and yield a hazard ratio >1.0 because people whose diabetes progresses are more likely to develop cardiovascular disease, an indication for statins.

Below, we replicate this study to show how immortal time bias can be introduced in cohort studies, quantify the relation between the extent of immortal time and the magnitude of the bias, determine the extent to which this bias accounted for the protective association previously reported, and show how immortal time bias can be prevented through time dependent analysis.

Demonstration of bias

We replicated Yee et al’s statin study using the same Saskatchewan Health databases. These computerised databases, generated by the province’s universal health programmes, provide information on dates of health coverage, sociodemographics, outpatient prescriptions, medical services and procedures, hospital discharge diagnoses, and vital statistics for about 91% of residents (roughly one million people).16

In accordance with the previous study, we identified a population based cohort of everyone aged 30 years and older, newly treated with a sulfonylurea or metformin between 1 January 1991 and 31 December 1996. The date of this first prescription was taken as cohort entry (start of follow-up). Individuals were excluded if they did not have at least one year of health coverage before cohort entry or had received oral hypoglycaemics or insulin during the year before entry. As in the previous study, we identified new users of statins by excluding those who had received a lipid lowering drug from three years before to six months after cohort entry. The remaining individuals were followed until study outcome, end of health coverage (because of death or emigration), death, or 31 December 1999 (end of study).

We identified the study outcome, starting insulin treatment, using the date of the first insulin prescription dispensed after cohort entry. The previous study used starting insulin as a surrogate end point for progression of diabetes; people who switch from oral hypoglycaemics to insulin are likely to have uncontrolled hyperglycaemia because of disease progression.17 Like the previous study, we excluded people who were taking insulin before their first statin prescription.

We identified all statin prescriptions dispensed during follow-up to determine individuals’ treatment status. As in the previous study, cohort members were classified as statin users if there was at least one year between the date of their first and last prescription; those with a shorter interval were considered non-users from an aetiological perspective.

To demonstrate and quantify the immortal time bias, we replicated the time fixed (time independent) analysis used by Yee et al to estimate the statin-insulin association and compared it with a simple time dependent analysis that corrected the misclassified immortal time. In the time fixed analysis, all person days between cohort entry and end of follow-up were classified as treated for those who met the statin user definition, regardless of the date on which they met this definition and as untreated for non-users (fig 2⇓). In the simple time dependent analysis, person days of follow-up were correctly classified as untreated until the intended treatment definition of “one year of use” was met, and as treated thereafter (fig 2⇓). We initially used Poisson regression to quantify the magnitude of the misclassified immortal person time and estimate the statin-insulin association, and then used the Cox proportional hazards model.18 19 In the Cox model, hazard ratios were adjusted for the potentially confounding effects of determinants of diabetes progression.

Fig 2 Depiction of typical statin user and non-user and sources of immortal time bias introduced by the time fixed analysis (top). All person days of follow-up were classified as treated for those who satisfied the statin user definition regardless of when the treatment definition was fulfilled. However, in the time dependent analysis (bottom) person days of follow-up were classified as untreated until the one year of statin use definition was met, and as treated thereafter

• Download figure
• Open in new tab
• Download powerpoint

To assess the relation between the amount of immortal time and the magnitude of the bias, as well as determine the extent to which different sources of immortal time accounted for the previously reported protective effect of statins on diabetes progression, we repeated the time fixed and time dependent analyses using Cox proportional hazards model correcting cumulatively for each period of immortal time. The first period corresponded to the first six months of follow-up during which cohort members could not receive a statin by design (fig 2⇑). The second period was from the end of this exclusionary phase until the date of the first statin prescription, and the third was the time needed, after the first prescription, to fulfil the intended “statin user” definition of at least one year of use.

Validation of bias

To validate the presence of the immortal time bias, we repeated the same study and analyses in the same cohort but with different treatments of interest: non-steroidal anti-inflammatory drugs and gastric acid suppressive drugs (histamine-2 (H2) receptor antagonists or proton pump inhibitors). These drugs were chosen because they are commonly prescribed and have no known beneficial effects on diabetes progression or the need to start insulin.

Quantification

The cohort of adults newly treated with an oral hypoglycaemic was comparable in size and clinical profile to that of the previous study (table 1⇓). During an average follow up of 4.9 years, 532 (4.6%) met the definition of statin users (at least one year of use), and 522 (4.5%) had received statins for less than one year and were classified as non-users from an aetiological perspective. An additional 10 607 were classified as non-users because they did not receive any statin prescriptions during follow-up. The mean time to first statin prescription (immortal periods 1 and 2) was 3.1 years for statin users and 4.4 years for non-users who received at least one prescription (fig 3⇓). During follow-up, 1418 (12.2%) people started insulin treatment (study outcome), some during periods of immortal time.

Table 1

Baseline characteristics of the study population according to the use of statins during follow-up. Values are mean (SD) unless stated otherwise

In the time fixed Poisson regression analysis, the immortal and untreated periods accounted for nearly 68% (2174 /3221 person years) of total follow-up time allocated to statin users and produced a crude rate ratio for starting insulin of 0.84 for statin users compared with non-users (table 2⇓). In contrast, the immortal time corrected crude rate ratio was 2.68. Similarly, statin users seemed to be at lower risk of progressing to insulin in the time fixed Cox analysis (adjusted hazard ratio 0.74, 95% confidence interval 0.58 to 0.95) but not in the time dependent analysis (1.97, 1.53 to 2.52).

Table 2

Distribution of person time and events according to use of statins before and after correcting for immortal time bias using Poisson regression and adjusted hazard ratios for starting insulin treatment

Table 3⇓ shows that the 26% risk reduction for progressing to insulin reported in the previous study (0.74, 0.58 to 0.95) was decreased to 18% (0.82, 0.64 to 1.05) after we corrected for the first immortal period and abolished after we corrected for the second (1.37, 1.07 to 1.76). The second period of immortal time, from the end of the six month exclusion period until the date of the first statin prescription, was the time when the largest proportion of non-user person time (42.7%) was incorrectly allocated to statin users in the time fixed analysis.

Table 3

Effect of correcting for different sources of immortal time bias from time fixed analysis on association between statin use and starting insulin

Table 4⇓ shows the results of the validation of the bias using non-steroidal anti-inflammatory drugs and gastrointestinal drugs as the treatments of interest. With the time fixed approach both treatments appeared to reduce the risk of diabetes progression. However, the protective associations disappeared after we corrected for the misclassified immortal time.

Table 4

Crude and adjusted hazard ratios for starting insulin treatment associated with use of non-steroidal anti-inflammatory drugs and gastric acid suppressive drugs before and after correcting for immortal time bias using Cox regression

Accounting for immortal time

We have shown that immortal time bias is introduced by the use of a time fixed analysis in cohort studies with periods of immortal time. In the statin and diabetes progression example, the immortal and untreated person time that was incorrectly allocated to the treated group in the time fixed analysis represented two thirds of total follow-up for statin users. This resulted in a spuriously low rate of events for this group compared with that for non-users. The beneficial effect of statins on the progression of diabetes in the previous study that used a time fixed analysis15 can therefore be ascribed to this bias.

The presence of immortal time bias is corroborated by the demonstration that agents with no known benefit on diabetes progression can be made to appear protective when subjected to the same design and time fixed analysis as that of the statin-insulin study. This demonstrates that this bias is the result of systematic error introduced by the inappropriate accounting of immortal follow-up time, and is, therefore, not specific to pharmacoepidemiological studies.

Immortal time bias has been previously described.22 However, we have shown that more complex designs can introduce new sources of immortal time that, in combination with an incorrect time fixed analysis, individually contribute to the magnitude of the bias. We have also provided additional evidence of the direct relation between the duration of the immortal period and the magnitude of the bias. Unlike previous examples, where the immortal periods were short and events occurred early,23 24 the current study entailed a long period of immortality during which many outcome events occurred. This resulted in a significant distortion of the association under study.

Our example of immortal time bias was one of treatment misclassification. However, this bias can also be introduced when periods of immortal time are differentially excluded from the analysis (selection bias). This occurs when the start of follow-up is defined as the start of treatment for the treated group and the date of diagnosis for the untreated or comparator group (fig 1⇑). Differential exclusion can also arise from the use of a hierarchical approach to determining treatment status. For example, in a study of inhaled corticosteroids for chronic obstructive pulmonary disease, the start of follow-up, defined as the date of the first prescription dispensed, was considerably later after diagnosis for users of inhaled corticosteroids than users of bronchodilators (the comparator) since users of inhaled corticosteroids were identified first, and a large proportion of them had been previously treated with a bronchodilator (that is, survived a previous treatment).25 This resulted in an apparent 38-52% decreased risk of death among the corticosteroid group. The use of a time dependent analysis eliminated the spurious protective association.24

Several approaches have been proposed to prevent immortal time bias including using a time dependent analysis as we have done here,7 studying only “survivors” of the immortal period by moving the start of follow-up to the end of the immortal period,8 and moving the start of follow-up to the date the treatment definition is met for users and a date assigned according to the distribution of users’ immortal time for non-users.26 Alternatively, a time matched, nested case-control analysis of the cohort can be used. This analytical technique has not only been shown to provide an unbiased estimate of the hazard ratio that would be obtained from a traditional time to event analysis of the full cohort,27 28 29 its inherent time dependent nature means that it is also free of immortal time bias. In addition, the nested case-control approach is much less susceptible to selection bias than the classic case-control study since controls are known to represent the source population that gave rise to the cases (that is, the underlying cohort), and the analysis can include all of the cases from the source population.

Other sources of bias

Our corrected results are also subject to bias, particularly confounding by indication. As diabetes progresses, individuals are more likely to develop cardiovascular disease, an indication for statins. By definition, those at higher risk of progressing are also more likely to be prescribed a statin and the statin-insulin association would therefore be expected to be confounded. Consequently, our objective was not to quantify the true nature of the statin-insulin association but rather to demonstrate how immortal time bias is introduced, delineate its impact on the previously reported statin-insulin association, and show how to prevent this bias. For this reason, our finding of a rate ratio of 1.97 for statin users in the corrected analysis should not be interpreted as evidence of an increased risk of progressing to insulin.

Confounding by indication may also explain the raised rate ratios that we observed for non-steroidal anti-inflammatory and gastrointestinal drugs. Individuals who progress and develop diabetic neuropathy and gastroparesis may be more likely to receive these drugs to treat associated symptoms of pain and gastrointestinal discomfort.

Our use of a simple dichotomous definition of statin users in the time dependent analyses may have resulted in residual misclassification of treatment status. To remain true to the original study’s treatment definition, we assumed that individuals were treated for the remainder of their follow-up once they had satisfied the statin user definition. However, some individuals may have become “non-users” later in their follow-up because long term adherence to statins is known to be low.30 31 Consequently, later events may have been incorrectly attributed to statin users rather than to non-users. The long duration of follow-up and high rate of late events may have accentuated the effect of this differential misclassification. This may also explain why the associations studied were all >1.0 after we had corrected for immortal time bias.

Conclusion

An increased awareness of immortal time bias is warranted given the frequency with which this bias is being observed, the wide variety of interventions and health outcomes that have been implicated, and the potential detrimental impact that such biased findings can have on clinical practice and health policy by promoting the use of potentially ineffective therapies or interventions. Moreover, this bias is not specific to studies of drug effects. Consequently, all cohort studies should be assessed for the presence of immortal time bias using appropriate validity criteria (box 2).

Box 2: Criteria for identifying immortal time bias in cohort studies

• Was treatment status determined after the start of follow-up or defined using follow-up time?
• Was the start of follow-up different for the treated and untreated (or comparator) group relative to the date of diagnosis?
• Were the treatment groups identified hierarchically (one group before the other)?
• Were subjects excluded on the basis of treatment identified during follow-up?
• Was a time fixed analysis used?

Notes

Cite this as: BMJ 2010;340:b5087

Footnotes

• This study is based on non-identifiable data provided by the Saskatchewan Department of Health. The interpretation and conclusions contained herein do not necessarily represent those of the Government of Saskatchewan or the Saskatchewan Department of Health.
• Contributors: All authors were involved in the concept and design of the study. LEL and SS acquired the data, LEL and AK conducted the analysis; LEL, JAH, AK, and SS interpreted the data, LEL wrote the first draft of the manuscript, and all authors contributed to subsequent revisions of the manuscript and approved the final version. LEL is the guarantor.
• Funding: SS is the recipient of a distinguished investigator award from the Canadian Institutes for Health Research. The institutes had no involvement in the design, analysis, interpretation, and writing of this study, or the decision to submit the manuscript.
• Competing interests: None declared.
• Ethics approval: The study was approved by the ethics review board of McGill University.
• Provenance and peer review: Not commissioned; externally peer reviewed.

References

1. ↵
   Schade R, Andersohn F, Suissa S, Haverkamp W, Garbe E. Dopamine agonists and the risk of cardiac-valve regurgitation. N Engl J Med2007;356:29-38.
2. ↵
   Lévesque LE, Brophy JM, Zhang B. The risk for myocardial infarction with cyclooxygenase-2 inhibitors: a population study of elderly adults. Ann Intern Med2005;142:481-9.
3. ↵
   Hippisley-Cox J, Coupland C. Risk of myocardial infarction in patients taking cyclo-oxygenase-2 inhibitors or conventional non-steroidal anti-inflammatory drugs: population based nested case-control analysis. BMJ2005;330:1366.
4. ↵
   Lévesque LE, Brophy JM, Zhang B. Time variations in the risk of myocardial infarction among elderly users of COX-2 inhibitors. CMAJ2006;174:1563-9.
5. ↵
   Strom BL. Study designs available for pharmacoepidemiology studies. In: Strom BL, ed. Pharmacoepidemiology. 4th ed. John Wiley, 2005:17-28.
6. ↵
   Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ. Users’ guides to the medical literature. IX. A method for grading health care recommendations. JAMA1995;274:1800-4.
7. ↵
   Suissa S. Immortal time bias in observational studies of drug effects. Pharmacoepidemiol Drug Saf2007;16:241-9.
8. ↵
   Rothman KJ, Greenland S. Cohort studies. In: Rothman KJ, Greenland S, eds. Modern epidemiology. 2nd ed. Lippincott-Raven, 1998:79-91.
9. ↵
   Sin DD, Tu JV. Inhaled corticosteroids and the risk of mortality and readmission in elderly patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med2001;164:580-4.
10. ↵
    Jackevicius CA, Li P, Tu JV. Prevalence, predictors, and outcomes of primary non-adherence after acute myocardial infarction. Circulation2008;117:1028-36.
11. ↵
    Sheehy O, LeLorier J, Rinfret S. Restrictive access to clopidogrel and mortality following coronary stent implantation. CMAJ2008;178:413-20.
12. ↵
    Trojano M, Pellegrini F, Fuiani A, Paolicelli D, Zipoli V, Zimatore GB, et al. New natural history of interferon-beta-treated relapsing multiple sclerosis. Ann Neurol2007;61:300-6.
13. ↵
    Nichol KL, Nordin JD, Nelson DB, Mullooly JP, Hak E. Effectiveness of influenza vaccine in the community-dwelling elderly. N Engl J Med2007;357:1373-81.
14. ↵
    Sin DD, McAlister FA. The effects of beta-blockers on morbidity and mortality in a population-based cohort of 11 942 elderly patients with heart failure. Am J Med2002;113:650-6.
15. ↵
    Yee A, Majumdar SR, Simpson SH, McAlister FA, Tsuyuki RT, Johnson JA. Statin use in type 2 diabetes mellitus is associated with a delay in starting insulin. Diabet Med2004;21:962-7.
16. ↵
    Downey W, Stang M, Beck P, Osei W, Nichol JL. Health services databases in Saskatchewan. In: Strom BL, ed. Pharmacoepidemiology. 4th ed. John Wiley, 2005:295-310.
17. ↵
    Standards of medical care in diabetes. Diabetes Care2004;27(suppl 1):S15-35.
18. ↵
    Kleinbaum DG, Kupper LL, Muller KE, Nizam A, eds. Poisson regression analysis. In: Applied regression analysis and other multivariable methods. 3rd ed. Duxbury Press, 1998:687-709.
19. ↵
    Cox DR. Regression models and life-tables. J R Stat Soc Ser B 1972;34:187-220.
20. Von Korff M, Wagner EH, Saunders K. A chronic disease score from automated pharmacy data. J Clin Epidemiol1992;45:197-203.
21. Schneeweiss S, Seeger JD, Maclure M, Wang PS, Avorn J, Glynn RJ. Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data. Am J Epidemiol2001;154:854-64.
22. ↵
    Suissa S. Immortal time bias in pharmacoepidemiology. Am J Epidemiol2008;167:492-9.
23. ↵
    Suissa S. Effectiveness of inhaled corticosteroids in chronic obstructive pulmonary disease: immortal time bias in observational studies. Am J Respir Crit Care Med2003;168:49-53.
24. ↵
    Suissa S. Inhaled steroids and mortality in COPD: bias from unaccounted immortal time. Eur Respir J2004;23:391-5.
25. ↵
    Soriano JB, Vestbo J, Pride NB, Kiri V, Maden C, Maier WC. Survival in COPD patients after regular use of fluticasone propionate and salmeterol in general practice. Eur Respir J2002;20:819-25.
26. ↵
    Zhou Z, Rahme E, Abrahamowicz M, Pilote L. Survival bias associated with time-to-treatment initiation in drug effectiveness evaluation: a comparison of methods. Am J Epidemiol2005;162:1016-23.
27. ↵
    Liddell FDK, McDonald JC, Thomas DC. Methods of cohort analysis: appraisal by application to asbestos mining. J R Stat Soc Ser A 1977;140:469-91.
28. ↵
    Breslow NE, Day NE, eds. Fitting models to continuous data. Statistical methods in cancer research. Vol.2. The design and analysis of cohort studies. IARC Scientific Publications, 1987:178-229.
29. ↵
    Suissa S. Novel approaches to pharmacoepidemiology study design and statistical analysis. In: Strom BL, ed. Pharmacoepidemiology. 4th ed. John Wiley, 2005:811-29.
30. ↵
    Jackevicius CA, Mamdani M, Tu JV. Adherence with statin therapy in elderly patients with and without acute coronary syndromes. JAMA2002;288:462-7.
31. ↵
    Ellis JJ, Erickson SR, Stevenson JG, Bernstein SJ, Stiles RA, Fendrick AM. Suboptimal statin adherence and discontinuation in primary and secondary prevention populations. J Gen Intern Med2004;19:638-45.