Refining PREVENT prediction models for 10-year risk of cardiovascular disease using measures of anxiety and depression (original) (raw)

Research

Open Access

CMAJ January 13, 2025 197 (1) E1-E8; DOI: https://doi.org/10.1503/cmaj.240996

Loading

Abstract

Background: Anxiety and depression are associated with cardiovascular disease (CVD). We aimed to investigate whether adding measures of anxiety and depression to the American Heart Association Predicting Risk of Cardiovascular Disease Events (PREVENT) predictors improves the prediction of CVD risk.

Methods: We developed and internally validated risk prediction models using 60% and 40% of the cohort data from the UK Biobank, respectively. Mental health predictors included baseline depressive symptom score and self-reported and record-based history of anxiety and depression diagnoses before the baseline. We identified CVD events using hospital admission and death certificate data over a 10-year period from baseline. We determined incremental predictive values by adding the mental health predictors to the PREVENT predictors using Harrell’s C-indices, sensitivity, specificity, and net reclassification improvement indices. We used a threshold of 10-year risk of incident CVD of greater than 5%.

Results: Of the 502 366 UK Biobank participants, we included 195 489 in the derivation set and 130 326 in the validation set. In the validation set, the inclusion of all mental health measures, except self-reported anxiety, produced a very modest increase in the C-index and specificity while sensitivity remained unchanged. Among these mental health predictors, depressive symptom score produced the greatest improvements in both C-index (difference of 0.005, 95% confidence interval 0.004–0.006) and specificity (difference of 0.89%). Depressive symptom score showed similar small improvements in female and male validation sets.

Interpretation: Our findings suggest that the inclusion of measures of depression and anxiety in PREVENT would have little additional effect on the risk classification of CVD at the population level and may not be worthwhile.

The global prevalence of cardiovascular disease (CVD) exceeded half a billion people in 2019.1 Risk assessment plays a key role in targeting prevention to reduce this burden of disease. New CVD risk prediction equations — the American Heart Association Predicting Risk of Cardiovascular Disease Events (PREVENT) equations — were developed by integrating predictors relevant to cardiovascular, kidney, and metabolic risks (e.g., obesity, diabetes, chronic kidney disease)2,3 because these conditions increasingly cluster together and are associated with CVD development, including heart failure, especially in racial and ethnic minority groups.2,3

Anxiety and depression are the most common mental health conditions worldwide.4 Each of these conditions has increased in prevalence by around 50% over the past 20 years and they are likely to increase the risk of CVD.4 Meta-analyses showed around 50% excess risk of incident CVD among people with anxiety or depression,5,6 which remained after adjusting for traditional CVD risk factors such as smoking, diabetes, and hypertension.5,7 We previously corroborated these findings using self-reported and record-based histories of anxiety and depression diagnoses and further showed that these 2 mental health conditions were independently associated with CVD.8 Importantly, baseline depression symptoms were also associated with CVD.9

Incorporating these various measures of anxiety and depression in PREVENT may detect high-risk groups that were overlooked by established predictors. We sought to determine whether including these mental health conditions with the PREVENT predictors improved the prediction of CVD risk.

Methods

Study design

We developed and internally validated prediction models of 10-year risk of CVD to investigate whether adding measures of anxiety and depression to the PREVENT predictors improves performance.

In 2024, the American Heart Association’s PREVENT equations were developed to predict the risk of first CVD event for the general adult population aged 30–79 years in the United States.2 The PREVENT equations were based on a large contemporary sample and included heart failure and kidney function as a CVD outcome and predictor, respectively, unlike previous pooled cohort equations. Calculated risks from PREVENT are intended to inform risk discussions between patients and clinicians and help guide therapeutic strategies. Acceptable risk thresholds are expected to be defined by future guidelines.3 In addition to the total CVD prediction model and core predictors, the PREVENT equations include optional models (e.g., atherosclerotic CVD, heart failure) and predictors (e.g., social deprivation index, glycated hemoglobin A1c [HbA1c], urine albumin:creatinine ratio). Detailed information about the PREVENT equations can be found elsewhere.2,3

Data sources

We developed prediction models using cohort data from the UK Biobank. Between 2007 and 2010, UK Biobank recruited more than 500 000 participants (a 5.5% response rate) from the general population aged 40–69 years.10,11 Participants were asked to undergo a baseline assessment at 1 of 22 centres across England, Scotland, and Wales. With the consent of participants, UK Biobank linked the baseline data to routine National Health Service databases to provide medical history and follow-up information on health events.12,13 The National Health Service Information Centre (England and Wales) and the National Health Service Central Register (Scotland) provided death certificate data; the Hospital Episode Statistics (England and Wales) and the Scottish Morbidity Records (Scotland) provided hospital admission data. Data were linked by UK Biobank staff using the National Health Service number (England and Wales) and Community Health number (Scotland).

In our study, we excluded participants if they had a history of CVD before the baseline assessment or missing values for the predictors. We also considered participants to have missing values if they had extreme clinical values for systolic blood pressure (< 90 mm Hg or > 200 mm Hg), total cholesterol (< 130 mg/dL or > 320 mg/dL), high-density-lipoprotein cholesterol (HDL-C, < 20 mg/dL or > 100 mg/dL), or estimated glomerular filtration rate (eGFR, < 0.1 or > 99.9 percentiles).

Outcome

The outcome was incident CVD, comprising coronary artery disease, stroke, and heart failure (International Classification of Diseases, 10th Revision [ICD-10] codes I20–25, I42.0, I42.6, I42.7, I42.9, I50, I60–64, or I110). Non-CVD death was a competing risk. Death certificate data were available up to November 2022. The hospital admission data were available up to October 2022 for England, May 2022 for Wales, and August 2022 for Scotland. Our analysis was restricted to a 10-year follow-up period. We censored participants at the end of the 10-year follow-up period unless they developed CVD or were censored earlier because of non-CVD death.

Predictors

We selected predictors for mental health conditions (depressive symptom score and self-reported and recorded history of anxiety and depression diagnoses) and those used in the PREVENT equations (age, sex, systolic blood pressure, HDL-C, non-HDL-C, eGFR, diabetes, current smoking, antihypertensive treatment, and statin treatment). Depressive symptom score was derived from the 4 items of the Patient Health Questionnaire-9 (PHQ-9),14,15 self-reported at the baseline assessment using a touchscreen. The questionnaire asked, “Over the past 2 weeks, how often have you felt down, depressed or hopeless?”; “Over the past 2 weeks, how often have you had little interest or pleasure in doing things?”; “Over the past 2 weeks, how often have you felt tense, fidgety or restless?”; and “Over the past 2 weeks, how often have you felt tired or had little energy?”. For each question, participants selected either “not at all” (scored 0), “several days” (scored 1), “more than half of the days” (scored 2), or “nearly every day” (scored 3). Finally, the answers were summed so that the total score ranged from 0 to 12, with higher scores indicating more severe conditions. Self-reported anxiety and depression diagnoses were confirmed during a verbal interview at the baseline assessment if participants responded to a touchscreen questionnaire that they had been told by a doctor that they had a serious illness or disability. In the interview, a trained nurse asked about past and current mental health conditions and coded these using an ICD-10 coding tree.16 Recorded anxiety and depression diagnoses before the baseline assessment were ascertained through record linkage to hospital admission data and defined as an ICD-10 code of F40–43 and F32–33, respectively; therefore, this captured only people admitted to hospital for anxiety or depression.

Age, sex, and current smoking status were self-reported by participants using a touchscreen questionnaire at the baseline assessment. Age was treated as a continuous variable, while sex and current smoking were coded as binary variables (male or female and yes or no, respectively). A trained nurse measured systolic blood pressure at the baseline assessment. Total cholesterol and HDL-C were measured on baseline blood samples and non-HDL-C was calculated by subtracting HDL-C from total cholesterol. Creatinine level was measured using baseline blood samples and used to calculate eGFR using the Chronic Kidney Disease Epidemiology Collaboration 2021 creatinine equation.17 Systolic blood pressure, HDL-C, non-HDL-C, and eGFR were continuous variables. Diabetes was defined as an HbA1c level of 6.5% or higher on baseline blood samples, self-reported use of glucose-lowering medication, or self-reported physician diagnosis, and was categorized as yes or no. Antihypertensive treatment and statin treatment were self-reported using the baseline touchscreen questionnaire and categorized as yes or no.

Statistical analyses

We randomly split our data set into the derivation set (60%) and the validation set (40%). In the derivation set, we evaluated the associations between the predictors and the outcome by single-predictor and multivariable models, including the PREVENT predictors with each of the mental health predictors. We also included anxiety, depression, and depressive symptom score together to assess whether associations remained after adjusting for other mental health predictors.

In the validation set, we evaluated and compared incremental predictive values of the mental health predictors with the PREVENT predictors using Harrell’s C-index; sensitivity, specificity, and positive and negative predictive values; and net reclassification indices by applying the models developed in the derivation set (Appendix 1, Supplementary Table 1, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.240996/tab-related-content). We estimated the 10-year CVD risk from these coefficients and 10-year baseline survival (Appendix 1, Supplementary Table 1). We assessed calibration using the calibration slope and the expected:observed event probability ratio. We calculated all predictive performance measures (except the C-index, which does not require a risk threshold) by setting a threshold of 10-year risk of incident CVD greater than 5% because this is the minimum level of risk at which to consider statin therapy initiation, as recommended by the 2019 American College of Cardiology and American Heart Association guideline.18 We also evaluated incremental predictive values separately in male and female validation sets.

We used Cox proportional hazards models to calculate coefficients and 95% confidence intervals (CIs) associated with each predictor of CVD. We calculated the 95% CIs of the C-index and net reclassification improvement indices using the variance and percentile bootstrap method, respectively. In subsequent analyses, we used a penalized spline basis for depressive symptom scores to assess whether considering nonlinearity improved the predictive ability. We used interaction terms of anxiety and depression to assess their joint contribution to prediction. For descriptive purposes, we calculated events per parameter for both the derivation data set (n = 1052) and validation data set (n = 709) by including all predictors and confirmed these to well exceed the conventional threshold of 10.

We evaluated incremental predictive values of interaction between mental health and PREVENT predictors in 2 stages. First, a mental health predictor and each of the PREVENT predictors were used to create interaction terms. We compared models including these interaction terms and selected the one with the lowest Akaike Information Criterion (AIC) result. We then examined the predictive performance for those selected models.

In sensitivity analyses, we repeated the main analysis using another random split of participants between the derivation and validations sets and again using imputed data. Despite the small proportions of missing values of all data in our study (3.8%), we compared those included and excluded from our analysis and examined whether data imputation changed our main results using missForest.19,20 Lastly, we evaluated the predictive performance of the publicly available PREVENT equations after logistic calibration and model revisions. We adjusted the intercept and all predictor regression coefficients of publicly available PREVENT equations by 1 overall adjustment factor (i.e., calibration slope) by fitting a new logistic regression model in our data set including the linear predictor as the only predictor. We revised the model by fitting logistic regression models, including each of the mental health conditions in addition to the linear predictor.

We conducted all analyses using R version 4.0.2 with the survival (3.3–1), compareC (1.3.2), nricens (1.6), pROC (1.18.5), and missForest (1.5) packages.

Ethics approval

The original UK Biobank study was approved by the North-West Multi-Centre Research Ethics Committee (no. 11/NW/0382). The investigation conformed to the principles outlined in the Declaration of Helsinki. Informed consent was obtained from all participants included in the study. We conducted our study under the UK Biobank application number 71392.

Results

Of the 502 366 UK Biobank participants, we excluded 176 551 participants: 164 803 because of missing predictor values and 11 748 because of history of CVD before the baseline assessment. Consequently, we included 325 815 participants in the analyses (Figure 1). In the derivation set, 195 489 participants had 15 787 CVD events over a mean follow-up time of 9.5 years. Participants who had a CVD event were older and more likely to be male and current smokers (Table 1). They had higher systolic blood pressure and lower eGFR and were more likely to have diabetes, take antihypertensive or statin treatment, and have mental health conditions. In the validation set, 130 326 participants had 10 639 CVD events over a mean follow-up time of 9.5 years. Overall, the baseline characteristics of participants in the validation set were similar to those of the derivation set.

Table 1:

Participant baseline characteristics

All mental health predictors were associated with CVD in the single predictor models, and this association remained after adjusting for the PREVENT predictors (Table 2). When additionally adjusting for self-reported or recorded anxiety and depression and depressive symptom score together, the magnitude of the association between depressive symptom score and CVD changed little while the associations with anxiety and depression were attenuated. In the validation set, inclusion of all mental health predictors, except for self-reported anxiety, modestly increased the C-index compared with the original PREVENT predictors (Table 3 and Appendix 1, Supplementary Table 2). The largest improvements were observed in models that included depressive symptom score, whether included in a linear (C-index difference 0.00502, 95% CI 0.00393–0.00611) or nonlinear (C-index difference 0.00516, 95% CI 0.00403–0.00629) model. This result was only slightly improved by adding histories of anxiety and depression together. The inclusion of depressive symptom score showed the largest, but nonetheless modest, improvement in specificity (0.89%); sensitivity remained unchanged with inclusion of any of the mental health predictors (Table 4). Consequently, the model including depressive symptom score yielded the largest overall (1.14, 95% CI 0.69–1.55) and nonevent (0.89, 95% CI 0.74–1.02) net reclassification indices (Table 5). Nonlinear modelling of depressive symptoms changed the results little (Appendix 1, Supplementary Tables 3 and 4). All main models were well calibrated (Appendix 1, Supplementary Table 5).

Table 2:

Associations of Predicting Risk of Cardiovascular Disease Events (PREVENT) predictors and mental health conditions with development of cardiovascular disease in the derivation set

Table 3:

Comparisons of C-indices between Predicting Risk of Cardiovascular Disease Events (PREVENT) and models including mental health conditions in the validation set

Table 4:

Sensitivity, specificity, and positive and negative predictive values of Predicting Risk of Cardiovascular Disease Events (PREVENT) and models including mental health conditions in the validation set*

Table 5:

Net reclassification indices in the validation set for models including mental health conditions in addition to Predicting Risk of Cardiovascular Disease Events (PREVENT) predictors

The sex-specific prediction performance evaluations are shown in Appendix 1, Supplementary Tables 6–11. Generally, the pattern was consistent, whereby the depressive symptom score, in either linear or nonlinear form, produced modest improvements in the C-index and specificity.

Based on the AIC, we selected 5 models that included an interaction term of mental health and PREVENT predictors (Appendix 1, Supplementary Table 12). The predictive performance of these selected models were modest and similar to those without interaction terms (Appendix 1, Supplementary Tables 13 and 14).

In sensitivity analyses, repeating the analysis with another random split yielded results consistent with those of our main analysis (Appendix 1, Supplementary Tables 15–17). Comparing characteristics of included and excluded participants, we found that excluded participants were more likely to be female, have diabetes, be current smokers, and be taking antihypertensive and statin treatments (Appendix 1, Supplementary Table 18). Predictors with the most missing data were total cholesterol (and therefore, non-HDL-C) and HDL-C (Appendix 1, Supplementary Table 19). Overall, the analyses based on imputed data yielded results consistent with those of our main analysis (Appendix 1, Supplementary Tables 20–23). Finally, adding mental health conditions to the PREVENT equations after logistic calibration led to modest improvements in the area under the curve overall, with the greatest improvement observed for the model with depressive symptom score (Appendix 1, Supplementary Table 24).

Interpretation

We found that the depressive symptom score was associated with CVD after adjusting for the PREVENT predictors, as well as history of anxiety and depression. Including the depressive symptom score showed the greatest, but still very modest, improvement in the performance of the model with PREVENT predictors across all validation sets (overall and by sex) in terms of C-index and specificity. From our findings, models including depressive symptom score were more likely to differentiate those at higher risk of CVD from those at lower risk and, in line with American College of Cardiology and American Heart Association prevention guideline, for every 1000 people at lower risk, about 9 would no longer be incorrectly classified as high risk. This suggests that the inclusion of depression and anxiety in PREVENT equations would have relatively little additional effect on the risk classification of CVD at the population level and may not be worthwhile.

However, according to the American Heart Association, new CVD risk predictors should be specific markers that identify a targeted therapeutic pathway or actionable response and be routinely available in primary care settings.2,3 In keeping with this recommendation, the depressive symptom score can help identify previously undiagnosed depression and serves as a measure of the severity of the symptom; depression can be managed using a range of interventions such as antidepressants, psychotherapy, and physical exercise.21–23 Importantly, the depressive symptom score could be readily available from reliable information, at relatively low cost and without requiring any invasive procedure, using the 4 items of PHQ-9, in keeping with guideline-recommended routine screening in primary care 24 and could therefore be fairly easily considered when considering CVD risk prevention, despite its moderate contribution. Among many self-report tools (e.g., the Beck Depression Inventory), PHQ-9 has been validated most often,25,26 and an ultra-short form of this tool, the 4-item PHQ, would be convenient and acceptable in primary care settings, given the demands on providers’ time.27–29

Although the lack of evidence for PREVENT hampers direct comparisons, 2 studies have given relatively consistent findings with our study by investigating the addition of common mental health conditions to CVD risk prediction. A score developed by machine learning using data from a single-centre survey of people with type 2 diabetes from 1 institution in China — which included self-rated anxiety and depression, as well as traditional CVD risk factors — showed high discrimination.30 This study also found that anxiety and depression ranked in the top 5 predictors based on mean impact values. A 2023 preprint study using UK Biobank data showed that some general mental health measures, such as anxious feeling and insomnia, were in the top 5 of 200 predictors of CVD risk based on a discrimination measure.31

Limitations

We developed our models using the UK Biobank cohort data; participants in this cohort are more likely to be White, affluent, and healthy than the general population of the United Kingdom.11 Before generalizing our findings to the UK population or other populations, further validation should be conducted. We split data into derivation and validation sets, which reduced sample size. However, we confirmed that events per parameter exceeded the conventional threshold. We based results on the minimum risk threshold for considering statin therapy initiation as per the American College of Cardiology and American Heart Association prevention guideline because, to date, no guidelines have been tailored to the PREVENT equations. As these results are sensitive to the defined risk threshold, this should be re-evaluated in future guidelines. Using recorded anxiety and depression diagnoses identified only a small number of participants, likely because this captured only severe cases requiring hospital admission. However, we also used self-reported diagnoses and the depressive symptom score to include a broader range of these conditions. The measures used for this study may not identify people with anxiety or depression as well as other established mental health instruments, such as the Beck Depression Inventory or the Generalized Anxiety Disorder Scale, or diagnostic interviews, which could reduce their prediction performance. However, the measures used in this study were drawn from real-world settings (e.g., electronic health records, short self-reports), since they are of lower cost and can be implemented relatively easily. Lastly, we excluded participants with missing predictors, which could bias our results. However, the absolute differences in the characteristics between those included and excluded were small (a maximum of 4 percentage points). Because blood samples were taken routinely from all participants in the UK Biobank, the most likely reason for missing data for serum biomarkers such as HDL-C would be technical difficulties relating to the sample, which are likely to have happened randomly.32 The analysis based on imputed data did not meaningfully change our findings.

Conclusion

Our findings suggest that the inclusion of measures of depression and anxiety in PREVENT would have little additional effect on the risk classification of CVD at the population level and may not be worthwhile. Investigating broader mental health conditions using more established tools or diagnostic interview data could be the focus of future studies to further refine CVD risk classification.

Acknowledgements

The authors thank all participants and staff of UK Biobank. UK Biobank was established by the Wellcome Trust, Medical Research Council, Department of Health, Scottish Government, and Northwest Regional Development Agency. UK Biobank has also had funding from the Welsh Assembly Government and the British Heart Foundation. The authors are also grateful to the Medical Research Council and the University of Edinburgh/University of Glasgow. This work was supported by the Medical Research Council (MR/N013166/1-LGH/MS/MED2525).

Footnotes

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is non-commercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/

References

  1. ↵
  2. ↵
  3. ↵
  4. ↵
    GBD 2019 Mental Disorders Collaborators. Global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet Psychiatry 2022;9:137–50.
  5. ↵
  6. ↵
  7. ↵
  8. ↵
  9. ↵
  10. ↵
  11. ↵
  12. ↵
  13. ↵
  14. ↵
  15. ↵
  16. ↵
  17. ↵
  18. ↵
  19. ↵
  20. ↵
  21. ↵
  22. ↵
  23. ↵
    US Preventive Services Task Force; Barry MJ, Nicholson WK, Silverstein M, et al. Screening for depression and suicide risk in adults: US Preventive Services Task Force recommendation statement. JAMA 2023;329:2057–67.
  24. ↵
  25. ↵
    Accuracy of Patient Health Questionnaire-9 (PHQ-9) for screening to detect major depression: individual participant data meta-analysis. BMJ 2019; 365:l1781. doi:10.1136/bmj.l1781.
  26. ↵
  27. ↵
  28. ↵
  29. ↵
  30. ↵