Comparison of Trends in Sepsis Incidence and Coding Using Administrative Claims Versus Objective Clinical Data (original) (raw)
Journal Article
,
1
Department of Population Medicine
,
Harvard Medical School and Harvard Pilgrim Health Care Institute
2
Division of Infectious Diseases
,
Brigham and Women's Hospital
,
Boston, Massachusetts
*Correspondence: Chanu Rhee, MD, Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 133 Brookline Ave, 6th Floor, Boston, MA 02215 (crhee1@partners.org).
Search for other works by this author on:
,
1
Department of Population Medicine
,
Harvard Medical School and Harvard Pilgrim Health Care Institute
Search for other works by this author on:
,
1
Department of Population Medicine
,
Harvard Medical School and Harvard Pilgrim Health Care Institute
Search for other works by this author on:
,
1
Department of Population Medicine
,
Harvard Medical School and Harvard Pilgrim Health Care Institute
Search for other works by this author on:
for the Centers for Disease Control and Prevention Epicenters Program
1
Department of Population Medicine
,
Harvard Medical School and Harvard Pilgrim Health Care Institute
2
Division of Infectious Diseases
,
Brigham and Women's Hospital
,
Boston, Massachusetts
Search for other works by this author on:
Accepted:
11 September 2014
Published:
25 September 2014
Cite
Chanu Rhee, Michael V. Murphy, Lingling Li, Richard Platt, Michael Klompas, for the Centers for Disease Control and Prevention Epicenters Program, Comparison of Trends in Sepsis Incidence and Coding Using Administrative Claims Versus Objective Clinical Data, Clinical Infectious Diseases, Volume 60, Issue 1, 1 January 2015, Pages 88–95, https://doi.org/10.1093/cid/ciu750
Close
Navbar Search Filter Mobile Enter search term Search
At 2 academic hospitals, the incidence of hospitalizations with sepsis claims rose dramatically from 2003 to 2012, but hospitalizations with objective clinical markers remained stable or decreased. Claims became more sensitive, and septicemia diagnoses were increasingly applied to nonbacteremic patients.
Background. National reports of a dramatic rise in sepsis incidence are largely based on analyses of administrative databases. It is unclear if these estimates are biased by changes in coding practices over time.
Methods. We calculated linear trends in the annual incidence of septicemia, sepsis, and severe sepsis at 2 academic hospitals from 2003 to 2012 using 5 different claims methods and compared case identification rates to selected objective clinical markers, including positive blood cultures, vasopressors, and/or lactic acid levels.
Results. The annual incidence of hospitalizations with sepsis claims increased over the decade, ranging from a 54% increase for the method combining septicemia, bacteremia, and fungemia codes (P < .001 for linear trend) to a 706% increase for explicit severe sepsis/septic shock codes (P = .001). In contrast, the incidence of hospitalizations with positive blood cultures decreased by 17% (P = .006), and hospitalizations with positive blood cultures with concurrent vasopressors and/or lactic acidosis remained stable (P = .098). The sensitivity of sepsis claims for capturing hospitalizations with positive blood cultures with concurrent vasopressors and/or lactic acidosis increased (P < .001 for all methods), whereas the proportion of septicemia hospitalizations with positive blood cultures decreased from 50% to 30% (P < .001).
Conclusions. The incidence of hospitalizations with sepsis codes rose dramatically while hospitalizations with corresponding objective clinical markers remained stable or decreased. Coding for sepsis has become more inclusive, and septicemia diagnoses are increasingly being applied to patients without positive blood cultures. These changes likely explain some of the apparent rise in sepsis incidence and underscore the need for more reliable surveillance methods.
Multiple reports suggest that the incidence of septicemia, sepsis, and severe sepsis has been increasing steadily for the past several decades [1–4]. Data from the National Hospital Discharge Survey, for example, recently demonstrated a doubling of septicemia and sepsis rates in the United States from 2000 to 2008 [4]. Commonly cited explanations for this emerging crisis include the aging population, increasing use of immunosuppression and invasive procedures, and the spread of multidrug-resistant pathogens [5–7]. Most existing studies on temporal trends, however, are based on analyses of administrative data. It is therefore possible that some of the observed increase in incidence is due to changes in diagnosis and coding practices rather than true increases in disease frequency [8–10].
An accurate understanding of trends in sepsis burden has critical public health and policy implications, particularly in light of the proliferation of sepsis quality improvement initiatives, regulatory mandates for sepsis care, and public reporting for sepsis-related outcomes [10]. However, estimates of changing sepsis burden over time may be particularly susceptible to shifts in diagnosis and coding patterns for several reasons. The clinical definition of sepsis includes subjective components, most notably the clinician's opinion on whether or not the patient's clinical syndrome is due to infection and, in the case of severe sepsis, whether or not acute organ dysfunction is present and attributable to infection. Estimating sepsis rates using claims could compound this uncertainty because education and care improvement campaigns, public reporting requirements, and reimbursement incentives are exerting continual pressures on the ways clinicians and hospitals diagnose and code for sepsis.
Imprecise and changing use of terminology may also be a source of bias. For example, the term “septicemia” technically refers to infection in the blood (from the Greek αἷμα, heme = blood) but in practice has been used variably to describe infections with and without positive blood cultures [11, 12]. Due to this confusion, the American College of Chest Physicians and the Society of Critical Care Medicine recommended discarding the term in 1991 [13], but it is still often used interchangeably with sepsis and remains a major component of coding schemes for epidemiologic studies of sepsis [1–4, 14, 15].
Our aim was to examine trends in sepsis incidence and coding patterns by comparing claims data to objective clinical markers for “septicemia” and severe sepsis, such as positive blood cultures, vasopressor use, and lactic acidosis. We focused on vasopressors and lactic acidosis because they represent an abnormal state of hypoperfusion, regardless of baseline organ function, and unambiguously equate to severe sepsis/septic shock in the presence of positive blood cultures with significant pathogens.
METHODS
Setting, Population, and Data Source
We identified all patients aged ≥18 years admitted to Massachusetts General Hospital and Brigham and Women's Hospital between 1 January 2003 and 31 December 2012. Massachusetts General Hospital (950 beds) and Brigham and Women's Hospital (779 beds) are academic hospitals located in Boston, Massachusetts. We retrieved comprehensive clinical data on all patients who had either a blood culture drawn and/or International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) discharge diagnosis codes indicative of septicemia, sepsis, or severe sepsis (defined below). We retrieved patients' demographics, ICD-9-CM codes, medications, laboratory results, and dates of admission, discharge, and death from the hospital's Research Patient Data Registry [16], a centralized clinical data warehouse. We obtained blood culture data from the clinical microbiology laboratory data. We derived patients' comorbidities from their ICD-9-CM and diagnosis-related group codes using the methods of Elixhauser et al and Charlson et al [17, 18]. We calculated the total number of adult inpatient discharges per year by searching for unique records associated with a diagnosis-related group code.
Claims Definitions
Septicemia, Sepsis, and Severe Sepsis
We examined several different claims-based definitions (Table 1): (1) septicemia codes (038) [4]; (2) the “Martin” definition of sepsis, consisting of codes that imply a disseminated bloodstream infection (septicemia, bacteremia, and fungemia) [1]; (3) “explicit” severe sepsis (995.92) and septic shock (785.52) codes; and (4) 2 commonly cited “implicit” code combinations used to denote severe sepsis: the “Angus” and “Dombrovskiy” definitions. The Angus definition uses 1286 codes for infection and 13 codes for acute organ dysfunction; if a code from both categories is present, the patient is labeled as having severe sepsis [14]. We used a published, modified version that also labels patients as Angus-positive if explicit severe sepsis or septic shock codes are present [19]. The Dombrovskiy definition uses a similar scheme of infection and organ dysfunction, but mainly relies on septicemia codes [2]. The ICD-9-CM codes for sepsis (995.91), severe sepsis (995.92), and septic shock (785.52) were introduced in October 2002, before the beginning of our study period.
Table 1.
Claims Definitions and Corresponding Objective Clinical Markers for Comparison
Claims Definition | ICD-9-CM Codesa | Clinical Markers |
---|---|---|
Septicemia | 038 | Positive blood culturesb |
Martin sepsis | 020.0, 038, 112.5, 112.81, 117.9, 790.7 | Positive blood cultures |
Angus severe sepsis | Infection codes: 001–005, 008–018, 020–027, 030–041, 090–098, 100–104, 110–112, 114–118, 320, 322, 324, 325, 420, 421, 451, 461–465, 481, 482, 485, 486, 491.21, 494, 510, 513, 540–542, 562.01, 562.03, 562.11, 562.13, 566, 567, 569.5, 569.83, 572.0, 572.1, 575.0, 590, 597, 599.0, 601, 614, 615, 616, 681–683, 686, 711.0, 730, 790.7, 996.6, 998.5, 993.3ANDOrgan dysfunction codes: 96.7, 286.6, 286.9, 287.4, 287.5, 293, 348.1, 348.3, 458, 570, 573.4, 584, 785.5OR995.92, 785.52 alone | Positive blood culture + vasopressorc use or lactic acid ≥4.0 mmol/L within ±1 d of culture |
Dombrovskiy severe sepsis | Infection codes: 003.1, 020.2, 022.3, 036.2, 036.3, 038.0–038.4, 038.8, 038.9, 054.5, 098.89, 112.5, 785.52, 995.91, 995.92ANDOrgan dysfunction codes: 286.6, 286.9, 287.5, 293.0, 297.4, 348.1, 348.3, 427.5, 458.0, 458.8, 458.9, 518.81, 518.82, 570, 572.2, 573.4, 584, 780.01, 785.5, 786.09, 799.1, 796.3 | Positive blood culture + vasopressor use or lactic acid ≥4.0 mmol/L within ±1 d of culture |
Explicit severe sepsis/septic shock | 785.52, 995.92 | Positive blood culture + vasopressor use or lactic acid ≥4.0 mmol/L within ±1 d of culture |
Hypotension/shock | 427.5, 458.0, 458.8, 458.9, 785.5, 796.3 | Vasopressor use |
Metabolic/lactic acidosis | 276.2 | Lactic acid ≥4.0 mmol/L |
Claims Definition | ICD-9-CM Codesa | Clinical Markers |
---|---|---|
Septicemia | 038 | Positive blood culturesb |
Martin sepsis | 020.0, 038, 112.5, 112.81, 117.9, 790.7 | Positive blood cultures |
Angus severe sepsis | Infection codes: 001–005, 008–018, 020–027, 030–041, 090–098, 100–104, 110–112, 114–118, 320, 322, 324, 325, 420, 421, 451, 461–465, 481, 482, 485, 486, 491.21, 494, 510, 513, 540–542, 562.01, 562.03, 562.11, 562.13, 566, 567, 569.5, 569.83, 572.0, 572.1, 575.0, 590, 597, 599.0, 601, 614, 615, 616, 681–683, 686, 711.0, 730, 790.7, 996.6, 998.5, 993.3ANDOrgan dysfunction codes: 96.7, 286.6, 286.9, 287.4, 287.5, 293, 348.1, 348.3, 458, 570, 573.4, 584, 785.5OR995.92, 785.52 alone | Positive blood culture + vasopressorc use or lactic acid ≥4.0 mmol/L within ±1 d of culture |
Dombrovskiy severe sepsis | Infection codes: 003.1, 020.2, 022.3, 036.2, 036.3, 038.0–038.4, 038.8, 038.9, 054.5, 098.89, 112.5, 785.52, 995.91, 995.92ANDOrgan dysfunction codes: 286.6, 286.9, 287.5, 293.0, 297.4, 348.1, 348.3, 427.5, 458.0, 458.8, 458.9, 518.81, 518.82, 570, 572.2, 573.4, 584, 780.01, 785.5, 786.09, 799.1, 796.3 | Positive blood culture + vasopressor use or lactic acid ≥4.0 mmol/L within ±1 d of culture |
Explicit severe sepsis/septic shock | 785.52, 995.92 | Positive blood culture + vasopressor use or lactic acid ≥4.0 mmol/L within ±1 d of culture |
Hypotension/shock | 427.5, 458.0, 458.8, 458.9, 785.5, 796.3 | Vasopressor use |
Metabolic/lactic acidosis | 276.2 | Lactic acid ≥4.0 mmol/L |
Abbreviation: ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification.
a Where 3- or 4-digit codes are listed, all associated subcodes were included. Codes were taken from diagnoses on hospital discharge.
b Blood cultures positive for common commensal organisms (Corynebacterium species other than C. diphtheriae, Bacillus species other than B. anthracis, Propionibacterium species, coagulase-negative staphylococci, viridans group streptococci, Aerococcus species, and Micrococcus species) were disregarded, unless they were isolated from at least 2 cultures drawn on separate occasions on 2 consecutive calendar days.
c Vasopressors were defined as dopamine, norepinephrine, vasopressin, phenylephrine, or epinephrine.
Table 1.
Claims Definitions and Corresponding Objective Clinical Markers for Comparison
Claims Definition | ICD-9-CM Codesa | Clinical Markers |
---|---|---|
Septicemia | 038 | Positive blood culturesb |
Martin sepsis | 020.0, 038, 112.5, 112.81, 117.9, 790.7 | Positive blood cultures |
Angus severe sepsis | Infection codes: 001–005, 008–018, 020–027, 030–041, 090–098, 100–104, 110–112, 114–118, 320, 322, 324, 325, 420, 421, 451, 461–465, 481, 482, 485, 486, 491.21, 494, 510, 513, 540–542, 562.01, 562.03, 562.11, 562.13, 566, 567, 569.5, 569.83, 572.0, 572.1, 575.0, 590, 597, 599.0, 601, 614, 615, 616, 681–683, 686, 711.0, 730, 790.7, 996.6, 998.5, 993.3ANDOrgan dysfunction codes: 96.7, 286.6, 286.9, 287.4, 287.5, 293, 348.1, 348.3, 458, 570, 573.4, 584, 785.5OR995.92, 785.52 alone | Positive blood culture + vasopressorc use or lactic acid ≥4.0 mmol/L within ±1 d of culture |
Dombrovskiy severe sepsis | Infection codes: 003.1, 020.2, 022.3, 036.2, 036.3, 038.0–038.4, 038.8, 038.9, 054.5, 098.89, 112.5, 785.52, 995.91, 995.92ANDOrgan dysfunction codes: 286.6, 286.9, 287.5, 293.0, 297.4, 348.1, 348.3, 427.5, 458.0, 458.8, 458.9, 518.81, 518.82, 570, 572.2, 573.4, 584, 780.01, 785.5, 786.09, 799.1, 796.3 | Positive blood culture + vasopressor use or lactic acid ≥4.0 mmol/L within ±1 d of culture |
Explicit severe sepsis/septic shock | 785.52, 995.92 | Positive blood culture + vasopressor use or lactic acid ≥4.0 mmol/L within ±1 d of culture |
Hypotension/shock | 427.5, 458.0, 458.8, 458.9, 785.5, 796.3 | Vasopressor use |
Metabolic/lactic acidosis | 276.2 | Lactic acid ≥4.0 mmol/L |
Claims Definition | ICD-9-CM Codesa | Clinical Markers |
---|---|---|
Septicemia | 038 | Positive blood culturesb |
Martin sepsis | 020.0, 038, 112.5, 112.81, 117.9, 790.7 | Positive blood cultures |
Angus severe sepsis | Infection codes: 001–005, 008–018, 020–027, 030–041, 090–098, 100–104, 110–112, 114–118, 320, 322, 324, 325, 420, 421, 451, 461–465, 481, 482, 485, 486, 491.21, 494, 510, 513, 540–542, 562.01, 562.03, 562.11, 562.13, 566, 567, 569.5, 569.83, 572.0, 572.1, 575.0, 590, 597, 599.0, 601, 614, 615, 616, 681–683, 686, 711.0, 730, 790.7, 996.6, 998.5, 993.3ANDOrgan dysfunction codes: 96.7, 286.6, 286.9, 287.4, 287.5, 293, 348.1, 348.3, 458, 570, 573.4, 584, 785.5OR995.92, 785.52 alone | Positive blood culture + vasopressorc use or lactic acid ≥4.0 mmol/L within ±1 d of culture |
Dombrovskiy severe sepsis | Infection codes: 003.1, 020.2, 022.3, 036.2, 036.3, 038.0–038.4, 038.8, 038.9, 054.5, 098.89, 112.5, 785.52, 995.91, 995.92ANDOrgan dysfunction codes: 286.6, 286.9, 287.5, 293.0, 297.4, 348.1, 348.3, 427.5, 458.0, 458.8, 458.9, 518.81, 518.82, 570, 572.2, 573.4, 584, 780.01, 785.5, 786.09, 799.1, 796.3 | Positive blood culture + vasopressor use or lactic acid ≥4.0 mmol/L within ±1 d of culture |
Explicit severe sepsis/septic shock | 785.52, 995.92 | Positive blood culture + vasopressor use or lactic acid ≥4.0 mmol/L within ±1 d of culture |
Hypotension/shock | 427.5, 458.0, 458.8, 458.9, 785.5, 796.3 | Vasopressor use |
Metabolic/lactic acidosis | 276.2 | Lactic acid ≥4.0 mmol/L |
Abbreviation: ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification.
a Where 3- or 4-digit codes are listed, all associated subcodes were included. Codes were taken from diagnoses on hospital discharge.
b Blood cultures positive for common commensal organisms (Corynebacterium species other than C. diphtheriae, Bacillus species other than B. anthracis, Propionibacterium species, coagulase-negative staphylococci, viridans group streptococci, Aerococcus species, and Micrococcus species) were disregarded, unless they were isolated from at least 2 cultures drawn on separate occasions on 2 consecutive calendar days.
c Vasopressors were defined as dopamine, norepinephrine, vasopressin, phenylephrine, or epinephrine.
Hypotension/Shock and Lactic Acidosis
We compiled the different hypotension and cardiovascular shock codes used in the Angus, Dombrovskiy, and Martin studies into 1 set of codes (Table 1). We also examined the code for metabolic/lactic acidosis used in the Martin study (276.2).
Objective Clinical Markers of Septicemia and Severe Sepsis
Positive Blood Cultures
As a comparison for septicemia and the Martin codes, we examined the incidence of all hospitalizations with at least 1 positive blood culture, and the sensitivity and positive predictive value (PPV) of septicemia and Martin codes for identifying these hospitalizations. As per Centers for Disease Control and Prevention criteria, positive blood cultures with common commensal organisms were disregarded, unless they were isolated from at least 2 cultures drawn on separate occasions on 2 consecutive calendar days (Table 1) [20].
Positive Blood Cultures With Concurrent Vasopressors or Lactic Acidosis
We examined the incidence of hospitalizations with positive blood cultures with either vasopressors (defined as dopamine, norepinephrine, vasopressin, phenylephrine, or epinephrine) or a lactic acid level ≥4.0 mmol/L within ±1 day of the positive culture (hereafter referred to as positive BC + pressor or LA ± 1 day). Because severe sepsis can occur without bacteremia and with more mild forms of organ dysfunction, we focused this comparison on the sensitivity of claims rather than PPV. We examined trends in the sensitivity of all of the claims definitions because this unambiguous septic event captures the entire spectrum of septicemia, sepsis, and severe sepsis/septic shock.
Hypotension/Shock and Lactic Acidosis
Within the cohort of patients with blood cultures drawn or a sepsis-related ICD-9-CM discharge code, we examined whether hypotension/shock codes have become more sensitive over time by examining the use of vasopressors. We focused on sensitivity rather than PPV because many patients with hypotension do not require vasopressors. Similarly, we examined the sensitivity of the lactic acidosis code for identifying patients with a lactic acid level ≥4.0 mmol/L.
Analyses
We assessed for linear trends in incidence, claims sensitivity and PPV, and mortality between 2003 and 2012. Average annual changes in outcomes were calculated using fitted linear trends; annual changes in incidence were calculated using 2003 values as the baseline for comparison. Exact binomial 95% confidence intervals (CIs) were calculated for estimated proportions for sensitivity and PPV. All analyses were performed using SAS software version 9.3 (SAS Institute, Cary, North Carolina). We considered P < .05 to be statistically significant and used 2-tailed tests. This study was approved by the Partners Healthcare Institutional Review Board.
RESULTS
Over the 10-year period, there were 1 002 815 adult hospital discharges. The demographic, clinical, and microbiological characteristics of patients with different sepsis claims and objective clinical markers are shown in Table 2.
Table 2.
Characteristics and Outcomes of Patients With Sepsis Claims and Objective Clinical Markers of Septicemia and Severe Sepsis/Septic Shock, 2003–2012
Patient Characteristics and Outcomes | Septicemia | Martin | Angus | Dombrovskiy | Severe Sepsis/Septic Shock | Positive BC | Positive BC + Pressor or LA ± 1 d |
---|---|---|---|---|---|---|---|
No. of discharges, 2003–2012 (N = 1 002 815) | 18 254 (2%) | 29 409 (4%) | 60 770 (9%) | 13 943 (2%) | 9426 (1%) | 20 578 (3%) | 4119 (0.6%) |
Male | 10 038 (55%) | 16 234 (55%) | 31 793 (52%) | 7864 (56%) | 5287 (56%) | 11 269 (55%) | 2436 (59%) |
Non–white race | 3996 (22%) | 6514 (22%) | 12 670 (21%) | 2953 (21%) | 2011 (21%) | 4793 (23%) | 945 (23%) |
Mean age | 63.0 | 61.4 | 65.7 | 63.8 | 63.9 | 59.8 | 61.7 |
Mean Charlson score | 3.8 | 3.8 | 3.7 | 3.8 | 3.8 | 3.7 | 3.8 |
Comorbidities | |||||||
Diabetes | 2514 (14%) | 4524 (15%) | 10 908 (18%) | 1603 (12%) | 1002 (11%) | 3513 (16%) | 550 (13%) |
Heart failure | 3742 (21%) | 5426 (18%) | 14 744 (24%) | 3146 (23%) | 2142 (23%) | 3183 (15%) | 874 (21%) |
Liver disease | 1083 (6%) | 1854 (6%) | 3667 (6%) | 932 (7%) | 563 (6%) | 1409 (7%) | 334 (8%) |
Lung disease | 2159 (12%) | 3423 (12%) | 10 303 (17%) | 1583 (11%) | 1037 (11%) | 2299 (11%) | 434 (11%) |
Lymphoma | 891 (5%) | 1763 (6%) | 2667 (4%) | 681 (5%) | 442 (5%) | 1303 (6%) | 205 (5%) |
Metastatic cancer | 2061 (11%) | 3193 (11%) | 5688 (9%) | 1419 (10%) | 904 (10%) | 2532 (12%) | 454 (11%) |
Renal disease | 2691 (15%) | 4586 (16%) | 10 394 (17%) | 1962 (14%) | 1288 (14%) | 2788 (14%) | 595 (14%) |
≥1 BC drawn | 17 675 (97%) | 28 472 (97%) | 47 111 (78%) | 13 539 (97%) | 9156 (97%) | … | … |
Positive BCsa | 6806 (37%) | 14 028 (48%) | 9378 (15%) | 4997 (36%) | 3194 (34%) | … | … |
Gram positive | 3218 (18%) | 7515 (26%) | 5025 (8%) | 2309 (17%) | 1454 (15%) | 11 320 (55%) | 2053 (50%) |
Gram negative | 2594 (14%) | 4662 (16%) | 2953 (5%) | 1764 (13%) | 1159 (12%) | 6675 (32%) | 1361 (33%) |
Fungi | 272 (1%) | 668 (2%) | 590 (1%) | 395 (3%) | 220 (2%) | 974 (5%) | 251 (6%) |
Multiple | 721 (4%) | 1179 (4%) | 805 (1%) | 527 (4%) | 359 (4%) | 1598 (8%) | 452 (11%) |
Mycobacterial | 1 (0.01%) | 4 (0.01%) | 5 (0.01%) | 2 (0.01%) | 2 (0.02%) | 11 (0.05%) | 2 (0.05%) |
Vasopressor use | 12 092 (61%) | 13 327 (45%) | 23 587 (39%) | 10 149 (73%) | 7608 (81%) | 7013 (34%) | 3925 (95%) |
Median hospital LOS (IQR) | 12 (6–22) | 11 (6–22) | 11 (6–20) | 13 (7–25) | 14 (7–25) | 12 (7–24) | 15 (8–29) |
Hospital mortality | 4889 (27%) | 5398 (18%) | 8582 (14%) | 4659 (33%) | 3488 (37%) | 2782 (14%) | 1427 (35%) |
Patient Characteristics and Outcomes | Septicemia | Martin | Angus | Dombrovskiy | Severe Sepsis/Septic Shock | Positive BC | Positive BC + Pressor or LA ± 1 d |
---|---|---|---|---|---|---|---|
No. of discharges, 2003–2012 (N = 1 002 815) | 18 254 (2%) | 29 409 (4%) | 60 770 (9%) | 13 943 (2%) | 9426 (1%) | 20 578 (3%) | 4119 (0.6%) |
Male | 10 038 (55%) | 16 234 (55%) | 31 793 (52%) | 7864 (56%) | 5287 (56%) | 11 269 (55%) | 2436 (59%) |
Non–white race | 3996 (22%) | 6514 (22%) | 12 670 (21%) | 2953 (21%) | 2011 (21%) | 4793 (23%) | 945 (23%) |
Mean age | 63.0 | 61.4 | 65.7 | 63.8 | 63.9 | 59.8 | 61.7 |
Mean Charlson score | 3.8 | 3.8 | 3.7 | 3.8 | 3.8 | 3.7 | 3.8 |
Comorbidities | |||||||
Diabetes | 2514 (14%) | 4524 (15%) | 10 908 (18%) | 1603 (12%) | 1002 (11%) | 3513 (16%) | 550 (13%) |
Heart failure | 3742 (21%) | 5426 (18%) | 14 744 (24%) | 3146 (23%) | 2142 (23%) | 3183 (15%) | 874 (21%) |
Liver disease | 1083 (6%) | 1854 (6%) | 3667 (6%) | 932 (7%) | 563 (6%) | 1409 (7%) | 334 (8%) |
Lung disease | 2159 (12%) | 3423 (12%) | 10 303 (17%) | 1583 (11%) | 1037 (11%) | 2299 (11%) | 434 (11%) |
Lymphoma | 891 (5%) | 1763 (6%) | 2667 (4%) | 681 (5%) | 442 (5%) | 1303 (6%) | 205 (5%) |
Metastatic cancer | 2061 (11%) | 3193 (11%) | 5688 (9%) | 1419 (10%) | 904 (10%) | 2532 (12%) | 454 (11%) |
Renal disease | 2691 (15%) | 4586 (16%) | 10 394 (17%) | 1962 (14%) | 1288 (14%) | 2788 (14%) | 595 (14%) |
≥1 BC drawn | 17 675 (97%) | 28 472 (97%) | 47 111 (78%) | 13 539 (97%) | 9156 (97%) | … | … |
Positive BCsa | 6806 (37%) | 14 028 (48%) | 9378 (15%) | 4997 (36%) | 3194 (34%) | … | … |
Gram positive | 3218 (18%) | 7515 (26%) | 5025 (8%) | 2309 (17%) | 1454 (15%) | 11 320 (55%) | 2053 (50%) |
Gram negative | 2594 (14%) | 4662 (16%) | 2953 (5%) | 1764 (13%) | 1159 (12%) | 6675 (32%) | 1361 (33%) |
Fungi | 272 (1%) | 668 (2%) | 590 (1%) | 395 (3%) | 220 (2%) | 974 (5%) | 251 (6%) |
Multiple | 721 (4%) | 1179 (4%) | 805 (1%) | 527 (4%) | 359 (4%) | 1598 (8%) | 452 (11%) |
Mycobacterial | 1 (0.01%) | 4 (0.01%) | 5 (0.01%) | 2 (0.01%) | 2 (0.02%) | 11 (0.05%) | 2 (0.05%) |
Vasopressor use | 12 092 (61%) | 13 327 (45%) | 23 587 (39%) | 10 149 (73%) | 7608 (81%) | 7013 (34%) | 3925 (95%) |
Median hospital LOS (IQR) | 12 (6–22) | 11 (6–22) | 11 (6–20) | 13 (7–25) | 14 (7–25) | 12 (7–24) | 15 (8–29) |
Hospital mortality | 4889 (27%) | 5398 (18%) | 8582 (14%) | 4659 (33%) | 3488 (37%) | 2782 (14%) | 1427 (35%) |
Data are presented as No. (%) unless otherwise specified.
Abbreviations: BC, blood culture; IQR, interquartile range; LA, lactic acidosis; LOS, length of stay.
a The breakdown of positive blood cultures refers to the first positive blood culture in the hospitalization, except for “Positive BC + Pressor or LA ± 1 d,” where it refers to the first positive blood culture associated with concurrent vasopressor use or lactic acidosis within ±1 day.
Table 2.
Characteristics and Outcomes of Patients With Sepsis Claims and Objective Clinical Markers of Septicemia and Severe Sepsis/Septic Shock, 2003–2012
Patient Characteristics and Outcomes | Septicemia | Martin | Angus | Dombrovskiy | Severe Sepsis/Septic Shock | Positive BC | Positive BC + Pressor or LA ± 1 d |
---|---|---|---|---|---|---|---|
No. of discharges, 2003–2012 (N = 1 002 815) | 18 254 (2%) | 29 409 (4%) | 60 770 (9%) | 13 943 (2%) | 9426 (1%) | 20 578 (3%) | 4119 (0.6%) |
Male | 10 038 (55%) | 16 234 (55%) | 31 793 (52%) | 7864 (56%) | 5287 (56%) | 11 269 (55%) | 2436 (59%) |
Non–white race | 3996 (22%) | 6514 (22%) | 12 670 (21%) | 2953 (21%) | 2011 (21%) | 4793 (23%) | 945 (23%) |
Mean age | 63.0 | 61.4 | 65.7 | 63.8 | 63.9 | 59.8 | 61.7 |
Mean Charlson score | 3.8 | 3.8 | 3.7 | 3.8 | 3.8 | 3.7 | 3.8 |
Comorbidities | |||||||
Diabetes | 2514 (14%) | 4524 (15%) | 10 908 (18%) | 1603 (12%) | 1002 (11%) | 3513 (16%) | 550 (13%) |
Heart failure | 3742 (21%) | 5426 (18%) | 14 744 (24%) | 3146 (23%) | 2142 (23%) | 3183 (15%) | 874 (21%) |
Liver disease | 1083 (6%) | 1854 (6%) | 3667 (6%) | 932 (7%) | 563 (6%) | 1409 (7%) | 334 (8%) |
Lung disease | 2159 (12%) | 3423 (12%) | 10 303 (17%) | 1583 (11%) | 1037 (11%) | 2299 (11%) | 434 (11%) |
Lymphoma | 891 (5%) | 1763 (6%) | 2667 (4%) | 681 (5%) | 442 (5%) | 1303 (6%) | 205 (5%) |
Metastatic cancer | 2061 (11%) | 3193 (11%) | 5688 (9%) | 1419 (10%) | 904 (10%) | 2532 (12%) | 454 (11%) |
Renal disease | 2691 (15%) | 4586 (16%) | 10 394 (17%) | 1962 (14%) | 1288 (14%) | 2788 (14%) | 595 (14%) |
≥1 BC drawn | 17 675 (97%) | 28 472 (97%) | 47 111 (78%) | 13 539 (97%) | 9156 (97%) | … | … |
Positive BCsa | 6806 (37%) | 14 028 (48%) | 9378 (15%) | 4997 (36%) | 3194 (34%) | … | … |
Gram positive | 3218 (18%) | 7515 (26%) | 5025 (8%) | 2309 (17%) | 1454 (15%) | 11 320 (55%) | 2053 (50%) |
Gram negative | 2594 (14%) | 4662 (16%) | 2953 (5%) | 1764 (13%) | 1159 (12%) | 6675 (32%) | 1361 (33%) |
Fungi | 272 (1%) | 668 (2%) | 590 (1%) | 395 (3%) | 220 (2%) | 974 (5%) | 251 (6%) |
Multiple | 721 (4%) | 1179 (4%) | 805 (1%) | 527 (4%) | 359 (4%) | 1598 (8%) | 452 (11%) |
Mycobacterial | 1 (0.01%) | 4 (0.01%) | 5 (0.01%) | 2 (0.01%) | 2 (0.02%) | 11 (0.05%) | 2 (0.05%) |
Vasopressor use | 12 092 (61%) | 13 327 (45%) | 23 587 (39%) | 10 149 (73%) | 7608 (81%) | 7013 (34%) | 3925 (95%) |
Median hospital LOS (IQR) | 12 (6–22) | 11 (6–22) | 11 (6–20) | 13 (7–25) | 14 (7–25) | 12 (7–24) | 15 (8–29) |
Hospital mortality | 4889 (27%) | 5398 (18%) | 8582 (14%) | 4659 (33%) | 3488 (37%) | 2782 (14%) | 1427 (35%) |
Patient Characteristics and Outcomes | Septicemia | Martin | Angus | Dombrovskiy | Severe Sepsis/Septic Shock | Positive BC | Positive BC + Pressor or LA ± 1 d |
---|---|---|---|---|---|---|---|
No. of discharges, 2003–2012 (N = 1 002 815) | 18 254 (2%) | 29 409 (4%) | 60 770 (9%) | 13 943 (2%) | 9426 (1%) | 20 578 (3%) | 4119 (0.6%) |
Male | 10 038 (55%) | 16 234 (55%) | 31 793 (52%) | 7864 (56%) | 5287 (56%) | 11 269 (55%) | 2436 (59%) |
Non–white race | 3996 (22%) | 6514 (22%) | 12 670 (21%) | 2953 (21%) | 2011 (21%) | 4793 (23%) | 945 (23%) |
Mean age | 63.0 | 61.4 | 65.7 | 63.8 | 63.9 | 59.8 | 61.7 |
Mean Charlson score | 3.8 | 3.8 | 3.7 | 3.8 | 3.8 | 3.7 | 3.8 |
Comorbidities | |||||||
Diabetes | 2514 (14%) | 4524 (15%) | 10 908 (18%) | 1603 (12%) | 1002 (11%) | 3513 (16%) | 550 (13%) |
Heart failure | 3742 (21%) | 5426 (18%) | 14 744 (24%) | 3146 (23%) | 2142 (23%) | 3183 (15%) | 874 (21%) |
Liver disease | 1083 (6%) | 1854 (6%) | 3667 (6%) | 932 (7%) | 563 (6%) | 1409 (7%) | 334 (8%) |
Lung disease | 2159 (12%) | 3423 (12%) | 10 303 (17%) | 1583 (11%) | 1037 (11%) | 2299 (11%) | 434 (11%) |
Lymphoma | 891 (5%) | 1763 (6%) | 2667 (4%) | 681 (5%) | 442 (5%) | 1303 (6%) | 205 (5%) |
Metastatic cancer | 2061 (11%) | 3193 (11%) | 5688 (9%) | 1419 (10%) | 904 (10%) | 2532 (12%) | 454 (11%) |
Renal disease | 2691 (15%) | 4586 (16%) | 10 394 (17%) | 1962 (14%) | 1288 (14%) | 2788 (14%) | 595 (14%) |
≥1 BC drawn | 17 675 (97%) | 28 472 (97%) | 47 111 (78%) | 13 539 (97%) | 9156 (97%) | … | … |
Positive BCsa | 6806 (37%) | 14 028 (48%) | 9378 (15%) | 4997 (36%) | 3194 (34%) | … | … |
Gram positive | 3218 (18%) | 7515 (26%) | 5025 (8%) | 2309 (17%) | 1454 (15%) | 11 320 (55%) | 2053 (50%) |
Gram negative | 2594 (14%) | 4662 (16%) | 2953 (5%) | 1764 (13%) | 1159 (12%) | 6675 (32%) | 1361 (33%) |
Fungi | 272 (1%) | 668 (2%) | 590 (1%) | 395 (3%) | 220 (2%) | 974 (5%) | 251 (6%) |
Multiple | 721 (4%) | 1179 (4%) | 805 (1%) | 527 (4%) | 359 (4%) | 1598 (8%) | 452 (11%) |
Mycobacterial | 1 (0.01%) | 4 (0.01%) | 5 (0.01%) | 2 (0.01%) | 2 (0.02%) | 11 (0.05%) | 2 (0.05%) |
Vasopressor use | 12 092 (61%) | 13 327 (45%) | 23 587 (39%) | 10 149 (73%) | 7608 (81%) | 7013 (34%) | 3925 (95%) |
Median hospital LOS (IQR) | 12 (6–22) | 11 (6–22) | 11 (6–20) | 13 (7–25) | 14 (7–25) | 12 (7–24) | 15 (8–29) |
Hospital mortality | 4889 (27%) | 5398 (18%) | 8582 (14%) | 4659 (33%) | 3488 (37%) | 2782 (14%) | 1427 (35%) |
Data are presented as No. (%) unless otherwise specified.
Abbreviations: BC, blood culture; IQR, interquartile range; LA, lactic acidosis; LOS, length of stay.
a The breakdown of positive blood cultures refers to the first positive blood culture in the hospitalization, except for “Positive BC + Pressor or LA ± 1 d,” where it refers to the first positive blood culture associated with concurrent vasopressor use or lactic acidosis within ±1 day.
Trends in Incidence of Sepsis Claims Versus Objective Clinical Markers
The annual incidence of hospitalizations with sepsis-related claims definitions increased significantly over the decade (Table 3 and Figure 1A), ranging from an overall 54% increase from 2003 to 2012 for Martin sepsis codes (average annual increase, 4.7% [95% CI, 3.8%–5.5%]; P < .001 for linear trend) to a 706% increase for explicit severe sepsis/septic shock codes (average annual increase, 59.3% [95% CI, 39.9%–78.8%]; P = .001). In contrast, the incidence of hospitalizations with at least 1 positive blood culture decreased 17%, from 213 cases per 10 000 discharges to 176 cases in 2012 (average annual decrease, 3.2% [95% CI, 1.7%–4.7%]; P = .006; Figure 1B). There was a nonsignificant increase in incidence of hospitalizations with positive BC + pressor or LA ± 1 day, from 38 cases per 10 000 discharges in 2003 to 41 cases in 2012 (average annual increase, 0.9% [95% CI, .0%–1.8%]; P = .098). There was no significant change in incidence of hospitalizations with at least 1 blood culture drawn, with 1848 cases per 10 000 discharges in 2003 to 2044 cases in 2012 (average annual increase, 0.2% [95%, CI, −.7 to 1.2%]; P = .646).
Table 3.
Change From 2003 to 2012 in Incidence and Hospital Mortality Rates Associated With Sepsis Claims and Hospitalizations With Objective Clinical Markers of Septicemia and Severe Sepsis/Septic Shock
Claim | Incidence | Mortality | ||||||
---|---|---|---|---|---|---|---|---|
2003, per 10 000 (Total Cases) | 2012, per 10 000 (Total Cases) | 10-y Relative Change | P Value for Linear Trend | 2003 (Total Deaths) | 2012 (Total Deaths) | 10-y Relative Change | P Value for Linear Trend | |
Claims | ||||||||
Septicemia | 135 (1400) | 239 (2551) | 77% | .002 | 30.9% (432) | 23.4% (598) | −24% | <.001 |
Martin sepsis | 226 (2348) | 348 (3715) | 54% | <.001 | 20.7% (487) | 17.1% (635) | −17% | .012 |
Angus severe sepsis | 429 (4466) | 745 (7951) | 74% | <.001 | 17.6% (784) | 12.6% (1000) | −28% | <.001 |
Dombrovskiy severe sepsis | 92 (959) | 190 (2026) | 106% | <.001 | 41.1% (394) | 28.5% (578) | −31% | <.001 |
Explicit severe sepsis/septic shock | 17 (180) | 137 (1467) | 706% | .001 | 50.0% (90) | 32.2% (473) | −36% | <.001 |
Objective clinical markers | ||||||||
Positive BC | 213 (2218) | 176 (1881) | −17% | .006 | 15.7% (348) | 11.6% (218) | −26% | .036 |
Positive BC + pressor or LA ± 1 d | 38 (392) | 41 (433) | 8% | .098 | 42.4% (166) | 31.6% (137) | −25% | .057 |
Claim | Incidence | Mortality | ||||||
---|---|---|---|---|---|---|---|---|
2003, per 10 000 (Total Cases) | 2012, per 10 000 (Total Cases) | 10-y Relative Change | P Value for Linear Trend | 2003 (Total Deaths) | 2012 (Total Deaths) | 10-y Relative Change | P Value for Linear Trend | |
Claims | ||||||||
Septicemia | 135 (1400) | 239 (2551) | 77% | .002 | 30.9% (432) | 23.4% (598) | −24% | <.001 |
Martin sepsis | 226 (2348) | 348 (3715) | 54% | <.001 | 20.7% (487) | 17.1% (635) | −17% | .012 |
Angus severe sepsis | 429 (4466) | 745 (7951) | 74% | <.001 | 17.6% (784) | 12.6% (1000) | −28% | <.001 |
Dombrovskiy severe sepsis | 92 (959) | 190 (2026) | 106% | <.001 | 41.1% (394) | 28.5% (578) | −31% | <.001 |
Explicit severe sepsis/septic shock | 17 (180) | 137 (1467) | 706% | .001 | 50.0% (90) | 32.2% (473) | −36% | <.001 |
Objective clinical markers | ||||||||
Positive BC | 213 (2218) | 176 (1881) | −17% | .006 | 15.7% (348) | 11.6% (218) | −26% | .036 |
Positive BC + pressor or LA ± 1 d | 38 (392) | 41 (433) | 8% | .098 | 42.4% (166) | 31.6% (137) | −25% | .057 |
Abbreviations: BC, blood culture; LA, lactic acidosis.
Table 3.
Change From 2003 to 2012 in Incidence and Hospital Mortality Rates Associated With Sepsis Claims and Hospitalizations With Objective Clinical Markers of Septicemia and Severe Sepsis/Septic Shock
Claim | Incidence | Mortality | ||||||
---|---|---|---|---|---|---|---|---|
2003, per 10 000 (Total Cases) | 2012, per 10 000 (Total Cases) | 10-y Relative Change | P Value for Linear Trend | 2003 (Total Deaths) | 2012 (Total Deaths) | 10-y Relative Change | P Value for Linear Trend | |
Claims | ||||||||
Septicemia | 135 (1400) | 239 (2551) | 77% | .002 | 30.9% (432) | 23.4% (598) | −24% | <.001 |
Martin sepsis | 226 (2348) | 348 (3715) | 54% | <.001 | 20.7% (487) | 17.1% (635) | −17% | .012 |
Angus severe sepsis | 429 (4466) | 745 (7951) | 74% | <.001 | 17.6% (784) | 12.6% (1000) | −28% | <.001 |
Dombrovskiy severe sepsis | 92 (959) | 190 (2026) | 106% | <.001 | 41.1% (394) | 28.5% (578) | −31% | <.001 |
Explicit severe sepsis/septic shock | 17 (180) | 137 (1467) | 706% | .001 | 50.0% (90) | 32.2% (473) | −36% | <.001 |
Objective clinical markers | ||||||||
Positive BC | 213 (2218) | 176 (1881) | −17% | .006 | 15.7% (348) | 11.6% (218) | −26% | .036 |
Positive BC + pressor or LA ± 1 d | 38 (392) | 41 (433) | 8% | .098 | 42.4% (166) | 31.6% (137) | −25% | .057 |
Claim | Incidence | Mortality | ||||||
---|---|---|---|---|---|---|---|---|
2003, per 10 000 (Total Cases) | 2012, per 10 000 (Total Cases) | 10-y Relative Change | P Value for Linear Trend | 2003 (Total Deaths) | 2012 (Total Deaths) | 10-y Relative Change | P Value for Linear Trend | |
Claims | ||||||||
Septicemia | 135 (1400) | 239 (2551) | 77% | .002 | 30.9% (432) | 23.4% (598) | −24% | <.001 |
Martin sepsis | 226 (2348) | 348 (3715) | 54% | <.001 | 20.7% (487) | 17.1% (635) | −17% | .012 |
Angus severe sepsis | 429 (4466) | 745 (7951) | 74% | <.001 | 17.6% (784) | 12.6% (1000) | −28% | <.001 |
Dombrovskiy severe sepsis | 92 (959) | 190 (2026) | 106% | <.001 | 41.1% (394) | 28.5% (578) | −31% | <.001 |
Explicit severe sepsis/septic shock | 17 (180) | 137 (1467) | 706% | .001 | 50.0% (90) | 32.2% (473) | −36% | <.001 |
Objective clinical markers | ||||||||
Positive BC | 213 (2218) | 176 (1881) | −17% | .006 | 15.7% (348) | 11.6% (218) | −26% | .036 |
Positive BC + pressor or LA ± 1 d | 38 (392) | 41 (433) | 8% | .098 | 42.4% (166) | 31.6% (137) | −25% | .057 |
Abbreviations: BC, blood culture; LA, lactic acidosis.
Figure 1.
A, Annual incidence of hospitalizations with discharge codes for septicemia, sepsis (Martin definition), and severe sepsis (Angus, Dombrovskiy definitions). B, Annual incidence of hospitalizations with blood cultures positive for a significant pathogen, with or without concurrent vasopressors or lactic acidosis. Numbers at the end of each trend line represent the fitted average annual percentage change compared to the 2003 incidence rate, with associated 95% confidence intervals. Abbreviations: BC, blood culture; LA, lactic acidosis.
Trends in Hospital Mortality
The in-hospital mortality of patients meeting all sepsis claims definitions decreased significantly over the 10-year period (Table 3). Hospital mortality rates also declined for patients with positive blood cultures, from 15.7% to 11.6% (average annual decrease, 0.3% [95% CI, .1%–.5%]; P = .036) and for patients with positive BC + pressor or LA ± 1 day, from 42.4% to 31.6% (average annual decrease, 0.7% [95% CI, .1%–1.3%]; P = .057).
Trends in Sensitivity and PPV of Claims Versus Objective Clinical Markers
The change in claims performance over time is summarized in Table 4. The sensitivity of both septicemia and Martin sepsis codes for identifying hospitalizations with positive blood cultures increased, but was statistically significant only for Martin codes. When using positive BC + pressor or LA ± 1 day as a reference, the sensitivity of all claims-based septicemia, sepsis, and severe sepsis definitions increased significantly over time (Table 4 and Figure 2). This was most dramatic for explicit severe sepsis/septic shock codes, increasing from 13.3% (52/392 cases; 95% CI, 10.1%–17.0%) in 2003 to 60.3% (261/433 cases; 95% CI, 55.5%–64.9%; P < .001 for linear trend), and also for the Dombrovskiy severe sepsis definition, increasing from a sensitivity of 52.8% (207/392 cases; 95% CI, 47.7%–57.8%) to 72.1% (312/433 cases; 95% CI, 67.6%–76.2%; P < .001). The PPV of both septicemia and Martin sepsis codes for identifying hospitalizations with positive blood cultures decreased significantly over the decade (Figure 2).
Table 4.
Changing Sensitivity and Positive Predictive Value of Sepsis Claims Definitions for Identifying Corresponding Objective Clinical Markers, 2003–2012
Claims Definition | Clinical Markers for Comparison | Claims Performance Characteristic | 2003, % (No. Cases) [95% CI] | 2012, % (No. Cases) [95% CI] | Fitted Annual Changea [95% CI] | Value for Linear Trend |
---|---|---|---|---|---|---|
Septicemia | Positive BC | Sensitivity | 31.8% (706/2218) [29.9–33.8] | 40.7% (765/1881) [38.4–42.9] | 0.9% [.2–1.7] | .056 |
Positive BC + pressor or LA ±1 d | Sensitivity | 57.6% (226/392) [52.6–62.6] | 77.1% (334/433) [72.9–81.0] | 1.4% [1.1–1.7] | <.001 | |
Positive BC | PPV | 50.4% (706/1400) [47.8–53.1] | 30.0% (765/2551) [28.2–31.8] | −2.1% [−1.8 to −2.4] | <.001 | |
Martin sepsis (septicemia, bacteremia, fungemia) | Positive BC | Sensitivity | 61.2% (1358/2218) [59.2–63.3] | 77.3% (1453/1881) [75.3–79.1] | 1.9% [1.6–2.1] | <.001 |
Positive BC + pressor or LA ±1 d | Sensitivity | 74.7% (293/392) [70.1–79.0] | 88.7% (384/433) [85.3–91.5] | 2.2% [1.8–2.7] | <.001 | |
Positive BC | PPV | 57.8% (1358/2348) [55.8–59.8] | 39.1% (1453/3715) [37.5–40.7] | −2.0% [−2.2; to −1.8] | <.001 | |
Angus severe sepsis | Positive BC + pressor or LA ±1 d | Sensitivity | 74.7% (293/392) [70.1–79.0] | 85.2% (369/433) [81.5–88.4] | 1.1% [.9–1.3] | <.001 |
Dombrovskiy severe sepsis | Positive BC + pressor or LA ±1 d | Sensitivity | 52.8% (207/392) [47.7–57.8] | 72.1% (312/433) [67.6–76.2] | 2.2% [1.8–2.7] | <.001 |
Explicit severe sepsis + septic shock | Positive BC + pressor or LA ±1 d | Sensitivity | 13.3% (52/392) [10.1–17.0] | 60.3% (261/433) [55.5–64.9] | 3.8% [2.6–4.9] | <.001 |
Hypotension/shock | Vasopressor use (any) | Sensitivity | 19.0% (742/3909) [17.8–20.3] | 33.7% (1858/5515) [32.4–35.0] | 1.8% [1.7–1.9] | <.001 |
Metabolic/LA | Lactic acid ≥4.0 mmol/L | Sensitivity | 9.3% (32/346) [6.4–12.8] | 16.6% (69/417) [13.1–20.5] | 1.0% [.4–1.5] | .013 |
Claims Definition | Clinical Markers for Comparison | Claims Performance Characteristic | 2003, % (No. Cases) [95% CI] | 2012, % (No. Cases) [95% CI] | Fitted Annual Changea [95% CI] | Value for Linear Trend |
---|---|---|---|---|---|---|
Septicemia | Positive BC | Sensitivity | 31.8% (706/2218) [29.9–33.8] | 40.7% (765/1881) [38.4–42.9] | 0.9% [.2–1.7] | .056 |
Positive BC + pressor or LA ±1 d | Sensitivity | 57.6% (226/392) [52.6–62.6] | 77.1% (334/433) [72.9–81.0] | 1.4% [1.1–1.7] | <.001 | |
Positive BC | PPV | 50.4% (706/1400) [47.8–53.1] | 30.0% (765/2551) [28.2–31.8] | −2.1% [−1.8 to −2.4] | <.001 | |
Martin sepsis (septicemia, bacteremia, fungemia) | Positive BC | Sensitivity | 61.2% (1358/2218) [59.2–63.3] | 77.3% (1453/1881) [75.3–79.1] | 1.9% [1.6–2.1] | <.001 |
Positive BC + pressor or LA ±1 d | Sensitivity | 74.7% (293/392) [70.1–79.0] | 88.7% (384/433) [85.3–91.5] | 2.2% [1.8–2.7] | <.001 | |
Positive BC | PPV | 57.8% (1358/2348) [55.8–59.8] | 39.1% (1453/3715) [37.5–40.7] | −2.0% [−2.2; to −1.8] | <.001 | |
Angus severe sepsis | Positive BC + pressor or LA ±1 d | Sensitivity | 74.7% (293/392) [70.1–79.0] | 85.2% (369/433) [81.5–88.4] | 1.1% [.9–1.3] | <.001 |
Dombrovskiy severe sepsis | Positive BC + pressor or LA ±1 d | Sensitivity | 52.8% (207/392) [47.7–57.8] | 72.1% (312/433) [67.6–76.2] | 2.2% [1.8–2.7] | <.001 |
Explicit severe sepsis + septic shock | Positive BC + pressor or LA ±1 d | Sensitivity | 13.3% (52/392) [10.1–17.0] | 60.3% (261/433) [55.5–64.9] | 3.8% [2.6–4.9] | <.001 |
Hypotension/shock | Vasopressor use (any) | Sensitivity | 19.0% (742/3909) [17.8–20.3] | 33.7% (1858/5515) [32.4–35.0] | 1.8% [1.7–1.9] | <.001 |
Metabolic/LA | Lactic acid ≥4.0 mmol/L | Sensitivity | 9.3% (32/346) [6.4–12.8] | 16.6% (69/417) [13.1–20.5] | 1.0% [.4–1.5] | .013 |
Abbreviations: BC, blood culture; CI, confidence interval; LA, lactic acidosis; PPV, positive predictive value.
a Average annual change was calculated over the 10-year period using fitted linear trends.
Table 4.
Changing Sensitivity and Positive Predictive Value of Sepsis Claims Definitions for Identifying Corresponding Objective Clinical Markers, 2003–2012
Claims Definition | Clinical Markers for Comparison | Claims Performance Characteristic | 2003, % (No. Cases) [95% CI] | 2012, % (No. Cases) [95% CI] | Fitted Annual Changea [95% CI] | Value for Linear Trend |
---|---|---|---|---|---|---|
Septicemia | Positive BC | Sensitivity | 31.8% (706/2218) [29.9–33.8] | 40.7% (765/1881) [38.4–42.9] | 0.9% [.2–1.7] | .056 |
Positive BC + pressor or LA ±1 d | Sensitivity | 57.6% (226/392) [52.6–62.6] | 77.1% (334/433) [72.9–81.0] | 1.4% [1.1–1.7] | <.001 | |
Positive BC | PPV | 50.4% (706/1400) [47.8–53.1] | 30.0% (765/2551) [28.2–31.8] | −2.1% [−1.8 to −2.4] | <.001 | |
Martin sepsis (septicemia, bacteremia, fungemia) | Positive BC | Sensitivity | 61.2% (1358/2218) [59.2–63.3] | 77.3% (1453/1881) [75.3–79.1] | 1.9% [1.6–2.1] | <.001 |
Positive BC + pressor or LA ±1 d | Sensitivity | 74.7% (293/392) [70.1–79.0] | 88.7% (384/433) [85.3–91.5] | 2.2% [1.8–2.7] | <.001 | |
Positive BC | PPV | 57.8% (1358/2348) [55.8–59.8] | 39.1% (1453/3715) [37.5–40.7] | −2.0% [−2.2; to −1.8] | <.001 | |
Angus severe sepsis | Positive BC + pressor or LA ±1 d | Sensitivity | 74.7% (293/392) [70.1–79.0] | 85.2% (369/433) [81.5–88.4] | 1.1% [.9–1.3] | <.001 |
Dombrovskiy severe sepsis | Positive BC + pressor or LA ±1 d | Sensitivity | 52.8% (207/392) [47.7–57.8] | 72.1% (312/433) [67.6–76.2] | 2.2% [1.8–2.7] | <.001 |
Explicit severe sepsis + septic shock | Positive BC + pressor or LA ±1 d | Sensitivity | 13.3% (52/392) [10.1–17.0] | 60.3% (261/433) [55.5–64.9] | 3.8% [2.6–4.9] | <.001 |
Hypotension/shock | Vasopressor use (any) | Sensitivity | 19.0% (742/3909) [17.8–20.3] | 33.7% (1858/5515) [32.4–35.0] | 1.8% [1.7–1.9] | <.001 |
Metabolic/LA | Lactic acid ≥4.0 mmol/L | Sensitivity | 9.3% (32/346) [6.4–12.8] | 16.6% (69/417) [13.1–20.5] | 1.0% [.4–1.5] | .013 |
Claims Definition | Clinical Markers for Comparison | Claims Performance Characteristic | 2003, % (No. Cases) [95% CI] | 2012, % (No. Cases) [95% CI] | Fitted Annual Changea [95% CI] | Value for Linear Trend |
---|---|---|---|---|---|---|
Septicemia | Positive BC | Sensitivity | 31.8% (706/2218) [29.9–33.8] | 40.7% (765/1881) [38.4–42.9] | 0.9% [.2–1.7] | .056 |
Positive BC + pressor or LA ±1 d | Sensitivity | 57.6% (226/392) [52.6–62.6] | 77.1% (334/433) [72.9–81.0] | 1.4% [1.1–1.7] | <.001 | |
Positive BC | PPV | 50.4% (706/1400) [47.8–53.1] | 30.0% (765/2551) [28.2–31.8] | −2.1% [−1.8 to −2.4] | <.001 | |
Martin sepsis (septicemia, bacteremia, fungemia) | Positive BC | Sensitivity | 61.2% (1358/2218) [59.2–63.3] | 77.3% (1453/1881) [75.3–79.1] | 1.9% [1.6–2.1] | <.001 |
Positive BC + pressor or LA ±1 d | Sensitivity | 74.7% (293/392) [70.1–79.0] | 88.7% (384/433) [85.3–91.5] | 2.2% [1.8–2.7] | <.001 | |
Positive BC | PPV | 57.8% (1358/2348) [55.8–59.8] | 39.1% (1453/3715) [37.5–40.7] | −2.0% [−2.2; to −1.8] | <.001 | |
Angus severe sepsis | Positive BC + pressor or LA ±1 d | Sensitivity | 74.7% (293/392) [70.1–79.0] | 85.2% (369/433) [81.5–88.4] | 1.1% [.9–1.3] | <.001 |
Dombrovskiy severe sepsis | Positive BC + pressor or LA ±1 d | Sensitivity | 52.8% (207/392) [47.7–57.8] | 72.1% (312/433) [67.6–76.2] | 2.2% [1.8–2.7] | <.001 |
Explicit severe sepsis + septic shock | Positive BC + pressor or LA ±1 d | Sensitivity | 13.3% (52/392) [10.1–17.0] | 60.3% (261/433) [55.5–64.9] | 3.8% [2.6–4.9] | <.001 |
Hypotension/shock | Vasopressor use (any) | Sensitivity | 19.0% (742/3909) [17.8–20.3] | 33.7% (1858/5515) [32.4–35.0] | 1.8% [1.7–1.9] | <.001 |
Metabolic/LA | Lactic acid ≥4.0 mmol/L | Sensitivity | 9.3% (32/346) [6.4–12.8] | 16.6% (69/417) [13.1–20.5] | 1.0% [.4–1.5] | .013 |
Abbreviations: BC, blood culture; CI, confidence interval; LA, lactic acidosis; PPV, positive predictive value.
a Average annual change was calculated over the 10-year period using fitted linear trends.
Figure 2.
Annual change from 2003 to 2012 in sensitivity of septicemia, Martin-defined sepsis, and severe sepsis codes for identifying hospitalizations with positive blood cultures with concurrent vasopressors and/or lactic acidosis (A), and positive predictive value for septicemia and Martin sepsis codes (septicemia + bacteremia + fungemia) for identifying hospitalizations with positive blood cultures (B). Numbers at the end of each trend line represent the fitted average annual percentage change, with associated 95% confidence intervals.
The sensitivity of hypotension/shock codes for capturing hospitalizations with vasopressor use increased significantly over the 10-year period, as did the sensitivity of the metabolic/lactic acid code for capturing hospitalizations with documented lactic acid levels ≥4.0 mmol/L (Table 4).
DISCUSSION
We found that the incidence of hospitalizations coded for septicemia, sepsis, and severe sepsis using multiple claims definitions increased steadily over a 10-year period, but this was accompanied by a decrease in hospitalizations with positive blood cultures and no significant change in the incidence of positive blood cultures associated with concurrent vasopressors and/or lactic acidosis. Our selected objective clinical markers do not allow us to directly calculate rates of nonbacteremic sepsis, but our findings suggest that diagnosis and coding practices are changing over time in ways that may lead to overestimates of the rate of change in sepsis incidence. First, the sensitivity of all 5 sepsis-related claims methods for identifying patients with an unambiguous septic event—positive blood cultures with concomitant vasopressor requirement and/or lactic acidosis—increased. Second, the sensitivity of hypotension/shock and acidosis codes for capturing patients with objective evidence of these forms of hypoperfusion increased. Last, the proportion of patients coded with septicemia that had positive blood cultures decreased steadily. This was also true for patients meeting the Martin sepsis definition, which consists of codes that imply a disseminated blood-borne infection.
To our knowledge, this is the first study examining temporal changes in the performance of claims for tracking sepsis incidence by comparing them to objective patient-level data. Both Whittaker et al and Iwashnya et al found that claims had poor sensitivity when compared to severe sepsis diagnosed clinically using the international consensus definition [19, 21]. However, these studies looked at the overall performance of claims only in a small, fixed time period, rather than examining whether their performance has changed over a long period of time. Notably, our findings of increasing but still imperfect claims sensitivity when compared to an unambiguous severe sepsis event, bacteremia with concurrent vasopressors and/or lactic acidosis, also support the notion that underdiagnosis and undercoding are still problems when estimating sepsis burden with claims. Lindenauer et al [9] used a nationally representative administrative database to show that incidence and mortality of pneumonia as a principal diagnosis declined significantly over a 7-year period, but incidence and mortality of sepsis or respiratory failure with pneumonia as a secondary diagnosis increased. When the 3 pneumonia coding methods were combined, changes in incidence and mortality were minimal, suggesting that changes in how pneumonia and sepsis were being coded was an important component of observed trends [9].
The many limitations of claims-based methods for sepsis surveillance compel a search for alternatives. The increasing availability of electronic health record systems could facilitate large-scale surveillance using objective clinical markers akin to the ones evaluated in our study. Positive blood cultures with vasopressors or lactic acidosis are attractive candidates because they are clinically meaningful, objective, and ascertainable using electronic health record data. This method is most suitable, however, for tracking trends over time rather than estimating the total burden of severe sepsis at any one time because <50% of patients with severe sepsis have bacteremia and milder cases do not develop lactic acidosis or require vasopressors [22–24]. Future work could explore surveillance strategies for capturing severe sepsis cases in nonbacteremic patients and patients who have other types of organ dysfunction.
Although our findings suggest that the rise in severe sepsis incidence may be overestimated by claims methods, we did find a trend toward increasing incidence of positive blood cultures associated with concurrent vasopressors and/or lactic acidosis. A recent large study by Kaukonen et al also suggested that severe sepsis incidence truly is increasing, though perhaps at a rate lower than that suggested by claims-based reports [25]. These investigators examined trends in severe sepsis cases admitted to intensive care units in Australia and New Zealand from 2000 to 2012 using standardized criteria for sepsis and organ failure abstracted by nurses at the bedside, and found an increase in incidence of >50%.
We also found decreasing hospital mortality in patients with objective markers of sepsis, paralleling the decrease in mortality found with sepsis claims in our study and in other non-claims-based reports. For example, the study by Kaukonen et al also found a substantial decline in sepsis-related mortality over time, even when adjusting for severity of illness [25]. In addition, Stevenson et al examined trends in severe sepsis mortality from 1991 to 2009 by analyzing usual-care participants from 36 multicenter randomized trials, reasoning that these patients were rigorously selected and thus less subject to misclassification than claims data, and found decreasing mortality rates that mirrored those seen with national claims data [8]. Both those studies, and our findings in patients with objective clinical markers of sepsis, provide evidence that declining sepsis mortality is not solely the result of a lower threshold for sepsis diagnosis and coding over time. Rather, declining mortality likely reflects improvements in sepsis care, presumably as a result of increased clinical awareness leading to earlier antibiotic administration, sepsis bundles, and the dissemination of other evidence-based practices at our hospitals and worldwide.
Our study has several limitations. First, we only analyzed data from 2 academic centers. Nonetheless, claims-based estimates in our hospitals closely mirrored nationwide trends in sepsis incidence, prevalence, and mortality [4, 26]. Second, we did not review charts to corroborate objective markers to clinical histories, or to examine vital signs for traditional signs of the systemic inflammatory response syndrome. However, it is clinically intuitive that a patient with bacteremia with concurrent vasopressors or lactic acidosis meets criteria for severe sepsis/septic shock. Third, based on this study, we can only make inferences about incidence trends in nonbacteremic severe sepsis. Prior prospective studies have reported on rates of positive blood cultures in severe sepsis or septic shock [22–24], but it is unknown if this proportion is changing over time. However, our finding that septicemia codes are increasingly being applied to nonbacteremic patients strongly suggests a change in how this ambiguous term is being applied. Furthermore, if sepsis claims have increased in sensitivity over time for a septic event as obvious as bacteremia with concurrent vasopressors or lactic acidosis, it is possible that clinicians are increasingly likely to diagnose severe sepsis in more subtle cases without bacteremia or shock. In addition, our finding of increasing sensitivity for at least 2 forms of organ dysfunction has implications for all severe sepsis cases. Fourth, our findings about trends in incidence of hospitalizations with positive blood cultures might be confounded if clinicians' thresholds for ordering blood cultures have been changing over time. However, we found no significant change in the rate of hospitalizations with blood culture orders. Fifth, we did not have preceding clinical data for patients transferred from other hospitals, nor could we identify the proportion of patients transferred from other hospitals. If an increasing number of septic patients were transferred over time and only had positive cultures prior to transfer, we might have underestimated the incidence of patients with objective clinical markers and the proportion of patients with septicemia codes who had positive blood cultures. Last, we had limited information about other clinical events during hospitalizations with positive blood cultures with concurrent vasopressors and/or lactic acidosis that might have influenced the likelihood of being coded as severe sepsis. A prior study showed that more severely ill patients are more likely to be appropriately coded as having severe sepsis [21]. On the other hand, we found a decline in associated hospital mortality of patients with these objective markers over time, suggesting that increasingly severe presentations were unlikely to account for the more sensitive coding.
In conclusion, over a 10-year period, we found a marked discrepancy in trends in incidence rates for hospitalizations with septicemia, sepsis, and severe sepsis codes compared to hospitalizations with positive blood cultures alone and with concurrent vasopressors and/or lactic acidosis. Furthermore, the sensitivity of sepsis and acute organ dysfunction codes increased significantly compared with objective clinical markers, while septicemia codes were increasingly applied to nonbacteremic patients. These findings support the concern that changing diagnosis and coding practices undermine the validity of sepsis trends derived from administrative data, and underscore the need for more objective methods to better track changes in sepsis rates over time.
Notes
Acknowledgments. We thank David Hooper, MD, Division of Infectious Diseases Infection Control Unit at Massachusetts General Hospital, for his assistance in obtaining the blood culture data needed for this study.
Financial support. This work was supported by a research grant from the Prevention Epicenters Program of the Centers for Disease Control and Prevention (grant number 5U54CK0001-03). C. R. received support from the National Institutes of Health (T32 AI007061).
Potential conflicts of interest. M. V. M. is conducting a trial in which participating hospitals are receiving product contributions from Sage Inc and Molnlycke. All other authors report no potential conflicts.
All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
References
1
Martin
GS
Mannino
DM
Eaton
S
Moss
M
The epidemiology of sepsis in the United States from 1979 through 2000
N Engl J Med
2003
348
1546
54
2
Dombrovskiy
VY
Martin
AA
Sunderram
J
Paz
HL
Rapid increase in hospitalization and mortality rates for severe sepsis in the United States: a trend analysis from 1993 to 2003
Crit Care Med
2007
35
1244
50
3
Kumar
G
Kumar
N
Taneja
A
et al.
Nationwide trends of severe sepsis in the 21st century (2000–2007)
Chest
2011
140
1223
31
4
Hall
MJ
Williams
SN
DeFrances
CJ
Golosinskiy
A
Inpatient care for septicemia or sepsis: a challenge for patients and hospitals
NCHS Data Brief
2011
62
1
8
5
Danai
PA
Moss
M
Mannino
DM
Martin
GS
The epidemiology of sepsis in patients with malignancy
Chest
2006
129
1432
40
6
Girard
TD
Opal
SM
Ely
EW
Insights into severe sepsis in older patients: from epidemiology to evidence-based management
Clin Infect Dis
2005
40
719
27
7
Esper
AM
Martin
GS
Extending international sepsis epidemiology: the impact of organ dysfunction
Crit Care
2009
13
120
8
Stevenson
EK
Rubenstein
AR
Radin
GT
Wiener
RS
Walkey
AJ
Two decades of mortality trends among patients with severe sepsis: a comparative meta-analysis
Crit Care Med
2014
42
625
31
9
Lindenauer
PK
Lagu
T
Shieh
MS
Pekow
PS
Rothberg
MB
Association of diagnostic coding with trends in hospitalizations and mortality of patients with pneumonia, 2003–2009
JAMA
2012
307
1405
13
10
Rhee
C
Gohil
S
Klompas
M
Regulatory mandates for sepsis care—reasons for caution
N Engl J Med
2014
370
1673
6
11
Odeh
M
Sepsis, septicaemia, sepsis syndrome, and septic shock: the correct definition and use
Postgrad Med J
1996
72
66
12
Bone
RC
Let's agree on terminology: definitions of sepsis
Crit Care Med
1991
19
973
6
13
Bone
RC
Balk
RA
Cerra
FB
et al.
Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine
Chest
1992
101
1644
55
14
Angus
DC
Linde-Zwirble
WT
Lidicker
J
Clermont
G
Carcillo
J
Pinsky
MR
Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care
Crit Care Med
2001
29
1303
10
15
Lagu
T
Rothberg
MB
Shieh
MS
Pekow
PS
Steingrub
JS
Lindenauer
PK
Hospitalizations, costs, and outcomes of severe sepsis in the United States 2003 to 2007
Crit Care Med
2012
40
754
61
16
Murphy
SN
Chueh
HC
A security architecture for query tools used to access large biomedical databases
Proc AMIA Symp
2002
552
6
17
Elixhauser
A
Steiner
C
Harris
DR
Coffey
RM
Comorbidity measures for use with administrative data
Med Care
1998
36
8
27
18
Charlson
ME
Pompei
P
Ales
KL
MacKenzie
CR
A new method of classifying prognostic comorbidity in longitudinal studies: development and validation
J Chronic Dis
1987
40
373
83
19
Iwashyna
TJ
Odden
A
Rohde
J
et al.
Identifying patients with severe sepsis using administrative claims: patient-level validation of the angus implementation of the international consensus conference definition of severe sepsis
Med Care
2014
52
e39
43
20
Centers for Disease Control and Prevention, National Healthcare Safety Network
Patient safety component manual
21
Whittaker
SA
Mikkelsen
ME
Gaieski
DF
Koshy
S
Kean
C
Fuchs
BD
Severe sepsis cohorts derived from claims-based strategies appear to be biased toward a more severely ill patient population
Crit Care Med
2013
41
945
53
22
Brun-Buisson
C
Doyon
F
Carlet
J
Bacteremia and severe sepsis in adults: a multicenter prospective survey in ICUs and wards of 24 hospitals. French Bacteremia-Sepsis Study Group
Am J Respir Crit Care Med
1996
154
617
24
23
Sands
KE
Bates
DW
Lanken
PN
et al.
Epidemiology of sepsis syndrome in 8 academic medical centers
JAMA
1997
278
234
40
24
Bone
RC
Fisher
CJ
Jr
Clemmer
TP
Slotman
GJ
Metz
CA
Balk
RA
Sepsis syndrome: a valid clinical entity. Methylprednisolone Severe Sepsis Study Group
Crit Care Med
1989
17
389
93
25
Kaukonen
KM
Bailey
M
Suzuki
S
Pilcher
D
Bellomo
R
Mortality related to severe sepsis and septic shock among critically ill patients in Australia and New Zealand, 2000–2012
JAMA
2014
311
1308
16
26
Gaieski
DF
Edwards
JM
Kallan
MJ
Carr
BG
Benchmarking the incidence and mortality of severe sepsis in the United States
Crit Care Med
2013
41
1167
74
© The Author 2014. Published by Oxford University Press on behalf of the Infectious Diseases Society of America. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.
Topic:
Citations
Views
Altmetric
Metrics
Total Views 3,275
2,476 Pageviews
799 PDF Downloads
Since 1/1/2017
Month: | Total Views: |
---|---|
January 2017 | 10 |
February 2017 | 41 |
March 2017 | 20 |
April 2017 | 9 |
May 2017 | 21 |
June 2017 | 11 |
July 2017 | 20 |
August 2017 | 23 |
September 2017 | 13 |
October 2017 | 25 |
November 2017 | 26 |
December 2017 | 54 |
January 2018 | 63 |
February 2018 | 48 |
March 2018 | 61 |
April 2018 | 46 |
May 2018 | 48 |
June 2018 | 50 |
July 2018 | 48 |
August 2018 | 48 |
September 2018 | 47 |
October 2018 | 41 |
November 2018 | 24 |
December 2018 | 33 |
January 2019 | 33 |
February 2019 | 35 |
March 2019 | 52 |
April 2019 | 54 |
May 2019 | 57 |
June 2019 | 37 |
July 2019 | 47 |
August 2019 | 61 |
September 2019 | 55 |
October 2019 | 44 |
November 2019 | 17 |
December 2019 | 24 |
January 2020 | 36 |
February 2020 | 54 |
March 2020 | 26 |
April 2020 | 39 |
May 2020 | 29 |
June 2020 | 22 |
July 2020 | 46 |
August 2020 | 50 |
September 2020 | 47 |
October 2020 | 42 |
November 2020 | 57 |
December 2020 | 22 |
January 2021 | 60 |
February 2021 | 27 |
March 2021 | 30 |
April 2021 | 12 |
May 2021 | 20 |
June 2021 | 21 |
July 2021 | 30 |
August 2021 | 9 |
September 2021 | 29 |
October 2021 | 25 |
November 2021 | 18 |
December 2021 | 22 |
January 2022 | 27 |
February 2022 | 28 |
March 2022 | 44 |
April 2022 | 37 |
May 2022 | 27 |
June 2022 | 29 |
July 2022 | 39 |
August 2022 | 32 |
September 2022 | 37 |
October 2022 | 43 |
November 2022 | 28 |
December 2022 | 27 |
January 2023 | 28 |
February 2023 | 27 |
March 2023 | 35 |
April 2023 | 23 |
May 2023 | 33 |
June 2023 | 23 |
July 2023 | 32 |
August 2023 | 37 |
September 2023 | 50 |
October 2023 | 29 |
November 2023 | 29 |
December 2023 | 32 |
January 2024 | 30 |
February 2024 | 37 |
March 2024 | 36 |
April 2024 | 39 |
May 2024 | 19 |
June 2024 | 37 |
July 2024 | 43 |
August 2024 | 39 |
September 2024 | 37 |
October 2024 | 47 |
November 2024 | 16 |
Citations
129 Web of Science
×
Email alerts
Related articles in PubMed
Citing articles via
More from Oxford Academic