Diagnosis of Legionella Infection (original) (raw)

Journal Article

,

L. Barth Reller

Section Editor

Search for other works by this author on:

,

Melvin P. Weinstein

Section Editor

Search for other works by this author on:

Department of Pathology, Christchurch School of Medicine and Health Sciences, and Microbiology Unit, Canterbury Health Laboratories

,

Christchurch, New Zealand

Reprints or correspondence: Dr. David R. Murdoch, Microbiology Unit, Canterbury Health Laboratories, PO Box 151, Christchurch, New Zealand (david.murdoch@cdhb.govt.nz).

Search for other works by this author on:

Accepted:

08 October 2002

Published:

01 January 2003

Cite

L. Barth Reller, Melvin P. Weinstein, David R. Murdoch, Diagnosis of Legionella Infection, Clinical Infectious Diseases, Volume 36, Issue 1, 1 January 2003, Pages 64–69, https://doi.org/10.1086/345529
Close

Navbar Search Filter Mobile Enter search term Search

Abstract

Legionellae, which are important causes of pneumonia in humans, continue to be incorrectly labeled as exotic pathogens. The ability to diagnose Legionella infection is limited by the nonspecific nature of clinical features and the shortcomings of diagnostic tests. Despite recent improvements, existing diagnostic tests for Legionella infection either lack sensitivity for detecting all clinically important legionellae or are unable to provide results within a clinically useful time frame. Understanding local Legionella epidemiology is important for making decisions about whether to test for Legionella infection and which diagnostic tests to use. In most situations, the use of both the urinary antigen test plus sputum culture is the best diagnostic combination. Polymerase chain reaction (PCR) is a promising tool, but standardized assays are not commercially available. Further work needs to focus on the development of urinary antigen tests assays that detect a wider range of pathogenic legionellae and on the development of standardized PCR assays.

Legionnaires disease continues to have the reputation of being an exotic infection. On the contrary, when systematically sought, Legionella species are consistently recognized as one of the common causes of pneumonia. Outside of the research setting, however, confirmed diagnoses of legionellosis are infrequent. This failure to diagnose legionnaires disease in routine practice is largely the result of 3 factors: the inability to clinically and radiographically distinguish legionnaires disease from other causes of pneumonia, the failure to order diagnostic tests for Legionella infection, and the shortcomings of available diagnostic tests. Legionnaires disease is more accurately described as an elusive diagnosis rather than an exotic infection.

Although diagnostic methods have improved during the 25 years since Legionella pneumophila was first described, no currently available test is able to diagnose legionnaires disease in a timely fashion with a high degree of sensitivity and specificity. Indeed, some authors have challenged the routine ordering of any microbiological tests for patients with pneumonia, supported by data suggesting that such tests do not significantly influence the choice of antibiotic therapy and patient outcome [1, 2]. Recently, Marrie [3] argued that this approach to the diagnosis of legionnaires disease was wrong. Failure to test for Legionella infection may delay the recognition of outbreaks of legionnaires disease, may miss changes in the epidemiology of the disease (including the development of antimicrobial resistance), and may prevent persons from claiming compensation for occupationally acquired infection. Of importance, it is possible to select a subset of patients for whom the yield from diagnostic tests for Legionella infection is high and who would benefit from a specific diagnosis.

This review focuses on current diagnostic tests for Legionella infection, with a particular emphasis on the tests that provide a diagnosis in a time frame that will affect initial infection management. While reviewing the various diagnostic tests for Legionella infection, it is important to bear several factors in mind. First, specialized tests are required to diagnose Legionella infection, and these must be specifically requested by the clinician. Second, it is important to appreciate the distinction between the performance of a test in a research laboratory with what can be realistically achieved in a local diagnostic laboratory. Considerable interlaboratory variation has been documented for the ability to culture legionellae [4], and this is also likely to extend to other tests. Third, interpretation of the performance of diagnostic tests is hindered by the lack of a suitable “gold standard.” Calculated sensitivity and specificity data will vary with different comparison standards. Fourth, the usefulness of diagnostic tests is influenced by local Legionella epidemiology. L. pneumophila serogroup 1 is the predominant cause of legionellosis in many, if not most, areas of the world, and infection with this organism is easier to diagnose than is infection with other Legionella species and serogroups. In some regions, other species and serogroups are more important. For example, Legionella longbeachae is a major cause of legionellosis in Australia and New Zealand [5], where it is often associated with exposure to potting mix [6]. Infection with this species will not be detected by current urinary antigen tests and will be missed by laboratories that perform serological assays only for L. pneumophila.

Culture

Culture diagnosis requires special media, adequate processing of specimens, and technical expertise (table 1). Several days are required to obtain a positive result, with most Legionella colonies being detected within 3–5 days. The standard medium used to culture legionellae is buffered charcoal yeast extract (BCYE) agar supplemented with α-ketoglutarate, with or without antimicrobial agents. This medium provides iron and L-cysteine, both of which are essential for the growth of legionellae. Growth of some Legionella species (e.g., Legionella micdadei and Legionella bozemanii) is enhanced by supplementation of BCYE agar with bovine serum albumin [7], and addition of indicator dyes to the media may aid identification. Culture media containing cefamandole will inhibit the growth of Legionella species that do not produce β-lactamase, such as L. micdadei and L. bozemanii [8].

Diagnostic tests for Legionella infection.

Table 1

Diagnostic tests for Legionella infection.

Legionellae can be isolated from a variety of sample types, although lower respiratory tract secretions (e.g., sputum and bronchoscopy samples) are the samples of choice. The major limitation of sputum culture is that fewer than one-half of patients with legionnaires disease produce sputum [9–12]. Other factors influence the sensitivity of culture once a sputum sample is obtained. Legionella bacteria may survive poorly in respiratory secretions, and these samples should be processed promptly. Some patients with legionnaires disease produce sputum that has relatively little purulence; these samples may be rejected by laboratories that discard sputum samples containing few polymorphonuclear leukocytes [13]. Consequently, rejection criteria should not be applied to sputum samples sent for Legionella culture. The experience of laboratory staff is also important, and laboratories experienced at Legionella culture are more likely to recover the organism.

Estimated sensitivities of sputum culture range from <10% to ∼80% and vary according to different comparison standards and by individual laboratories [9, 14–16]. In practice, the better results are likely to be achieved only by laboratories with a special interest in Legionella infection, and sensitivities are usually <50% when serological findings are used as the standard. Bronchoscopic samples are likely to produce a greater diagnostic yield than are expectorated sputum samples.

Legionella species can be isolated from blood cultures, but the yield is poor. Growth of legionellae is maintained by routine blood culture media but may not activate the alarm of commercial blood culture machines [17]. Consequently, blind subcultures onto solid media are required. Overall, the yield of blood cultures is low and is unlikely to influence clinical management.

Direct Fluorescent Antibody (Dfa) Staining

DFA staining can detect legionellae in respiratory secretions and tissue samples. This technique has the advantage of providing a result within 2–4 h, but it is technically demanding and should be performed by experienced laboratory personnel. Reported sensitivities of DFA staining vary, are consistently less than that of culture, and are less precisely known for species other than L. pneumophila. For lower respiratory tract secretions, sensitivities generally have a range of 25%–66%, with bronchoalveolar lavage fluid specimens having a higher yield than either transtracheal or sputum samples [15, 18, 19]. The specificity of DFA staining has been estimated to be ∼94% [18], although the test is likely to be less specific in inexperienced hands. False-positive results may occur because of cross-reactions with other bacteria, including Bacteroides fragilis, Pseudomonas species, Stenotrophomonas species, and Flavobacterium species. Cross-reactions may be less of a problem when monoclonal-antibody DFA agents are used. Problems with sensitivity and specificity have limited the use of DFA staining, and a positive DFA result in the absence of other supporting evidence is now generally not accepted as sufficient for the diagnosis of Legionella infection.

Urinary Antigen Detection

Detection of soluble Legionella antigen in urine specimens is a rapid method that provides an early diagnosis of Legionella infection and has been a useful tool for the investigation of outbreaks of legionnaires disease [20]. Commercial kits that use both RIA and EIA methodologies have been available for several years and have similar performance characteristics. Recently, an immunochromatographic assay (NOW Legionella Urinary Antigen Test; Binax) has been developed that has similar sensitivity and specificity to EIA [21]. This test is easy to perform and can provide a result within 15 min.

For the detection of L. pneumophila serogroup 1, urinary antigen tests have sensitivities in the range of 70%–100% and specificities approaching 100% [22]. Sensitivities can be increased by as much as 20% by 25-fold concentration of urine samples before testing [23]. The major disadvantage with these tests is their inability to reliably detect organisms other than L. pneumophila serogroup 1. Broad-spectrum assays that detect soluble antigens from a wide variety of Legionella species have been developed but are not commercially available. The Biotest Legionella Urine Antigen EIA (Biotest) is intended to detect legionellae other than L. pneumophila serogroup 1, but it does so less reliably than it detects L. pneumophila serogroup 1 [24]. False-positive urinary antigen results have occurred in patients with serum sickness [25].

Legionella antigenuria can be detected as early as 1 day after onset of symptoms and persists for days to weeks. In one instance, excretion of antigen was documented to occur for >300 days [26]. Soluble Legionella antigens have also been detected in samples other than urine, including samples of sputum, lung tissue, serum, and pleural fluid, although the use of such samples has not been fully evaluated.

Urinary antigen testing is now an established and valuable tool for the diagnosis of legionnaires disease, particularly in regions where L. pneumophila serogroup 1 is the most common cause of the disease. In locations where only a minority of infections are caused by L. pneumophila serogroup 1, currently available urinary antigen assays contribute less to existing laboratory tests.

Serological Testing

Serological testing for Legionella infection is a valuable epidemiological tool but has little impact on clinical decision making because of the time delay before a result is available. The antibodies produced in response to infection are generally a mixture of IgA, IgM, and IgG, and tests should detect all types for optimal sensitivity. The measurement of specific IgM is an unreliable marker of acute infection, because IgM antibodies can persist for long periods of time. Seroconversion may take several weeks, which is a major limitation of serological testing. In most cases, a 4-fold increase in antibody titer is detected within 3–4 weeks, but in some cases, this may take >10 weeks [27]. Obtainment of convalescent-phase serum samples too early undoubtedly results in many false-negative results and probably partially accounts for the 20%–30% of patients with legionnaires disease who supposedly do not develop a detectable antibody response [18]. However, it is clear that a proportion of people with proven Legionella infection do not have detectable seroconversion [15]. In practice, clinicians should be encouraged to obtain convalescent-phase serum samples 3 weeks after the onset of illness for testing in parallel with serum obtained during the acute phase. If there is no seroconversion after this time period and Legionella infection is still suspected, an additional convalescent-phase sample should be obtained.

Of the various antibody detection methods that are available, indirect immunofluorescence is the standard reference test. A 4-fold or greater increase in reciprocal antibody titer to ⩾128 is considered diagnostic. Acute-phase reciprocal antibody titers of ⩾256 in the presence of pneumonia were once considered sufficient for a presumptive diagnosis, but this has been shown to be unreliable [28], especially given the high prevalence of Legionella antibody positivity among some people without clinical evidence of legionellosis. There is virtually no role for testing single serum samples.

Another disadvantage of serological testing is the inability to accurately detect all Legionella species and serogroups. Although seroconversion to L. pneumophila serogroup 1 is generally regarded as being highly predictive of disease, the sensitivity and specificity of seroconversion to other species and serogroups has not been rigorously confirmed. Furthermore, cross-reactive antibody formation among members of the family Legionellaceae can make it difficult to determine the infecting species or serogroup. Some patients with non–L. pneumophila serogroup 1 infection will have seroconversion to L. pneumophila serogroup 1 [29]. Conversely, sequence-based identification of Legionella PCR products from patients who have had seroconversion to Legionella species other than L. pneumophila indicate that some were likely to have been infected with L. pneumophila (unpublished observations). Cross-reactive antibodies are also occasionally found in patients with infections caused by non-Legionella bacteria, including pseudomonads, mycobacteria, Bacteroides species, and Campylobacter species. An interesting cross-reaction occurs between L. bozemanii and Rickettsia typhi as a result of a shared antigen [30].

Nucleic Acid Amplification

Recently, DNA detection techniques have shown promise for the rapid diagnosis of Legionella infection. PCR enables specific amplification of minute amounts of Legionella DNA, provides results within a short time frame, and has the potential to detect infections caused by any Legionella species and serogroup. Currently, Legionella PCR is only available in a limited number of laboratories that use a variety of in-house assays.

PCR has been successfully used to detect Legionella DNA in a range of environmental and clinical samples. When testing samples from the lower respiratory tract, PCR has repeatedly been shown to have a sensitivity equal to or greater than culture [31–33]. Indeed, PCR could be considered the test of choice for patients who produce sputum. The role of PCR for testing other sample types is less clear. Legionella DNA can be detected in urine, serum, and leukocyte samples obtained from patients with legionnaires disease with sensitivities of 30%–86% [34–37]. The sensitivity of PCR is likely to increase when testing samples that are obtained early in the course of illness and when testing >1 sample type from each patient [34]. Throat swabs may also be a suitable sample for PCR testing, but this application has only been evaluated in a single study [38].

Further work is needed to establish a standard PCR method that will be robust enough to be used outside the setting of a research laboratory. Its application to nonrespiratory samples is particularly attractive, because this will circumvent the problem of patients who do not produce sputum samples. The development of an optimal PCR assay has been complicated by the intermittent contamination of some commercial DNA extraction kits with Legionella DNA [39]. Use of these kits to process samples may result in false-positive results and highlights the importance of including appropriate controls in each assay run.

Testing Strategy

Because legionnaires disease cannot be clinically or radiographically distinguished from other causes of pneumonia, the decision to test for Legionella infection can be difficult and is often made erratically. It is important to be familiar with local epidemiology. Some laboratories in areas where Legionella species are a common cause of pneumonia have elected to routinely culture all sputum samples obtained from patients with pneumonia on Legionella media. Few laboratories, however, would be able to justify the additional cost associated with routine Legionella culture. In most locations, the incidence of Legionella infection is unknown, and the decision to order diagnostic tests for Legionella infection is usually limited to at-risk patients, to patients with severe pneumonia, and to outbreak scenarios. It is certainly possible to select a subset of patients for whom the yield from Legionella diagnostic tests is likely to be relatively high. This group includes elderly persons, smokers, immunosuppressed individuals, those with chronic lung disease, patients who reside in hospitals with _Legionella-_colonized water supplies, and individuals exposed to potting mix. The incidence of legionnaires disease is higher among patients with severe pneumonia, and all patients with pneumonia who are admitted to an intensive care unit should be tested for this infection.

In locations where L. pneumophila serogroup 1 is the predominant cause of Legionella infection, or during an outbreak of infection with L. pneumophila serogroup 1, the urinary antigen test is a particularly valuable diagnostic tool. The development of a commercial urinary antigen test that also reliably detects other human Legionella pathogens would be a major advance and would likely make this the diagnostic test of choice in almost all settings. In geographic locations where legionellae other than L. pneumophila serogroup 1 are numerically important pathogens, current urinary antigen tests are still useful but should not be used as the sole diagnostic tool. If available, Legionella PCR combined with urinary antigen testing is likely to be the best initial testing strategy that will detect all Legionella species and provide results within a time frame that will affect clinical management.

Where PCR is unavailable, the urinary antigen test combined with culture of lower respiratory tract samples is the best test combination. Culture remains an important diagnostic tool, but its relatively low sensitivity and the reliance on the availability of a lower respiratory tract sample make it inadequate as a sole diagnostic test. Although serological testing has no impact on initial management, it can be useful if a specific diagnosis is not made during the acute phase of infection, but both acute- and convalescent-phase samples must be tested in parallel. During a suspected outbreak of legionnaires disease, an aggressive approach to diagnosis is warranted, which would involve the use of a combination of testing methods.

Legionella species may occasionally cause Pontiac fever, an acute, febrile, nonpneumonic illness characterized by a high attack rate, short incubation period, and rapid recovery. The diagnosis of Pontiac fever usually relies on the recognition of typical clinical features during an outbreak situation, and the diagnosis is confirmed by serological testing of affected persons. The use of culture is largely limited to testing environmental samples to determine the source of the outbreak, although L. pneumophila serogroup 1 has been isolated from a tracheal aspirate from a child with Pontiac fever [40].

Conclusions

Legionella infection is undoubtedly underrecognized. Diagnosis relies on the use of specialized tests, often in combination. Urinary antigen tests, sputum culture, and PCR testing of lower respiratory tract samples are the most important diagnostic tools for detection of Legionella infection early in the course of illness. The development of urinary antigen test assays that detect a wider range of pathogenic legionellae and the development of standardized PCR assays will be major advances in Legionella diagnostics. The increased availability and use of improved diagnostic tests will help better characterize the epidemiology of legionnaires disease, including the true incidence and geographic variation.

Acknowledgments

I am grateful to Steve Chambers for helpful comments on the article in manuscript.

References

1

, , , , , .

Initial microbiologic studies did not affect outcome in adults hospitalized with community-acquired pneumonia

,

Am J Respir Crit Care Med

,

1999

, vol.

160

(pg.

346

-

8

)

2

, , , , .

Nonvalue of the initial microbiological studies in the management of nonsevere community-acquired pneumonia

,

Chest

,

2001

, vol.

119

(pg.

181

-

4

)

3

.

Diagnosis of Legionellaceae as a cause of community-acquired pneumonia: “...continue to treat first and not bother to ask questions later”—not a good idea

,

Am J Med

,

2001

, vol.

110

(pg.

73

-

5

)

4

.

Legionnaires' disease

,

Clin Infect Dis

,

1993

, vol.

16

(pg.

741

-

9

)

5

, , , et al.

Distribution of Legionella species and serogroups isolated by culture in patients with sporadic community-acquired legionellosis: an international collaborative study

,

J Infect Dis

,

2002

, vol.

186

(pg.

127

-

8

)

6

, , .

Distribution of Legionella longbeachae serogroup 1 and other legionellae in potting soils in Australia

,

Appl Environ Microbiol

,

1990

, vol.

56

(pg.

2984

-

8

)

7

, , , , .

Increased recovery of Legionella micdadei and Legionella bozemanii on buffered charcoal yeast extract agar supplemented with albumin

,

J Clin Microbiol

,

1990

, vol.

28

(pg.

616

-

8

)

8

, , , .

Growth of 28 Legionella species on selective culture media: a comparative study

,

J Clin Microbiol

,

1993

, vol.

31

(pg.

2764

-

8

)

9

, , , et al.

Comparative study of the clinical presentation of Legionella pneumonia and other community-acquired pneumonias

,

Chest

,

1998

, vol.

113

(pg.

1195

-

200

)

10

, , , .

Legionella species community-acquired pneumonia: a review of 56 hospitalized adult patients

,

Chest

,

1996

, vol.

109

(pg.

1243

-

9

)

11

, .

Legionnaires' disease: a review of 79 community acquired cases in Nottingham

,

Thorax

,

1986

, vol.

41

(pg.

635

-

40

)

12

, , , , , .

Legionnaires' disease: clinical features of the epidemic in Philadelphia

,

Ann Intern Med

,

1979

, vol.

90

(pg.

509

-

17

)

13

, .

Danger of sputum purulence screens in culture of Legionella species

,

J Clin Microbiol

,

1994

, vol.

32

(pg.

209

-

10

)

14

, , , , .

Legionella, Chlamydia pneumoniae and Mycoplasma infection in patients admitted to Christchurch Hospital with pneumonia

,

N Z Med J

,

1999

, vol.

112

(pg.

222

-

4

)

15

, , , , .

Diagnosis of Legionnaires' disease: an update of laboratory methods with new emphasis on isolation by culture

,

JAMA

,

1983

, vol.

250

(pg.

1981

-

5

)

16

, , , et al.

Microbial etiology of acute pneumonia in hospitalized patients

,

Chest

,

1992

, vol.

101

(pg.

1005

-

12

)

17

, , , , .

Isolation of Legionella pneumophila from blood with the BACTEC system: a prospective study yielding positive results

,

J Clin Microbiol

,

1985

, vol.

22

(pg.

422

-

4

)

18

, , .

Laboratory diagnosis of Legionnaires' disease

,

Am Rev Respir Dis

,

1980

, vol.

121

(pg.

317

-

27

)

19

, , .

Direct immunofluorescence in the diagnosis of Legionnaires' disease

,

Chest

,

1981

, vol.

79

(pg.

566

-

70

)

20

, , , , .

Rapid diagnosis of Legionnaires' disease using an immunochromatographic assay for Legionella pneumophila serogroup 1 antigen in urine during an outbreak in the Netherlands

,

J Clin Microbiol

,

2000

, vol.

38

(pg.

2738

-

9

)

21

, , , et al.

Evaluation of a rapid immunochromatographic assay for the detection of Legionella antigen in urine samples

,

Eur J Clin Microbiol Infect Dis

,

1999

, vol.

18

(pg.

896

-

8

)

22

, .

Legionella urinary antigen testing: potential impact on diagnosis and antibiotic-therapy

,

Diagn Microbiol Infect Dis

,

1996

, vol.

24

(pg.

129

-

39

)

23

, , , et al.

Comparison of the Binax Legionella urinary antigen enzyme immunoassay (EIA) with the Biotest Legionella urine antigen EIA for detection of Legionella antigen in both concentrated and nonconcentrated urine samples

,

J Clin Microbiol

,

1998

, vol.

36

(pg.

2718

-

22

)

24

, , , et al.

A multicenter evaluation of the Biotest Legionella urinary antigen EIA

,

Clin Microbiol Infect

,

1998

, vol.

4

(pg.

359

-

65

)

25

, , , , , .

Case of false-positive results of the urinary antigen test for Legionella pneumophila

,

Clin Infect Dis

,

1999

, vol.

29

(pg.

953

-

4

)

26

, , .

Onset and duration of urinary antigen excretion in Legionnaires disease

,

J Clin Microbiol

,

1984

, vol.

20

(pg.

605

-

7

)

27

, , , , .

Delayed seroconversion in Legionnaire's disease

,

Lancet

,

1988

, vol.

2

8609

pg.

513

28

, , , et al.

Reevaluation of the definition of Legionnaires' disease: use of the urinary antigen assay

,

Clin Infect Dis

,

1995

, vol.

20

(pg.

1286

-

91

)

29

, , , , , .

Reactivity of serum from patients with suspected legionellosis against 29 antigens of Legionellaceae and _Legionella_-like organisms by indirect immunofluorescence assay

,

J Infect Dis

,

1983

, vol.

147

(pg.

23

-

31

)

30

, , , , .

Western blot analysis of immune response to Legionella bozemanii antigens

,

J Clin Pathol

,

1991

, vol.

44

(pg.

932

-

5

)

31

, , , et al.

Detection of Legionella spp. in bronchoalveolar lavage fluids by DNA amplification

,

J Clin Microbiol

,

1992

, vol.

30

(pg.

920

-

4

)

32

, , , , .

Detection of Legionella species in respiratory specimens using PCR with sequencing confirmation

,

J Clin Microbiol

,

2000

, vol.

38

(pg.

1709

-

12

)

33

, , , .

Enzyme-linked immunoassay for detection of PCR-amplified DNA of legionellae in bronchoalveolar fluid

,

J Clin Microbiol

,

1995

, vol.

33

(pg.

1247

-

52

)

34

, , , et al.

Use of the polymerase chain reaction to detect Legionella DNA in urine and serum samples from patients with pneumonia

,

Clin Infect Dis

,

1996

, vol.

23

(pg.

475

-

80

)

35

, , , et al.

Detection of Legionella DNA in human and guinea pig urine samples by the polymerase chain reaction

,

Eur J Clin Microbiol Infect Dis

,

1995

, vol.

14

(pg.

25

-

33

)

36

, , , , , .

Diagnostic relevance of the detection of Legionella DNA in urine samples by the polymerase chain reaction

,

Eur J Clin Microbiol Infect Dis

,

1999

, vol.

18

(pg.

716

-

22

)

37

, , , .

Detection of Legionella DNA in guinea pig peripheral leukocytes, urine and plasma by the polymerase chain reaction

,

Eur J Clin Microbiol Infect Dis

,

1999

, vol.

18

(pg.

445

-

7

)

38

, , , , .

Diagnosis of Legionella pneumophila, Mycoplasma pneumoniae, or Chlamydia pneumoniae lower respiratory infection using the polymerase chain reaction on a single throat swab specimen

,

Diagn Microbiol Infect Dis

,

1996

, vol.

24

(pg.

7

-

14

)

39

, , , et al.

Qiagen DNA extraction kits for sample preparation for Legionella PCR are not suitable for diagnostic purposes

,

J Clin Microbiol

,

2002

, vol.

40

pg.

1126

40

, , , , , .

An outbreak of Pontiac fever among children following use of a whirlpool

,

Clin Infect Dis

,

1998

, vol.

26

(pg.

1374

-

8

)

© 2003 by the Infectious Diseases Society of America

Topic:

I agree to the terms and conditions. You must accept the terms and conditions.

Submit a comment

Name

Affiliations

Comment title

Comment

You have entered an invalid code

Thank you for submitting a comment on this article. Your comment will be reviewed and published at the journal's discretion. Please check for further notifications by email.

Citations

Views

Altmetric

Metrics

Total Views 37,383

34,528 Pageviews

2,855 PDF Downloads

Since 12/1/2016

Month: Total Views:
December 2016 7
January 2017 75
February 2017 254
March 2017 165
April 2017 144
May 2017 96
June 2017 171
July 2017 137
August 2017 208
September 2017 198
October 2017 77
November 2017 218
December 2017 970
January 2018 1,266
February 2018 1,244
March 2018 1,272
April 2018 1,160
May 2018 1,175
June 2018 1,083
July 2018 1,077
August 2018 1,081
September 2018 966
October 2018 897
November 2018 947
December 2018 573
January 2019 654
February 2019 627
March 2019 774
April 2019 535
May 2019 525
June 2019 588
July 2019 664
August 2019 578
September 2019 529
October 2019 543
November 2019 446
December 2019 333
January 2020 532
February 2020 454
March 2020 328
April 2020 457
May 2020 245
June 2020 314
July 2020 271
August 2020 406
September 2020 286
October 2020 369
November 2020 401
December 2020 348
January 2021 298
February 2021 277
March 2021 374
April 2021 398
May 2021 266
June 2021 187
July 2021 213
August 2021 267
September 2021 279
October 2021 318
November 2021 320
December 2021 208
January 2022 203
February 2022 185
March 2022 291
April 2022 224
May 2022 278
June 2022 196
July 2022 191
August 2022 216
September 2022 202
October 2022 277
November 2022 274
December 2022 172
January 2023 174
February 2023 212
March 2023 292
April 2023 298
May 2023 241
June 2023 174
July 2023 213
August 2023 259
September 2023 216
October 2023 246
November 2023 201
December 2023 199
January 2024 237
February 2024 222
March 2024 289
April 2024 256
May 2024 189
June 2024 167
July 2024 169
August 2024 157
September 2024 190

Citations

189 Web of Science

×

Email alerts

More on this topic

Citing articles via

More from Oxford Academic