Comparison of the Biofire FilmArray RP, Genmark eSensor RVP, Luminex xTAG RVPv1, and Luminex xTAG RVP Fast Multiplex Assays for Detection of Respiratory Viruses (original) (raw)
Abstract
There are several U.S. FDA-cleared molecular respiratory virus panels available today, each with advantages and disadvantages. This study compares four multiplex panels, the BioFire Diagnostics FilmArray RP (respiratory panel), the GenMark Dx eSensor RVP (respiratory viral panel), the Luminex xTAG RVPv1, and the Luminex xTAG RVP fast. Three hundred specimens (200 retrospective and 100 consecutive) were tested using all four platforms to determine performance characteristics. The overall sensitivity and specificity, respectively, and 95% confidence interval (CI; in parentheses) for each panel were as follows: FilmArray RP, 84.5% (79.2, 88.6) and 100% (96.2, 100); eSensor RVP, 98.3% (95.5, 99.5) and 99.2% (95.4, 100); xTAG RVPv1, 92.7% (88.5, 95.4) and 99.8% (96.0, 100); and xTAG RVP fast, 84.4% (78.5, 88.9) and 99.9% (96.1, 100). The sensitivity of each assay fluctuated by viral target, with the greatest discrepancies noted for adenovirus and influenza virus B detection. Hands-on time and time to result were recorded and ease of use was assessed to generate a complete profile of each assay.
INTRODUCTION
Respiratory viral infections remain a leading cause of medical visits and can contribute significantly to morbidity and mortality. Most respiratory viruses present with similar symptoms, making a diagnosis difficult without laboratory testing. Accurate and timely identification of respiratory viruses benefits the patient, particularly when the pathogen is one for which therapy exists, such as influenza viruses (1). Additional benefits of respiratory viral identification include epidemiologic tracking of local outbreaks or epidemics, applying appropriate infection control measures to admitted patients (e.g., droplet and/or contact precautions and considerations in creating cohorts), and decreasing the use of unnecessary antimicrobial therapy, when appropriate (2, 3).
Current diagnostic techniques for the detection and identification of respiratory viruses are somewhat limited. Although rapid antigen testing offers quick results, the sensitivity and specificity of rapid antigen testing vary greatly (4, 5). The sensitivities of two rapid influenza virus tests for the detection of influenza virus A/H1N1/2009 averaged 21% (6), and respiratory syncytial virus (RSV) rapid antigen sensitivity as low as 59% has been reported (7). Direct fluorescent antibody testing and viral culture have a greater degree of sensitivity and have the advantage of detecting several viruses simultaneously; however, both are time consuming and demand significant experience to perform the testing and correctly interpret the results. To close the gap between sensitivity and time to result, amplification-based technologies have been developed. Several nucleic acid amplification tests are available as in vitro diagnostics for respiratory virus detection. Some assays only detect a single viral type, such as adenovirus (AdV) (Hologic Gen-Probe, San Diego, CA), parainfluenza viruses (PIV) (Hologic Gen-Probe), metapneumovirus (MPV) (Hologic Gen-Probe and Quidel, San Diego, CA), and influenza viruses (Cepheid, Sunnyvale, CA; IQuum, Marlborough, MA; and Qiagen, Gaithersburg, MD). Other assays offer limited multiplex testing, such as for influenza viruses and RSV (Focus Diagnostics, Cypress, CA; Nanosphere, Northbrook, IL; and Hologic Gen-Probe) and for RSV and metapneumovirus (Quidel). Currently, four molecular multiplex respiratory virus panels for the simultaneous detection of more than three respiratory viruses have been cleared by the U.S. FDA, namely, the FilmArray RP (respiratory panel) (BioFire Diagnostics, Salt Lake City, UT), the eSensor RVP (respiratory viral panel) (GenMark Dx, Carlsbad, CA), the Luminex xTAG RVPv1, and the Luminex xTAG RVP fast (Luminex Molecular Diagnostics, Austin, TX). Table 1 summarizes the details of these four FDA-cleared assays.
Table 1.
Details for the four FDA-cleared respiratory panels
Assay_a_ | Manufacturer | Methodology | Preextraction required | Viruses reported_b_ |
---|---|---|---|---|
FilmArray RP_c_ | BioFire Diagnostics | Endpoint melt curve analysis | No | AdV; CoV HKU1, NL63; influenza virus A (H1/2009, H1, H3); influenza virus B; MPV; PIV1, -2, -3, -4; RSV; RhV/EV |
eSensor RVP | GenMark Dx | Voltammetry | Yes | AdV (C, B/E); influenza virus A (H1/2009, H1, H3); influenza virus B; MPV; PIV1, -2, -3; RSV (A/B); RhV |
xTAG RVPv1 | Luminex Molecular Diagnostics | Fluorescence-labeled bead array | Yes | AdV; influenza virus A (H1, H3); influenza virus B; MPV; PIV1, -2, -3; RSV (A/B); RhV/EV |
xTAG RVP fast | Luminex Molecular Diagnostics | Fluorescence-labeled bead array | Yes | AdV; influenza virus A (H1, H3); influenza virus B; MPV; RSV; RhV/EV |
Studies comparing these molecular multiplex platforms to in-house molecular methods have found overall sensitivities and specificities, respectively, of 89.4% and 99.6% for the FilmArray RP (8), 95.4% and 99.7% for the eSensor RVP (9), 91.2% and 99.7% for the xTAG RVPv1 (10), and 78.8% and 99.6% for the xTAG RVP fast (11). Although there are additional published data on the performance of each of these four respiratory platforms, it is difficult to compare assay performance among the multiplex panels since the comparative method varies from study to study (12–17). To our knowledge, there has not been a comprehensive analysis of how these multiplex respiratory viral panels compare to each other. This study aims to compare the four FDA-cleared molecular multiplex respiratory viral panels with regard to sensitivity, specificity, and workflow parameters, such as hands-on time, time to result, and relative ease of use.
MATERIALS AND METHODS
Study specimens.
The nasopharyngeal (NP) swab specimens (n = 300) used in this study were collected from 294 symptomatic patients. Specimens were from the following age groups: ≤5 years (n = 105), 6 to 11 years (n = 31), 12 to 17 years (n = 18), and ≥18 years (n = 146). Positive (n = 161) and negative (n = 39) retrospective specimens (n = 200) were collected between August 2007 and November 2011, and consecutive specimens (n = 100) were collected in January 2012. Retrospective specimens were tested at the time of collection by the method requested by the clinician (laboratory-developed test [LDT], n = 72; xTAG RVPv1, n = 83; LDT and xTAG RVPv1, n = 6; viral culture, n = 1; Xpert Flu [Cepheid, Sunnyvale, CA], n = 38). The consecutive specimens were initially assayed using the xTAG RVPv1. Flocked swabs transported in universal transport medium (Becton, Dickinson, Sparks, MD) were used for all NP specimens collected starting in 2009. Otherwise, routine NP swabs in viral transport medium (Remel, Lenexa, KS) were used. All specimens were stored at −70°C. A true prospective study could not be performed due to the inability to predict the time period with greatest respiratory virus prevalence and variability. Consequently, we archived all NP swabs submitted for respiratory virus panel testing between November 2011 and April 2012 at −70°C. The positivity rate and number of circulating viruses was later determined to be highest in mid-January (35% positivity and 7 viruses circulating). Therefore, to mimic a prospective study, 100 consecutive NP swabs received beginning on 9 January 2012 were retrieved from the freezer for study inclusion.
Respiratory virus testing.
All NP swabs were tested with the four respiratory viral panels according to their respective package inserts. Extraction, when required, was performed using the bioMérieux EasyMag (Durham, NC) and was completed within 24 h of the samples being thawed. The assays were initiated within 24 h of extraction. Samples were kept at 4°C until all testing was complete, which occurred in less than 5 days. Standard eSensor and xTAG batches consisted of 21 specimens plus applicable controls. Process controls included the addition of bacteriophage MS-2 to all samples prior to extraction; bacteriophage lambda was used as a run control for the xTAG assays, as well as external positive and no-template controls. The FilmArray RP contained its own internal controls within each pouch—an RNA process control and a second-stage PCR control. Time studies were conducted by two technologists independently logging elapsed times for each step of the assay over the course of two to three batches; mean times are reported.
Reference result.
The interpretation of results was performed according to the individual package inserts. Those specimens with indeterminate or equivocal results were repeated according to the manufacturers' suggestions (FilmArray, n = 1, GenMark, n = 3, and xTAG RVPv1, n = 13). If the repeat result was positive or negative, the final result was documented as the second result obtained as recommended by the manufacturers. However, if the sample was again indeterminate/equivocal, the sample was excluded from the sensitivity analysis (n = 1), with the exception of the observed parainfluenza virus 1 (PIV1) and PIV3 cross-reactivity on the xTAG RVPv1, discussed below. These results were recorded as PIV3 positive only. A true positive was defined as being positive by two or more of the platforms used or positive by a laboratory-developed test. The xTAG assays were considered one platform (xTAG RVPv1 and xTAG RVP fast) for the purposes of defining a true positive. LDTs were available for adenovirus (18), enterovirus (EV) (19), influenza virus A and B (20), and RSV A and B (21). We previously determined that the enterovirus LDT does not cross-react with rhinovirus (RhV) (data not shown). LDTs were performed when there was not a positive consensus from the multiplex platforms (i.e., only one platform was positive). Adenovirus typing was performed by sequencing the hexon gene according to the protocol of Lu and Erdman (22). The Prodesse ProFAST+ assay (Hologic Gen-Probe) was performed according to the recommendations in the manufacturer's package insert for influenza virus A subtype analyses.
Statistical analysis.
To assess pairwise differences in sensitivity and specificity, P values were determined by Student's t test using GraphPad Prism 6.0 (GraphPad, La Jolla, CA). P values for comparative analyses of all four tests were determined using analysis of variance (ANOVA) (GraphPad). Confidence intervals were calculated by the modified Wald method (GraphPad).
This study was approved by the Institutional Review Board of the University of North Carolina at Chapel Hill.
RESULTS
Method comparison.
The archived retrospective specimens were selected to challenge the sensitivities of the assays. Positive NP specimens were selected to represent as many of the targets as possible. Of 200 specimens, 161 (80.5%) were consensus positives (e.g., positive for at least one of the viruses detectable by at least two platforms). The consecutive arm of the study was designed to simulate real laboratory conditions during a period of relatively high respiratory disease and to moderate the bias that is inherent in using archived specimens for comparative studies. Of the 100 consecutive NP swabs, 38 were positive for at least one of the detectable viruses, as follows: AdV (n = 3), influenza virus A/H3 (n = 1), MPV (n = 10), PIV1 (n = 2), RSV (n = 8), and RhV/EV (n = 17). No influenza virus B, PIV2, or PIV3 positives were detected during this study period. Since positive specimens were limited during the consecutive arm, the sensitivity and specificity analyses were performed using all specimens combined (n = 300). The sensitivity data for the four multiplex respiratory virus assays are summarized in Table 2.
Table 2.
Sensitivity per target for each of the four respiratory viral panels as determined using NP swabs
Virus | No. of true-positive specimens (n = 300 specimens tested) | % Sensitivity (95% CI) of: | |||
---|---|---|---|---|---|
FilmArray RP | eSensor RVP | xTAG | |||
RVPv1 | RVP fast | ||||
AdV | 35 | 57.1 (40.8, 72.0) | 100 (88.2, 100) | 74.3 (57.8, 86.0) | 82.9 (66.9, 92.3) |
Influenza virus | |||||
A | 30 | 86.2_a_ (68.8, 95.1) | 100 (86.5, 100) | 100 (86.5, 100) | 86.7 (69.7, 95.3) |
A H1/09 | 16 | 73.3_a_ (47.6, 89.5) | 100 (77.3, 100) | 100 (77.3, 100) | 81.3 (56.2, 94.2) |
A H3 | 14 | 100 (74.9, 100) | 100 (74.9, 100) | 92.9 (66.5, 99.9) | 78.6 (51.7, 93.2) |
B | 22 | 77.3 (56.2, 90.3) | 100 (82.5, 100) | 95.5 (76.5, 99.9) | 45.5 (26.9, 65.4) |
MPV | 26 | 96.2 (79.6, 99.9) | 100 (84.8, 100) | 100 (84.8, 100) | 100 (84.8, 100) |
PIV | |||||
1 | 14 | 100 (74.9, 100) | 100 (74.9, 100) | 100 (74.9, 100) | NA_b_ |
2 | 13 | 92.3 (64.6, 99.9) | 100 (73.4, 100) | 100 (73.4, 100) | NA |
3 | 13 | 100 (73.4, 100) | 100 (73.4, 100) | 100 (73.4, 100) | NA |
RSV | |||||
A | 22 | 86.4 (65.8, 96.1) | 100 (82.5, 100) | 86.4 (65.8, 96.1) | 86.4 (65.8, 96.1) |
B | 14 | 100 (74.9, 100) | 100 (74.9, 100) | 92.9 (66.5, 99.9) | 85.7 (58.8, 97.2) |
RhV/EV | 43 | 83.7 (69.7, 92.2) | 90.7 (77.8, 96.9) | 93.0 (80.7, 98.3) | 93.0 (80.7, 98.3) |
The FilmArray RP had sensitivities greater than 92% for the following targets: influenza virus A/H3, MPV, PIV1, PIV2, PIV3, and RSV B. However, AdV (57.1%), influenza virus A H1/2009 (73.3%), influenza virus B (77.3%), RSV A (86.4%), and RhV/EV (83.7%) were detected with less sensitivity by the FilmArray. As we had no comparator method for the coronaviruses or PIV4, these targets were not included in the analysis. The eSensor RVP sensitivities were 100% for all targets with the exception of RhV (90.7%). The sensitivities of xTAG RVPv1 were greater than 92% for all targets except AdV (74.3%) and RSV A (86.4%). The RVP fast version of the assay was less sensitive overall; all targets showed <92% sensitivity, with the exception of MPV (100%) and RhV/EV (93.0%). Since the xTAG RVP fast does not detect PIV1, PIV2, or PIV3, we could not assess their individual performance on this platform.
Specificities were high for all assays. The FilmArray RP was 100% specific for all targets (95% confidence interval [CI], 96.2, 100). The eSensor recorded false-positive results for AdV (n = 3), influenza virus B (n = 1), MPV (n = 2), PIV2 (n = 2), RSV B (n = 2), and RhV (n = 7). Sporadic false-positive signals were detected by the xTAG RVPv1 and xTAG RVP fast for AdV (1 each), MPV (1 each), and RhV (xTAG RVPv1, n = 2, and xTAG RVP fast, n = 1). The overall specificities (95% CIs) for the eSensor RVP, xTAG RVPv1, and xTAG RVP fast tests were 99.2% (95.4, 100), 99.8% (96.0, 100), and 99.9% (96.1, 100), respectively.
Coinfections.
Thirty samples were dually positive, and one was positive for three viruses, representing a coinfection rate of 10%. Twenty-one of the dual positives involved RhV/EV. Specific details on the coinfections identified by each platform can be found in Table 3. Overall, the eSensor RVP detected the most coinfections (96.8%), followed by the xTAG RVPv1 (71.0%), FilmArray RP (61.3%), and xTAG RVP fast (54.8%).
Table 3.
Combinations of multiple viruses identified during the study and how many instances were detected by each assay
Viral combination_b_ | No. of coinfections detected by: | No. of true-positive results | |||
---|---|---|---|---|---|
FilmArray RP | eSensor RVP | xTAG | |||
RVPv1 | RVP fast | ||||
AdV + InfA H1/2009 | 0 | 1 | 0 | 0 | 1 |
AdV + MPV | 0 | 1 | 0 | 1 | 1 |
AdV + PIV2 | 1 | 1 | 1 | NA_a_ | 1 |
AdV + RhV/EV | 4 | 6 | 5 | 5 | 6 |
AdV + RSV A | 1 | 4 | 2 | 3 | 4 |
InfA H1/2009 + RhV/EV | 1 | 1 | 1 | 1 | 1 |
InfA H1/2009 + RSV B | 1 | 2 | 2 | 2 | 2 |
MPV + RhV/EV | 1 | 1 | 1 | 1 | 1 |
PIV1 + RhV/EV | 2 | 1 | 1 | NA | 2 |
PIV2 + RhV/EV | 2 | 4 | 3 | NA | 4 |
PIV3 + RhV/EV | 0 | 1 | 0 | NA | 1 |
RSV A + RhV/EV | 5 | 5 | 5 | 4 | 5 |
RSV B + RhV/EV | 1 | 1 | 1 | 0 | 1 |
AdV + MPV + RSV A | 0 | 1 | 0 | 0 | 1 |
Total | 19 | 30 | 22 | 17 | 31 |
Adenovirus.
The greatest variability observed was for AdV detection. All AdV-positive samples (n = 35) were typed by sequencing the hexon gene. Samples with low levels of virus (determined by LDT cycle threshold [_CT_] values of >35) were not able to be typed (n = 4). These four samples were not detected by the xTAG RVPv1, xTAG RVP fast, or FilmArray RP but were detected by the eSensor RVP and our adenovirus LDT (Table 4). The eSensor RVP detected 100% of adenovirus-positive specimens and accurately identified them as species C or species B/E when compared to the results of hexon gene sequencing. With 57.1% sensitivity for AdV detection, the FilmArray RP detected all of the AdV type 3 and 4 specimens but only detected 4 of 7 of AdV type 1, 2 of 7 of AdV type 2, 1 of 3 AdV type 5, and 1 of 2 AdV type 21 specimens. The xTAG RVPv1 detected all of the AdV type 4 and 21 and 5 of 7 AdV type 1, 6 of 7 AdV type 2, 7 of 8 AdV type 3, and 2 of 3 AdV type 5 specimens, while xTAG RVP fast missed one specimen each of AdV type 1 and AdV type 3.
Table 4.
Results by adenovirus type and platform_a_
AdV type | No. of specimens positive by: | Total positive | |||
---|---|---|---|---|---|
FilmArray RP | eSensor RVP | xTAG | |||
RVPv1 | RVP fast | ||||
1 | 4 | 7 | 5 | 6 | 7 |
2 | 2 | 7 | 6 | 7 | 7 |
3 | 8 | 8 | 7 | 7 | 8 |
4 | 4 | 4 | 4 | 4 | 4 |
5 | 1 | 3 | 2 | 3 | 3 |
21 | 1 | 2 | 2 | 2 | 2 |
Untypeable | 0 | 4 | 0 | 0 | 4 |
All | 20 | 35 | 26 | 29 | 35 |
Rhinovirus/enterovirus.
The FilmArray RP and the xTAG assays show cross-reactivity between RhV and EV, while the eSensor RVP is specific for RhV. Therefore, we investigated whether the 90.7% eSensor sensitivity for RhV was due to the lack of EV cross-reactivity. We tested false-negative eSensor RVP RhV specimens using our enterovirus-specific LDT. Three of 4 (75%) false-negative eSensor RVP RhV specimens were positive by our enterovirus LDT. Therefore, these do not represent false negatives for RhV detection.
Age group-specific performance.
We compared the overall sensitivities and specificities for each platform by age groups: ≤5 years, 6 to 11 years, 12 to 17 years, and ≥18 years. The eSensor RVP did not have any statistically significant differences by age group, and none of the assays showed differences in specificity by age group. The xTAG RVP fast and FilmArray RP both showed higher sensitivities for the ≥18-year age group than for the <18-year age group (xTAG RVP fast, 89.8% versus 81.5% [P = 0.038], and FilmArray, 91.9% versus 80.6% [P = 0.004]). The FilmArray RP also demonstrated different sensitivities for the ≤5-year age group and the ≥18-year age group (80.2% versus 91.1%; P = 0.01). For the xTAG RVPv1, the age group that had the lowest sensitivity (85.5%) was the ≤5-year group, and this was significantly different than the sensitivities of the 6- to 17-year (100%; P < 0.001) and the ≥18-year (97.3%; P = 0.002) groups.
DISCUSSION
Without doubt, the commercial availability of multiplex molecular respiratory viral panels represents a significant advancement in the laboratory diagnosis of respiratory viral illnesses. Nonetheless, each system has its advantages and disadvantages, and each user should determine which system is appropriate for their specific laboratory and/or patient population. Since not all laboratories are able to do a side-by-side study of the four FDA-cleared molecular respiratory viral panels, we have attempted to demonstrate the relative performance of each assay, including a workflow assessment. Table 5 shows the four assays as they compare in terms of hands-on time, time to result, and number of steps (as an indicator of ease of use).
Table 5.
Workflow analysis of the four platforms
Parameter | Time_a_ (h) or other value as indicated using: | |||
---|---|---|---|---|
FilmArray RP | eSensor RVP | xTAG | ||
RVPv1 | RVP fast | |||
Off-board extraction_b_ | No | Yes | Yes | Yes |
No. of steps | 1 | 3 | 5 | 2 |
Hands-on time | 0.05 | 0.92 | 1.2 | 0.75 |
Instrument time | 1.1 | 5.0 | 5.5 | 2.75 |
Time to assay completion | 1.1 | 6.0 | 6.6 | 3.5 |
Total time to result | 1.2 | 7.2 | 7.8 | 4.8 |
No. of samples processed in 8 h per instrument | 7 | 21 | 21 | 21 |
BioFire's FilmArray RP was the easiest to use and had the shortest time to result. For MPV, influenza virus A/H3, PIV1, PIV2, PIV3, and RSV B, its sensitivities were >92%, whereas the sensitivities for AdV, influenza virus A/H1N1/2009, influenza virus B, RSV A, and RhV/EV were lower. The assay's AdV sensitivity was significantly lower than those of the eSensor (P < 0.0001) and xTAG RVP fast (P < 0.05) but similar to that of the xTAG RVPv1 (P = 0.13); its influenza virus A/H1N1/2009 sensitivity was lower than those of the eSensor and xTAG RVPv1 (P < 0.05) but comparable to that of the xTAG RVP fast (P = 0.67); and its influenza virus B sensitivity was lower than that of the eSensor (P < 0.05) only. However, the FilmArray RP was significantly more sensitive than the xTAG RVP fast for influenza virus B (P < 0.05), and there were no statistically significant differences in sensitivity rates between the assays for RSV A or RhV/EV. The FilmArray RP was 100% specific. An extended FDA-cleared panel of 20 targets is now available that includes Chlamydophila pneumoniae, Mycoplasma pneumoniae, and Bordetella pertussis, making the FilmArray RP the panel with the most targets and the only one to include bacterial targets. We did not assess the performance of these additional targets in this study. As the only closed system in our study and the only system to provide only qualitative results, the FilmArray RP is amenable to testing outside a dedicated high-complexity molecular laboratory and is appropriately labeled by the FDA as being of moderate complexity. This is in contrast to the other three panels evaluated in this study that are FDA cleared as high-complexity tests and are not closed systems. On the downside, each FilmArray instrument can only test one sample at a time, limiting its capacity for high-throughput testing and requiring the need for more than one instrument in most laboratories.
The GenMark eSensor RVP was the most sensitive of the four assays, detecting 100% of the samples known to be positive for AdV, influenza virus A, influenza virus B, MPV, PIV1, PIV2, PIV3, RSV A, and RSV B. The eSensor RVP is the only assay to report adenovirus species (AdV C or AdV B/E), doing so with 100% accurate results compared to our sequencing results. However, the loss of enterovirus detection is a potential limitation, particularly for laboratories that routinely test for enteroviruses in respiratory specimens. The increased sensitivity of the eSensor RVP is countered by a lower specificity than the other three assays (P < 0.05). Whether the false positives represent increased analytical sensitivity, greater variability in assay performance, or the ease with which runs can be contaminated cannot be determined. Although the eSensor RVP does provide a quantitative value in nanoamps (nA), the package insert states that it is not intended to provide a quantitative value for the virus present. Furthermore, Pierce and Hodinka concluded that an accurate quantitative correlation between real-time PCR CT values and nA signal strength could not be made (9). Similarly, our observed nA signal strengths for false positives were not statistically different from those obtained for true positives. The workflow for the eSensor RVP is a slight improvement over that of the predicate xTAG RVPv1, with two fewer steps and less hands-on time. However, the time to result is still over 7 h, and postamplification material is still manipulated, increasing the risk of laboratory contamination. The eSensor RVP is best suited for batched testing in a high-complexity laboratory with significant molecular experience.
xTAG RVPv1 was the first large multiplex panel cleared by the FDA. With multiple steps, it had the longest hands-on time and longest time to result. Although its sensitivities for AdV (74%) and RSV A (86%) were somewhat lower, there was no statistical difference between the xTAG RVPv1 and the FilmArray RP or xTAG RVP fast for AdV, and there were no statistical differences in RSV A sensitivities between any of the tests (P = 0.81). Similar to the eSensor RVP, the xTAG RVP assay provides quantitative results in median fluorescent intensity (MFI) units, but the test is only intended to provide qualitative results, and we have observed that these MFI units are not linear compared to the CT values of real-time PCR (data not shown). However, specimens with high titers of PIV3 also caused a positive PIV1 signal; thus, the ratio of these signals must be assessed in determining PIV3 calls. The xTAG RVPv1 is best utilized as a batched test performed in an experienced molecular laboratory, as there are multiple steps where postamplification material is manipulated.
Although the xTAG RVP fast does decrease the time to result by 3 h relative to the xTAG RVPv1 and reduces the number of hands-on steps, it does not offer PIV detection, and there is a significant drop in overall sensitivity. The sensitivities were below 92% for all targets but MPV and RhV/EV, and importantly, the influenza virus B sensitivity was 45.5%. There were minimal specificity concerns with the xTAG RVP fast.
It is known that the detection of adenovirus presents a significant challenge to the molecular multiplex tests. While not all cause respiratory infection, there are presently 57 recognized human adenovirus serotypes. Based on the respective package inserts, the xTAG RVPv1 has decreased detection for species C (serotypes 1, 2, 5, and 6) and serotypes 7A (species B) and 41 (species F), while the FilmArray RP has decreased detection of serotypes 2 and 6 (species C). Thus, the lower sensitivities we observed for these two assays are not surprising, though the 57.1% sensitivity we observed for the FilmArray RP is significantly lower than what is stated in the package insert (88.9%). Although the FilmArray RP did miss five AdV serotype 2 specimens, it also missed three serotype 1 specimens, two serotype 5 specimens, and four specimens that were unable to be typed due to low viral quantity (Table 4). The observed sensitivity of the xTAG RVPv1 for adenovirus (74.3%) is very similar to the package insert data of 78.3%. The serotypes of AdV missed by the xTAG RVPv1 were 1, 3, 5, and untypeable (Table 4). The risk of offering a test with lower sensitivity for adenovirus will be dependent on the patient population served and whether additional testing for adenovirus is offered.
The xTAG RVPv1 and xTAG RVP fast will subtype influenza virus A-positive samples as seasonal H1 or seasonal H3. The eSensor RVP and FilmArray RP will also subtype H1N1/2009. Two specimens positive for the influenza virus A matrix gene only (subtype negative) by xTAG RVP and xTAG RVP fast were actually seasonal H3, as established by the other tests and confirmed by the Prodesse ProFAST+ assay. Four samples initially showed influenza virus A-positive/subtype-negative results by the eSensor RVP. All four were lower-level positives (nA of 44.7 to 59.0; for all positives, the mean nA was 152.6 [95% CI of 135.2, 167.1]). As recommended by the package insert, the samples were tested again, and all were negative. With the disappearance of seasonal H1 (which was oseltamivir resistant), there is not currently a clinical need to offer influenza virus A subtype information. However, subtype information is helpful for our epidemiology and infection control colleagues, and it may prove to be beneficial should differences in antiviral susceptibility develop.
Our study is partially limited by the inability to do a true prospective study. Unfortunately for the study, the respiratory season (2011-2012) during which the study was performed was not as robust as past seasons, including the noticeable absence of significant numbers of influenza virus-positive specimens. Therefore, the numbers per target included in the consecutive study are relatively low, making it difficult to ascertain the significance of differing performance among the assays in a true clinical setting. However, the same trends in sensitivity were observed in both the retrospective specimens and the archived, consecutive specimens (e.g., lower adenovirus and RSV sensitivities [data not shown]). Furthermore, the extended freezer time and freeze-thaw cycle for some of the specimens could affect assay performance. To minimize any differential effect on assay performance, we tested thawed specimens on all four platforms simultaneously and did not use a retrospective reference result, but it should be noted that such results may vary from the results for freshly obtained specimens that have not been frozen.
Another limitation is the use of a composite gold standard. Ideally, individual LDTs for each target would also be performed on every specimen to establish the reference result, but this was not possible in our laboratory due to the lack of LDTs for every target, as well as financial constraints. Thus, we used “positive by at least two platforms” as the reference result by which we calculated sensitivity and specificity. As such, an assay with greater sensitivity than all other assays tested would show lower specificity. There is also inherent bias associated with selecting archived specimens for comparative studies. The frozen specimens had primarily been tested with the xTAG RVPv1, although we did attempt to select as many specimens as possible that had been tested by LDTs. Thus, the retrospective sensitivity data may favor the xTAG RVPv1. Lastly, the eSensor RVP assay that was used was the research-use-only (RUO) version, as the assay had not yet been FDA cleared at the time of this study. We did not, however, analyze any RUO targets in the present work and note that the chemistry for target detection and the threshold for positivity (3 nA) are the same as for the in vitro diagnostics version. Even considering the study limitations, we assert that this head-to-head comparison of the four commercially available respiratory viral panels provides useful information regarding the relative performance of each assay.
Lastly, a significant consideration for clinical laboratories is the comparison of the relative costs of these assays. Although there is a paucity of data regarding the cost effectiveness of implementing molecular multiplex testing for respiratory viruses, Mahony et al. (23) demonstrated a Can$291 per inpatient cost saving by eliminating testing algorithms and using a molecular multiplex assay as the primary test. Although all three platforms require specialized instrumentation, none of the vendors require that the equipment be a capital purchase. In terms of reagents, the relative costs are FilmArray RP > eSensor RVP > xTAG RVPv1. This is also the order for the number of targets offered. However, note that the list is reversed when ranking hands-on time (Table 5). When all expenses are considered, the three competitive multiplex assays offer nearly the same cost per target. Therefore, when considering an FDA-cleared multiplex respiratory panel, it is important to focus on what is best for your patient population and your laboratory. Future generations of these tests and/or the development of new products promise to improve the ability to offer accurate, rapid, and cost-effective laboratory diagnoses for respiratory tract illnesses.
ACKNOWLEDGMENTS
We acknowledge the kind generosity of BioFire Diagnostics, GenMark Dx, and Luminex Molecular Diagnostics for providing the reagents used in this study.
M.B.M. has received speaker honoraria from Abbott Molecular (Luminex distributor), BioFire Technologies, and GenMark Dx. M.B.M. is a scientific advisory board member for BioFire Technologies.
Footnotes
Published ahead of print 13 March 2013
REFERENCES
- 1.Templeton KE. 2007. Why diagnose respiratory viral infection? J. Clin. Virol. 40(Suppl 1):S2–S4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Fox JD. 2007. Respiratory virus surveillance and outbreak investigation. J. Clin. Virol. 40(Suppl 1):S24–S30 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Krasinski K, LaCouture R, Holzman RS, Waithe E, Bonk S, Hanna B. 1990. Screening for respiratory syncytial virus and assignment to a cohort at admission to reduce nosocomial transmission. J. Pediatr. 116:894–898 [DOI] [PubMed] [Google Scholar]
- 4.Centers for Disease Control and Prevention 2012. Evaluation of 11 commercially available rapid influenza diagnostic tests—United States, 2011-2012. MMWR Morb. Mortal. Wkly. Rep. 61:873–876 [PubMed] [Google Scholar]
- 5.Uyeki TM, Prasad R, Vukotich C, Stebbins S, Rinaldo CR, Ferng YH, Morse SS, Larson EL, Aiello AE, Davis B, Monto AS. 2009. Low sensitivity of rapid diagnostic test for influenza. Clin. Infect. Dis. 48:e89–e92 [DOI] [PubMed] [Google Scholar]
- 6.Ginocchio CC, Zhang F, Manji R, Arora S, Bornfreund M, Falk L, Lotlikar M, Kowerska M, Becker G, Korologos D, de Geronimo M, Crawford JM. 2009. Evaluation of multiple test methods for the detection of the novel 2009 influenza A (H1N1) during the New York City outbreak. J. Clin. Virol. 45:191–195 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Goodrich JS, Miller MB. 2007. Comparison of Cepheid's analyte-specific reagents with BD Directigen for detection of respiratory syncytial virus. J. Clin. Microbiol. 45:604–606 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Pierce VM, Elkan M, Leet M, McGowan KL, Hodinka RL. 2012. Comparison of the Idaho Technology FilmArray system to real-time PCR for detection of respiratory pathogens in children. J. Clin. Microbiol. 50:364–371 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Pierce VM, Hodinka RL. 2012. Comparison of the GenMark Diagnostics eSensor respiratory viral panel to real-time PCR for detection of respiratory viruses in children. J. Clin. Microbiol. 50:3458–3465 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Pabbaraju K, Tokaryk KL, Wong S, Fox JD. 2008. Comparison of the Luminex xTAG respiratory viral panel with in-house nucleic acid amplification tests for diagnosis of respiratory virus infections. J. Clin. Microbiol. 46:3056–3062 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Gadsby NJ, Hardie A, Claas EC, Templeton KE. 2010. Comparison of the Luminex respiratory virus panel fast assay with in-house real-time PCR for respiratory viral infection diagnosis. J. Clin. Microbiol. 48:2213–2216 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Babady NE, Mead P, Stiles J, Brennan C, Li H, Shuptar S, Stratton CW, Tang YW, Kamboj M. 2012. Comparison of the Luminex xTAG RVP fast assay and the Idaho Technology FilmArray RP assay for detection of respiratory viruses in pediatric patients at a cancer hospital. J. Clin. Microbiol. 50:2282–2288 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Couturier MR, Barney T, Alger G, Hymas WC, Stevenson JB, Hillyard D, Daly JA. 19 February 2013. Evaluation of the FilmArray respiratory panel for clinical use in a large children's hospital. J. Clin. Lab. Anal. [Epub ahead of print.] doi:10.1002/jcla.21576 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Hammond SP, Gagne LS, Stock SR, Marty FM, Gelman RS, Marasco WA, Poritz MA, Baden LR. 2012. Respiratory virus detection in immunocompromised patients with FilmArray respiratory panel compared to conventional methods. J. Clin. Microbiol. 50:3216–3221 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Loeffelholz MJ, Pong DL, Pyles RB, Xiong Y, Miller AL, Bufton KK, Chonmaitree T. 2011. Comparison of the FilmArray respiratory panel and Prodesse real-time PCR assays for detection of respiratory pathogens. J. Clin. Microbiol. 49:4083–4088 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Pabbaraju K, Wong S, Tokaryk KL, Fonseca K, Drews SJ. 2011. Comparison of the Luminex xTAG respiratory viral panel with xTAG respiratory viral panel fast for diagnosis of respiratory virus infections. J. Clin. Microbiol. 49:1738–1744 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Rand KH, Rampersaud H, Houck HJ. 2011. Comparison of two multiplex methods for detection of respiratory viruses: FilmArray RP and xTAG RVP. J. Clin. Microbiol. 49:2449–2453 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Heim A, Ebnet C, Harste G, Pring-Akerblom P. 2003. Rapid and quantitative detection of human adenovirus DNA by real-time PCR. J. Med. Virol. 70:228–239 [DOI] [PubMed] [Google Scholar]
- 19.Monpoeho S, Dehee A, Mignotte B, Schwartzbrod L, Marechal V, Nicolas JC, Billaudel S, Ferre V. 2000. Quantification of enterovirus RNA in sludge samples using single tube real-time RT-PCR. Biotechniques 29:88–93 [DOI] [PubMed] [Google Scholar]
- 20.Popowitch EB, Rogers E, Miller MB. 2011. Retrospective and prospective verification of the Cepheid Xpert influenza virus assay. J. Clin. Microbiol. 49:3368–3369 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.van Elden LJ, van Loon AM, van der Beek A, Hendriksen KA, Hoepelman AI, van Kraaij MG, Schipper P, Nijhuis M. 2003. Applicability of a real-time quantitative PCR assay for diagnosis of respiratory syncytial virus infection in immunocompromised adults. J. Clin. Microbiol. 41:4378–4381 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Lu X, Erdman DD. 2006. Molecular typing of human adenoviruses by PCR and sequencing of a partial region of the hexon gene. Arch. Virol. 151:1587–1602 [DOI] [PubMed] [Google Scholar]
- 23.Mahony JB, Blackhouse G, Babwah J, Smieja M, Buracond S, Chong S, Ciccotelli W, O'Shea T, Alnakhli D, Griffiths-Turner M, Goeree R. 2009. Cost analysis of multiplex PCR testing for diagnosing respiratory virus infections. J. Clin. Microbiol. 47:2812–2817 [DOI] [PMC free article] [PubMed] [Google Scholar]