Concordance Study of 3 Direct-to-Consumer Genetic-Testing Services (original) (raw)

BACKGROUND

Several companies offer direct-to-consumer (DTC) genetic testing to evaluate ancestry and wellness. Massive-scale testing of thousands of single-nucleotide polymorphisms (SNPs) is not error free, and such errors could translate into misclassification of risk and produce a false sense of security or unnecessary anxiety in an individual. We evaluated 3 DTC services and a genomics service that are based on DNA microarray or solution genotyping with hydrolysis probes (TaqMan® analysis) and compared the test results obtained for the same individual.

METHODS

We evaluated the results from 3 DTC services (23andMe, deCODEme, Navigenics) and a genomics-analysis service (Expression Analysis).

RESULTS

The concordance rates between the services for SNP data were >99.6%; however, there were some marked differences in the relative disease risks assigned by the DTC services (e.g., for rheumatoid arthritis, the range of relative risk was 0.9–1.85). A possible reason for this difference is that different SNPs were used to calculate risk for the same disease. The reference population also had an influence on the relative disease risk.

CONCLUSIONS

Our study revealed excellent concordance between the results of SNP analyses obtained from different companies with different platforms, but we noted a disparity in the data for risk, owing to both differences in the SNPs used in the calculation and the reference population used. The larger issues of the utility of the information and the need for risk data that match the user's ethnicity remain, however.

Numerous companies now offer direct-to-consumer (DTC)3 (or direct access) genetic testing to evaluate ancestry, health, and wellness (1). This type of massive-scale testing of thousands of SNPs is a complex multistep analytical procedure and thus is liable to error. Such errors could translate into, for example, a misclassification of risk that in turn could produce a false sense of security or unnecessary anxiety in an individual. A previous study that investigated the concordance of SNP data from 23andMe and Navigenics showed a level of agreement of 99.7% and some considerable differences in the assigned relative risk of disease (2). The objective of the present study was to undertake a more wide-ranging study. To this end, we evaluated 3 DTC services and a genomics service that used DNA microarray analysis or genotyping with hydrolysis probes (TaqMan® analysis) and compared the test results obtained for the same individual (3).

Samples obtained from the same healthy volunteer were tested at 3 DTC genetic-testing services (23andMe, Mountain View, CA; deCODEme, Reykjavik, Iceland; Navigenics, Foster City, CA) and a genomics-analysis service (Expression Analysis, Durham, NC).

Samples for the DTC services were collected according to their respective instructions and with the sample-collection kits provided. Saliva samples were sent to 23andMe and Navigenics, and buccal swabs were sent to deCODEme. For Expression Analysis, samples of DNA were extracted from whole blood in our laboratory with the chemagic Magnetic Separation Module robot (chemagen Biopolymer-Technologie). The platform used for the analysis depended on the service. 23andMe and deCODEme used DNA microarrays from Illumina, with the HumanHap 550+ Genotyping BeadChip (approximately 5.78 × 105 SNPs analyzed) and Human1M-Duo DNA Analysis BeadChip (approximately 1.1 × 106 SNPs), respectively. Navigenics used Applied Biosystems TaqMan® Genotyping Assays (approximately 120 SNPs). Expression Analysis used 2 platforms: the Affymetrix Genome-Wide Human SNP Array 6.0 (approximately 9.09 × 105 SNPs) and the Illumina HumanOmniExpress BeadChip (approximately 7.33 × 105 SNPs). The DTC services performed the disease risk analyses. 23andMe and deCODEme allow customers to change their ancestry information for the disease risk analysis by using the Web page for selecting the optimum reference population. The ancestry information for Navigenics is fixed, however, and their analysis primarily uses the reference population of Americans with northern and western European ancestry. To better compare the results from 23andMe and deCODEme with those from Navigenics, we selected “Northern Europe” for 23andMe and “European Ancestry” for deCODEme.

We used Microsoft Access to compare the SNP data. Data in the .csv file format were imported directly into Access. Other data formats were first converted into .csv format (e.g., Affymetrix .cel files). For Navigenics, we copied the raw data directly from the Web page to obtain a .csv file. Some minor editing was required in some cases (e.g., remove header, convert minus to plus strand, or alphabetize the SNP list). We created a query in Access's Design View and compared pairs of tables to determine the number of SNPs common to the 2 files and the number of matching SNPs. The concordance rate was calculated by dividing the number of matched SNPs by the number of SNPs compared.

We compared the relative disease risks obtained from the DTC services. 23andMe and deCODEme provide relative disease risks on their Web pages. For Navigenics, we calculated relative risk by dividing the individual risk by the mean risk of the reference population. For deCODEme, we also investigated the influence of the reference population on relative disease risk by changing the ancestry information of the individual on the Web page.

Table 1 summarizes the concordance rates for the SNP data. The concordance rates were >99.6% in all the comparisons. In particular, the concordance rates for comparing Illumina microarrays were >99.9%. The concordance rates between Illumina and Affymetrix microarrays were 99.6%–99.7%. Concordance rates for SNP data for comparisons of DNA microarray analysis and TaqMan analysis were 100% in all of the comparisons, although the numbers of SNPs compared were much smaller (<100 SNPs).

Concordance rates for SNP data (concordant SNPs/total SNPs).

Table 1.

Concordance rates for SNP data (concordant SNPs/total SNPs).

Table 1.

Concordance rates for SNP data (concordant SNPs/total SNPs).

Fig. 1 shows the variation in relative disease risk assigned by the different DTC services. There were some marked differences in relative disease risk (see the Data Supplement that accompanies the online version of this Brief Communication at http://www.clinchem.org/content/vol57/issue3 for SNPs used in determining risk and the statistical significance of differences in the risk estimates).

Relative disease risk assigned by 3 DTC services for a series of diseases evaluated by all 3 services.

Fig. 1.

Values in parentheses indicate the number of SNPs analyzed (3a, results for 2 of 5 SNPs were unavailable, i.e., no base calls).

For example, the estimates for rheumatoid arthritis indicate a protective effect in 1 case (relative risk, 0.9) and a deleterious effect of the genotype on disease risk in the other 2 cases (relative risk, 1.3 and 1.85). Similarly, the disease risk for atrial fibrillation varied from 0.8 to 1.8. For age-related macular degeneration, deCODEme and Navigenics results produced relative disease risk values of approximately 1.1 (1.10 and 1.13, respectively); however, 23andMe produced a relative risk for this disease of 1.61. Fig. 1 also shows the number of SNPs used in the calculation of relative disease risk, and an examination of this figure reveals that the number of SNPs used in the risk calculation depended on the service. This finding might be one of the reasons for the differences in evaluations of relative disease risk. In addition, relative disease risks varied considerably, depending on the ancestry information. In the case of the deCODEme results, for example, the relative disease risk for rheumatoid arthritis was 1.86 when we used “European Ancestry”; however, the value changed to 1.15 when we used “African Ancestry.” The corresponding relative risks for colorectal cancer were 1.01 and 1.46, respectively.

DTC genetic testing is a highly controversial area, and several recent reports (4, 5) and statements by professional societies (6) have addressed this type of testing.

Concerns have been voiced over the lack of involvement of a clinician or genetic counselor in such testing (7), the regulation of such tests (8, 9), ethical and legal issues (10, 11), the variance in and the validity of risk assessment (2, 12), and the reliability of the overall analytical process and the possibility of genotyping errors (13–15).

Our study found a >99.6% concordance in SNP genotypes among 3 DTC services (23andMe, deCODEme, Navigenics) and a genomics-analysis service (Expression Analysis). This result agrees with a previous study that found a 99.7% concordance between DTC tests performed by 2 providers (23andMe and Navigenics) (2). Although this degree of concordance is very high, 100% accuracy must be the goal, because an error in the base call for a single SNP could produce a change in risk classification leading to a false sense of security or to an incorrect indication of increased risk.

The validity and importance of risk assessed by genotyping is a highly charged issue (12). We assessed the influence of ethnicity on risk classification. For some DTC services, the risk calculated applied only to Caucasian individuals, thus leading to the possibility of a misleading risk assessment for non-Caucasian individuals. Our investigation into the effect of selecting ethnicities different from that of the study individual on the risk calculations revealed the potential magnitude of this problem.

In conclusion, our study has revealed excellent concordance between the results of SNP analyses from different companies performed on different platforms. Nonetheless, for some of the diseases investigated, there is large variation in relative disease risks reported by the different companies. The larger issues of the utility of the information and the need for risk data that match the user's ethnicity remain.

Previous Presentations: Imai K, Kricka LJ, Fortina P. Concordance Study of Direct-To-Consumer (DTC) Genetic Testing. 2010 AACC Annual Meeting Poster Session. B-97.

3 Nonstandard abbreviations:

• DTC
• SNP
  single-nucleotide polymorphism.

Author Contributions:All authors confirmed they have contributed to the intellectual content of this paper and have met the following 3 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; and (c) final approval of the published article.

Authors' Disclosures or Potential Conflicts of Interest:No authors declared any potential conflicts of interest.

Role of Sponsor: The funding organizations played no role in the design of study, choice of enrolled patients, review and interpretation of data, or preparation or approval of manuscript.

References

Genetics and Public Policy Center

.

DTC genetic tests, 05/28/2010

.

(Accessed September 2010).

Ng

PC

,

Murray

SS

,

Levy

S

,

Venter

JC

.

An agenda for personalized medicine

.

Nature

2009

;

461

:

724

–

6

.

Imai

K

,

Kricka

LJ

,

Fortina

P

.

Concordance study of direct-to-consumer (DTC) genetic testing [Abstract]

.

Clin Chem

2010

;

56

(

S6

):

A83

.

Henen

L

,

Sauter

A

,

van den Cruyce

E

.

Direct to consumer genetic testing study

.

Brussels

:

European Parliament

;

2008

.

82

p.

(Accessed November 2010).

Human Genetics Commission

.

A Common Framework of Principles for direct-to-consumer genetic testing services

.

London

:

Human Genetics Commission

;

2010

.

(Accessed December 2010).

Hudson

K

,

Javitt

G

,

Burke

W

,

Byers

P

.

ASHG statement on direct-to-consumer genetic testing in the United States

.

Obstet Gynecol

2007

;

110

:

1392

–

5

.

Norrgard

K

.

DTC genetic testing for diabetes, breast cancer, heart disease and paternity

.

Nat Educ

2008

;

1

.

(Accessed November 2010).

Gniady

JA

.

Regulating direct-to-consumer genetic testing: protecting the consumer without quashing a medical revolution

.

Fordham Law Rev

2008

;

76

:

2429

–

75

.

Patch

C

,

Sequeiros

J

,

Cornel

MC

.

Genetic horoscopes: Is it all in the genes? Points for regulatory control of direct-to-consumer genetic testing

.

Eur J Hum Genet

2009

;

17

:

857

–

9

.

Samuel

GN

,

Jordens

CFC

,

Kerridge

I

.

Direct-to-consumer personal genome testing: ethical and regulatory issues that arise from wanting to ‘know’ your DNA

.

Intern Med J

2010

;

40

:

220

–

4

.

Tamir

S

.

Direct-to-consumer genetic testing: ethical-legal perspectives and practical considerations

.

Med Law Rev

2010

;

18

:

213

–

38

.

Janssens

AC

,

Gwinn

M

,

Bradley

LA

,

Oostra

BA

,

van Duijn

CM

,

Khoury

MJ

.

A critical appraisal of the scientific basis of commercial genomic profiles used to assess health risks and personalize health interventions

.

Am J Hum Genet

2008

;

82

:

593

–

9

.

Pompanon

F

,

Bonin

A

,

Bellemain

E

,

Taberlet

P

.

Genotyping errors: causes, consequences and solutions

.

Nat Rev Genet

2005

;

6

:

847

–

59

.

Berthier-Schaad

Y

,

Kao

WH

,

Coresh

J

,

Zhang

L

,

Ingersoll

RG

,

Sterphens

R

,

Smith

MW

.

Reliability of high-throughput genotyping of whole genome amplified DNA in SNP genotyping studies

.

Electrophoresis

2007

;

28

:

2812

–

7

.

The Spittoon [23andMe blog]

.

Update on 23andMe and Committee on Energy and Commerce of the United States House of Representatives

.

(Accessed November 2010).

© 2011 The American Association for Clinical Chemistry