Empirical comparison of cross-platform normalization methods for gene expression data - PubMed (original) (raw)

Comparative Study

Empirical comparison of cross-platform normalization methods for gene expression data

Jason Rudy et al. BMC Bioinformatics. 2011.

Abstract

Background: Simultaneous measurement of gene expression on a genomic scale can be accomplished using microarray technology or by sequencing based methods. Researchers who perform high throughput gene expression assays often deposit their data in public databases, but heterogeneity of measurement platforms leads to challenges for the combination and comparison of data sets. Researchers wishing to perform cross platform normalization face two major obstacles. First, a choice must be made about which method or methods to employ. Nine are currently available, and no rigorous comparison exists. Second, software for the selected method must be obtained and incorporated into a data analysis workflow.

Results: Using two publicly available cross-platform testing data sets, cross-platform normalization methods are compared based on inter-platform concordance and on the consistency of gene lists obtained with transformed data. Scatter and ROC-like plots are produced and new statistics based on those plots are introduced to measure the effectiveness of each method. Bootstrapping is employed to obtain distributions for those statistics. The consistency of platform effects across studies is explored theoretically and with respect to the testing data sets.

Conclusions: Our comparisons indicate that four methods, DWD, EB, GQ, and XPN, are generally effective, while the remaining methods do not adequately correct for platform effects. Of the four successful methods, XPN generally shows the highest inter-platform concordance when treatment groups are equally sized, while DWD is most robust to differently sized treatment groups and consistently shows the smallest loss in gene detection. We provide an R package, CONOR, capable of performing the nine cross-platform normalization methods considered. The package can be downloaded at http://alborz.sdsu.edu/conor and is available from CRAN.

PubMed Disclaimer

Figures

Figure 1

Figure 1

Mean-mean plots for MAQC group A ILM and AFX data after cross-platform normalization. Scatter or sunflower plots for MAQC treatment group A for each normalization method for the ILM and AFX data. Mean expression level of sample A assays on the AFX platform is plotted against mean expression level on the ILM platform. Plot titles indicate the cross-platform normalization method performed. Red lines are the line y = x, provided for visual comparison. Sub-titles indicate the _r_2 value for the plot.

Figure 2

Figure 2

Mean-mean plots for MAQC ILM and AFX data without cross-platform normalization. Scatter plots for each MAQC treatment group. Single platform plots were produced from random non-overlapping subsets of seven assays each selected from the MAQC data set for that platform and treatment group. Red lines are the line y = x, provided for visual comparison. Sub-titles indicated the _r_2 value for the plot.

Figure 3

Figure 3

ROC-like curves for the reduced MAQC data set. ROC-like curves for the seven non-discrete cross platform normalization methods are applied to the reduced MAQC data set. Horizontal axes represent false discovery rate (FDR), while vertical axes represent the proportion of genes found to be differentially expressed between treatment groups A and B. Horizontal and vertical lines represent the areas used to compute under-detection (u) and over-detection (o), respectively, although no substantial areas of vertical lines are visible. The "union INT DWD" and "intersection INT DWD" curves represent the intersections (in the sense of gene sets) of the red curve with the yellow and green curves, respectively.

Figure 4

Figure 4

Concordance (_r_2) for normalization methods applied to the MAQC data. Plot titles give the source platforms of the data being normalized. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45]. Subtitles show ranking of the methods. A: resample1, B: resample2, C: dwd, D: eb.par, E: gq, F: mrs, G: qn, H: distran, I: xpn3, J: xpn6, K: xpn9, L: xpn_mod6, M: no.norm. Inequalities in sub-titles are significant at the 0.5/_n_2 level, where n is the number of methods in each sub-figure (including controls), by two-sided Mann-Whitney U-test. Commas indicate the difference in ranking is not significant. Numbers indicate the number of gene clusters used for XPN, e.g. xpn6 means XPN was performed using 6 gene clusters.

Figure 5

Figure 5

Over-detection (o) for normalization methods applied to the MAQC data. Plot titles give the source platforms of the data being normalized. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45]. Subtitles show ranking of the methods. A: resample1, B: resample2, C: dwd, D: eb.par, E: gq, F: mrs, G: qn, H: distran, I: xpn3, J: xpn6, K: xpn9, L: xpn_mod6, M: no.norm. Inequalities in sub-titles are significant at the .05/_n_2 level, where n is the number of methods in each sub-figure (including controls), by two-sided Mann-Whitney U-test. Commas indicate the difference in ranking is not significant. Numbers indicate the number of gene clusters used for XPN, e.g. xpn6 means XPN was performed using 6 gene clusters.

Figure 6

Figure 6

Under-detection (u) for normalization methods applied to the MAQC data. Plot titles give the source platforms of the data being normalized. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45]. Subtitles show ranking of the methods. A: resample1, B: resample2, C: dwd, D: eb.par, E: gq, F: mrs, G: qn, H: distran, I: xpn3, J: xpn6, K: xpn9, L: xpn_mod6, M: no.norm. Inequalities in sub-titles are significant at the .05/_n_2 level, where n is the number of methods in each sub-figure (including controls), by two-sided Mann-Whitney U-test. Commas indicate the difference in ranking is not significant. Numbers indicate the number of gene clusters used for XPN, e.g. xpn6 means XPN was performed using 6 gene clusters.

Figure 7

Figure 7

Normalization methods applied to the human sperm AFX and ILM data. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45]. Subtitles show ranking of the methods. Inequalities in sub-titles are significant at the .05/_n_2 level, where n is the number of methods in each sub-figure (including controls), by two-sided Mann-Whitney U-test. Commas indicate the difference in ranking is not significant. Numbers indicate the number of gene clusters used for XPN, e.g. xpn6 means XPN was performed using 6 gene clusters.

Figure 8

Figure 8

Normalization methods applied to the MAQC data with unequal treatment groups. Labels on the _x_-axis indicate method names and the treatment group sizes. Numbers indicate the sizes of treatment groups A and B, respectively, on the AFX platform and B and A, respectively, on the ILM platform, in terms of number of assays. For example, the label "method.m.n" indicates that the method "method" was applied to a data set containing m group A AFX assays, n group B AFX assays, m group B ILM assays, and n group A ILM assays. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45].

Figure 9

Figure 9

DWD applied to human sperm data with missing treatment groups. One treatment group was removed from each platform's data set before DWD was performed. Subtitles indicate the groups retained. Scrambled: Same as transfer, but parameters were randomly re-ordered before being used for cross-platform normalization. Self.transfer: Full human sperm data set was used for training. Transfer: MAQC data set was used for training. No.transfer: No additional training set was used. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45]. Differences between all pairs are significant at the .05/_n_2 level, where n is the number of methods in each sub-figure (including controls), by two-sided Mann-Whitney U-test.

Figure 10

Figure 10

DWD platform parameter transfer. One treatment group was used as a training set for DWD to transform another treatment group in the MAQC and human sperm data sets. Titles indicate treatment groups used. Scrambled: Same as transfer, but parameters were randomly re-ordered before being used for cross-platform normalization. Self.transfer: Full human sperm data set was used for training. Transfer: Indicated treatment group was used as a training set. No.transfer: No additional training set was used. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45].

Figure 11

Figure 11

Correlations between DWD parameters. DWD parameters were obtained for each treatment group. Bars represent median correlation values between DWD parameters and whiskers represent interquartile ranges. AFX-ILM A-2 and B-2 represent independent resamples of the A and B data. Results represent 100 bootstrap iterations.

Figure 12

Figure 12

P-values for ANOVA. Histogram of _p_-values for treatment-platform interaction terms of model (4).

Similar articles

Cited by

References

    1. Schena M, Shalon D, Davis RW, Brown P. Qantitative monitoring of gene expression patterns with a complementary DNA microarray. Science. 1995;270:467–479. doi: 10.1126/science.270.5235.467. - DOI - PubMed
    1. Velculescu VE, Zhang L, Vogelstein B, Kinzler KW. Serial analysis of gene expression. Science. 1995;270(5235):484–487. doi: 10.1126/science.270.5235.484. - DOI - PubMed
    1. Wang Z, Gerstein M, Snyder M. RNA-Seq: a revolutionary tool for transcriptomics. Nature Reviews Genetics. 2009;10:57–63. doi: 10.1038/nrg2484. - DOI - PMC - PubMed
    1. Hu Z, Fan C, Oh D, Marron J, He X, Qaqish B, Livasy C, Carey L, Reynolds E, Dressler L. The molecular portraits of breast tumors are conserved across microarray platforms. BMC Genomics. 2006;7:96. doi: 10.1186/1471-2164-7-96. - DOI - PMC - PubMed
    1. Jiang H, Deng Y, Chen H, Tao L, Sha Q. Joint analysis of two microarray gene-expression data sets to select lung adenocarcinoma marker genes. BMC Bioinformatics. 2004;5:81. doi: 10.1186/1471-2105-5-81. - DOI - PMC - PubMed

Publication types

MeSH terms

LinkOut - more resources