Empirical comparison of cross-platform normalization methods for gene expression data - PubMed (original) (raw)
Comparative Study
Empirical comparison of cross-platform normalization methods for gene expression data
Jason Rudy et al. BMC Bioinformatics. 2011.
Abstract
Background: Simultaneous measurement of gene expression on a genomic scale can be accomplished using microarray technology or by sequencing based methods. Researchers who perform high throughput gene expression assays often deposit their data in public databases, but heterogeneity of measurement platforms leads to challenges for the combination and comparison of data sets. Researchers wishing to perform cross platform normalization face two major obstacles. First, a choice must be made about which method or methods to employ. Nine are currently available, and no rigorous comparison exists. Second, software for the selected method must be obtained and incorporated into a data analysis workflow.
Results: Using two publicly available cross-platform testing data sets, cross-platform normalization methods are compared based on inter-platform concordance and on the consistency of gene lists obtained with transformed data. Scatter and ROC-like plots are produced and new statistics based on those plots are introduced to measure the effectiveness of each method. Bootstrapping is employed to obtain distributions for those statistics. The consistency of platform effects across studies is explored theoretically and with respect to the testing data sets.
Conclusions: Our comparisons indicate that four methods, DWD, EB, GQ, and XPN, are generally effective, while the remaining methods do not adequately correct for platform effects. Of the four successful methods, XPN generally shows the highest inter-platform concordance when treatment groups are equally sized, while DWD is most robust to differently sized treatment groups and consistently shows the smallest loss in gene detection. We provide an R package, CONOR, capable of performing the nine cross-platform normalization methods considered. The package can be downloaded at http://alborz.sdsu.edu/conor and is available from CRAN.
Figures
Figure 1
Mean-mean plots for MAQC group A ILM and AFX data after cross-platform normalization. Scatter or sunflower plots for MAQC treatment group A for each normalization method for the ILM and AFX data. Mean expression level of sample A assays on the AFX platform is plotted against mean expression level on the ILM platform. Plot titles indicate the cross-platform normalization method performed. Red lines are the line y = x, provided for visual comparison. Sub-titles indicate the _r_2 value for the plot.
Figure 2
Mean-mean plots for MAQC ILM and AFX data without cross-platform normalization. Scatter plots for each MAQC treatment group. Single platform plots were produced from random non-overlapping subsets of seven assays each selected from the MAQC data set for that platform and treatment group. Red lines are the line y = x, provided for visual comparison. Sub-titles indicated the _r_2 value for the plot.
Figure 3
ROC-like curves for the reduced MAQC data set. ROC-like curves for the seven non-discrete cross platform normalization methods are applied to the reduced MAQC data set. Horizontal axes represent false discovery rate (FDR), while vertical axes represent the proportion of genes found to be differentially expressed between treatment groups A and B. Horizontal and vertical lines represent the areas used to compute under-detection (u) and over-detection (o), respectively, although no substantial areas of vertical lines are visible. The "union INT DWD" and "intersection INT DWD" curves represent the intersections (in the sense of gene sets) of the red curve with the yellow and green curves, respectively.
Figure 4
Concordance (_r_2) for normalization methods applied to the MAQC data. Plot titles give the source platforms of the data being normalized. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45]. Subtitles show ranking of the methods. A: resample1, B: resample2, C: dwd, D: eb.par, E: gq, F: mrs, G: qn, H: distran, I: xpn3, J: xpn6, K: xpn9, L: xpn_mod6, M: no.norm. Inequalities in sub-titles are significant at the 0.5/_n_2 level, where n is the number of methods in each sub-figure (including controls), by two-sided Mann-Whitney U-test. Commas indicate the difference in ranking is not significant. Numbers indicate the number of gene clusters used for XPN, e.g. xpn6 means XPN was performed using 6 gene clusters.
Figure 5
Over-detection (o) for normalization methods applied to the MAQC data. Plot titles give the source platforms of the data being normalized. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45]. Subtitles show ranking of the methods. A: resample1, B: resample2, C: dwd, D: eb.par, E: gq, F: mrs, G: qn, H: distran, I: xpn3, J: xpn6, K: xpn9, L: xpn_mod6, M: no.norm. Inequalities in sub-titles are significant at the .05/_n_2 level, where n is the number of methods in each sub-figure (including controls), by two-sided Mann-Whitney U-test. Commas indicate the difference in ranking is not significant. Numbers indicate the number of gene clusters used for XPN, e.g. xpn6 means XPN was performed using 6 gene clusters.
Figure 6
Under-detection (u) for normalization methods applied to the MAQC data. Plot titles give the source platforms of the data being normalized. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45]. Subtitles show ranking of the methods. A: resample1, B: resample2, C: dwd, D: eb.par, E: gq, F: mrs, G: qn, H: distran, I: xpn3, J: xpn6, K: xpn9, L: xpn_mod6, M: no.norm. Inequalities in sub-titles are significant at the .05/_n_2 level, where n is the number of methods in each sub-figure (including controls), by two-sided Mann-Whitney U-test. Commas indicate the difference in ranking is not significant. Numbers indicate the number of gene clusters used for XPN, e.g. xpn6 means XPN was performed using 6 gene clusters.
Figure 7
Normalization methods applied to the human sperm AFX and ILM data. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45]. Subtitles show ranking of the methods. Inequalities in sub-titles are significant at the .05/_n_2 level, where n is the number of methods in each sub-figure (including controls), by two-sided Mann-Whitney U-test. Commas indicate the difference in ranking is not significant. Numbers indicate the number of gene clusters used for XPN, e.g. xpn6 means XPN was performed using 6 gene clusters.
Figure 8
Normalization methods applied to the MAQC data with unequal treatment groups. Labels on the _x_-axis indicate method names and the treatment group sizes. Numbers indicate the sizes of treatment groups A and B, respectively, on the AFX platform and B and A, respectively, on the ILM platform, in terms of number of assays. For example, the label "method.m.n" indicates that the method "method" was applied to a data set containing m group A AFX assays, n group B AFX assays, m group B ILM assays, and n group A ILM assays. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45].
Figure 9
DWD applied to human sperm data with missing treatment groups. One treatment group was removed from each platform's data set before DWD was performed. Subtitles indicate the groups retained. Scrambled: Same as transfer, but parameters were randomly re-ordered before being used for cross-platform normalization. Self.transfer: Full human sperm data set was used for training. Transfer: MAQC data set was used for training. No.transfer: No additional training set was used. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45]. Differences between all pairs are significant at the .05/_n_2 level, where n is the number of methods in each sub-figure (including controls), by two-sided Mann-Whitney U-test.
Figure 10
DWD platform parameter transfer. One treatment group was used as a training set for DWD to transform another treatment group in the MAQC and human sperm data sets. Titles indicate treatment groups used. Scrambled: Same as transfer, but parameters were randomly re-ordered before being used for cross-platform normalization. Self.transfer: Full human sperm data set was used for training. Transfer: Indicated treatment group was used as a training set. No.transfer: No additional training set was used. Boxes show the interquartile range and whiskers extend to an additional 1.5 times the interquartile range. Values outside the whiskers are plotted as circles. Notches are drawn such that non-overlapping notches are strong evidence of differing medians [61,45].
Figure 11
Correlations between DWD parameters. DWD parameters were obtained for each treatment group. Bars represent median correlation values between DWD parameters and whiskers represent interquartile ranges. AFX-ILM A-2 and B-2 represent independent resamples of the A and B data. Results represent 100 bootstrap iterations.
Figure 12
P-values for ANOVA. Histogram of _p_-values for treatment-platform interaction terms of model (4).
Similar articles
- Transcriptomic Harmonization as the Way for Suppressing Cross-Platform Bias and Batch Effect.
Borisov N, Buzdin A. Borisov N, et al. Biomedicines. 2022 Sep 18;10(9):2318. doi: 10.3390/biomedicines10092318. Biomedicines. 2022. PMID: 36140419 Free PMC article. Review. - Merging two gene-expression studies via cross-platform normalization.
Shabalin AA, Tjelmeland H, Fan C, Perou CM, Nobel AB. Shabalin AA, et al. Bioinformatics. 2008 May 1;24(9):1154-60. doi: 10.1093/bioinformatics/btn083. Epub 2008 Mar 5. Bioinformatics. 2008. PMID: 18325927 - Cross-species and cross-platform gene expression studies with the Bioconductor-compliant R package 'annotationTools'.
Kuhn A, Luthi-Carter R, Delorenzi M. Kuhn A, et al. BMC Bioinformatics. 2008 Jan 17;9:26. doi: 10.1186/1471-2105-9-26. BMC Bioinformatics. 2008. PMID: 18201381 Free PMC article. - Novel software package for cross-platform transcriptome analysis (CPTRA).
Zhou X, Su Z, Sammons RD, Peng Y, Tranel PJ, Stewart CN Jr, Yuan JS. Zhou X, et al. BMC Bioinformatics. 2009 Oct 8;10 Suppl 11(Suppl 11):S16. doi: 10.1186/1471-2105-10-S11-S16. BMC Bioinformatics. 2009. PMID: 19811681 Free PMC article. - AnyExpress: integrated toolkit for analysis of cross-platform gene expression data using a fast interval matching algorithm.
Kim J, Patel K, Jung H, Kuo WP, Ohno-Machado L. Kim J, et al. BMC Bioinformatics. 2011 Mar 17;12:75. doi: 10.1186/1471-2105-12-75. BMC Bioinformatics. 2011. PMID: 21410990 Free PMC article.
Cited by
- Comparison and development of cross-study normalization methods for inter-species transcriptional analysis.
Feldman S, Ner-Gaon H, Treister E, Shay T. Feldman S, et al. PLoS One. 2024 Sep 10;19(9):e0307997. doi: 10.1371/journal.pone.0307997. eCollection 2024. PLoS One. 2024. PMID: 39255285 Free PMC article. - Uniformly shaped harmonization combines human transcriptomic data from different platforms while retaining their biological properties and differential gene expression patterns.
Borisov N, Tkachev V, Simonov A, Sorokin M, Kim E, Kuzmin D, Karademir-Yilmaz B, Buzdin A. Borisov N, et al. Front Mol Biosci. 2023 Sep 6;10:1237129. doi: 10.3389/fmolb.2023.1237129. eCollection 2023. Front Mol Biosci. 2023. PMID: 37745690 Free PMC article. - Assessing equivalent and inverse change in genes between diverse experiments.
Neums L, Koestler DC, Xia Q, Hu J, Patel S, Bell-Glenn S, Pei D, Zhang B, Boyd S, Chalise P, Thompson JA. Neums L, et al. Front Bioinform. 2022 Sep 21;2:893032. doi: 10.3389/fbinf.2022.893032. eCollection 2022. Front Bioinform. 2022. PMID: 36304274 Free PMC article. - Transcriptomic Harmonization as the Way for Suppressing Cross-Platform Bias and Batch Effect.
Borisov N, Buzdin A. Borisov N, et al. Biomedicines. 2022 Sep 18;10(9):2318. doi: 10.3390/biomedicines10092318. Biomedicines. 2022. PMID: 36140419 Free PMC article. Review. - Microarray Data Preprocessing: From Experimental Design to Differential Analysis.
Federico A, Saarimäki LA, Serra A, Del Giudice G, Kinaret PAS, Scala G, Greco D. Federico A, et al. Methods Mol Biol. 2022;2401:79-100. doi: 10.1007/978-1-0716-1839-4_7. Methods Mol Biol. 2022. PMID: 34902124
References
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources