The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements (original) (raw)

Accession codes

Accessions

Gene Expression Omnibus

References

  1. Lesko, L.J. & Woodcock, J. Translation of pharmacogenomics and pharmacogenetics: a regulatory perspective. Nat. Rev. Drug Discov. 3, 763–769 (2004).
    Article CAS Google Scholar
  2. Frueh, F.W. Impact of microarray data quality on genomic data submissions to the FDA. Nat. Biotechnol. 24, 1105–1107 (2006).
    Article CAS Google Scholar
  3. Dix, D.J. et al. A framework for the use of genomics data at the EPA. Nat. Biotechnol. 24, 1108–1111 (2006).
    Article CAS Google Scholar
  4. Tan, P.K. et al. Evaluation of gene expression measurements from commercial microarray platforms. Nucleic Acids Res. 31, 5676–5684 (2003).
    Article CAS Google Scholar
  5. Ramalho-Santos, M., Yoon, S., Matsuzaki, Y., Mulligan, R.C. & Melton, D.A. “Stemness”: transcriptional profiling of embryonic and adult stem cells. Science 298, 597–600 (2002).
    Article CAS Google Scholar
  6. Ivanova, N.B. et al. A stem cell molecular signature. Science 298, 601–604 (2002).
    Article CAS Google Scholar
  7. Miller, R.M. et al. Dysregulation of gene expression in the 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine-lesioned mouse substantia nigra. J. Neurosci. 24, 7445–7454 (2004).
    Article CAS Google Scholar
  8. Fortunel, N.O. et al. Comment on “'Stemness': transcriptional profiling of embryonic and adult stem cells” and “a stem cell molecular signature”. Science 302, 393 author reply 393 (2003).
  9. Miklos, G.L. & Maleszka, R. Microarray reality checks in the context of a complex disease. Nat. Biotechnol. 22, 615–621 (2004).
    Article CAS Google Scholar
  10. Frantz, S. An array of problems. Nat. Rev. Drug Discov. 4, 362–363 (2005).
    Article CAS Google Scholar
  11. Marshall, E. Getting the noise out of gene arrays. Science 306, 630–631 (2004).
    Article CAS Google Scholar
  12. Michiels, S., Koscielny, S. & Hill, C. Prediction of cancer outcome with microarrays: a multiple random validation strategy. Lancet 365, 488–492 (2005).
    Article CAS Google Scholar
  13. Ein-Dor, L., Zuk, O. & Domany, E. Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer. Proc. Natl. Acad. Sci. USA 103, 5923–5928 (2006).
    Article CAS Google Scholar
  14. Petersen, D. et al. Three microarray platforms: an analysis of their concordance in profiling gene expression. BMC Genomics 6, 63 (2005).
    Article Google Scholar
  15. Dobbin, K.K. et al. Interlaboratory comparability study of cancer gene expression analysis using oligonucleotide microarrays. Clin. Cancer Res. 11, 565–572 (2005).
    CAS PubMed Google Scholar
  16. Irizarry, R.A. et al. Multiple-laboratory comparison of microarray platforms. Nat. Methods 2, 345–350 (2005).
    Article CAS Google Scholar
  17. Larkin, J.E., Frank, B.C., Gavras, H., Sultana, R. & Quackenbush, J. Independence and reproducibility across microarray platforms. Nat. Methods 2, 337–344 (2005).
    Article CAS Google Scholar
  18. Kuo, W.P. et al. A sequence-oriented comparison of gene expression measurements across different hybridization-based technologies. Nat. Biotechnol. 24, 832–840 (2006).
    Article CAS Google Scholar
  19. Shi, L. et al. QA/QC: challenges and pitfalls facing the microarray community and regulatory agencies. Expert Rev. Mol. Diagn. 4, 761–777 (2004).
    Article Google Scholar
  20. Shi, L. et al. Cross-platform comparability of microarray technology: intra-platform consistency and appropriate data analysis procedures are essential. BMC Bioinformatics 6 Suppl. 2, S12 (2005).
    Article Google Scholar
  21. Ji, H. & Davis, R.W. Data quality in genomics and microarrays. Nat. Biotechnol. 24, 1112–1113 (2006).
    Article CAS Google Scholar
  22. Canales, R.D. et al. Evaluation of DNA microarray results with quantitative gene expression platforms. Nat. Biotechnol. 24, 1115–1122 (2006).
    Article CAS Google Scholar
  23. Shippy, R. et al. Using RNA sample titrations to assess microarray platform performance and normalization techniques. Nat. Biotechnol. 24, 1123–1131 (2006).
    Article CAS Google Scholar
  24. Patterson, T.A. et al. Performance comparison of one-color and two-color platforms within the MicroArray Quality Control (MAQC) project. Nat. Biotechnol. 24, 1140–1150 (2006).
    Article CAS Google Scholar
  25. Tong, W. et al. Evaluation of external RNA controls for the assessment of microarray performance. Nat. Biotechnol. 24, 1132–1139 (2006).
    Article CAS Google Scholar
  26. Guo, L. et al. Rat toxicogenomic study reveals analytical consistency across microarray platforms. Nat. Biotechnol. 24, 1162–1169 (2006).
    Article CAS Google Scholar
  27. Mecham, B.H. et al. Sequence-matched probes produce increased cross-platform consistency and more reproducible biological results in microarray-based gene expression measurements. Nucleic Acids Res. 32, e74 (2004).
    Article Google Scholar
  28. Carter, S.L., Eklund, A.C., Mecham, B.H., Kohane, I.S. & Szallasi, Z. Redefinition of Affymetrix probe sets by sequence overlap with cDNA microarray probes reduces cross-platform inconsistencies in cancer-associated gene expression measurements. BMC Bioinformatics 6, 107 (2005).
    Article Google Scholar
  29. Draghici, S., Khatri, P., Eklund, A.C. & Szallasi, Z. Reliability and reproducibility issues in DNA microarray measurements. Trends Genet. 22, 101–109 (2006).
    Article CAS Google Scholar
  30. Irizarry, R.A., Wu, Z. & Jaffee, H.A. Comparison of Affymetrix GeneChip expression measures. Bioinformatics 22, 789–794 (2006).
    Article CAS Google Scholar
  31. Pruitt, K.D., Tatusova, T. & Maglott, D.R. NCBI Reference Sequence (RefSeq): a curated non-redundant sequence database of genomes, transcripts and proteins. Nucleic Acids Res. 33, D501–D504 (2005).
    Article CAS Google Scholar
  32. Thierry-Mieg, D. & J, T.-M. AceView: a comprehensive cDNA-supported gene and transcripts annotation. Genome Biology 7, Suppl. 1, S12 (2006).
    Article Google Scholar
  33. Bammler, T. et al. Standardizing global gene expression analysis between laboratories and across platforms. Nat. Methods 2, 351–356 (2005).
    Article Google Scholar
  34. Harr, B. & Schlotterer, C. Comparison of algorithms for the analysis of Affymetrix microarray data as evaluated by co-expression of genes in known operons. Nucleic Acids Res. 34, e8 (2006).
    Article Google Scholar
  35. Tusher, V.G., Tibshirani, R. & Chu, G. Significance analysis of microarrays applied to the ionizing radiation response. Proc. Natl. Acad. Sci. USA 98, 5116–5121 (2001).
    Article CAS Google Scholar
  36. Thompson, K.L. et al. Use of a mixed tissue RNA design for performance assessments on multiple microarray formats. Nucleic Acids Res. 33, e187 (2005).
    Article Google Scholar
  37. Baker, S.C. et al. The External RNA Controls Consortium: a progress report. Nat. Methods 2, 731–734 (2005).
    Article CAS Google Scholar
  38. Reid, L.H. The value of a proficiency testing program to monitor performance in microarray laboratories. Pharm. Discov. 5, 20–25 (2005).
    Google Scholar
  39. Ball, C.A. et al. Standards for microarray data. Science 298, 539 (2002).
    Article CAS Google Scholar
  40. Tong, W. et al. ArrayTrack–supporting toxicogenomic research at the U.S. Food and Drug Administration National Center for Toxicological Research. Environ. Health Perspect. 111, 1819–1826 (2003).
    Article CAS Google Scholar
  41. Tong, W. et al. Development of public toxicogenomics software for microarray data management and analysis. Mutat. Res. 549, 241–253 (2004).
    Article CAS Google Scholar

Download references

Acknowledgements

All MAQC participants freely donated their time and reagents for the completion and analysis of the MAQC project. Participants from the National Institutes of Health (NIH) were supported by the Intramural Research Program of NIH, Bethesda, Maryland. D.H. thanks Ian Korf for BLAST discussions. This study utilized a number of computing resources, including the high-performance computational capabilities of the Biowulf PC/Linux cluster at the NIH (http://biowulf.nih.gov/) as well as resources at the analysis sites.

Author information

Authors and Affiliations

  1. National Center for Toxicological Research, US Food and Drug Administration, 3900 NCTR Road, Jefferson, 72079, Arkansas, USA
    Leming Shi, Leming Shi, Weida Tong, Yvonne P Dragan, James J Chen, Xiao-hui Fan, James C Fuscoe, Weigong Ge, Lei Guo, Tao Han, Stephen C Harris, Nan Mei, Baitang Ning, Tucker A Patterson, Zhenqiang Su & William Slikker Jr
  2. Expression Analysis, Inc., 2605 Meridian Parkway, Durham, 27713, North Carolina, USA
    Laura H Reid & Wendell D Jones
  3. GE Healthcare, 7700 S. River Parkway, Suite 2603, Tempe, 85284, AZ, USA
    Richard Shippy & Alan Brunner
  4. Affymetrix, Inc., 3420 Central Expressway, Santa Clara, 95051, California, USA
    Janet A Warrington, Christopher Davies, Xu Guo, Chunmei Liu, Tim McDaniel & Yaron Turpaz
  5. Illumina,Inc. 9885 Towne Centre Drive, San Diego, 92121, California, USA
    Shawn C Baker & Eugene Chudin
  6. Agilent Technologies, Inc., 5301 Stevens Creek Blvd., Santa Clara, 95051, California, USA
    Patrick J Collins, Paul K Wolber, Anne Bergstrom Lucas, John Corson, Glenda Delenstarr, Stephanie Fulmer-Smentek, Svetlana Shchegrova & Alex Wong
  7. Eppendorf Array Technologies, rue du Séminaire 20a, Namur, 5000, Belgium
    Francoise de Longueville, Vincent Bertholet & Christophe Van
  8. NCI Advanced Technology Center, 8717 Grovemont Circle, Bethesda, 20892, Maryland, USA
    Ernest S Kawasaki & Yonghong Wang
  9. Applied Biosystems, 850 Lincoln Centre Drive, Foster City, 94404, California, USA
    Kathleen Y Lee, Yongming Andrew Sun, Lu Zhang, Catalin C Barbacioru, Roger Canales, Raymond R Samaha & Chunlin Xiao
  10. Panomics, Inc., 6519 Dumbarton Circle, Fremont, 94555, California, USA
    Yuling Luo, Yunqing Ma, Botoul Maqsodi & Wen Yang
  11. Medical University of Ohio, 3000 Arlington Avenue, Toledo, 43614, Ohio, USA
    James C Willey
  12. Ambion, An Applied Biosystems Business, 2130 Woodward Street, Austin, 78744, Texas, USA
    Robert A Setterquist, Mike Wilson & David Dorris
  13. Stratagene Corp., 11011 North Torrey Pines Road, La Jolla, 92130, California, USA
    Gavin M Fischer & Natalia Novoradovskaya
  14. Office of Research and Development, US Environmental Protection Agency, 109 TW Alexander Drive, Research Triangle Park, 27711, North Carolina, USA
    David J Dix, J Christopher Corton, Susan Hester, Hongzu Ren, Mitch Rosen & Beena Vallanat
  15. Center for Drug Evaluation and Research, US Food and Drug Administration, 10903 New Hampshire Avenue, Silver Spring, 20993, Maryland, USA
    Felix W Frueh, Federico M Goodsaid, Shashi Amur, Michael S Orr, P Scott Pine, Barry A Rosenzweig, Karol L Thompson & Sue Jane Wang
  16. National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, 8600 Rockville Pike, Bethesda, 20894, Maryland, USA
    Damir Herman, Jean Thierry-Mieg & Danielle Thierry-Mieg
  17. University of Massachusetts-Boston, 100 Morrissey Boulevard, Boston, 02125, Massachusetts, USA
    Roderick V Jensen, Aron C Eklund, Michael J Lombardi & Adam Papallo
  18. Asuragen, Inc., 2150 Woodward, Austin, 78744, Texas, USA
    Charles D Johnson, Mike Wilson & Timothy S Davison
  19. A Division of Clinical Data, Cogenics™, Inc., 100 Perimeter Park Drive, Suite C, Morrisville, 27560, North Carolina, USA
    Edward K Lobenhofer, Patrick Hurban & Kenneth L Philips
  20. Center for Biologics Evaluation and Research, US Food and Drug Administration, 29 Lincoln Drive, Bethesda, 20892, Maryland, USA
    Raj K Puri & Jing Han
  21. Center for Devices and Radiological Health, US Food and Drug Administration, 2098 Gaither Road, Rockville, 20850, Maryland, USA
    Uwe Scherf & Zivana Tezak
  22. UCLA David Geffen School of Medicine, Transcriptional Genomics Core, Cedars-Sinai Medical Center, 8700 Beverly Boulevard, Los Angeles, 90048, California, USA
    Charles Wang, Xutao Deng & Jun Xu
  23. Solexa, Inc., 25861 Industrial Boulevard, Hayward, 94545, California, USA
    Lu Zhang & Gary P Schroth
  24. SAS Institute, Inc., 100 SAS Campus Drive, Cary, 27513, North Carolina, USA
    Wenjun Bao, Tzu-Ming Chu & Russ Wolfinger
  25. Vialogy Corp., 2400 Lincoln Avenue, Altadena, 91001, California, USA
    Cecilie Boysen & Bud Bromley
  26. Operon Biotechnologies, 2211 Seminole Drive, Huntsville, 35805, Alabama, USA
    Donna Brown
  27. Z-Tech Corp., 3900 NCTR Road, Jefferson, 72079, Arkansas, USA
    Xiaoxi Megan Cao, Hong Fang, Huixiao Hong, Roger G Perkins, Feng Qian, Hongmei Sun, Jie Wu & Qian Xie
  28. Center for Food Safety and Applied Nutrition, US Food and Drug Administration, 8401 Muirkirk Road, Laurel, 20708, Maryland, USA
    Thomas A Cebula, Scott A Jackson & J Eugene LeClerc
  29. CapitalBio Corp., 18 Life Science Parkway, Changping District, Beijing, 102206, China
    Jing Cheng & Liang Zhang
  30. Biogen Idec, 5200 Research Place, San Diego, 92122, California, USA
    Lisa J Croner
  31. US Environmental Protection Agency, Office of the Science Advisor, 1200 Pennsylvania Avenue, NW, DC 20460, Washington, USA
    Kathryn Gallagher
  32. Yale University, W.M. Keck Biotechnology Resource Laboratory, Microarray Resource, 300 George Street, New Haven, 06511, Connecticut, USA
    Janet Hager & Irina Tikhonova
  33. TeleChem ArrayIt, 524 E. Weddell Drive, Sunnyvale, 94089, California, USA
    Paul K Haje & Mark Schena
  34. Center for Veterinary Medicine, US Food and Drug Administration, 8401 Muirkirk Road, Laurel, 20708, Maryland, USA
    Heather C Harbottle
  35. Cold Spring Harbor Laboratory, 500 Sunnyside Boulevard, Woodbury, 11797, New York, USA
    Eli Hatchwell
  36. Burnham Institute, 10901 North Torrey Pines Road, La Jolla, 92037, California, USA
    Craig A Hauser
  37. Stanford University School of Medicine, 318 Campus Drive, Stanford, 94305, California, USA
    Hanlee Ji
  38. Gene Express, Inc., 975 Research Drive, Toledo, 43614, Ohio, USA
    Charles R Knight, Terry W Osborn & Elizabeth H Peters
  39. Department of Developmental Biology, Harvard School of Dental Medicine, 188 Longwood Avenue, Boston, 02115, Massachusetts, USA
    Winston P Kuo
  40. Vanderbilt University, 465 21st Avenue South, Nashville, 37232, Tennessee, USA
    Shawn Levy
  41. University Texas Southwestern Medical Center, 6000 Harry Hines Boulevard/ND6.504, Dallas, 75390, Texas, USA
    Quan-Zhen Li
  42. Department of Computer Science, University of Texas at Dallas, MS EC31 Richardson, 75083, Texas, USA
    Ying Liu
  43. GenUs BioSystems, Inc., 1808 Janke Drive Unit M, Northbrook, 60062, Illinois, USA
    Scott R Magnuson
  44. Norwegian Microarray Consortium, Rikshospitalet - Radiumhospitalet Health Centre, Montebello, Oslo, N0310, Norway
    Ola Myklebost
  45. Novartis, 250 Massachusetts Avenue, Cambridge, 02139, Massachusetts, USA
    Ron Peterson & Frank Staedtler
  46. Breast Medical Oncology Department-Unit 1354, MD Anderson Cancer Center, 1155 Pressler Street, Houston, 77230, Texas, USA
    Lajos Pusztai
  47. Luminex Corp., 12212 Technology Boulevard, 78727, Austin, Texas, USA
    Dave D Smith
  48. Children's Hospital Informatics Program at the Harvard-MIT Division of Health Sciences and Technology (CHIP@HST), Harvard Medical School, Boston, 02115, Massachusetts, USA
    Zoltan Szallasi
  49. Department of Physiology and Pharmacology, Wake Forest University School of Medicine, Medical Center Boulevard, Winston-Salem, 27157, North Carolina, USA
    Stephen J Walker
  50. Department of Bioengineering, University of Illinois at Urbana-Champaign, 1304 W. Springfield Avenue, Urbana, 61801, Illinois, USA
    Sheng Zhong
  51. Full Moon Biosystems, Inc., 754 N. Pastoria Avenue, Sunnyvale, 94085, California, USA
    Yaping Zong

Consortia

MAQC Consortium

Corresponding author

Correspondence toLeming Shi.

Ethics declarations

Competing interests

Many of the MAQC participants are employed by companies that manufacture gene expression products and/or perform testing services. J.C.W. is a consultant for and has significant financial interest in Gene Express, Inc.

Supplementary information

Rights and permissions

About this article

Cite this article

MAQC Consortium. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements.Nat Biotechnol 24, 1151–1161 (2006). https://doi.org/10.1038/nbt1239

Download citation