Cloud-scale differential gene expression for RNA-seq (original) (raw)
Site Map
Related Tools
| Bowtie: Ultrafast short read alignment |
|---|
| Crossbow: Cloud-scale genotyping |
| Contrail: Cloud-based de novo assembly |
| Hadoop: Open Source MapReduce |
| Tophat: RNA-Seq splice junction mapper |
| Cufflinks: Isoform assembly, quantitation |
Reference jars
| H. sapiens GRCh37/Ensembl 67 |
|---|
| s3://genome-idx/myrna/myrna-refs/human_ensembl_67.jar s3://genome-idx/myrna/myrna-refs//human_ensembl_67.cs.jar (color) |
| M. musculus NCBIM37/Ensembl 67 |
| s3://genome-idx/myrna/myrna-refs/mouse_ensembl_67.jar s3://genome-idx/myrna/myrna-refs/mouse_ensembl_67.cs.jar (color) |
| P. troglodytes v2.1/Ensembl 67 |
| s3://genome-idx/myrna/myrna-refs/chimp_ensembl_67.jar s3://genome-idx/myrna/myrna-refs/chimp_ensembl_67.cs.jar (color) |
| M. mulatta v1/Ensembl 67 |
| s3://genome-idx/myrna/myrna-refs/rhesus_ensembl_67.jar s3://genome-idx/myrna/myrna-refs/rhesus_ensembl_67.cs.jar (color) |
| R. norvegicus RGSC3.4/Ensembl 67 |
| s3://genome-idx/myrna/myrna-refs/rat_ensembl_67.jar s3://genome-idx/myrna/myrna-refs/rat_ensembl_67.cs.jar (color) |
| G. gallus WASHUC2/Ensembl 67 |
| s3://genome-idx/myrna/myrna-refs/chicken_ensembl_67.jar s3://genome-idx/myrna/myrna-refs/chicken_ensembl_67.cs.jar (color) |
| C. familiaris BROADD2/Ensembl 67 |
| s3://genome-idx/myrna/myrna-refs/dog_ensembl_67.jar s3://genome-idx/myrna/myrna-refs/dog_ensembl_67.cs.jar (color) |
| D. melanogaster BDGP5.13/Ensembl 67 |
| s3://genome-idx/myrna/myrna-refs/fly_ensembl_67.jar s3://genome-idx/myrna/myrna-refs/fly_ensembl_67.cs.jar (color) |
| C. elegans WS210/Ensembl 67 |
| s3://genome-idx/myrna/myrna-refs/worm_ensembl_67.jar s3://genome-idx/myrna/myrna-refs/worm_ensembl_67.cs.jar (color) |
| S. cerevisiae v1.01/Ensembl 67 |
| s3://genome-idx/myrna/myrna-refs/yeast_ensembl_67.jar s3://genome-idx/myrna/myrna-refs/yeast_ensembl_67.cs.jar (color) |
Related publications
- Langmead B, Hansen K, Leek J. Cloud-scale RNA-sequencing differential expression analysis with Myrna. Genome Biology 11:R83.
- Schatz M, Langmead B, Salzberg SL. Cloud computing and the DNA data race. Nature Biotechnology 2010 Jul;28(7):691-3.
- Langmead B, Schatz M, Lin J, Pop M, Salzberg SL. Searching for SNPs with cloud computing. Genome Biology 10:R134.
- Langmead B, Trapnell C, Pop M, Salzberg SL. Ultrafast and memory-efficient alignment of short DNA sequences to the human genome. Genome Biology 10:R25.
Authors
Links
- Bowtie sourceforge.net project
- SEQanswers
- J.H. Bloomberg School of Public Health
- JHSPH Biostatistics
Myrna on GitHub - 6/11/2013
- Myrna source now lives in a public GitHub repository.
Version 1.2.3 - October 17, 2013
- Fixed performance issue in PostprocessReduce when there are many samples.
- Made the overlap step's default partition length smaller for better load balance.
- Eliminated problematic types of Hadoop counters that could easily contribute to exceeding the maximum # counters allowed.
- Fixed bug whereby defunct matchMatrix function was used in Stats step.
- Updated SRA paths for the examples.
- Added Biostrings R package for automatic installation.
- Fixed a silent detection failure into the examples.
- Fixed a silent count failure.
- Fixed a minor failure that made myrna fail under Hadoop.
Version 1.2.1 - June 24, 2013
- Upgraded to R v3.0.1 and fix a DBI module incompatibility.
- Updated defaults to bowtie v1.0.0.
- Updated the examples to work with the newest public database updates.
- Updated some script to properly use the local R installation if present.
- Fixed fastaq-dump new version incompatibilities.
Version 1.2.0 - July 19, 2012
- Added support for Hadoop version 0.20.205.
- Dropped support for Hadoop versions prior to 0.20.
- Updated default Hadoop version for EMR jobs to 0.20.205
- Updated Bowtie version used to 0.12.8
- Updated R version used to 2.14.2
- Updated jar files to use Ensembl v67 (used to be v61). In the process, fixed an issue whereby $MYRNA_HOME/reftools scripts would die due to unexpected new format of Ensembl database schema.
- Fixed issues with streaming jar version parsing
- Fixed documentation bugs regarding the old --sra-toolkit option, which is superseded by the--fastq-dump option.
- Removed some diagnostic counters because Hadoop began to enforce an upper limit on the number of counters allowed per job. For instance, per-label summary statistics are no longer printed in the Normalize step.
Version 1.1.2 - May 23, 2011
- Updated jar files to use Ensembl v61 (used to be v58).
- Added--from-middle option that causes a read's overlap "influence" to be measured from the middle of the read rather than from one of the ends.
- Added--just-align and--resume-align options.--just-align causes Myrna to put the results of the Alignment phase in the--output directory and quit after the alignment phase. You can later "resume" Myrna by specifying this directory as the--input directory and specifying the--resume-align option.
- Fixed issue with.sra input whereby status output from fastq-dump would be interpreted as a read.
- Other minor bugfixes.
Version 1.1.1 - February 7, 2011
- Added support for the.sra file format, used by theSequence Read Archive. These files can now be specified in the manifest. Crossbow uses the fastq-convert tool from theSRA Toolkit to convert.sra files to FASTQ files in the preprocess stage.
- The examples that included defunct SRA FASTQ files were updated to point to new.sra files instead.
- Fixed labeling scheme in $MYRNA_HOME/example/human/full.manifest to use proper format.
- Removed the --union-constitutive gene footprint which was effectively superseded by the --union-intersection footprint.
- Fixed issues with the Ensembl.pl/Ensembl.R scripts that would cause them to abort when no exons were marked as constitutive.
- Upgraded Bowtie version to 0.12.7.
Version 1.0.9 - September 13, 2010
- Fixed bug in cloud mode that would cause crashing in the Map portion of the Postprocess stage
- Fixed example manifests that point to Short Read Archive files to use .bz2 instead of .gz extensions.
Version 1.0.8 - September 4, 2010
- Set the memory cap on the sort task to be inversely proportional to --cpus, to avoid memory footprint blowup on computers with more processors.
- Fixed a final issue that affected how Myrna handles quality value conversion.
- Fixed issue whereby bzip2'ed data would be handled incorrectly by the preprocessor.
- Fixed counter in Preprocess step that would erroneously refer to unpaired reads as paired. Also "Read data fetched to EC2" has been changed to "Read data fetched".
- In EMR mode, updated where user credentials are found; Amazon changed their path sometime around 8/30/2010.
- In EMR mode, updated the manner in which the bootstrap action is specified; the old way was disabled by Amazon sometime around 8/30/2010.
- Fixed issue whereby ReduceWrap.pl would crash in cases with a large number of bins (>10 million) .
- NOTE: The Short Read Archive (SRA) seems to be in the midst of a reorganization that includes files that were previously gzipped being replaced with versions zipped with bzip2. The files will sometimes disappear for a while. If you are having problems with an example where input reads come from the SRA, try renaming files in the manifest file as appropriate. If that doesn't work, please contact us.
Version 1.0.7 - August 27, 2010
- Fixed issue whereby the order of the arguments to bowtie would result in a crash when POSIXLY_CORRECT was set.
- Fixed --keep-all option, which was causing a crash.
- Fixed a lingering quality bug whereby qualities were converted immediately to phred33 but phred64 or solexa64 flags would be spuriously passed to Bowtie.
Version 1.0.6 - August 26, 2010
- Single-computer mode now copies the output that it writes to the console to a file myrna.local.(pid).out. Please include the contents of this file when reporting issues.
- Sorting in single-computer mode is now more portable; switched from command-line sort to pure-Perl File::Sort.
- More helpful output from MapWrap.pl and ReduceWrap.pl to make it easier to debug issues in single-computer-mode runs.
- Fixed issue where web form would incorrectly convert + signs in AWS secret key to spaces, causing some good credentials to fail verification.
- Fixed issue in preprocessor that mishandles copies when user's AWS secret key contains slash characters.