11/8/2017 Trinity De novo Transcriptome Assembly Workshop trinityrnaseq/rnaseq_trinity_tuxedo_workshop Wiki GitHub

Transcription

1 trinityrnaseq / RNASeq_Trinity_Tuxedo_Workshop Trinity De novo Transcriptome Assembly Workshop Brian Haas edited this page on Oct 17, revisions De novo RNA-Seq Assembly and Analysis Using Trinity and EdgeR The following details the steps involved in: Generating a Trinity de novo RNA Seq assembly Inspecting the assembly in the context of a reference genome (when one is available) Pages 4 Home Import and run workshop VM Trinity De novo Transcriptome Assembly Workshop Tuxedo Genome Guided Transcriptome Assembly Workshop Mapping reads and Trinity transcripts to a target genome sequence (when one is available) Clone this wiki locally Clone in Desktop Visualizing the aligned reads and transcripts in comparison to reference transcript annotations Identifying differentially expressed transcripts using EdgeR and various Trinity included helper utilities All required software and data are provided pre installed on a VirtualBox image Be sure to run the workshop VM and open a terminal After installing the VM, be sure to quickly update the contents of the rnaseq_workshop_data directory by: % cd RNASeq_Trinity_Tuxedo_Workshop % git pull This way, you ll have the latest content, including any recent bugfixes Data Content: This demo uses RNA Seq data corresponding to Schizosaccharomyces pombe fission yeast, involving paired end 76 base strand specific RNA Seq reads corresponding to four samples: Sp_log logarithmic growth, Sp_plat plateau phase, Sp_hs heat shock, and Sp_ds diauxic shift There are 'leftfq' and 'rightfq' FASTQ formatted Illlumina read files for each of the four samples All RNA Seq data sets can be found in the RNASEQ_data/ subdirectory Also included is a 'genomefa' file corresponding to a genome sequence, and annotations for reference genes 'genesbed' or 'genesgff3' These resources can be found in the GENOME_data/ subdirectory Note, although the genes, annotations, and reads represent genuine sequence data, they were artificially selected and organized for use in this tutorial, so as to provide varied levels of expression in a very small data set, which could be processed and analyzed within an approximately one hour time session and with minimal computing resources Automated Interactive or Manual Execution of Activities - You Decide! De novo Transcriptome Assembly Workshop 1/10

2 The commands to be executed are indicated below after a command prompt '%' You can type these commands into the terminal, or you can simply copy/paste these commands into the terminal to execute the operations To avoid having to copy/paste the numerous commands shown below into a unix terminal, the VM includes a script runtrinitydemopl that enables you to run each of the steps interactively To begin, simply run: % /runtrinitydemopl Note, by default and for convenience, the demo will show you the commands that are to be executed This way, you don t need to type them in yourself The protocol followed is that described here: Below, we refer to ${TRINITY_HOME}/ as the directory where the Trinity software is installed, and '%' as the command prompt De novo assembly of reads using Trinity To generate a reference assembly that we can later use for analyzing differential expression, we'll combine the read data sets for the different conditions together into a single target for Trinity assembly We do this by providing Trinity with a list of all the left and right fastq files to the left and right parameters as comma delimited like so: % ${TRINITY_HOME}/Trinity seqtype fq SS_lib_type RF \ left RNASEQ_data/Sp_logleftfqgz,RNASEQ_data/Sp_hsleftfqgz,RNASEQ_data/Sp_dsleftfqgz,RNAS \ right RNASEQ_data/Sp_logrightfqgz,RNASEQ_data/Sp_hsrightfqgz,RNASEQ_data/Sp_dsrightfqgz,R \ CPU 2 max_memory 1G Running Trinity on this data set may take 10 to 15 minutes You ll see it progress through the various stages, starting with Jellyfish to generate the k mer catalog, then followed by Inchworm, Chrysalis, and finally Butterfly Running a typical Trinity job requires ~1 hour and ~1G RAM per ~1 million PE reads You'd normally run it on a high memory machine and let it churn for hours or days The assembled transcripts will be found at trinity_out_dir/trinityfasta De novo Transcriptome Assembly Workshop 2/10

3 Just to look at the top few lines of the assembled transcript fasta file, you can run: % head trinity_out_dir/trinityfasta and you can see the Fasta formatted Trinity output: >TRINITY_DN0_c0_g1_i1 len=1894 path=[1872:0 1893] [ 1, 1872, 2] GTATACTGAGGTTTATTGCCTGTAACGGGCAAACTCGAGCAGTTCAATCACGAGGAGATT ACCAGAAAACACTCGCTATTGCTTTGAAAAAGTTTAGCCTTGAAGATGCTTCAAAATTCA TTGTATGCGTTTCACAGAGTAGTCGAATTAAGCTGATTACTGAAGAAGAATTTAAACAAA TTTGTTTTAATTCATCTTCACCGGAACGCGACAGGTTAATTATTGTGCCAAAAGAAAAGC CTTGTCCATCGTTTGAAGACCTCCGCCGTTCTTGGGAGATTGAATTGGCTCAACCGGCAG CATTATCATCACAGTCCTCCCTTTCTCCTAAACTTTCCTCTGTTCTTCCCACGAGCACTC AGAAACGAAGTGTCCGCTCAAATAATGCGAAACCATTTGAATCCTACCAGCGACCTCCTA GCGAGCTTATTAATTCTAGAATTTCCGATTTTTTCCCCGATCATCAACCAAAGCTACTGG AAAAAACAATATCAAACTCTCTTCGTAGAAACCTTAGCATACGTACGTCTCAAGGGCACA Examine assembly stats Capture some basic statistics about the Trinity assembly: % $TRINITY_HOME/util/TrinityStatspl trinity_out_dir/trinityfasta which should generate data like so Note your numbers may vary slightly, as the assembly results are not deterministic ################################ ## Counts of transcripts, etc ################################ Total trinity 'genes': 377 Total trinity transcripts: 384 Percent GC: 3866 ######################################## Stats based on ALL transcript contigs: ######################################## Contig N10: 3373 Contig N20: 2605 Contig N30: 2219 Contig N40: 1936 Contig N50: 1703 Median contig length: 772 Average contig: Total assembled bases: ##################################################### ## Stats based on ONLY LONGEST ISOFORM per 'GENE': ##################################################### Contig N10: 3373 Contig N20: 2605 Contig N30: 2216 Contig N40: 1936 Contig N50: 1695 Median contig length: 772 Average contig: Total assembled bases: Compare de novo reconstructed transcripts to reference annotations De novo Transcriptome Assembly Workshop 3/10

4 Since we happen to have a reference genome and a set of reference transcript annotations that correspond to this data set, we can align the Trinity contigs to the genome and examine them in the genomic context a Align the transcripts to the genome using GMAP First, prepare the genomic region for alignment by GMAP like so: % gmap_build d genome D k 13 GENOME_data/genomefa Now, align the Trinity transcript contigs to the genome, outputting in SAM format, which will simplify viewing of the data in our genome browser % gmap n 0 D d genome trinity_out_dir/trinityfasta f samse > trinity_gmapsam Note, you ll likely encounter warning messages such as No paths found for comp42_c0_seq1, which just means that GMAP wasn t able to find a high scoring alignment of that transcript to the targeted genome sequences Convert to a coordinate sorted BAM binary sam format like so: % samtools view Sb trinity_gmapsam > trinity_gmapbam Coordinate sort the bam file needed by most tools especially viewers as input : % samtools sort trinity_gmapbam trinity_gmap Now index the bam file to enable rapid navigation in the genome browser: % samtools index trinity_gmapbam b Align RNA-seq reads to the genome using Tophat Next, align the combined read set against the genome so that we ll be able to see how the input data matches up with the Trinity assembled contigs Do this by running TopHat like so: Prep the genome for running tophat % bowtie2 build GENOME_data/genomefa genome Now run tophat: % tophat2 I 300 i 20 genome \ RNASEQ_data/Sp_logleftfqgz,RNASEQ_data/Sp_hsleftfqgz,RNASEQ_data/Sp_dsleftfqgz,RNAS \ RNASEQ_data/Sp_logrightfqgz,RNASEQ_data/Sp_hsrightfqgz,RNASEQ_data/Sp_dsrightfqgz,R Index the tophat bam file needed by the viewer: % samtools index tophat_out/accepted_hitsbam c Visualize all the data together using IGV % igvsh g `pwd`/genome_data/genomefa `pwd`/genome_data/genesbed,`pwd`/tophat_out/accepted_hitsbam,`pwd`/trinity_gmapbam De novo Transcriptome Assembly Workshop 4/10

5 Does Trinity fully or partially reconstruct transcripts corresponding to the reference transcripts and yielding correct structures as aligned to the genome? Are there examples where the de novo assembly resolves introns that were not similarly resolved by the alignments of the short reads, and vice versa? Exit the IGV viewer to continue on with the tutorial/demo Abundance estimation using RSEM To estimate the expression levels of the Trinity reconstructed transcripts, we use the strategy supported by the RSEM software We first align the original rna seq reads back against the Trinity transcripts, then run RSEM to estimate the number of rna seq fragments that map to each contig Because the abundance of individual transcripts may significantly differ between samples, the reads from each sample must be examined separately, obtaining sample specific abundance values For the alignments, we use bowtie instead of tophat There are two reasons for this First, because we re mapping reads to reconstructed cdnas instead of genomic sequences, properly aligned reads do not need to be gapped across introns Second, the RSEM software is currently only compatible with gap free alignments The RSEM software is wrapped by scripts included in Trinity to facilitate usage in the Trinity framework Separately quantify transcript expression for each of the samples: The following script will run RSEM, which first aligns the RNA Seq reads to the Trinity transcripts using the Bowtie aligner, and then performs abundance estimation This process is % ${TRINITY_HOME}/util/align_and_estimate_abundancepl seqtype fq \ left RNASEQ_data/Sp_dsleftfqgz right RNASEQ_data/Sp_dsrightfqgz \ transcripts trinity_out_dir/trinityfasta \ output_prefix Sp_ds est_method RSEM aln_method bowtie \ trinity_mode prep_reference output_dir Sp_dsRSEM Once finished, RSEM will have generated two files: Sp_dsisoformsresults and Sp_dsgenesresults These files contain the Trinity transcript and component the Trinity analogs to Isoform and gene rna seq fragment counts and normalized expression values Examine the format of the Sp_dsisoformsresults file by looking at the top few lines of the file: % head head Sp_dsRSEM/Sp_dsisoformsresults yielding note your results may not be identical due to assembly differences : transcript_id gene_id length effective_length expected_count TPM FPKM IsoPct TRINITY_DN0_c0_g1_i1 TRINITY_DN0_c0_g TRINITY_DN0_c1_g1_i1 TRINITY_DN0_c1_g TRINITY_DN100_c0_g1_i1 TRINITY_DN100_c0_g De novo Transcriptome Assembly Workshop 5/10

6 TRINITY_DN100_c1_g1_i1 TRINITY_DN100_c1_g TRINITY_DN100_c2_g1_i1 TRINITY_DN100_c2_g TRINITY_DN101_c0_g1_i1 TRINITY_DN101_c0_g TRINITY_DN101_c1_g1_i1 TRINITY_DN101_c1_g TRINITY_DN101_c2_g1_i1 TRINITY_DN101_c2_g TRINITY_DN102_c0_g1_i1 TRINITY_DN102_c0_g Run RSEM on each of the remaining three samples and examine their outputs: Quantify expression for Sp_hs: % ${TRINITY_HOME}/util/align_and_estimate_abundancepl seqtype fq \ left RNASEQ_data/Sp_hsleftfqgz right RNASEQ_data/Sp_hsrightfqgz \ transcripts trinity_out_dir/trinityfasta \ output_prefix Sp_hs est_method RSEM aln_method bowtie \ trinity_mode prep_reference output_dir Sp_hsRSEM % head Sp_hsRSEM/Sp_hsisoformsresults Quantify expression for Sp_log: % ${TRINITY_HOME}/util/align_and_estimate_abundancepl seqtype fq \ left RNASEQ_data/Sp_logleftfqgz right RNASEQ_data/Sp_logrightfqgz \ transcripts trinity_out_dir/trinityfasta \ output_prefix Sp_log est_method RSEM aln_method bowtie \ trinity_mode prep_reference output_dir Sp_logRSEM % head Sp_logRSEM/Sp_logisoformsresults Quantify expression for Sp_plat: % ${TRINITY_HOME}/util/align_and_estimate_abundancepl seqtype fq \ left RNASEQ_data/Sp_platleftfqgz right RNASEQ_data/Sp_platrightfqgz \ transcripts trinity_out_dir/trinityfasta \ output_prefix Sp_plat est_method RSEM aln_method bowtie \ trinity_mode prep_reference output_dir Sp_platRSEM % head Sp_platRSEM/Sp_platisoformsresults Generate a counts matrix and perform cross-sample normalization: % ${TRINITY_HOME}/util/abundance_estimates_to_matrixpl est_method RSEM \ out_prefix Trinity_trans \ Sp_dsRSEM/Sp_dsisoformsresults \ Sp_hsRSEM/Sp_hsisoformsresults \ Sp_logRSEM/Sp_logisoformsresults \ Sp_platRSEM/Sp_platisoformsresults You should find a matrix file called 'Trinity_transcountsmatrix', which contains the counts of RNA Seq fragments mapped to each transcript Examine the first few lines of the counts matrix: De novo Transcriptome Assembly Workshop 6/10

7 % head n20 Trinity_transcountsmatrix In addition, a matrix containing normalized expression values is generated called 'Trinity_transTMMEXPRmatrix' These expression values arenormalized for sequencing depth and transcript length, and then scaled via TMM normalization under the assumption that most transcripts are not differentially expressed Differential Expression Using EdgeR To detect differentially expressed transcripts, run the Bioconductor package edger using our counts matrix: % ${TRINITY_HOME}/Analysis/DifferentialExpression/run_DE_analysispl \ matrix Trinity_transcountsmatrix \ method edger \ dispersion 01 \ output edger Examine the contents of the edger/ directory % ls ltr edger/ rw r r 961 Oct 14 21:08 Trinity_transcountsmatrixSp_ds_vs_Sp_hsSp_dsvsSp_hsEdgeRRscript rw r r 965 Oct 14 21:08 Trinity_transcountsmatrixSp_ds_vs_Sp_logSp_dsvsSp_logEdgeRRscript rw r r Oct 14 21:08 Trinity_transcountsmatrixSp_ds_vs_Sp_hsedgeRDE_resultsMA_n_Volcanopdf rw r r Oct 14 21:08 Trinity_transcountsmatrixSp_ds_vs_Sp_hsedgeRDE_results rw r r 965 Oct 14 21:08 Trinity_transcountsmatrixSp_hs_vs_Sp_logSp_hsvsSp_logEdgeRRscript rw r r Oct 14 21:08 Trinity_transcountsmatrixSp_ds_vs_Sp_platedgeRDE_resultsMA_n_Volcanopdf rw r r Oct 14 21:08 Trinity_transcountsmatrixSp_ds_vs_Sp_platedgeRDE_results rw r r 969 Oct 14 21:08 Trinity_transcountsmatrixSp_ds_vs_Sp_platSp_dsvsSp_platEdgeRRscript rw r r Oct 14 21:08 Trinity_transcountsmatrixSp_ds_vs_Sp_logedgeRDE_resultsMA_n_Volcanopdf rw r r Oct 14 21:08 Trinity_transcountsmatrixSp_ds_vs_Sp_logedgeRDE_results rw r r 973 Oct 14 21:08 Trinity_transcountsmatrixSp_log_vs_Sp_platSp_logvsSp_platEdgeRRscript rw r r Oct 14 21:08 Trinity_transcountsmatrixSp_hs_vs_Sp_platedgeRDE_resultsMA_n_Volcanopdf rw r r Oct 14 21:08 Trinity_transcountsmatrixSp_hs_vs_Sp_platedgeRDE_results rw r r 969 Oct 14 21:08 Trinity_transcountsmatrixSp_hs_vs_Sp_platSp_hsvsSp_platEdgeRRscript rw r r Oct 14 21:08 Trinity_transcountsmatrixSp_hs_vs_Sp_logedgeRDE_resultsMA_n_Volcanopdf rw r r Oct 14 21:08 Trinity_transcountsmatrixSp_hs_vs_Sp_logedgeRDE_results rw r r Oct 14 21:08 Trinity_transcountsmatrixSp_log_vs_Sp_platedgeRDE_resultsMA_n_Volcanopdf rw r r Oct 14 21:08 Trinity_transcountsmatrixSp_log_vs_Sp_platedgeRDE_results The files '*DE_results' contain the output from running EdgeR to identify differentially expressed transcripts in each of the pairwise sample comparisons Examine the format of one of the files, such as the results from comparing Sp_log to Sp_plat: % head edger/trinity_transcountsmatrixsp_log_vs_sp_platedgerde_results De novo Transcriptome Assembly Workshop 7/10

8 logfc logcpm PValue FDR TRINITY_DN194_c0_g1_i e e 18 TRINITY_DN200_c0_g1_i e e 17 TRINITY_DN191_c0_g1_i e e 15 TRINITY_DN145_c0_g1_i e e 15 TRINITY_DN172_c0_g1_i e e 15 TRINITY_DN160_c0_g2_i e e 15 TRINITY_DN16_c0_g1_i e e 14 TRINITY_DN67_c0_g2_i e e 14 TRINITY_DN243_c0_g1_i e e 14 These data include the log fold change logfc, log counts per million logcpm, P value from an exact test, and false discovery rate FDR The EdgeR analysis above generated both MA and Volcano plots based on these data Examine any of these 'edger/transcriptscountsmatrixconda_vs_condbedgerde_resultsma_n_volcanopdf' pdf files like so: % evince edger/trinity_transcountsmatrixsp_log_vs_sp_platedgerde_resultsma_n_volcanopdf Note, on linux, 'evince' is a handy viewer You might also use 'xpdf' to view pdf files On mac, try using the 'open' command from the command line instead Exit the chart viewer to continue How many differentially expressed transcripts do we identify if we require the FDR to be at most 005? You could import the tab delimited text file into your favorite spreadsheet program for analysis and answer questions such as this, or we could run some unix utilities and filters to query these data For example, a unix y way to answer this question might be: % sed '1,1d' edger/trinity_transcountsmatrixsp_log_vs_sp_platedgerde_results awk '{ if ($5 <= 005) print;}' wc l 69 Trinity facilitates analysis of these data, including scripts for extracting transcripts that are above some statistical significance FDR threshold and fold change in expression, and generating figures such as heatmaps and other useful plots, as described below Extracting differentially expressed transcripts and generating heatmaps De novo Transcriptome Assembly Workshop 8/10

9 Now let's perform the following operations from within the edger/ directory Enter the edger/ dir like so: % cd edger/ Extract those differentially expressed DE transcripts that are at least 4 fold differentially expressed at a significance of <= 0001 in any of the pairwise sample comparisons: % $TRINITY_HOME/Analysis/DifferentialExpression/analyze_diff_exprpl \ matrix /Trinity_transTMMEXPRmatrix P 1e 3 C 2 The above generates several output files with a prefix diffexprp1e 3_C2, indicating the parameters chosen for filtering, where P FDR actually is set to 0001, and fold change C is set to 2^ 2 or 4 fold These are default parameters for the above script See script usage before applying to your data Included among these files are: diffexprp1e 3_C2matrix : the subset of the FPKM matrix corresponding to the DE transcripts identified at this threshold The number of DE transcripts identified at the specified thresholds can be obtained by examining the number of lines in this file % wc l diffexprp1e 3_C2matrix 57 Note, the number of lines in this file includes the top line with column names, so there are actually 56 DE transcripts at this 4 fold and 1e 3 FDR threshold cutoff Also included among these files is a heatmap diffexprp1e 3_C2matrixlog2centeredgenes_vs_samples_heatmappdf as shown below, with transcripts clustered along the vertical axis and samples clustered along the horizontal axis % evince diffexprp1e 3_C2matrixlog2centeredgenes_vs_samples_heatmappdf Exit the PDF viewer to continue Extract transcript clusters by expression profile by cutting the dendrogram De novo Transcriptome Assembly Workshop 9/10

10 Extract clusters of transcripts with similar expression profiles by cutting the transcript cluster dendrogram at a given percent of its height ex 60%, like so: % $TRINITY_HOME/Analysis/DifferentialExpression/define_clusters_by_cutting_treepl \ Ptree 60 R diffexprp1e 3_C2matrixRData This creates a directory containing the individual transcript clusters, including a pdf file that summarizes expression values for each cluster according to individual charts: % evince diffexprp1e 3_C2matrixRDataclusters_fixed_P_60/my_cluster_plotspdf More information on Trinity and supported downstream applications can be found from the Trinity software website: De novo Transcriptome Assembly Workshop 10/10