Variant calling using SAMtools

Transcription

1 Variant calling using SAMtools Calling variants - a trivial use of an Interactive Session We are going to conduct the variant calling exercises in an interactive idev session just so you can get a feel for this mode of computing. Almost everything you see here can easily be bundled up into batch scripts and run in that mode. Introduction Variant calling, at first glance, is pretty simple: Map sequence reads to an appropriate reference, emitting BAM files; Generate pileups and look for evidence of structural variation; Correct for false-positives due to the sequencing technology. Common file formats used in variant detection are: BAM files containing alignments against a reference genome Reference FASTA files containing genome sequence VCF files to represent SNPs and small indels BED files for specifying regions of the genome Setting up Go to the Terminal shell window in which you have launched an idev session: 1. Set your BASE variable a. export BASE=/scratch/01374/vaughn/tacc_ngs 2. mkdir $WORK/variants && cd $WORK/variants 3. Copy your own BWA alignment file into place: cp $WORK/bwa-align/hs37d5_allseqs_bwa.bam. 4. Load up the modules we will need for this session a. module load samtools && export PATH=$PATH:$TACC_SAMTOOLS_DIR/bcftools Now, we're ready for some variant-hunting! Alignment statistics Before diving into interpretation of bioinformatics results, it's nice to get some summary statistics. SAMtools has a tool 'flagstat' that makes it easy to do this for BAM files. Run samtools flagstat hs37d5_allseqs_bwa.bam and you should get: in total (QC-passed reads + QC-failed reads) duplicates mapped (87.81%:nan%) paired in sequencing read read properly paired (0.88%:nan%) with itself and mate mapped singletons (7.82%:nan%) with mate mapped to a different chr with mate mapped to a different chr (mapq>=5) As an aside, the reads whose mates map to alternate chromosomes may be revealing structural rearrangement. Most likely not, but among these reads is where you would look for evidence of such a thing. Basic variant calling Variant calling is basically a three-step process:

2 1. First, samtools mpileup command transposes the mapped data in a sorted BAM file fully to genome-centric coordinates. It starts at the first base on the first chromosome for which there is coverage and prints out one line per base. Each line has information on every base observed in the raw data at that base position along with a lot of auxiliary information depending on which flags are set. It calculates the Bayseian prior probability given by the data, but does not estimate a real genotype. 2. Next, bcftools with a few options added uses the prior probability distribution and the data to calculate an actual genotype for the variants detected. 3. Finally, vcfutils.pl (or equivalent) is used to filter down the list of candidates according to some set of objective criteria. Here's a basic set of commands to generate a BCF of genotypes. First, be sure the BWA file is sorted and indexed samtools sort hs37d5_allseqs_bwa.bam hs37d5_allseqs_bwa-sorted samtools index hs37d5_allseqs_bwa-sorted.bam Then, call variants and write to a VCF file samtools mpileup -uf /corral-repl/utexas/bioiteam/tacc_ngs/human_variation/ref/hs37d5.fa \ hs37d5_allseqs_bwa-sorted.bam \ bcftools view -vcg - > hs37d5_allseqs_bwa.raw.vcf Open the 'hs37d5_allseqs_bwa.raw.vcf' file in a text editor and take a look around. Notice all the descriptive metadata in the comments (lines that start with the # character) - VCF is a very good format for bioinformatics! Question: What version of the VCF standard is this file The first line of the file is ##fileformat=vcfv4.1 and so this is a VCF 4.1 file Find the variant in chromosome 20 at position in the VCF file (It is probably the first variant record in the file). Hint If you are using 'less' to page through the VCF, you can hit the '?' key and type in and it will take you to the line where this polymorphism is found. Question: What is the reference base for this SNP and what is the polymorhism? The reference allele at this position is 'G' and the polymorphism is 'A' Question: What is the read depth supporting this polymorphism? Over in the INFO field, you will see a string DP=10. Looking up in the metadata at the top of the file, you see that DP is "Raw read depth". So, there are 10 reads supporting this polymorphism. Sounds like a potential winner! Question: What is the quality of this SNP? The QUAL field for this SNP is 39. VCF qualities are expressed on a PHRED scale, so this means there is a 1x10^-3.9^ chance that this SNP has been called in error. Pretty good, right? Inspecting base-level alignments So, you've identified a variant chr20: , it's got good quality and read support, and you want to verify it for yourself. You can use any number of BAM browsers (IGV, SeqMonk, etc) but you can also do this right from the terminal using a command bundled with samtools called 'tview'. Enter the following command

3 samtools tview hs37d5_allseqs_bwa-sorted.bam /corral-repl/utexas/bioiteam/tacc_ngs/human_variation/ref/hs37d5.fa You should see a window that resembles the following screen shot. Browser View 1 Hit the '?' key to pull up help, then hit '?' again to dismiss help. Let's go to the SNP we examined before. Type 'n' to turn on color coding of nucleotides. Hit 'g' and you will be asked to enter a position in the genome, then enter 20: in this box and hit 'Return' and you will be transported to the location in reference genome. The base will be the left-most column in the BAM browser. Browser View 2 Filtering your VCF file The vcfutils.pl script, bundled with bcftools inside SAMTools can provide useful stats and can perform some filtering of VCFs by specific criteria. Running vcfutils.pl $TACC_SAMTOOLS_DIR/bcftools/vcfutils.pl qstats hs37d5_allseqs_bwa.raw.vcf Output

4 QUAL #non-indel #SNPs #transitions #joint ts/tv #joint/#ref #joint/#non-indel Question: How many INDELs were identifed in your VCF file? Hint The 'grep' command may help you figure this out. grep -c "INDEL" hs37d5_allseqs_bwa.raw.vcf 48 So there are 48 indels that were identifiable in this data set.

5 Why filter? With any variant caller you use, there will be an inherent trade-off between sensitivity and specificity. Typically, you would carry forward as much data as practical at each processing step, deferring final judgement until later so you have more information. For example, you might not want to exclude low coverage regions in one sample from your raw data because you may be able to infer a genotype based on family information later. This typically leads to NGS pipelines that maximize sensitivity, leading to a substantial number of false positives. Filtering after variant calling ca n be useful to eliminate false positives based on all the data available after numerous analyses. In the samtools/bcftools world, the vcfutils.pl script provides a means to filter SNPs on many criteria. Exercises Now, you will explore some filter settings for vcfutils.pl varfilter to see how many SNPs get filtered out, using the linux tool xargs to do a parameter sweep. Filtering, counting, and parameter iteration # First - create a tiny shell script to run vcfutils, accepting a single parameter: echo "#\!/bin/bash" > tiny.sh echo "echo \"Sweeping with vcfutils.pl, min read depth of: \$1\" " >> tiny.sh echo "$TACC_SAMTOOLS_DIR/bcftools/vcfutils.pl varfilter -Q 20 -d \$1 hs37d5_allseqs_bwa.raw.vcf grep -v '^#' wc -l " >> tiny.sh # Now make it executable chmod +x tiny.sh # Use xargs to do a sweep of read depths echo xargs -n 1 tiny.sh Output Sweeping with vcfutils.pl, min read depth of: Sweeping with vcfutils.pl, min read depth of: Sweeping with vcfutils.pl, min read depth of: Sweeping with vcfutils.pl, min read depth of: Sweeping with vcfutils.pl, min read depth of: Sweeping with vcfutils.pl, min read depth of: This may be obvious, but there are fewer SNPs returned the higher read depth you require to support them. Homework: Try to update tiny.sh so that sweeps through Quality rather than read depth Exercise questions

6 You can use bedtools and some Linux built-in commands to compare your vcf file to one generated by BioITeam staff using some other data and means (you will need module load bedtools). Your file: hs37d5_allseqs_bwa.raw.vcf BioITeam staff VCF file: /corral-repl/utexas/bioiteam/tacc_ngs/human_variation/all.samtools.vcf Hints Investigate grep -c Try intersectbed and wc *Question: How many SNPs are in your VCF file and the staff-generated VCF file? Solution grep -c -v "^#" hs37d5_allseqs_bwa.raw.vcf 1507 grep -c -v "^#" /corral-repl/utexas/bioiteam/tacc_ngs/human_variation/all.samtools.vcf Question: How many SNPs are in common between the two files? Solution intersectbed -a hs37d5_allseqs_bwa.raw.vcf -b /corral-repl/utexas/bioiteam/tacc_ngs/human_variation/all.samtools.vcf wc -l 88 Question: What are the average and maximum quality values for the SNPs that are in your VCF file? Solution grep -v '^#' hs37d5_allseqs_bwa.raw.vcf awk 'BEGIN {max=0} {sum+=$6; if ($6>max) {max=$6}} END {print "Average qual: "sum/nr "\tmax qual: " max}' Average qual: Max qual: 113 There's probably no way you knew how to do this! Please check out the power of awk before you write YAPPS (Yet Another Perl or Python Script) - it's a lifesaver, and is named after a cute bird.