INTRODUCTION AUX FORMATS DE FICHIERS

Transcription

1 INTRODUCTION AUX FORMATS DE FICHIERS

2 Plan. Formats de séquences brutes.. Format fasta.2. Format fastq 2. Formats d alignements 2.. Format SAM 2.2. Format BAM 4. Format «Variant Calling» 4.. Format Varscan 4.2. Format VCF

3 Format Fasta

4 Format fastq séquence = 4 lignes dans le fichier ère ligne = identifiant de la séquence

5 Format fastq 4ème ligne = Qualité Appelée aussi Phred quality score (Sanger format) Probabilité qu'une base soit incorrecte

6 Format fastq Encodée en ASCII (allège le fichier)

7 Formats d alignements Plusieurs format existent SAM et BAM (= standards) ELAND (spécifique Illumina) MAQ Map

8 SAM Format: introduction NGS => a variety of new alignment tools : Bowtie (Langmead,B. et al (2009), Maq (Li,H. et al (2008), BWA (Li and Durbin, 2009),... SAM : a common alignment format that supports all sequence types and aligners SAM : Sequence Alignment/Map format A well-defined interface between alignment and downstream analyses

9 overview

10 @SQ name LN : ref seq name header ref seq length readname flag referencename readname2 flag referencename alignment... one tab-delimited line per alignment

11 Tab-delimited : SAM Fields

12 SAM Format (example) HWI-EAS337_3:7::45:27 63 C02HBa085P07_LR M = TAAGAACTTGGCTGATCGCCTACTTACTGCTTTTAC XA:i:0 MD:Z:36 NM:i:0

13 QNAME: Query name HWI-EAS337_3:7::45:27 63 C02HBa085P07_LR M = TAAGAACTTGGCTGATCGCCTACTTACTGCTTTTAC XA:i:0 MD:Z:36 NM:i:0

14 HWI-EAS337_3:7::45: (decimal) = (binairy) C02HBa085P07_LR40 -read is one of a pair each segment properly aligned 255 according to the aligner 36M -read is in second pair = -read n is mapped on reverse strand TAAGAACTTGGCTGATCGCCTACTTACTGCTTTTAC XA:i:0 MD:Z:36 NM:i:0

15

16 RNAME : reference sequence name HWI-EAS337_3:7::45:27 63 C02HBa085P07_LR M = TAAGAACTTGGCTGATCGCCTACTTACTGCTTTTAC XA:i:0 MD:Z:36 NM:i:0

17 POS : position on reference HWI-EAS337_3:7::45:27 63 C02HBa085P07_LR M = TAAGAACTTGGCTGATCGCCTACTTACTGCTTTTAC XA:i:0 MD:Z:36 NM:i:0

18 MAPQ : mapping quality It equals 0 log0 Pr{mapping position is wrong}, rounded to the nearest integer. A value 255 indicates that the mapping quality is not available. Zero value is the lowest quality. HWI-EAS337_3:7::45:27 63 C02HBa085P07_LR M = TAAGAACTTGGCTGATCGCCTACTTACTGCTTTTAC XA:i:0 MD:Z:36 NM:i:0

19 CIGAR : episode CIGAR : extended CIGAR string (Compact Idiosyncratic Gapped Alignment Report) Format: [0-9][MIDNSHP][0-9][MIDNSHP]... [0-9] : position M = match or mismatch (?!), I/D = insertion / deletion, N = skipped bases on reference, S/H = soft / hard clip (soft means nt's still appear in sequence field), P = padding e.g.: S8M means that the first (5'-most) nt is not part of the alignment, but the following 8 nt's are either matches or mis-matches. HWI-EAS337_3:7::45:27 63 C02HBa085P07_LR M = TAAGAACTTGGCTGATCGCCTACTTACTGCTTTTAC XA:i:0 MD:Z:36 NM:i:0

20 RNEXT : mate or not mate? ' = ' means the mate is mapped to the same reference sequence as the current read ' * ' means that the read is unpaired (has no mate) HWI-EAS337_3:7::45:27 63 C02HBa085P07_LR M = TAAGAACTTGGCTGATCGCCTACTTACTGCTTTTAC XA:i:0 MD:Z:36 NM:i:0

21 PNEXT : mate position ' 0 ' means no info is avalaible HWI-EAS337_3:7::45:27 63 C02HBa085P07_LR M = TAAGAACTTGGCTGATCGCCTACTTACTGCTTTTAC XA:i:0 MD:Z:36 NM:i:0

22 TLEN : insert size read mate insert size HWI-EAS337_3:7::45:27 63 C02HBa085P07_LR M = TAAGAACTTGGCTGATCGCCTACTTACTGCTTTTAC XA:i:0 MD:Z:36 NM:i:0

23 SEQ and QUAL : sequence and quality c.f. FASTQ format HWI-EAS337_3:7::45:27 63 C02HBa085P07_LR M = TAAGAACTTGGCTGATCGCCTACTTACTGCTTTTAC XA:i:0 MD:Z:36 NM:i:0

24 OPT : optional fields Follow the TAG:TYPE:VALUE format. TYPE is : [A(printable character); i(signed integer); f(floating point); z(printable string); H(hex string)] Ex : NM:i:0 edit distance, equal zero for this alignment HWI-EAS337_3:7::45:27 63 C02HBa085P07_LR M = TAAGAACTTGGCTGATCGCCTACTTACTGCTTTTAC XA:i:0 MD:Z:36 NM:i:0

25 CIGAR : episode 2 CIGAR : extended CIGAR string (Compact Idiosyncratic Gapped Alignment Report) Format: [0-9][MIDNSHP][0-9][MIDNSHP]... [0-9] : position M = match or mismatch (?!), I/D = insertion / deletion, N = skipped bases on reference, S/H = soft / hard clip (soft means nt's still appear in sequence field), P = padding e.g.: S8M means that the first (5'-most) nt is not part of the alignment, but the following 8 nt's are either matches or mis-matches. HWI-EAS337_3:7::45:27 63 C02HBa085P07_LR M = TAAGAACTTGGCTGATCGCCTACTTACTGCTTTTAC XA:i:0 MD:Z:36 NM:i:0

26 CIGAR : episode 2

27 BAM BAM = compressed SAM Indexed BAM : *.bam.bai Tools (post process, viewers) use indexed bam to avoid all information extraction

28 SAM Tools SAM Tools : a library and software package for parsing and manipulating alignments in the SAM/BAM format.

29 Picard Tools

30 Reference

31 Format VARSCAN 2.2 Chrom Position Ref Var Reads Reads2 VarFreq Strands Strands2 Qual Qual2 Pvalue chromosome name position (-based) reference allele at this position variant allele at this position reads supporting reference allele reads supporting variant allele frequency of variant allele by read count strands on which reference allele was observed strands on which variant allele was observed average base quality of reference-supporting read bases average base quality of variant-supporting read bases Significance of variant read count vs. expected baseline error

32 Varscan 2.2 Example Chrom Position Ref chr A chr A chr T chr A chr T chr T chr T Var G G A G C C C Reads Reads2 VarFreq Strands Strands2 Qual % % % % % % % Qual Pvalue

33 Format VARSCAN Chrom Position Ref Cons Reads Reads2 VarFreq Strands Strands2 Qual Qual2 Pvalue MapQual MapQual2 ReadsPlus ReadsMinus Reads2Plus Reads2Minus VarAllele chromosome name position (-based) reference allele at this position Consensus genotype of sample in IUPAC format. reads supporting reference allele reads supporting variant allele frequency of variant allele by read count strands on which reference allele was observed strands on which variant allele was observed average base quality of reference-supporting read bases average base quality of variant-supporting read bases Significance of variant read count vs. expected baseline error Average map quality of ref reads (only useful if in pileup) Average map quality of var reads (only useful if in pileup) Number of reference-supporting reads on + strand Number of reference-supporting reads on - strand Number of variant-supporting reads on + strand Number of variant-supporting reads on - strand Most frequent non-reference allele observed

34 VARSCAN Example Chrom C2HBa5G22_LR30 C02HBa0072A04_LR26 C02SLe008B07_LR335 C05HBa045P9_LR36 C06HBa027M7_LR66 C07HBa008L2_LR20 Pos Ref A G C G C A Cons R K T A M R R 0 0 R2 VarFreq Str 50% 50% 2 00% % 0 50% 50% Str2 Q Q Pval MapQ MapQ R+ R R R2- VarAllele 0 G 0 T 2 T 0 A A 0 G

35 Format VCF SAM = standard for alignment VCF = standard for storing sequence variation SNPs, indels, large structural variants Primary intention : to represent human genetic variation (000 Genome Project) Can be used in diffrent contexts

36 overview

37 header meta info starting with '##' body Meta info : provide a standardized description of tags and annotations used in the body section.

38 example mandatory fields #CHROM : chrom id #ID : unique identifier of variant #POS : position of the start of the variant #REF : refrence allele #ALT : comma seprated list of alternate non reference alleles allele #QUAL : phred quality score (?) #FILTER : site filtering information (?) #INFO : user extensible annotation

39

40 no mandatory fields #FORMAT : describe format of #SAMPLE(s) #FORMAT : infos found in the header Samples for this line : genotypes and read depth #SAMPLE: genotype ' ' (i.e. deletion) is on each allele and read depth is 3 #SAMPLE 2: genotype '2 ' (i.e remplacement) is on each allele and read depth is 29

41 example (2)

42 #INFO: found in the header Ancestral allele is 'T', variants found is HapMap2 membership #FORMAT Samples for this line : genotype #SAMPLE : one allele with genotype '0' (0 is reference) and one allele with genotype ' ' (SNP) #SAMPLE2 : genotype '2' (insertion) on each allele

43 large struct variant

44 VCF Tools VCF Tools = 2 modules Operations on VCF files : format validation, merging, comparing, intersecting Analyse SNP data in VCF format : allele frequencies, various Quality Control metrics GATK toolkit : alternative tools for VCF generation and manipulation

45 Reference

46 Sources & Ref Joe Fass and his «Next Generation Sequence Alignment» slides The Sequence Alignment/Map format and SAM tools. Li et al Bioinformatics The variant call format and VCFtools. 20 Bioinformatics Daneck et al.