István'Albert' ' Biochemistry$and$Molecular$Biology$$ and$bioinforma;cs$consul;ng$center$ $ Penn$State$

Transcription

1 2012%BMMB597D:AnalyzingNextGenera;onSequencingData Week1,Lecture3 István'Albert' ' BiochemistryandMolecularBiology andbioinforma;csconsul;ngcenter PennState

2 SomeProgrammingRequired Exis;ngsoOwaretoolscanrarelydoallsteps SourcedataindifferentfilesmaybeformaUed differently Weneedtobridgethedifferenceswithsimple transforma;ons Someprogrammingabilityisnecessaryforevery project

3 Possiblythefirstscrip;nglanguage:awk' appearedin1977,strongunix(1972)roots itistheprecursoroflanguagessuchasperl (1987)andPython(1989) Hasfallenintodisuseforawhile;thelarge textdatasetshaveleadtoaresurgenceofthe language

4 TheArtofUnixProgramming:TAOUP SeminalopensourcebookbyEricRaymond Notquiteright:The'more'things'change'the'more'they'stay'the'same.'

5 Thestructureofanawkprogram!!!!awk!!pattern!{!action!}!! lineoriented 1. trytomatchthepauerntotheline 2. Ifthereisamatchprocessthelineandproduceanew line defaultpauern=matcheverything

6 SimpleAwkExample nopauernspecified

7 Advice writepipelineswithclearinputsandoutput Keeptheflowofinforma;onfromleOtoright Itisaloteasiertomodify/alterlateron

8 Specialvariables Awkautoma;cally'splitstheinputby whitespace(spacesandtabs)andassigns namestothem: 0theen;reline 1firstfield 2secondfield NFthenumberoffields NRthenumberofthecurrentline

9 Thewhitespacecurse:spacesandtabs Manytoolswillautosplitbywhitespace!thiswas thoughttobeconvenientbutisalsothesourceof extremelysubtleerrors!leadstoacolumnshioina tabfileifafieldcontainsspaces Always'specify'the'character'to'be'split'by!' Thisreferstoprogramminglanguagesaswell! Donotusethesplit()methodswiththeirdefault behavior(python,perletc)unlessyouperfectly understandwhattheydo)

10 Customizeawktousetabsasboththe inputandoutputfieldseparator alias!awk="awk!4f!'\t'!4v!ofs='\t'"! Tip:youcanaddthistothe.profile'or'.bash_profile'fileinyourrootfoldersothat itisac;vateallthe;me Note:filenamesthatstartwithadot.areonlylistedifyoudoals'Ja'

11 RecalltheGFFformatfromlecture3 SearchforGFF3!hUp:// Tabseparatedwith9columns.MissingaUributesmaybereplacedwithadot!. 1. Seqid'(usuallychromosome) 2. Source(whereisthedatacomingfrom) 3. Type(usuallyatermfromthesequenceontology) 4. Start''(intervalstartrela;vetotheseqid) 5. End''''(intervalendrela;vetotheseqid) 6. Score'''(thescoreofthefeature,afloa;ngpointnumber) 7. Strand''(+/%/.) 8. Phase'''''''(usedtoindicatereadingframeforcodingsequences) 9. AVributes(semicolonseparatedaUributes!Name=ABC;ID=1) ExampleaUributespecifica;on:name=REB1;id=YP33546

12 WillaUempttoextractthename from9 th column%>theauribute matching ac;on SemicolonseparatedaUributesoneofthemisName=YAL069W;' 1. Splitbysemicolonandkeepthesecondelement,thensplitthisagainbythe= signandkeepthesecondelement

13 Youmayalso puttheprogramintoafile

14 Writeasingestepata;me andtestitevery;me

15

16 Createagfffilethatcontainsthegene nameinthetypecolumn

17 Makethatnewsimplergff:short.gff containsonlygenesandhasnameinthesecondcolumn

18 Intervallengthsforgenes

19 Operators +'J'*'/'fornumericalcontext %modulodivision(remainderofdivision) <space>''stringconcatena;on ==,'!='''equal,notequal ~,'!~''''''match,nomatch(regularexpressions)

20 SpecialpaUerns BEGIN!beforethestreamstarts END!aOerthestreamends

21 MulitplepaUerns!awkwilltrythem eachinturn

22 Advancedawk usuallynotneeded condi;onals:if' loops:for,while' break,con;nue' associa;ve'data'structures'(hash,'dic;onary)' Youcandoallthatthoughatthatpoint itisprobablybeuertolearnpython Butyoucan'do'a'lot'with'just'basic'awk! Awk s'power'comes'from'its'simplicity' 'one'liners!'

23 Afewawkresources Largenumberofresources,thequirkyname makesitverysearchable!how'to'do'x'with' awk? Howtouseawk: hup://sparky.rice.edu/awk.html

24 Homework3 1. Writetheawkcodethatfindsthelengthof thegenome(sumofallchromosomelenghts) fromthefeatures.gfffile. 2. Writetheawkcodethatfindsthelongest andshortestgeneinthefeatures.gfffile