Administriva# CS#133:#Databases# Logical#Plan#to#Physical#Plan# Goals#for#Today# Spring#2017# Lec#11# #2/21# Query#Evalua?on# Prof.

Transcription

1 Administriva CS133:Databases Spring2017 Lec11 2/21 QueryEvalua?on Prof.BethTrushkowsky Lab2 Part2duetomorrow FinalversionduenextWednesday Problemsets PS5dueThursday PS6outthisThursday Midterm Nextclass:reviewsession MidtermoutaSernextclass,due2/28(noclassthatday) GoalsforToday Discussalgorithmsforimplemen?ngqueryplan operators:selec?on,joins ReasonaboutcostinI/Os Understandhowexternalsor?ngandhashingcan beusedforthesealgorithms PrepareforfinalexercisesofLab2 LogicalPlantoPhysicalPlan Logicalqueryplanpar$allyshowsushowtoevaluatequery Missing:choiceofspecific'algorithm forexecu$ngoperators Access/path/choice: Sequen?alscan?Indexes? Join/algorithm/choice: Simplenestedloop Indexnestedloop Sort<merge?' Selec$ons:howtoapply mul?pleconstraints? Projec$ons:duplicateelim.?// bid=100'' Reserves' sname' sid=sid' ra8ng'>'5' Sailors'

2 Review:Rela?onalOpera?ons GeneralThemes Findingalgorithmsfor: Selec$on(σ) Projec$on(π) Last/$me:sor?ngorhashingto removeduplicateswhenprojec?ng WhatcanwedowhenalldatacannotfitinRAM Out:of:core/algorithms Conceptually:'form'par88ons'to'get' like 'things'together Cross:product/and/Join( ) Set:difference(b) Union( ) Aggrega$on(SUM,MIN,etc.)andGROUPBY Factorsinfluencingoperatorcost: Inputsize(numberofpages) Indexesavailable Bufferpoolspace ASerwecoverchoices'for'algorithms,wewilldiscuss howtoopmize/queriesformedbycomposingoperators SchemaforExamples Sailors(sid:integer,sname:string,ra$ng:integer,age:real) Reserves(sid:integer,bid:integer,day:date,rname:string) Similartooldschema;rnameaddedforvaria?ons. Reserves: Eachrecordis40byteslong 100recordperpage M=1000pages Sailors: Eachrecordis50byteslong, 80recordperpage N=500pages Thesenumbersalso onyourworksheet Oftheform SimpleSelec?ons σ R. attr value ( R) op SELECT* FROMReservesR WHERER.bid<100; Size'of'result'approximatedassize)of)R)*)reducon)factor Reduc?onfactor alsocalledselec$vity Sta?s?csinCatalogcanhelpuses?mate Howbesttoexecuteaselec?on?Dependson: Whatindexes/accesspathsareavailable Expectedsizeoftheresult (intermsofnumberoftuplesand/ornumberofpages)

3 SimpleSelec?ons:notusingindexes (Review) Withno'index,'unsorted: Mustscanthewholerela?on ForReserves=cost'is'M'(pagesinR)!1000I/Os. Withno'index,'sorted: Costofbinarysearch+numberofpagescontaining results ForReserves=log I/Os+%selec8vity*pages&' UsinganIndexforSelec?ons Costdependsonqualifyingtuplesandifindexclustered Costhastwocomponents: Findingqualifyingdataentries(typicallysmall,e.g.,findingleafintree) Pluscostofretrieving'records'(could'be'large'withoutclustering) Forexamplequeryon Reserves,if10%oftuplesqualify: (resultsizees?mate=100pagesor10,000'tuples) CLUSTERED'(Alt'2)' Lirlemore than100i/os Data'entries' (Index'File)' (Data'file)' UNCLUSTERED''(Alt'2)' Couldbemore than10,000i/os! Data'entries' Data'Records' Data'Records' Selec?onsusingIndex(cont) Important/refinement)for)unclustered/indexes: 1. Findqualifyingdataentries. 2.Sort'the'record'ids'ofthedatarecordstoberetrieved. 3.Fetchpagesin'order! Ensuresthateachdatapageislookedatjustonce(though ofsuchpageslikelytobehigherthanwithclustering). Whenusingthisrefinement,isusing theunclustered/index/ever/beder/than/ sequen$al/scan? GeneralSelec?onCondi?ons SELECT* FROMReservesR WHERER.bid=103ANDR.sid=42; AB+<tree'indexmatches(aconjunc?onof)termsif theterm(s)involveonlyarributesinaprefixofthe searchkey. E.g.,Indexon<a,)b,)c>''matchespredicate a=5)and)b=)3, butnot b=3 ' ForHashindex:indexmustinvolveallarributesin searchkey Why?

4 GeneralSelec?ons:TwoApproaches Whatifseveralindexesexistthatcouldbeused? Approach1:pick'one'indextouse Findthemost/selec$ve/access/path, retrievetuplesusingit, thenapplytheothercondi?ons Applyingothercondi?ons won timpactnumberof pagesfetched Most/selec$ve/access/ path:indexwithfile scanes?matedto requirefewestpagei/os GeneralSelec?ons:TwoApproaches Approach2:use'mul8ple'indexes Tousetwoormorematchingindexes (Alt2or3fordataentries): Getsets'of'record'ids'ofdatarecordsusingeachmatching index. Thenintersectthesesetsofrids. Retrievetherecordsandapplyanyremainingcondi?ons Example:day)<)8/9/94)AND/bid=103/AND/sid=42) SupposehaveB+'tree'index'on'day'andanindex'on'sid' Intersect:ridsusingindexondaywithridsusingindexonsid Thencheckbid=103 Exercise:Selec?on Exercise2 I. B+treeon<rname,day> II. B+treeon<day,rname> III. Hashindexon<day,rname> Disjunc?on: ifallcondi?onshaveanindex,usetheunionofrids! Butifevenoneofthemdoesnothaveindex,haveto dosequen?alscananyway JoinOperators Joinsareaverycommonqueryopera?on! Joinscanbeveryexpensive: ConsideraninnerjoinofRandSeachwith1Mrecords' How)many)tuples)in)the)answer)(worst)case)?) TwomainclassesofJOINalgorithms: Algorithmsthateffec?velyenumerate)cross)product) Algorithmsthatavoid)cross)product)bygexng like par??onstogether

5 EqualityJoinsonOneJoinColumn SELECT * FROM Reserves R, Sailors S WHERE R.sid=S.sid Rela?oninfo: M=1000pagesinR,t R =100tuplesperpage. N=500pagesinS,t S =80tuplesperpage. Inexamples,RisReservesandSisSailors. Cost)metric):ofI/Os (Wewillignorecostoffinaloutputfromquery) SimpleNestedLoopsJoin foreachtuplerinrdo foreachtuplesinsdo ifr i ==s j thenadd<r,s>toresult Foreachtupleintheouterrela?onR,wescan theen?reinnerrela?ons. Cost:(t R *M)*N+M =100,000* I/Os. Whatifsmallerrela?on(S)wasouter? (t s *N)*M+N=40,000* I/Os. PagebOrientedNestedLoopsJoin foreachpagep R inrdo Howmanybufferpool foreachpagep S insdo framesdoweneed? foreachtuplerinp R do foreachtuplesinp S do ifr i ==s j thenadd<r,s>toresult ForeachpageofR,geteachpageofS,andwriteout matchingpairsoftuples<r,s>,whererisinrbpageands isinsbpage. Whatisthecostofthisapproach?(TryExercise3) WithRasouter,cost=M*N+M=1000* Ifsmallerrela?on(S)isouter,cost=500* BlockNestedLoopsJoin PageborientedNLdoesn tuseallavailablebufferframes! Alterna?veapproach: Useonepageasaninput'bufferforscanningthe'inner'S, onepageastheoutputbuffer anduseallremainingpagestohold'block'of'outer'r' ForeachblockofR,scanthrougheachpageofSformatches R&S.'.'.' Input'buffer'for'S' block'of'r'tuples' (<'B<1'pages)'.'.'.' Output'buffer' JoinResult.'.'.'

6 BlockNestedLoopJoin:Examples Cost:/Scanofouter+outerblocks*scanofinner outerblocks=ceiling(pagesofouter/blocksize) WithReserves(R)asouter,and100'pages/block: ScanningRis1000I/Os;atotalof10blocks. PerblockofR,scanSailors(S);10*500'I/Os. Howmany?meswouldwe scansiftheblock'size'was'b' insteadof100? With100bpageblockofSailorsasouter: CostofscanningSis500I/Os;atotalof5blocks. PerblockofS,scanReserves:5*1000I/Os. AvoidingCrossbproduct Simple,Pageboriented,andBlockNestedbloop joinalgorithmseffec8vely'enumerate'the'cross< product' everypairoftuplesiscompared Next:algorithmsthatavoidcrossbproduct (forequalityjoins) tuplesinthetworela?onscanbethoughtofas belongingtoparons)' Indexprobe IndexNestedLoopsJoin Ifthereisanindexonthejoincolumnofonerela?on(sayS), canmakethatrela?ontheinnerandusetheindex Cost:M+((M*t R )*cost'of'finding'matching's'tuples) Typical probe costs: 1.2I/Osforhashindex 2b4I/OsforB+tree foreachtuplerinrdo foreachtuplesinswheres==r do add<r,s>toresult Probetofindmatching data)entries) Thecost'of'finding'S'tuples'(assumingAlt.(2)or(3)fordata entries)dependsonifindexisclustered Clustered:1I/OperpageofmatchingStuples. Unclustered:upto1I/OpermatchingStuple. Exercise4:IndexNestedLoops HaveHashbindex(Alt.2)onsidofSailors(asinner) ScanReserves:1000'page'I/Os,100*1000tuples. ForeachReservestuple:''' 1.2'I/Os'togetdataentryinindex, plus1'i/otoget[the)exactly)one]matchingsailorstuple Totalcost: *100,000=221,000'I/Os' Hashbindex(Alt.2)onsidofReserves(asinner): ScanSailors:500'page'I/Os,80*500tuples. ForeachSailorstuple: 1.2'I/Ostofindindexpagewithdataentries, plus'cost'of'retrieving'matching'reserves'tuples Assuminguniformdistribu?on,2.5reserva?onspersailor (100,000tuplesinR/40,000tuplesinS). Costofretrievingthemis1or2.5I/Osdependingonwhether theindexisclustered

7 i=j SortbMergeJoin(RS) SortRandSonthejoincolumn,thenscanthem todoa merge (onjoinfield),andoutputresult tuples. Par?cularlyusefulif oneorbothinputsarealreadysortedonjoinfield(s) outputisrequiredtobesortedonjoinfield(s) ExampleofSortbMergeJoin sid sname rating age 22 dustin yuppy lubber guppy rusty InstanceofSailors (outer) Supposejoiningonsid=sid sid bid day rname /4/96 guppy /3/96 yuppy /10/96 dustin /12/96 lubber /11/96 lubber /12/96 dustin InstanceofReserves (inner) CostforthisJOIN:SortS+SortR+(M+N) Thecostofmerging:typicallyM+N Why? Reflec?on:SortbMergeJoin Outerrela?onisscannedonce Eachpar88onofInnerrela?onisscanned once'per'matching'outer'tuple' Merge phasecanrequiresomebacktrackingif duplicatevaluesappearinjoincolumn Mul?plescansofanInnerpar??onwillprobably findneededpagesinbufferpool HashbJoin (thisvariant: GraceHashJoin ) Par??onboth rela?onsonthejoin arributesusinghash func?onh Rtuplesinpar??onR i willonlymatchs tuplesinpar??ons i. Foreachpar??oni ReadinallofR i, HashR i onh2 Scanthroughpagesof S i,probinghashtable formatches Original'' Rela8on'.'.'.' Disk' Par88ons' of'r'&'s' Disk' INPUT' hash' fn' h2' hash' func8on' h' B'main'memory'buffers' Hash'table'for'par88on' R i '('<'B<1'pages)' h2' Input'buffer' for's i' OUTPUT' 1' 2' B<1' Output'' 'buffer' B'main'memory'buffers' Par88ons' Disk' 1' 2' Join'Result' B<1'

8 Aggrega?onOpera?ons (AVG,MIN,etc.) Withoutgrouping: Ingeneral,requiresscanningtherela?on. Givenatreeindexwhosesearchkeyincludesallarributes intheselectorwhereclauses,candoindex<only'scan. Withgrouping: Sortongroupbbyarributes,thenscanrela?onand computeaggregateforeachgroup. Similarapproachbasedonhashingongroupbbyarributes. Canuseatreeindexwhosesearchkey covers all arributesinselect,whereandgroupbyclauses SimpleDb:InsertandDelete Insertoperator Insertstuplesitgetsfromitschildintoagivenrela?on Output:numberoftuplesinserted INSERT INTO Students(sid, name, gpa) VALUES (42, Alice, 3.5); A'tuple!' Deleteoperator Deletesthetuplesitgetsfromitschild Output:numberoftuplesdeleted AsktheBufferPooltoinsert/delete INSERT INTO HonorsStudents Possibly' SELECT sid,name,gpa FROM Students mul8ple' WHERE gpa > 3.8; tuples!'(or'0)'' DELETE FROM Students WHERE gpa < 2.0; SimpleDb:BufferPoolEvic?on Implementapageevic?onpolicy UpdategetPage()toevictifnecessary Flushingpages Beforeadirtypageisevicted,itschangesmustbe wrirentodisk! However,callingflushPage()shouldnotevict pages!!