PESIT Bangalore South Campus

Transcription

1 INTERNAL ASSESSMENT TEST - 2 Date : 19/09/2016 Max Marks : 50 Subject & Code : DATA WAREHOUSING AND DATA MINING(10IS74) Section : 7 th Sem. ISE A & B Name of faculty : Ms. Rashma.B.M Time : 11:30 to 1:00pm NOTE: Answer all 5 full questions. 1. What is data preprocessing? Explain the following techniques for performing data preprocessing. i) Sampling ii) Binarization & discretization Ans: Data preprocessing is one of the important steps in Data Mining which converts raw data to data mining algorithm compatible form. Following are various methods to accomplish data preprocessing: 1.Aggregation 2.Sampling 3.Dimensionality Reduction 4.Feature subset selection 5.Feature creation 6.Discretization and Binarization 7.Attribute Transformation Sampling is the main technique employed for data selection.it is often used for both the preliminary investigation of the data and the final data analysis. Sampling is used in data mining because processing the entire set of data of interest is too expensive or time consuming. i.simple Random Sampling There is an equal probability of selecting any particular item ii.sampling without replacement As each item is selected, it is removed from the population iii.sampling with replacement Objects are not removed from the population as they are selected for the sample. In sampling with replacement, the same object can be picked up more than once iv.stratified sampling Split the data into several partitions; then draw random samples from each partition(various class of data, previous method may miss required data hence stratified sampling). v.adaptive or progressive sampling used in predictive modeles

2 2. a For the following vectors X and Y calculate indicated similarity or distance measures, given X=(0,1,0,1) Y=(1,0,1,0) i) Cosine ii) correlation iii) Euclidean iv) Jaccard Refer to exercise problem done in class. ANS i)cosine =X.Y/ X * Y. Ans =0 ii)apply formula :correlation = covariance (x,y)/standard deviation(x)* standard deviation (y) ans= -1. iii)euclidean : ans: 2 dist = k = 1 ( p k q k ) iv)jaccard : J = (M 11 ) / (M 01 + M 10 + M 11 ). Ans=0 n 2 05 Marks 05 Marks 2. b Define the following : i) Association rule ii) Support & confidence iii) Maximal frequent itemset iii) Closed itemset v) Monotonicity property ANS i) Given a set of transactions T, the goal of association rule mining is to find all rules having support minsup threshold confidence minconf threshold ii) Support : gives ratio of association rule made to that of total no. of transactions Consider X->Y Support = count(x U Y)/count(T) Confidence : gives ratio of existence of inference to be true with respect to existence of rule antecedent. Consider X->Y Confidence = count(x U Y)/count(X). iii) Maximal frequent Item set: given inferred set of infrequent item set, the item set adjacent to inferred infrequent item set, that has maximal support are called maximal frequent itemset. iv) Closed itemset : all those item sets in a search space graph that has larger or equal or count compared to that of its adjacent nodes. v) X, Y : ( X Y ) s( X ) s( Y ) Anti monotone property

3 Monotone property that is converse of anti monotone property. 3. Apply apriori algorithm to find frequent item sets in the given Market basket analysis t1:beef, Chicken, Milk, t2:beef, Cheese, t3: Cheese, Boots, t4:beef, Chicken, Cheese, t5: Beef, Chicken, Clothes, Cheese, Milk, t6:chicken, Clothes, Milk, t7:chicken, Milk, Clothes ANS :1 st iteration : generation of 1-item set find respective counts Beef: Chicken: Milk: Cheese: Boots: Clothes: 2 nd iteration 2-itemset continue till you find k_max 4. With an appropriate example of contingency tables explain limitations of interest factor. f11 N N f11 ANS :Intrestingness = = ( f1 + N ) ( f+ 1 N ) f1 + f+ 1 Consider example where interestingness is very less even though actual values show it can be a strong frequent item set and explain in detail. 5. Apply FP-growth algorithm and show step wise generation to find frequent item sets ending in I5 where minimal support count is 2. T1:(I1,I2),T2:(I2,I3,I4), T3:(I1,I3,I4,I5),T4: (I1,I4,I5),T5: (I1,I2,I3),T6: (I1,I2,I3,I4), T7:(I1), T8: (I1,I2,I3), T9: (I1,I2,I4), T10:(I2,I3,I5). ANS: (This is just convention as problem is solved in terms of alphabets. NOTE: do not change lables in exam should solve the problems with given lables)

4 Consider I1->a, I2->b,I3->c, I4->d, I5->e.

5