EE 352 Lab 5 Cache Me If You Can

Transcription

1 EE 52 Lab 5 Cache Me If You Can Introduction In this lab you use your straightforward triple-nested loop implementation of a matrix multiply while implementing a second blocked version of matrix multiply and compare their performance benefits in regard to cache behavior. You will explore the interaction of algorithms and cache architecture. (Make sure you have both versions of your matrix multiply code working before exploring the caching effects.) You will use the MARS Cache Simulator tool to perform your experiments. 2 What you will learn This lab is intended to give you insight and intuition for how to write cache-conscious code and to understand the benefits of certain cache architecture parameters. Background Information and Notes Blocked Matrix Multiply: A traditional NxN matrix multiply can be implemented with a triple nested for loop. However, when N is large, the number of data points accessed before significant reuse is proportional to N 2. This can lead to cache performance and, in turn, overall performance. The advantage of cache memory is predicated on our ability to fit the working data set into the cache. Thus, we can break the large matrix (e.g. 8x8) into smaller matrices (e.g. 2x2) and define the matrix multiply operation recursively (i.e. calculate the overall matrix product by calculating the product of smaller blocks of the matrix. The code for doing this is straightforward though it may require some examination to see exactly what values are being accessed. We have attempted to provide visualization below in addition to the code. // b = block size, N = matrix dimension for(i=0; i < N; i=i+b) for(j=0; j < N; j=j+b) for(k=0; k < N; k=k+b) for(ii=i; ii < i+b; ii++) for(jj=j; jj < j+b; jj++) for(kk=k; kk < k+b; kk++) C[ii][jj] = C[ii][jj] + A[ii][kk] * B[kk][jj]; Last Revised: 6/7/204

2 Accesses performed when outer i,j = 0,0 Black border shows C[ii][jj] values being calculated, Yellow = A matrix access, Blue = B Matrix Accesses Accesses performed when outer i,j = 0,block_size Black border shows C[ii][jj] values being calculated, Yellow = A matrix access, Blue = B Matrix Accesses Figure Matrix accesses made in the blocked version. Each timestep shown is a new value of k. Total number of access made to each matrix A & B = 2b 2 where b = block size. Notice now that in the case of a large matrix (e.g. N = 024) we would only be accessing small blocks of data in any one iteration of the inner loops. Presumably this data could fit in cache and we would see better performance. We will explore this later in the lab. MARS Cache Simulator: MARS includes a data cache simulator that will allow you to provide a cache configuration for your program and have the simulator determine the hit rate and other statistics. The tool is available by clicking Tools.. Cache Simulator. You will need to click the Connect to MIPS button to tell the cache simulator to attach to/monitor activity from the MARS code simulator. 2 Last Revised: 6/7/204

3 The configurable parameters for the cache include Placement Policy (a.k.a. mapping scheme), Block Replacement Policy (always leave at LRU), Set size (a.k.a. K-Ways), Number of blocks in cache and cache block size. The total cache size (in bytes) is simply calculated as: (# of Blocks) * (Cache block size in words) * (4 bytes per word). Note: There is an oddity that statistics are not kept for any cache with 256 blocks or more even though those options exist in the drop down box. Thus, we will have to work around this. Everytime you run your program (you can just click the reset MIPS mem. and register button in the simulator) you should also be sure to RESET the cache simulator by clicking the Reset button. 4 Procedure Run the following experiments on the non-blocked (original) version of the matrix multiply. Notes: Hit rates = Percentages (not absolute counts). Use Excel or other tool to make all plots (no hand plots). a. Cache misses can be categorized as compulsory, capacity, or conflict misses. Determine the number of compulsory misses. To do this set your block size at 8 words per block. Think about what mapping scheme should be used if you do not want any conflict misses. Then continue to increase the size of the cache (i.e. number of blocks) until you can correctly determine the number of compulsory misses. (Think about how you would know the misses counted are all compulsory and not capacity misses). Also determine what the smallest cache size was that only produced compulsory misses. b. Given the smallest cache size that only produced compulsory misses, decrease the block size to 4 words per block and double the number of blocks (to keep the cache size the same). What happens to the number of cache misses? Are these still all compulsory misses? c. Using a total cache size of half of what you found from the previous two parts, vary the cache block size and number of blocks to keep the cache size constant and determine what the optimal block size is to produce the best hit rate. Create a table and an Excel X-Y Last Revised: 6/7/204

4 plot with all possible block sizes and the corresponding hit rates. Make the block size the horizontal axis and plot it on a log scale. d. Use four words per block and start with the previous cache size. Repeatedly reduce the cache size (i.e. number of blocks) by half until the cache size is 28 Bytes. Determine the hit rate for each cache size. e. For a 256 Bytes cache with 4 words per block, plot the hit rate as a function of associativity (k=, 2, 4, 8, 6). What is the optimal associativity setting? Run the following experiments on the blocked version of the matrix multiply using a 256 byte cache with 4 words per block. f. Create a table listing k (where k is the number ways) = {,2,4,8,6} down the vertical axis and the matrix blocking size = 2,, 4 and 6 along the horizontal axis. Fill in the hit rate for each table entry and create an Excel X-Y plot with a curve for each k value. Remember -way set associative is a direct mapping and in this case, 6-ways is a fully associative mapping. g. Discuss (compare and contrast) the results to the unblocked version from part e. 5 Review. Based on your plot from part f, what is the optimal blocking size for the 2x2 matrix multiply? Why do you think this might be? 2. Based on your plot from part f, assuming that increasing k (i.e. the associativity) is more and more expensive, what value of k would you suggest for the cache design (i.e. at what point do we start to see diminishing hit rate benefits from increasing k) 6 Submission. Submit your blocked matrix multiple file on blackboard and a PDF or Word document with answers to the questions posed in Part 2 along with the Review questions. Embed your Excel graphs into this document and provide appropriate. 4 Last Revised: 6/7/204

5 7 Lab Report Name: Score: (Detach and turn this sheet along with any other requested work or printouts). Turn in the data in well formatted tables and Excel plots from each part of the procedure. a. Number of Compulsory Misses: Smallest cache size (in bytes) that only had compulsory misses: Bytes b. When you double the blocks but decrease the block size to 4 words, what happens to the # of misses: Are these still all compulsory misses? c. Table with hit rates for all possible block sizes for the given cache size from previous parts. d. Table listing hit rates for progressively smaller caches? e. Table listing hit rates for different associativity (k-values) and corresponding plot. What is the optimal associativity (k-value)? f. Table listing k (where k is the number ways) = {,2,4,8,6} down the vertical axis and the matrix blocking size = 2,, 4 and 6 along the horizontal axis and corresponding plot. g. A few sentences comparing the benefits or costs of a blocked matrix multiply vs. a non-blocked implementation (i.e. compare the results in e and f). 2. Turn in answers to the review questions. Last Revised: 6/7/204 5

6 8 Grading Rubric Name: Score: Student Name: Item Outcome Score Max. ness A: Number of compulsory misses A: cache size B: answer & C: 2 C: Plot/formatting D: hit rate E: data 2 E: plot E: Associativity F: 2 F: Plot Discussion (=correct & obvious demonstration of clear understanding/reasoning, 2 = generally correct with some lack of reasoning/understanding, = little demonstration of mastery, 0 = demonstration of mastery) Requirement G Review Problem Review Problem 2 / / / Late Deductions Open Ended Comments: SubTotal 25 Total Req. / Mult Score 4 (Excellent) (Good) 2 (Poor) (Deficient) (0) Failure Guideline Blocked Works Works usually Fails in several Does not work Not 6 Last Revised: 6/7/204

7 Matrix Multiply Req. a. correctly all the time but fails in -2 cases Solution cases but not always Major Errors implemented b. 0.5 Solution Solution c d e Answer f and g Review Problem Review Problem 2 TOTAL 0.5 Plot and ly Formatted 0.5 Plot and ly Formatted 0.5 Plot and ly Formatted 2, Wellformatted plot and insightful good good is correct but plot is not formatted as specified is correct but plot is not formatted as specified is correct but plot is not formatted as specified data, but formatting or is only adequate adequate adequate or formatting or answer or answer or Last Revised: 6/7/204 7