Understanding and Exploring Memory Hierarchies

Transcription

1 Understanding and Exploring Memory Hierarchies Issued : Thursday 27th January 2011 Due : Friday 11th March 2011 at 4.00pm (at the ITO) This assignment represents the total practical component of the Computer Architecture module of Inf3. This practical contributes 25% of the overall mark for the module. It consists of a programming exercise culminating in a brief written report. Assessment of this practical will be based on the correctness and the clarity of the solution (see more details below), and on the completeness and clarity of the written report. This practical is to be solved individually to assess your competence on the subject. Please bear in mind the School of Informatics guidelines on plagiarism. You must return your solutions to the ITO before the due date shown above. This practical is handed out well in advance of the coverage of caching techniques in the lectures. The aim of handing out the practical so early in the semester is to maximise your opportunity for scheduling your time most effectively. You are strongly advised to read up on cache architectures in the course textbook and to start the practical as soon as possible. This will ease your workload approaching the common deadline around the end of week 9. 1 Introduction Program transformation techniques are often used to try to improve the cache hit rate of heavily used programs. A detailed understanding of the cache s dynamic behaviour is usually obtained by software simulation rather than monitoring of hardware. In this exercise you are asked to investigate the memory access behaviour of a pair of code fragments which are related by a simple program transformation. To do this, you will have to write a reasonably flexible cache simulator and also devise a mechanism for generating the address traces which will form its main input. You will need to undertake the following sub-tasks: 1. Generation of address traces. Your experiments will be fuelled by files which list the sequence of memory accesses made by programs. Such files are called traces. The required format of a trace file is described in section Implementation of a cache simulator. You must write a program which accepts a trace file as input and reports upon its behaviour. The cache configuration (number of blocks, size of blocks, associativity and so on) will be described as part of the trace file. The degree of flexibility allowed by your system is open ended, but the minimal credible system would implement a direct-mapped write-allocate write-through cache with the block size and the number of blocks controllable from the trace file. As well as the key statistic of hit-rate, your simulator will offer a selection of other output options, including access by access textual explanation of what s happening and the ability to display interesting aspects of the contents of the cache at any time. 1

2 3. Validation of the simulator. In order that both you and your tutor have high confidence in the validity of your results, you should provide a clear written description of the structure and operation of your simulator and a set of small, artificial examples which use the various output options to demonstrate that your simulator is working correctly. 4. Design and execution of experiments. The aim is to investigate the circumstances under which the transformed version of the test program has better cache behaviour than the original, and the extent of the improvement (the programs are shown in the Appendix). Think carefully about the experiments you run. Experiments on realistically sized caches by today s standards are unlikely to be feasible in practical terms, but the same effects should be visible on much smaller examples. 5. Writing a Report. You must report and discuss the results of your experiments. Did the transformed program outperform the simple original? Under what circumstances (if any)? Make sure that your data is presented for easy consumption (e.g. graphically). 2 Format of your submission Your submission should clearly indicate (in both the report and the code) which cache features you have simulated completely and which you have only partially completed. You should submit hard copy of the following items, neatly bound or otherwise collated and marked clearly with your name and your tutor s name, to the ITO before 4pm on Friday 11th March Source code listing of your simulator program, with explanatory comments. 2. Written description of the internal structure and workings of your simulator (1-2 pages depending upon complexity), explaining clearly which parts are complete and which incomplete. 3. Written description of your trace generation method (a few sentences). 4. Extracts of simple trace files used for validation and a description of expected and actual results on these, in order to convince your tutor that your simulations are accurate (a few pages including examples). 5. A report detailing the experiments you ran and giving a critical summary of the results (2-4 pages including results). You should provide enough information to make the experiments repeatable. Present your data clearly and concisely. 2

3 3 Target Architecture Our imaginary architecture has a very simple main memory model. There is only one data type, the integer, and one kind of addressable memory location, the word, which stores an integer. Data memory addresses start from 0. There is no virtual memory. Thus address 0 indexes the first integer in memory, address 1 the second integer and so on. Our cache blocks will correspond to some number of words (set in the trace file). The main point of this simple model is that your simulator doesn t have to concern itself with byte addressing within words, word alignment and so on. 4 Trace File Format Your simulator must accept its input from trace files which have the format described here. The first part of the trace describes the structure of the cache. This description consists of a sequence of lines, each starting with an integer, optionally followed by some comment text which describes the cache parameter being set (the text is just there for the reader s convenience and should be ignored by the simulator). The lines are as follows: 1. the first line describes the number of blocks in the cache 2. the second line describes the number of words in a cache block 3. the third line describes the set associativity (i.e. the number of blocks per set) 4. any subsequent lines in the first part are free for your own use in the later stages, when you make your simulator more complex, for example describing replacement policies for set-associative schemes (you should describe their use in your report) The second part of the trace contains the sequence of memory accesses to be simulated, one per line, together with commands which control the output generated by the simulator: 1. a memory access line consists of either l or s (indicating load or store respectively) followed by a space and then an integer indicating the memory address to be accessed. NB. We do not specify the actual data value being written, nor the register being used. Thus, your simulator cannot keep track of real data. All that matters in calculating hit rate is the sequence of addresses used. 2. an output control command is one of (a) v, indicating that full line-by-line explanation should be switched on (if it is currently off) or off (if it is currently on). The default is that it is off. 3

4 (b) p indicating that the complete content of the cache should be output in some suitable format (see below) (c) h indicating that the hit-rate achieved so far should be output For example, the following trace file indicates that the simulated cache has 8 words per block, and 16 blocks organised in a direct mapping (otherwise known as 1-way associative ). There is a sequence of six accesses of which the first three must be explained line by line in the output. The cache contents are printed after the fifth access and the hit rate is printed at the end. 16 (blocks in total) 8 (block size) 1 (1 way associative, in other words, direct mapped) v (switch on line by line explanation) l 17 l 29 s 17 v (switch off line by line explanation) l 25 l 35 p (print out the cache contents) s 29 h (print out the hit rate) Please notice that comments can take any form you like! Your simulator should ignore everything which follows the expected values on a line. 5 Output Format Output should be sent to standard output (though it will make sense to direct it to a file for large examples) and comes in four flavours: error messages, indicating that the simulator has found an erroneous input line, or that something silly is being asked for (e.g. associativity higher than cache size, negative addresses and so on) an indication of hit rate so far, triggered by an h input and printed out in some suitable format at your discretion a print out of the cache contents, triggered by a p input and displaying, in some suitable tabular format, everything your simulator knows about the current state of 4

5 the cache. This will include at least the tag and valid bit for the data stored in each block. As noted above, it will not include the real data values, because these are not provided by the trace. a full explanation of the behaviour of an access. This is generated for each access while the simulator is in full explanation mode, and should consist of some text describing what has just happened, for example A read to address 17 looked for tag 0 in block 2 and was a miss. or similar, at your discretion, to give as much information as you have. This will get more complicated and long-winded as you introduce more complex cache structures. Full explanation mode is initially switched off. Its state is toggled by v command. 6 Experiments A source-to-source program transformation known as loop fusion claims to improve the cache hit rate for suitably structured segments of code. Your task is to investigate this claim by experiment. Code fragment examples showing unfused and fused loops are presented in the appendix of this document. You will need to generate access traces for a number of runs of the fragments (e.g. varying N) and investigate their behaviour when simulated on a variety of cache configurations. It is up to you to decide exactly which experiments to conduct. You should be aiming to highlight trends which occur as the various parameters (cache size / structure and array size N) are varied, while focusing on the fundamental question of whether loop fusion seems to be helpful for this example. Note, choosing just one or two candidate values of N may give misleading results. 7 Generating Trace Files For your larger scale experiments it will be infeasible to write traces by hand. Instead, you will have to generate the access sequence sections of a trace by running a suitable adaptation of the source program being traced. Every time the original program would make a memory access, the adapted program should output a line of the trace. For the purposes of this exercise, you should only trace data accesses to the arrays a, b, c, and d. In other words, we are assuming that the compiler has placed the loop control variables i and j in registers and that instructions are cached separately (and are not of interest to us). You should assume that the arrays are laid out in memory one after another in alphabetical order, starting at address 0, with the contents of each array laid out row by row. Thus, a[0,0] is at address 0 and d[n-1,n-1] is at address 4N

6 8 Working in Stages This section contains some suggestions as to how you might tackle the work in stages. There is no obligation to follow these, but your report must make it clear exactly what your simulator can handle. However far you get, remember that one of the aims of the exercise is to experiment with loop fusion, so make sure to leave yourself time to do this. Stage 1 Restrict your simulator to direct mapped caches (in other words the set associativity must be 1 - reject trace files which do not have this property). You can test this out with hand generated traces. Use full explanation mode to verify that your simulator is doing what you think. Write trace files with easily predictable hit rates (e.g. think of a four access trace which uses four different addresses and has a 50% hit rate). Stage 2 For your report, you will need to generate some much longer traces from the given fused and unfused loops. Think about how to annotate these code fragments so that they generate an access trace in the prescribed format. You can easily add the cache parameter section of the trace by hand afterwards. Stage 3 Upgrade your simulator to cater for genuinely set-associative caches (i.e. with set size greater than 1). You could start by assuming some fixed scheme for resolving clashes (e.g. random) and then build in the ability to simulate other schemes (e.g. LRU). Add new commands to the input format to allow these to be specified as part of the trace and refine the output formats to explain their operation. Don t forget to explain what you ve done in your report. 6

7 Appendix - Test Programs a, b, c and d, are all N N arrays of integers. The unfused loops for (i=0; i<n; i++) for (j=0; j<n; j++) a[i][j] = 1/b[i][j] * c[i][j]; for (i=0; i<n; i++) for (j=0; j<n; j++) d[i][j] = a[i][j] + c[i][j]; The fused loops for (i=0; i<n; i++) for (j=0; j<n; j++) { a[i][j] = 1/b[i][j] * c[i][j]; d[i][j] = a[i][j] + c[i][j]; } Nigel Topham, January