CS377P Programming for Performance Single Thread Performance In-order Pipelines

Transcription

1 CS377P Programming for Performance Single Thread Performance In-order Pipelines Sreepathi Pai UTCS September 9, 2015

2 Outline 1 Introduction 2 Pipeline Preliminaries 3 The Inorder Pipeline and Performance Issues 4 Performance Tuning

3 Outline 1 Introduction 2 Pipeline Preliminaries 3 The Inorder Pipeline and Performance Issues 4 Performance Tuning

4 Compute-Bound programs CPU-bound programs (i.e. not I/O-bound) Compute-bound Memory-bound Compute-bound programs have a high ratio of ALU to memory operations (arithmetic intensity) do not wait for memory (data usually fits in cache) are limited by the CPU pipeline benefit the most from changing the CPU

5 Examples of Compute-bound Programs Numerical Software Cryptography Signal-processing Molecular Dynamics Parsers (?)

6 Performance of Compute-Bound Programs Goal: Achieve maximum IPC instructions per cycle property of the pipeline/cpu

7 Outline 1 Introduction 2 Pipeline Preliminaries 3 The Inorder Pipeline and Performance Issues 4 Performance Tuning

8 Example int count(const char *s) { int c = 0; while(*s!= \0 ) { if(*s == A ) c++; } s++; } return c; What determines the performance of the above code?

9 Same length, different number of A s Time (ns) Number of As

10 Same length, different number of A s (with CI) Time (ns) Number of As

11 Same length and number of A s, but different distribution Time (ns) start10k.dat end10k.dat random10k.dat

12 objdump -d d <count>: 40052d: 55 push %rbp 40052e: e5 mov %rsp,%rbp : d e8 mov %rdi,-0x18(%rbp) : c7 45 fc movl $0x0,-0x4(%rbp) 40053c: eb 14 jmp <count+0x25> 40053e: 48 8b 45 e8 mov -0x18(%rbp),%rax : 0f b6 00 movzbl (%rax),%eax : 3c 41 cmp $0x41,%al : jne 40054d <count+0x20> : fc 01 addl $0x1,-0x4(%rbp) 40054d: e8 01 addq $0x1,-0x18(%rbp) : 48 8b 45 e8 mov -0x18(%rbp),%rax : 0f b6 00 movzbl (%rax),%eax : 84 c0 test %al,%al 40055b: 75 e1 jne 40053e <count+0x11> 40055d: 8b 45 fc mov -0x4(%rbp),%eax : 5d pop %rbp : c3 retq

13 gas listing (partial) 4:br.c **** int count(const char *s) { 9.loc cfi_startproc pushq %rbp 12.cfi_def_cfa_offset cfi_offset 6, E5 movq %rsp, %rbp 15.cfi_def_cfa_register DE8 movq %rdi, -24(%rbp) 5:br.c **** int c = 0; 17.loc C745FC00 movl $0, -4(%rbp) :br.c **** while(*s!= \0 ) { 19.loc f EB14 jmp.l2 21.L4: 8:br.c **** if(*s == A ) 22.loc B45E8 movq -24(%rbp), %rax FB600 movzbl (%rax), %eax C41 cmpb $65, %al a 7504 jne.l3 9:br.c **** c++; 27.loc c 8345FC01 addl $1, -4(%rbp)

14 How to Read x86 Assembly in 2 minutes LABEL: instruction operands... where operand may be: immediate: $65 register: %rax implicit: push %rbp memory address: $.LC1 memory reference 1: (%rax) (equivalent to *rax) memory reference 2: -18(%rax) (eq. to *(rax - 18)) many more memory reference formats (see manual)

15 Outline 1 Introduction 2 Pipeline Preliminaries 3 The Inorder Pipeline and Performance Issues 4 Performance Tuning

16 The MIPS 5-stage Pipeline From Wikipedia: Instruction Fetch Instruction Decode Register Fetch Execute Address Calc. Memory Access Write Back IF ID EX MEM WB Next PC Adder Next SEQ PC RS1 RS2 Register File Next SEQ PC Zero? Branch taken MUX PC Memory IR IF / ID Sign Extend Imm ID / EX MUX MUX ALU EX / MEM Memory MEM / WB MUX WB Data

17 Stages Fetch Fetches instructions at (%pc) Increment (%pc) to point to next instruction Decode Identifies instruction Identifies operands (memory/registers/immediate) Execute Performs ALU operations Memory Performs Loads/Stores Writeback Retires instruction Results are visible in register file Stores are forwarded to memory

18 Fetch Performance : 3c 41 cmp $0x41,%al : jne 40054d <count+0x20> : fc 01 addl $0x1,-0x4(%rbp) 40054d: e8 01 addq $0x1,-0x18(%rbp) After fetching 0x400547, where should the next instruction be fetched from?

19 Decode/Execute Performance fdivp fdivp Above code executes: (A/B)/C Decode must handle hazards: Structural: not enough divide units Data: results (from previous instructions) not ready yet

20 Memory Performance mov (%rax), %rbx addl $1, %rbx What if data is not found in cache? Topic to be covered later under Caches

21 Writeback Performance Is this a significant bottleneck for in-order processors?

22 Outline 1 Introduction 2 Pipeline Preliminaries 3 The Inorder Pipeline and Performance Issues 4 Performance Tuning

23 What can you, as a programmer, do? For simple, in-order pipelines: Very little A compiler can nearly always outperform a programmer

24 Eliminating Branches Loop unrolling for(i = 0; i < n; i++) { body; } After unrolling the loop UNROLL times: for(i = 0; i < n - UNROLL; i+=unroll) { body; i += 1; body;... i += UNROLL - 1; body; } for(; i < n; i++) { body; } Compiler can do this: see gcc -funroll-loops

25 Eliminating Branches Inline Code Goal is to produce straight-line code: f() { body_f; } while(cond) { f(); } After inlining: while(cond) { body_f; } Compiler can do this as well: gcc -finline*.

26 Other Fetch Optimizations If convert branches replace branches with conditional/predicated instructions (cmove) (gcc -O0) : 3c 41 cmp $0x41,%al : jne <count+0x20> (gcc -O3) : 80 fa 41 cmp $0x41,%dl : 0f 44 c1 cmove %ecx,%eax Layout hot code nearby use compiler profile-guided optimization All of these optimizations are best done by the compiler. But check that the compiler is doing it!

27 Structural Hazards and Data Dependencies Data Dependencies change order of instructions to minimize waiting for data tweak algorithm if necessary Instruction Scheduling change order of instructions to minimize stalls best done by compiler (see gcc -march -mcpu -mtune) but examine anyway hand-assemble otherwise

28 Conclusion Compute-bound code requires: Steady supply of instructions to pipeline Minimum waiting/stalling for operations Compilers can do a very good job but always examine generated code! Handwriting assembly code is an alternative but time-consuming may as well report a bug to the compiler writers