Verification of Cache Coherency Formal Test Generation

Transcription

1 Dr. Monica Farkash NXP Semiconductors, Inc. EE 382M-11, Department of Electrical and Computer Engineering The University of Texas at Austin 1 Cache Coherency Caches and their coherency Challenge Verification of Cache Coherency Formal Test Generation 2 ECE department, University of Texas at Austin 1

2 Data storage to service future requests for that data Hardware implements cache as a block of memory for temporary storage of data likely to be used again CPUs and hard drives frequently use a cache, as do web browsers and web servers For our purpose we will assume a cache is a hardware block 3 Cache Coherence (coherency) refers to the consistency of data stored in local caches of a shared resource Clients memory contents is maintained in caches Same memory locations stored in multiple local caches Coherency plans give a way to maintain data correctness between accesses of these shared locations Cache coherence is intended to manage such conflicts and maintain consistency between cache and memory 4 ECE department, University of Texas at Austin 2

3 Various models and protocols have been devised for maintaining cache coherence Choice of the consistency model is crucial to designing a cache coherent system Coherence models differ in performance and scalability; each must be evaluated for every system design Protocol Examples MSI protocol MESI MOSI MOESI MERSI MESIF write-once Synapse Berkeley Firefly Dragon protocol 5 6 ECE department, University of Texas at Austin 3

4 The letters of the protocol name identify the possible states in which a cache line can be For MSI, each block contained inside a cache can have one of three possible states M: The block has been modified in the cache The data in the cache is then inconsistent with storage (e.g memory) A cache with a block in the M state needs to write the block to the storage when that block is evicted S: This block is unmodified and exists in at least one cache The cache can evict the data without writing it to the backing store I: This block is invalid The block must be fetched from memory or another cache if the block is to be stored in this cache 7 Cache Another Cache M: The block has been modified in the cache The data in the cache is then inconsistent with storage (e.g memory) A cache with a block in the M state needs to write the block to the storage when that block is evicted Invalid Event Modified S: This block is unmodified and exists in at least one cache The cache can evict the data without writing it to the backing store Shared I: This block is invalid The block must be fetched from memory or another cache if the block is to be stored in this cache Protocol Representation State Meaning 8 ECE department, University of Texas at Austin 4

5 Cache Another Cache If the block is in the M or S states, the cache supplies the data If the block is not in the cache (in the I state), it must verify that the line is not in the M state in any other cache. Different caching architectures handle this differently: bus architectures often perform snooping, where the read request is broadcast to all of the caches Other architectures include cache directories which have agents (a.k.a. directories) that know which caches last had copies of a particular cache block If another cache has the block in the M state, then that cache must write back the data to the backing store and go to the S or I states Once any M line is written back, the cache obtains the block from either the storage, or another cache with the data in the S state. The cache can then supply the data to the requestor, Protocol Representation Behavior Explained: READ REQUEST 9 Cache Another Cache Protocol Representation When a write request arrives at a cache for a block for a block in the M state, the cache writes and thus modifies the data locally block is in S:the cache must notify all other caches that might contain the block in the S state to evict the block. Then the data may be locally modified The notification may be via bus snooping or a directory block is in I: the cache must notify any other caches that might contain the block in the S or M states that they must evict the block If the block is in another cache in the M state, that cache must either write the data to the storage or supply it to the requesting cache If at this point the cache does not yet have the block locally, the block is read from the storage before being modified in the cache After the data is modified, the cache block is in the "M" state Behavior Explained: WRITE REQUEST 10 ECE department, University of Texas at Austin 5

6 A state machine for the Read and Write Operations If a pair of cache blocks are found in one of the X then it is a bug Usage: Formal Methods: Can I ever get into one of the X states? Checkers (Assertions): While running tests do I ever get into one of the X combinations ECE department, University of Texas at Austin 6

7 13 14 ECE department, University of Texas at Austin 7

8 Cache Coherency ONE ADDRESS!!! Data in the system is coherent Everybody will see the latest version for the data for that address in the memory Memory Consistency TWO ADDRESSES!!! The order in which changes are seen by different cores Different CPUs can see different order in which changes happen 15 Sequential Mem consistency Model Initial condition: A=0 B=0 (1) A = 1 (2) print (B) (3) B = 1 (4) print (A) Potential interleaving of operations: (1) (2) (3) (4) => is printed 01 (3) (4) (1) (2) => is printed 01 (1) (3) (2) (4) => is printed 11 (3) (1) (2) (4) => is printed 11. NEVER ECE department, University of Texas at Austin 8

9 Initial condition: A=0 B=0 (1) A = 1 (2) print (B) (3) B = 1 (4) print (A) There is no reason to wait with the print until the write A=1 reaches the memory => you can see 00 Weak order ARM - Collier tests Collier tests (ARM) 17 Formal Methods Protocol Formal Validation Proprietary protocols Develop a model of the protocol! High level Deadlock /Livelock Protocol Implementation Validation The implemented version extract the model Run rules Equivalence checking protocol <-> implementation 18 ECE department, University of Texas at Austin 9

10 Testing Generate tests to cover cache dedicated scenarios Run tests (simulation/emulation) Check correctness Metric coverage 19 Concurrency One test for each client act independently Strict ordering of (some) events to generate desired scenarios Generation offline: need to control the order of events during run-time Cache Another Cache 20 ECE department, University of Texas at Austin 10

11 Take advantage of known state to generate interesting scenarios TCG Multi-core => stop other activity DUT Delays testbench 21 Runtime controls DUT Control over what is being run testbench Complex scenarios (proactive not reactive) Coverage of scenarios small suite Accountability Runtime controls to partially order events 22 ECE department, University of Texas at Austin 11

12 Test contents Random generation efficiency issue Scenario = sequence of events cache dedicated scenario = cover the protocol PSWG (maybe) future language Client 1 Client 2 23 Local Knowledge Low Level SOC COMPLEXITY OF THE MODEL USED IN TEST/STIMULI GENERATION Specification + Executable model ge n g g e e g n n e n gen ge n gen gen GRAPH- BASED /Obsidian /GenesysPro One single model Contains ALL relevant info - Choices/Decisions - Test Contents/Consequences /RAPTOR ECE department, University of Texas at Austin 12

13 Testbench valid cond GRAPH BASED Executable + Spec model COMPLEXITY OF THE MODEL USED IN TEST/STIMULI GENERATION ge n g e n g eg ne n gen TREK /RAPTOR gen ge n gen Perspec System Verifier /Obsidian Questa infact /GenesysPro Different methods of generation Graph based (Cadence, Mentor Graphics, Breker) Path through the graph = test decisions Adapting the graph generation to protocol needs Need long tests Many passages 26 ECE department, University of Texas at Austin 13

14 TCG - Depth 27 A company might have several types of TCG Depends on the level (unit-subsystem-system) Eg. major types TCG on the same product All are constraint solvers at some level Behavioral model based (C++ model) Obsidian (ARM)/ GenesysPro (IBM) / Raptor (Freescale) - requires specialization CPU processors Very complex all instruction sets specific Graph Based Can be extended to cover CPU Easier to use Reactive (runtime) testbench 28 ECE department, University of Texas at Austin 14

15 Testing Generate tests to cover cache dedicated scenarios Challenges Concurrency Scenarios Depth Tool types (graph) - companies Run tests (simulation/emulation) Check correctness Metric coverage 29 TCG <-> Checking needs TCG self checking tests Runtime controls TCG online can immediately validate result Random contents (?) =>Test independent Checking Test independent Checking Online (runtime) runs with the HW & test Passive observer reads values while HW executes Decides if what it sees is correct What means sees? What means correct? Different names/implementation Everybody has it in one shape or form!!! 30 ECE department, University of Texas at Austin 15

16 A Trace-Driven Validation Methodology for Multi-Processor SoCs Jay Bhadra a.o test Pseudo-random sequences User preferences Reference MP System run beh. model MP model (RTL/gates/C model /full-chip/block level) Comparison and analysis pass, fail, reason run implementation Problem Cannot fully check pseudo-random MP programs with data races > static vs. dynamic static scenario core0 #1: a = 11; #2: b = 33; core0 #1: a = 11; #2: b = 33; core1 #3: c = 22; #4: d = 44; dynamic scenario core1 #3: a = 22; #4: b = 44; 32 ECE department, University of Texas at Austin 16

17 tdvm uses abstraction Takes a global view of the MP SoC Specification employs MP abstraction > Abstract C++ reference model not cycle-accurate Implementation TM Slide 6 employs abstraction via traces > MP models are simulated and abstract traces are obtained > Uses abstract information on coherency events CPU instructions, bus transactions Performs assume-guarantee > Aims at detecting MP related system-level bugs 33 stimuli simulation environment MP SoC model (RTL) Tracers Abstract simulation traces (impl) Global order calculator Abstract MP SoC in C++ (spec) Expected res. generator Checker tdvm environment FAIL PASS Abstract error trace 34 ECE department, University of Texas at Austin 17

18 core0 core1 #1: a=11; #3: a = 22; #2 b=33; #4: b = 44; simulation environment stimuli MP SoC model (RTL) Tracers Abstract simulation traces (impl) Global order calculator Abstract MP SoC in C++ (spec) Expected res. generator Checker tdvm environment FAIL PASS Abstract error trace 35 core0 core1 #1: a=11; #3: a = 22; #2 b=33; #4: b = 44; stimuli simulation environment MP SoC model (RTL) Tracers Abstract simulation traces (impl) a = 22, b = 44 Global order calculator Abstract MP SoC in C++ (spec) Expected res. generator Checker tdvm environment FAIL PASS Abstract error trace 36 ECE department, University of Texas at Austin 18

19 core0 core1 #1: a=11; #3: a = 22; #2 b=33; #4: b = 44; stimuli simulation environment MP SoC model (RTL) Tracers Abstract simulation traces (impl) a = 22, b = 44 order = 1; 3; 4; 2 Global order calculator Abstract MP SoC in C++ (spec) Expected res. generator Checker tdvm environment FAIL PASS Abstract error trace 37 core0 core1 #1: a=11; #3: a = 22; #2 b=33; #4: b = 44; stimuli simulation environment MP SoC model (RTL) Tracers Abstract simulation traces (impl) a = 22, b = 44 order = 1; 3; 4; 2 Global order calculator Abstract MP SoC in C++ (spec) Expected res. generator Checker tdvm environment a = 22, b = 33 FAIL PASS Abstract error trace 38 ECE department, University of Texas at Austin 19

20 LSU L1$ CPU tracer tdvm Abs sim CPU tracer LSU L1$ L2$/BIU traces L2$/BIU core0 Bus core1 tracer System memory bus Coherency bus MP SoC platform manager 39 stimuli simulation environment MP SoC model (RTL) Tracers Abstract simulation traces (impl) Global order calculator Abstract MP SoC in C++ (spec) Expected res. generator Checker tdvm environment FAIL PASS Abstract error trace 40 ECE department, University of Texas at Austin 20

21 Instruction time stamps issue time (it), internal perform time (ipt), perform time (pt), completion time (ct) LOAD Instruction: {issued} it = t1 {l1 hit} pt = t2 {l1 miss l2 hit} pt = t3 {l2 miss bus req issued} {trans visible} pt = t5 {trans done} {data access} {instr complete} ct = t8 t1 t2 t3 t4 t5 t6 t7 t8 41 Instruction time stamps issue time (it), internal perform time (ipt), perform time (pt), completion time (ct) STORE Instruction: {issued} it = t1 {allocate store Q} ipt = t2 {instr complete} ct = t3 {l1 hit} pt = t4 {l1 miss l2 hit} pt = t5 {bus {trans req visible} issued} pt = t7 {bus trans done} {data updated} t1 t2 t3 t4 t5 t6 t7 t8 t9 42 ECE department, University of Texas at Austin 21

22 Bus transaction time stamps bus issue time (bit), bus presentation time (prt), bus perform time (bpt), bus completion time (bct) BUS Transaction: {issued} bit = t1 {trans presented to cores devices} prt = t2 {response obtained} {trans visible} bpt = t4 {data obtained or no data} bct = t5 t1 t2 t3 t4 t5 The global order of events can be computed by sorting instruction and bus transaction time-stamps 43 stimuli simulation environment MP SoC model (RTL) Tracers Abstract simulation traces (impl) Global order calculator Abstract MP SoC in C++ (spec) Expected res. generator Checker tdvm environment FAIL PASS Abstract error trace C++ spec ECE department - UT at Austin 44 ECE department, University of Texas at Austin 22

23 plat1 plat2 plat3 Platform (mp-soc) C++ spec Represents inheritance ECE department - UT at Austin 45 device coherency processor p1 bus p2 bus c1 c2 c3 C++ spec Represents inheritance ECE department - UT at Austin 46 ECE department, University of Texas at Austin 23

24 device coherency processor p1 bus p2 bus c1 c2 {1} {2+} c3 plat1 plat2 plat3 Represents inheritance platform C++ spec Represents composition ECE department - UT at Austin 47 device coherency processor p1 bus p2 bus c1 c2 {1} {2+} c3 plat1 plat2 plat3 Represents inheritance platform C++ spec Represents composition ECE department - UT at Austin 48 ECE department, University of Texas at Austin 24

25 device coherency processor p1 bus p2 bus c1 c2 {1} {1+} c3 {1+} plat1 plat2 plat3 Represents inheritance platform C++ spec Represents composition ECE department - UT at Austin 49 coherency event bus op instruction p1 bus op p2 bus op presentation PowerPC core other core read flush pre_ld syn clean write load add originator target snooper C++ spec ECE department - UT at Austin 50 ECE department, University of Texas at Austin 25

26 Coherency checks static or dynamic validation checks for both true or false sharing platforms Barrier order check i1 ; i2; b ; i3 ; i4 Collision order check Producer/consumer order check Completion order check checks for in-order execution Matching transaction check matches load/store/touch instructions with bus transactions Mutex check checks reservation granules for each thread Synchronization check pairs up all sync instructions with sync bus transactions 51 Testing Generate tests to cover cache dedicated scenarios Challenges: concurrency, scenarios, depth Tool types (graph) - companies Run tests (simulation/emulation) Check correctness Challenges: Independent trace based checker internal model Metric/coverage 52 ECE department, University of Texas at Austin 26

27 Test Generator Coverage Accountability Coverage on the generation model (graph) Ex. Test x {Graph Transitions} Coverage per transition Overall suite coverage Run-time monitoring Run-time events coverage (Assertions) Implementation coverage Source code coverage RTL functional / toggle 53 Caches Protocols Protocol Validation Formal Testing Generation Run Check correctness Metric/coverage 54 ECE department, University of Texas at Austin 27