Testbench and Simulation

Transcription

1 Testbench and Simulation Graphics: Alexandra Nolte, Gesine Marwedel, 2003

2 Focus of this Class Understand the simulator Event Based Simulation Testbenches and verification Approaches and metodologies Examples:

3 Introduction Purpose of Verification: Discover as many potential bugs in the design as reasonable before sending chip out for fabrication Do this by simulating chip (and chip components) in Verilog Why is verification important? Chip fab might cost $4M and take 8 weeks Very expensive and time consuming to iterate chip fab! Want to get prototype correct in one to two fab cycles FPGAs can rely more on using the prototype for debug But, note, it is more difficult to debug hardware than a simulation

4 Introduction Verification consumes more than 60% of design resources People, compute cycles Verification mainly done with pre synthesis code Though some simulation, and other checks, are done to make sure the netlist is correct With increased reuse of existing Intellectual Property ( IP ), verification has become very challenging IP = Predesigned blocks, internally developed, purchased or obtained from open source Debugging is often harder than design! Focus of these Notes Primarily on verification tasks likely to be performed by module level designer, and code constructs commonly used

5 Verification Tools and Methods It is impossible to know that you have eliminated all the bugs in a design Thus it is important to use a variety of tools, techniques and methods that give you a high probability of discovering bugs And to have a plan to apply them! Get as many avenues of attack as possible Available tools and methods include: Simulation through test fixtures Including mixed level simulation Inserting and tracking assertions Formal verification Emulation

6 Simulations Through Test Fixtures Basic concept: Apply vectors to design as stimulus Observe outputs, and internal nodes, for correct functionality Key Questions: Where to you get the vectors? How do you observe the outputs? What are the available coding styles?

7 Sources of Verification Vectors. From expected functionality Vectors designed to exercise expected functions of chip or block From specification or understanding of function of chip/block Prioritized from must work to would like to work 2. From Higher Level Model Obtain vectors for individual blocks from a higher level behavioral model E.g. C model developed for project Example: Run video stream through C model of MPEG encoder Extract examples from this to run through Motion Estimator C model here is an example of a reference behavioral model

8 Sources of Simulation Vectors 3. Vectors added specifically as a result of production of verification plan E.g. Vectors specifically designed to test difficult aspects of design Features that were hard to design Modules are more likely to be buggy E.g. Bus arbiters E.g. vectors designed to increase the coverage of the design Increase code and functional coverage 4. Random vectors Run random vectors Compare results with same vectors run in a higher level model

9 Sources of Simulation Vectors 5. System level vectors simulating the chip in its entirety Important to do a LOT of this Very slow and time consuming While design is incomplete, can be a mixed behavioral (e.g. C) and RTL simulation Using Verilog Programming Language Interface (PLI) Requires good behavioral models for interface chips Memories, etc.

10 Observing Correctness. Observe in Waveform Viewer 2. Observing results of assertions 3. Try to write self checking test fixtures, that analyze the results and inform you of correctness. Useful as it means you can automatically check other parts of a design when you redesign some portion. #0 dec = ; #28 if (zero == b) $display ( Check passed ) else $display ( Error: Check FAILED ); Try to take to a higher level. i.e. Incorporate `understanding of function into self checking feature integer testdata; // test data being used integer ExpectedDelay; // expected delay for test data initial begin testdata = 4; in = testdata;... ExpectedDelay = testdata * 0; #ExpectedDelay if (zero == b) $display ( Check passed ) else $display ( Error: Check FAILED );

11 Verilog Code for Test Fixtures Approaches Can use any syntactically correct code Choose test vector generation approach: On the fly generation: Use continuous loops for repetitive signals Use simple assignments for signals with few transitions (e.g. reset) Use tasks to generate specific waveform sets Read vectors stored as constants in an array Read vectors from a file Choose timing approach: Relative Timing, or Absolute Timing Generate clock separately from vectors Whenever possible check simulation results within test fixture Against a stored set of expected results, or Against an internal model of expected behavior

12 Examples On the fly generation Use a task to generate an often repeated vector set task refresh; // generate a RAS before CAS refresh cycle output RAS, CAS; begin // assume RAS and CAS high on entry #5 RAS = 0; #5 RAS = ; #0 CAS = 0; #5 CAS = ; #45; // allow refresh to complete end initial begin refresh (RAS, CAS);

13 Assertions Code blocks that check for correct and incorrect behavior Put inside RTL code (but do not synthesize) Usually inserted by designer System Verilog allows more concise assertions, but can also be written in normal Verilog Example (Verlog95): // synopsys off ifdef Assertions_on // check ONE bus request granted ONE clock cyle after any reqest clock) if (( request) & (~ grant)) // request, no active grants clock) // wait one cycle if (~ grant) $display( ERROR: bus access not granted ); else if ((grant[0] + grant[] + grant[2] + grant[3])>) $display ( ERROR: multiple buses accesses granted ); end endif // synopsys on

14 Assertion Types Concurrent: the property must be true throughout a simulation Procedural: incorporated in procedural code apply only for a limited time 4

15 Procedural Assertions Immediate : at the time of statement execution Strobed : schedule the evaluation of the expression for the end of current timescale to let the glitches settle down Not in functions Clocked : being triggered by an event or sampling clock 5

16 Immediate assertions Severity level $fatal : terminates the simulation with an error code. The first argument shall be consistent with the argument to $finish. $error $warning $info assert (myfunc(a,b)) count = count + ; else >event; [ identifier : ] assert ( expression ) [ pass_statement ] [ else fail_statement ] 6

17 System Functions $onehot (<expression>) returns true if only one and only one bit of expression is high. $onehot0(<expression>) returns true if at most one bit of expression is low. $inset (<expression>, <expression> {, <expression> } ) returns true if the first expression is equal to at least one of the subsequent expression arguments. $insetz(<expression>,<expression> {, <expression> } ) returns true if the first expression is equal to at least other expression argument. $isunknown(<expression>) returns true if any bit of the expression is x. 7

18 Assertion example a sequence of conditions that span multiple clock cycles 8

19 Formal Verification Equivalency Checking Determines that two designs are logically equivalent Examples: RTL and netlist Different netlists after non design coding changes Often used to help verify output of synthesis Model Checking Trying to prove or disprove that a circuit possesses a property that is part of a more abstract, higher level specification E.g. Correct design capture of a Finite State Automata Requires good capture of specification in a suitable language

20 Simulation Engines There are never enough simulation cycles to complete verification. Event based Verilog simulator Most general but slowest 2. Cycle based Verilog simulator Slightly less general but faster 3. Verilog simulator hardware accelerator Use hardware as a co processor to accelerate simulation of Verilog (that does not have a lot of I/O i.e. not all signals captured) 4. Emulation i.e. Build a multi FPGA system that can emulate the standard cell ASIC, though at a slower clock rate Allows very complete verification (except for timing critical issues) but takes a lot of engineering resource

21 Verification Metrics How do you know your chip is ready for fabrication? You can never know you are bug free! General solution: When cost (and opportunity cost) of more verification is higher than the cost of using the first silicon to complete the debug process i.e. When it is quicker and cheaper to build the chip to find the remaining bugs Note: Some bugs can be worked around with firmware Common Metrics:. Bug discovery rate 2. Code coverage 3. Functional Coverage 4. Assertion coverage

22 Verification Metrics Code Coverage Has every line of code been simulated? What percentage of possible paths have been simulated? E.g. All alternatives in an if then sequence What percentage of possible state sequences have been simulated? Requires instrumentation of code and appropriate data collecting and reporting tools Functional Coverage Have all the functions in the specification been simulated? E.g. All interface modes in a USB interface Requires writing of code (SystemVerilog or integrated via PLI) to monitor the hardware that implements these functions and data collecting within the test fixture

23 Verification Environment Definitions Creates stimulus Executes transactions Supplies data to the DUT Test Transactor Driver Checks correctness Scoreboard Assertions Testbench Verification Environment Checker Monitor Identifies transactions Observes data from DUT DUT

24 Coverage Driven Verification Measure progress using functional coverage With VIP Coverage-Driven Methodology Productivity gain % Coverage Goal Self-checking random environment development time Directed Methodology Time

25 Graphics: Alexandra Nolte, Gesine Marwedel, 2003 Universität Dortmund Event Based Simulation

26 Analog Circuit Simulation Divide time into slices Update information in whole circuit at each slice Used by SPICE Allows detailed modeling of current and voltage Computationally intensive and slow Don t need this level of detail for most digital logic simulation 26

27 Digital Simulation Could update every signal on input change Could update just the full path on input change Don t even need to do that much work!

28 Event Driven Simulation When an input to the simulating circuit changes, put it on a changed list Loop while the changed list isn t empty: Remove a signal from the changed list For each sink of the signal Recompute its new output(s) For any output(s) that have changed value, add that signal to the changed list When the changed list is empty, keep simulation results until next input change 28

29 Simulation Update only if changed Some circuits are very large Updating every signal => very slow simulation Event driven simulation is much faster! 29

30 Graphics: Alexandra Nolte, Gesine Marwedel, 2003 Universität Dortmund Testbenches

31 Simulation of Verilog Need to verify your design Unit Under Test (UUT) Use a testbench! Special Verilog module with no ports Generates or routes inputs to the UUT Outputs information about the results Inputs UUT Outputs (Response) OR Stimulus Inputs UUT Outputs (Response) Testbench Testbench

32 Simulation Example module adder4b (sum, c_out, a, b, c_in); input [3:0] a, b; input c_in; output [3:0] sum; output c_out; assign {c_out, sum} = a + b + c_in; endmodule a[3:0] b[3:0] c_in 4 4 adder4b 4 sum[3:0] c_out 32

33 Simulation Example Testbenches frequently named t_<uut name> t_adder4b a[3:0] b[3:0] c_in 4 4 adder4b (UUT) 4 sum[3:0] c_out 33

34 Example `timescale ns /00ps // time_unit/time_precision module t_adder4b; reg[8:0] stim; // inputs to UUT are regs wire[3:0] S; // outputs of UUT are wires UUT wire C_out; all inputs grouped into // instantiate UUT single vector (not adder4b(s, C_out, stim[8:5], stim[4:], stim[0]); required) Behav. Verilog: do this once // stimulus generation initial begin end endmodule not an apostrophe! stim = 9'b ; // at 0 ns #0 stim = 9'b0000; // at 0 ns #0 stim = 9'b0000; // at 20 ns #0 stim = 9'b0000; // at 30 ns #0 stim = 9'b0000; // at 40 ns #0 $stop; // at 50 ns stops simulation timing control for simulation see response to each of these input vectors 34

35 Overview Simulation and Testbenches Simulation Overview Timing Controls and Delay Testbenches Verilog Shortcuts Concatenating Vectors Arrays of Instances Introduction to RTL Verilog (Continuous Assignments) 35

36 Timing Controls For Simulation Can put delays in a Verilog design Gates, wires, even behavioral statements! SIMULATION Used to approximate real operation while simulating Used to control testbench SYNTHESIS Synthesis tool IGNORES these timing controls Cannot tell a gate to wait.5 nanoseconds! Delay is a result of physical properties! The only timing that can be (easily) controlled is on a clockcycle basis Can tell synthesizer to attempt to meet cycle time restriction 36

37 No Delay vs. Unit Delay When no timing controls specified: no delay Unrealistic even electrons take time to move OUT is updated at same time A and/or B change: and(out, A, B) Unit delay often used Not accurate either, but closer Depth of circuit does affect speed! Easier to see how changes propagate through circuit OUT is updated unit after A and/or B change: and # A0(OUT, A, B); #0 delay Don t use this. 37

38 Unit Delay Example A B C Y Z No Delay: Y and Z change at same time as A, B, and C! Unit Delay: Y changes unit after B, C Unit Delay: Z changes unit after A, Y T A B C Y Z No Delay T A B C Y Z Unit Delay 0 0 x x x

39 Types Of Delays Inertial Delay (Gates) Suppresses pulses shorter than delay amount In reality, gates need to have inputs held a certain time before output is accurate This models that behavior Transport Delay (Nets) Time of flight from source to sink Short pulses transmitted 39

40 Delay Examples wire #5 net_; // 5 unit transport delay and #4 (z_out, x_in, y_in); // 4 unit inertial delay assign #3 z_out = a & b; // 3 unit inertial delay wire #2 z_out; // 2 unit transport delay and #3 (z_out, x_in, y_in); // 3 for gate, 2 for wire wire #3 c; // 3 unit transport delay assign #5 c = a & b; // 5 for assign, 3 for wire 40

41 Delays In Testbenches Most common use in class A single testbench will test many possibilities Want to examine each case separately Spread them out over time Use to generate a clock signal Example later in lecture 4

42 Overview Simulation and Testbenches Simulation Overview Timing Controls and Delay Testbenches Introduction to RTL Verilog (Continuous Assignments) 42

43 Testbench Basics Instantiate the unit being tested (UUT) Provide input to that unit Usually a number of different input combinations! Watch the results (outputs of UUT) Can watch ModelSim Wave window Can print out information to the screen or to a file 43

44 Print Test Information A number of system calls to output info $monitor Give a list of nets and variables to monitor Output the given values every time a one of them changes $display, $strobe Output a value at a specific time during simulation Can use formatting strings with these commands Also have system calls that write to files Only have meaning in simulation System calls are ignored in synthesis 44

45 Output String Formatting Formatting string %h, %H hex %d, %D decimal %o, %O octal %b, %B binary %t time $monitor( %t: %b %h %h %h %b\n, $time, c_out, sum, a, b, c_in); Can get more details from Verilog standard 45

46 Output Example `timescale ns /00ps module t_adder4b; reg[8:0] stim; wire[3:0] S; wire C4; // time_unit/time_precision // inputs to UUT are regs // outputs of UUT are wires // instantiate UUT adder4b(s, C4, stim[8:5], stim[4:], stim[0]); // monitor statement initial $monitor($time, C4, S, stim[8:5], stim[4:], stim[0]); // stimulus generation initial begin end endmodule All values will run together; easier to read with formatting string stim = 9'b ; // at 0 ns #0 stim = 9'b0000; // at 0 ns #0 stim = 9'b0000; // at 20 ns #0 stim = 9'b0000; // at 30 ns #0 stim = 9'b0000; // at 40 ns #0 $stop; // at 50 ns stops simulation 46

47 Output Example `timescale ns /00ps module t_adder4b; reg[8:0] stim; wire[3:0] S; wire C4; // time_unit/time_precision // inputs to UUT are regs // outputs of UUT are wires // instantiate UUT adder4b(s, C4, stim[8:5], stim[4:], stim[0]); // monitor statement initial $monitor( %t %b %d %d %d %b, $time, C4, S, stim[8:5], stim[4:], stim[0]); // stimulus generation initial begin end endmodule Formatted with spaces between values, vectors shown as decimal stim = 9'b ; // at 0 ns #0 stim = 9'b0000; // at 0 ns #0 stim = 9'b0000; // at 20 ns #0 stim = 9'b0000; // at 30 ns #0 stim = 9'b0000; // at 40 ns #0 $stop; // at 50 ns stops simulation 47

48 Exhaustive Testing For combinational designs with few inputs Test ALL combinations of inputs to verify output Could enumerate all test vectors, but don t Generate them using a for loop! reg [4:0] x; initial begin for (x = 0; x < 6; x = x + ) #5 // need a delay here! end Need to use reg type for x. Why? It s a variable assigned a value in a behavioral block What would happen without the delay? 48

49 Why Loop Vector Has Extra Bit Want to test all vectors 0000 to reg [3:0] x; initial begin for (x = 0; x < 6; x = x + ) #5 // need a delay here! end If x is 4 bits, it only gets up to => 5 00 => 0 => 0 => => 0000 => 000 x is never >= 6 so loop goes forever! 49

50 Exhaustive Example: UUT module Comp_4_str(A_gt_B, A_lt_B, A_eq_B, A, B); output A_gt_B, A_lt_B, A_eq_B; input [3:0] A, B; // Code to compare A to B and set outputs // A_gt_B, A_lt_B, A_eq_B accordingly endmodule 50

51 Exhaustive Example: Testbench module t_comp_4_str(); wire A_gt_B, A_lt_B, A_eq_B; reg [4:0] A, B; // sized to prevent loop wrap around wire [3:0] A_bus, B_bus; assign A_stim = A[3:0]; // display only 4 bit values assign B_stim = B[3:0]; Comp_4_str M (A_gt_B, A_lt_B, A_eq_B, A_stim, B_stim); // UUT initial $monitor( %t A: %h B: %h AgtB: %b AltB: %b AeqB: %b, $time, A_stim, B_stim, A_gt_B, A_lt_B, A_eq_B); initial #2000 $finish; // end simulation, quit program note multiple initial blocks initial begin #5 for (A = 0; A < 6; A = A + ) begin // exhaustive test of valid inputs for (B = 0; B < 6; B = B + ) begin #5; // may want to test x s and z s end // first for end // second for end // initial endmodule 5

52 Generating Clocks Hard way: initial begin #5 clk = 0; #5 clk = ; #5 clk = 0; (repeat hundreds of times) end Better way: initial begin clk = 0; forever #5 clk = ~clk; end 52

53 FSM Testing Response to input vector depends on state For each state: Check all transitions For Moore, check output at each state For Mealy, check output for each transition This includes any transitions back to same state! Can be time consuming to traverse FSM repeatedly 53

54 Example : Gray Code Counter Write a 3 bit Grey Code counter Write a testbench to test a 3 bit gray code counter. module gray_counter(out, up down, en, clk, rst_n); In this module, rst_n is asynchronous active low en input is used to enable the counter: if 0 the counter is freezed up_down input signal is used to choose to increment or decrement ( increment, 0 decrement) 54

55 Gray Code Counter Only bit can switch at the same time!!!! Widely used in asynchronous blocks (avoid hazard and glitches) IN OUT How to implement it? Counter + Binary to Grey encoder

56 Gray Code Counter + sel Up-down 3 bit FF Binary To Gray clk rst_n en en

57 Gray Code Counter Counter reg [2:0] counter; clk, negedge rst_n) begin if(rst_n== b0) counter <= 3 b000; else if(en == b) begin case(sel) b: counter <= counter+ b; b0: counter <= counter- b; default: counter+ b; endcase end else; end

58 Gray Code Counter + sel 3 bit FF Binary To Grey clk Up-down rst_n en en Sequential And Comb Comb

59 Gray Code Counter Encoder Truth table IN = [A,B,C] OUT out[0] out[] out[2] AB C Karnaugh Maps AB C AB C out[2] = A out[] = AB +A B = A^B OUT = (IN >> ) ^ IN; out[0] = BC +B C = B^C

60 Gray Code Counter Encoder counter[2] counter[] counter[0] counter_out[2] counter_out[] counter_out[0]

61 Gray Counter : Code module GreyCounter ( input wire input wire input wire input wire output wire [2:0] ) reg [2:0] clk, rst_n, sel, en, counter_out counter; clk, negedge rst_n) begin if(rst_n== b0) counter <= 3 b000; else if(en == b) begin case(sel) b: counter <= counter+ b; b0: counter <= counter b; default: counter+ b; endcase end else; end assign counter_out = (counter>>) ^ counter; endmodule

62 Testing Gray Counter stim Test Fix up-down en DUT counter_out_dut clk rst_n DUT == Error Golden Model counter_out_gm

63 Be careful not to change your inputs on the active clock edge of your flipflops! Solution : Gray Code Counter Test module t_gray_counter(); wire [2:0] out_gc, out_gm; reg clk, rst; reg sel, en; gray_counter GC(clk, rst_n, sel, en, out_gc); golden_model GM(clk,rst_n,sel, en, out_gm); initial $monitor( %t out: %b, sel: %b, en: %b, rst: %b, $time, out, sel, en, rst); initial #00 $stop; // end simulation // UUT // GoldenModel // clk not displayed initial begin clk = 0; forever #5 clk = ~clk; // What is the clock period? end initial begin rst = 0; #0 rst = ; // When does rst change relative to clk); sel = ; en=; end // initial always@(posedge clk) begin if(out_gc!= out_gm) $error( Output Mismach at %t ns, $time); end endmodule 63

64 Simulation: Gray Code Counter Test # 0 out: xxx rst_n: 0 // reset system # 5 out: 000 rst_n: 0 // first positive edge # 0 out: 000 rst_n: // release reset # 5 out: 00 rst_n: // next positive edge # 25 out: 0 rst_n: # 35 out: 00 rst_n: # 45 out: 0 rst_n: # 55 out: rst_n: # 65 out: 0 rst_n: # 75 out: 00 rst_n: # 85 out: 000 rst_n: # 95 out: 00 rst_n: 64

65 Force/Release In Testbenches Allows you to override value FOR SIMULATION Does NOT apply to hardware Can t tell the synthesis tools when you get here, make this value become 3 d5 Synthesizer won t allow force release How does this help testing? Pinpoint bug by controlling other signals Can use with FSMs to override state Force to a state Test all edges/outputs for that state Force the next state to be tested, and repeat Can also use simulator GUI to force values 65

66 Force/Release Example assign y = a & b; assign z = y c; initial begin a = 0; b = 0; c = 0; #5 a = 0; b = ; c = 0; #5 force y = ; #5 b = 0; #5 release y; #5 $stop; end t a b c y z

67 Review Questions What is the difference between transport and inertial delay? When is each used? What does it mean to exhaustively test a design? Why are force/release statements useful? How do you generate a clock with a period of 6 time units? 67