Acceptance Testing. How CSlim and FitNesse Can Help You Test Your Embedded System. Doug Bradbury Software Craftsman, 8th Light

Transcription

1 Acceptance Testing How CSlim and FitNesse Can Help You Test Your Embedded System Doug Bradbury Software Craftsman, 8th Light 1

2 Tutorial Environment git clone git://github.com/dougbradbury/c_learning.git cd c_learning./bootstrap.sh or with a live CD: cp -R cslim_agile_package c_clearning cd c_learning git pull./bootstrap.sh 2

3 Overview Talk w/ exercises: Acceptance Tests Tutorial: Writing Acceptance tests Tutorial: Fitnesse Tutorial: CSlim Talk: Embedded Systems Integration Bonus Topics 3

4 Introductions Who are you? Where do you work? What experience do you have with... embedded systems? acceptance testing? FitNesse and Slim? 4

5 Objectives As a result of this course you will be able to: Understand the purposes of acceptance testing; Use acceptance tests to define and negotiate scope on embedded systems projects; Integrate a CSlim Server into your embedded systems; 5

6 Objectives (cont) As a result of this course you will be able to: Add CSlim fixtures to your embedded system; Write Fitnesse tests to drive the execution of CSlim fixtures; Write and maintain suites of tests in a responsible manner. 6

7 Points on a star How many points does this star have? 7

8 Star Point Specification Points on a star are counted by the number of exterior points. 8

9 Points on a star How many points does this star have? 9

10 By Example

11 Points on a star Now, how many points does this star have? 11

12 Robo-draw Pick a partner... 12

13 Acceptance Testing Collaboratively producing examples of what a piece of software is supposed to do 13

14 Unit Tests help you build the code right. Acceptance Tests help you build the right code. 14

15 Acceptance Tests A collaborative derivation of scope 15

16 Acceptance Test Living Documentation 16

17 Acceptance Test A medium for communication 17

18 Acceptance Tests Tests 18

19 Acceptance Tests Entire System 19

20 Fail Don t test the entire system 20

21 Acceptance Tests User Interface 21

22 Acceptance Tests Load / Performance 22

23 Acceptance Tests Hardware 23

24 Acceptance Tests Algorithm 24

25 Acceptance Tests QA Tool 25

26 Acceptance Tests Collaboratively produced examples A definition of scope Living documentation A medium for communication 26

27 Why you really need ATs Твой софт - говно! 27

28 Workflow Tests Given, When, Then Preconditions, Action, Results 28

29 Workflow example Given a new car traveling at 30 mph When I fully apply the breaks Then the car should stop within 35 yards 29

30 An example, by example A treadmill controller walkncode.com 30

31 Treadmill Requirements It controls the speed of the treadmill. It increments and decrements the speed. The max speed is configurable. It keeps track of total distance walked. It keeps track of calories burned. 31

32 Ambiguities mph, kph, fps? inc dec amounts? range for max speed? distance in miles, km? formula for tracking calories? 32

33 Treadmill by example Given treadmill speed is 1.0 mph When I increment the speed Then the speed should be 1.1 mph 33

34 Your Turn Write an example for the decrement speed scenario. Write an example for the max speed scenario 34

35 Decrement Speed Given treadmill speed is 1.0 mph When I decrement the speed Then the speed should be 0.9 mph 35

36 Maximum Speed Given treadmill speed is 3.0 mph Given max speed is 3.0 mph When I increment the speed Then the speed should be 3.0 mph 36

37 Calculation Tests A series of inputs and outputs Usually captured in table form 37

38 Division Test numerator denominator quotient?

39 Calories Burned Test speed time calories?

40 Your Turn Write a calculation example for distance traveled. 40

41 Distance Travelled speed time distance?

42 Cumulative Distance speed time distance?

43 When Do you stop? 43

44 Break git clone git://github.com/dougbradbury/c_learning.git cd c_learning./bootstrap.sh or with a live CD: cp -R cslim_agile_package c_clearning cd c_learning git pull./bootstrap.sh 44

45 FitNesse Ecosystem 45

46 Starting FitNesse java -jar fitnesse.jar -p e 0 Options Port: -p 8080 No History: -e 0 46

47 FitNesse Tutorial Follow along! 47

48 It s a wiki 48

49 Editing a page Try typing a WikiWord (at least 2 capital letters) - Then save 49

50 Creating a new page Click on the? to create that new page 50

51 Editing a new page 51

52 Tables Tables in FitNesse are defined with pipes 52

53 Tables Save your table 53

54 User Guide Learn more about the wiki markup language 54

55 Let s write some examples 55

56 Another way to add pages 56

57 Three types of pages Normal - Contain only text and links Test - Executable pages Suite - a SubWiki of tests (or other suites) 57

58 Let s view the new test 58

59 Test Pages Behold, the test button 59

60 Calculation Test Edit this test and create a decision table 60

61 Push the button! Green means passed 61

62 Try some other values Red means failure 62

63 Line up your columns Try the format button 63

64 Workflow Test Create a new test with a script table 64

65 Workflow Test Run 65

66 Workflow failure 66

67 Exercise Create a TreadmillSuite Create the following tests TestIncrementTreadmill (workflow) TestDecrementTreadmill (workflow) TestMaxSpeed (workflow) TestCumulativeDistance (calculation) TestTotalDistance (calculation) 67

68 68

69 69

70 70

71 71

72 72

73 It s Fixture Time cslim/fixtures/fixtures.c 73

74 It s Fixture Time cslim/fixtures/decisiontableexample.c 74

75 Fixture Objects cslim/fixtures/decisiontableexample.c 75

76 Fixture functions cslim/fixtures/decisiontableexample.c 76

77 SlimList cslim/include/cslim/slimlist.h 77

78 Returning a Value cslim/fixtures/decisiontableexample.c All return values are strings. 78

79 Extra functions Decision tables have a few optional functions. 79

80 Order of Execution all setters execute() all getters reset() 80

81 Let s build a fixture CheatSuite.TreadmillControlSuite 81

82 TestIncrementSpeed Exceptions are yellow 82

83 TestIncrementSpeed Let s create a fixture called Treadmill 83

84 Create a Fixture cd treadmill/fixtures/ cp FixtureTemplate.c Treadmill.c sed -i '' s/examplefixture/treadmill/g Treadmill.c 84

85 Register the Fixture fixtures/fixturemain.c 85

86 ReBuild the at target %> make at 86

87 Run the Test again Fixture was found! 87

88 The First Method giventreadmillspeed 88

89 The First Method treadmill/fixtures/treadmill.c 89

90 IncrementSpeed Implement Increment Speed on your own. 90

91 targetspeed Return a value to make the test fail. 91

92 targetspeed Failure 92

93 Treadmill Api include/treadmill/api.h 93

94 Declare an instance treadmill/fixtures/treadmill.c 94

95 Create & Destroy treadmill/fixtures/treadmill.c 95

96 Reading from a SlimList treadmill/fixtures/treadmill.c 96

97 No Parameters treadmill/fixtures/treadmill.c 97

98 Result treadmill/fixtures/treadmill.c 98

99 ReBuild the at target %> make at 99

100 Woot! Green is good 100

101 Exercise Add fixture methods to Treadmill.c to support: TestDecrementSpeed TestMaxSpeed 101

102 Catch up If you need to catch up with the group cp cheat_fixtures/treadmill.c fixtures 102

103 Calculation Test 103

104 Create a new fixture cd treadmill/fixtures/ cp FixtureTemplate.c TreadmillCumulativeDistance.c sed -i '' s/examplefixture/ TreadmillCumulativeDistance/g TreadmillCumulativeDistance.c 104

105 Decision table naming speed => setspeed time => settime distance? => distance 105

106 Make it red. Implement stubs for: settime setspeed distance 106

107 Does anybody really know what time it is? We will use a link-time seam to mock out uptime. include/hardware/uptime.h 107

108 FakeUptime mocks/hardware/fakeuptime.h mocks/hardware/fakeuptime.c 108

109 Back to our fixture Use the uptimemillis extern to set the current time fixtures/treadmillcumulativedistance.c 109

110 Exercise Finish the TreadmillCumulativeDistance fixture. Extra credit: TreadmillDistance fixture. (hint: you ll need to use the reset function) 110

111 Cumulative Distance fixtures/treadmillcumulativedistance.c 111

112 Cumulative Distance fixtures/treadmillcumulativedistance.c 112

113 Cumulative Distance fixtures/treadmillcumulativedistance.c 113

114 Cumulative Distance fixtures/treadmillcumulativedistance.c 114

115 Total Distance fixtures/treadmilldistance.c 115

116 Break Take

117 System Architecture Web App Message Protocol Front Panel Fixtures Domain API Application Production Build AT build Hardware Abstraction Hardware Mock HW 117

118 Hardware Abstraction Remember, acceptance tests are not about testing the hardware. 118

119 Hardware Abstraction include/hardware/pwm.h 4 void Pwm_Create(void); 5 void Pwm_Destroy(void); 6 7 void Pwm_Start(); 8 void Pwm_Stop(); 9 void Pwm_SetDutyCycle(double percent); 10 void Pwm_SetPeriod(int microseconds); 119

120 Hardware Abstraction mocks/hardware/fakepwm.h 4 #include "Pwm.h" 5 6 extern int FakePwm_isRunning; 7 extern int FakePwm_period; 8 extern double FakePwm_dutyCycle; 120

121 Hardware Abstraction mocks/hardware/fakepwm.c 3 int FakePwm_isRunning = 0; 4 double FakePwm_dutyCycle = 0.0; 5 int FakePwm_period = 0; 6 11 void Pwm_Start(void) { FakePwm_isRunning = 1; } 13 void Pwm_Stop(void) { FakePwm_isRunning = 0; } 15 void Pwm_SetDutyCycle(double percent) { FakePwm_dutyCycle = percent; } 17 void Pwm_SetPeriod(int us) { FakePwm_period = us; } 121

122 UI abstraction The UI uses the same API as the tests Click / Press 122

123 UI Abstraction Detect button press... Api_IncrementTargetSpeed();

124 UI Abstraction What about Displays? LCD Application 124

125 UI Abstraction Turn the dependency around with an observer LCD Application 125

126 UI Abstraction Observer Pattern void onspeedchange(double speed) { /*update speed display */ } Api_registerSpeedObserver(&onSpeedChange); 126

127 Messaging Uses the same API as everyone else. Test message parsing independently. 127

128 Timers Can t wait on real time Timer also hit the same API 128

129 Interrupts Generate events in system that can be simulated 129

130 Then what s the point? Acceptance tests are not entire system tests. They are a software collaboration medium. 130

131 Running on target 131

132 Serial Bridge What about running on a target without an ethernet stack? 132

133 Running on target host processor embedded target TCP Serial Fitnesse Slim Bridge Slim Server 133

134 On Target Demo 134

135 Porting CSlim 135

136 Porting CSlim Implement a TCP socket Server only needs to handle one connection at a time calls Slim_HandleConnection with pointers to send / recv functions 136

137 You can help Still some features missing (sut, libraries) Contribute your port back to the project Help reduce memory usage Slim(mer)? 137

138 Bonus topics Multi-parameter script table functions Returning Lists Query tables 138

139 Multiple Parameters Function name alternates with parameters 139

140 Multiple Parameters Function name alternates with parameters 140

141 Multiple Parameters Function name alternates with parameters 141

142 Returning Lists cslim/fixtures/querytableexample

143 Query Tables Rows of Records Order doesn t matter 143

144 Query Tables Return a List of records A record is a list or key, value pairs A key, value pair is a two element list ( ) ((id, 1), (pay, 1000)) ((id 2), (pay, 1500)) 144

145 Query Tables The list must be serialized and returned from the query function. cslim/fixtures/querytableexample.c 145

146 Recommended Reading Specification by Example - Gojko Adzic Test Driven Development for Embedded C - James Grenning Fit - For Developing Software - Mugridge / Cunningham FitNesse Users Guide 146