Department of Computer Science University of Tartu Formal Methods (2013)
The Service Literally: Who then? Reverse telephone directory (Annuaire inversé) offered by France Telecom. It is an automated phone service. Seems no longer active, but similar online services exist in many countries. Our task: create a decent test suite for this system. Case study by Utting & Legeard.
Informal Requirements I The system answers a call with its welcome message: Welcome! Press key. Message is repeated 3 times. Each time with 6 seconds timeout. If is not pressed, system plays the notallow message: Buy a new phone, idiot! It then hangs up. If is pressed, system gives the enter message. Enter a 10 digit number followed by # Repeated 3 times; timeout 20 secs.
Informal Requirements II If no digits followed by # are pressed, system plays the bye message. If a number followed by # is received, then... If the number is special short number (112) an explanation for that number is given press to enter another number! Else if number is not 10 digits: play error message goto enter process Otherwise look up number in database. If number does not exist: play sorry and then goto enter process. If number does exist: goto info menu.
Informal Requirements III The info menu starts with the info message: Press 1 to spell the name, press 2 to hear the address, press for another search. And this is what it does: 1 & 2: respond and return to info menu. : back to the enter process. No keys: repeat info 3 times with 6 sec timeout, then bye-bye. At any time: caller may hang up.
A Finite State Machine A finite state machine with input/output (Mealy Machine): 1. Finite set of states S. 2. Finite set of input symbol I. 3. Finite set of output symbols O. 4. State transitions δ: S I S 5. Output function λ: S I O. We write a transition as follows: s 1 i/o s 2 δ(s 1, i) = s 2 λ(s 1, i) = o
Modelling as an FSM? We want a model to generate test scripts to be followed by a human tester. Input alphabet I represent tester actions. Output set O are expected responses. We care about system logic, not database content... only check a couple of representative phone numbers! What do we abstract away? What are the individual input events?
What we really care about Some special keys: 1, 2, # and. Representative phone numbers: 18 fire brigade short number. num1 05000030000 (unused) num2 01837395826 (a friend) bad 0123456789 (too short). Additional actions: dial dial! wait wait until service times out.
Specific output symbols welcome Welcome! Press key. fire 18 is the fire brigade. Press to start over. sorry The number 05000030000 is not found. Try again! name The number 01837395826 belongs to Carla Bruni. Press 1 to spell the name, 2 to hear... addr The address is Élysée Palace, Paris. Press 1 to spell... spell Bruni is spelled B-R-U-N-I. Press 1 to spell the name...
Example Sequence Our inputs and expected responses: dial/welcome wait/welcome /enter num2/name 2/addr wait/info wait/info wait/bye
The FSM model Now, we draw it... How do we deal with timeouts. (Not so elegant!) Make the output function total:. Is the transition function total?
What do we gain? Have you ever used an automated phone service? Transition are slow. We would like to check each transition. We would like to spend as little time on the phone as possible. Solution: transition tour provided by the Directed Chinese Postman Algorithm. This gives a transition tour with 4 tests and 61 transition in total.
I get following transitions dial/welcome, */enter, wait/enter, bad/error, wait/enter, num1/sorry, wait/enter, 18/fire, */enter, num2/name, wait/info, */enter, wait/enter, num2/name, wait/info, 2/addr, wait/info, 1/spell, wait/info, wait/info, */enter, wait/enter, wait/enter, bad/error, wait/enter, wait/enter, num1/sorry, wait/enter, wait/enter, 18/fire, */enter, wait/enter, wait/enter, num2/name, wait/info, wait/info, 2/addr, wait/info, wait/info, 1/spell, 2/addr, 1/spell, wait/info, wait/info, wait/bye; dial/welcome, wait/welcome, */enter, bad/error, num1/sorry, wait/enter, wait/enter, wait/bye; dial/welcome, wait/welcome, wait/welcome, */enter, num2/name, */enter, 18/fire, wait/bye; dial/welcome, wait/welcome, wait/welcome, wait/notallow.
Extended Finite State Machines Extended FSM has 1. Visible states like the FSM. 2. Internal state variables. Transitions now additionally update state variables and may have guards. Key point is the two-level abstraction: 1. Visible states drive the test generation. 2. Internal state allow more precise modeling of the system. Of course, test coverage metrics w.r.t. visible states only.
ModelJUnit Object getstate(): Returns the current visible state, i.e., the part of EFSM s fields that test generator should see. @Action void m i (): annotated action method, updates state (both internal and visible) and calls SUT methods. boolean m i Guard(): when this returns true m i may be called. Note: test generator observes via getstate() and uses reflection to make a random enabled transition.
EFSM of : State public class QuiDonc implements FsmModel { public enum State { Start, Star, Enter, Emerg, Info }; private State currstate; // visible state private int timeouts; // internal state public String WELCOME = "Welcome..." } public String getstate() { return currstate.tostring(); // + timeouts; } public void reset(boolean testing) { timeouts = 0; currstate = State.Start; }...
EFSM of : Action public class QuiDonc implements FsmModel {... public boolean starguard() { return currstate == State.Star currstate == State.Emerg currstate == State.Info; } public @Action void star() { out.println("*/"+enter); // Normally: Execute SUT! currstate = State.Enter; timeouts = 0; }...
EFSM of : Action public class QuiDonc implements FsmModel {... // no guards; always enabled public @Action void wait_() { // not overriding Object.wait() timeouts++; if (timeouts >= 3) { if (currstate == State.Star) out.println("wait/"+notallow); else out.println("wait/"+bye); currstate = State.Start; timeouts = 0; } else out.println("wait/...(repeat msg)..."); }... }
What s next? Download ModelJUnit and try the simple set examples. Download Thimbley s CPP and try the transition tour.