Distributed TLC. Markus A. Kuppe. TLA + Community Event, ABZ 2014 Toulouse, June 3, /33

Transcription

1 1/33 Distributed TLC Markus A. Kuppe TLA + Community Event, ABZ 2014 Toulouse, June 3, 2014

2 2/33 Outline Introduction Distributed TLC When and why? Ad-hoc distributed TLC Use case Demo Common problems Tuning Conclusion Cloud distributed TLC Use case Demo Conclusion Summary and Outlook Summary Outlook

3 3/33 Outline Introduction Distributed TLC When and why? Ad-hoc distributed TLC Use case Demo Common problems Tuning Conclusion Cloud distributed TLC Use case Demo Conclusion Summary and Outlook Summary Outlook

4 /33 Highlevel TLC FPS Fingerprint Set (contains a hash of all already explored states) SQ State Queue (contains all unexplored states) WORKER Set of worker nodes (nodes/cores doing heavy calculations) φ Safety properties (as dened by the model)

5 4/33 Highlevel TLC FPS Fingerprint Set (contains a hash of all already explored states) SQ State Queue (contains all unexplored states) WORKER Set of worker nodes (nodes/cores doing heavy calculations) φ Safety properties (as dened by the model)

6 4/33 Highlevel TLC FPS Fingerprint Set (contains a hash of all already explored states) SQ State Queue (contains all unexplored states) WORKER Set of worker nodes (nodes/cores doing heavy calculations) φ Safety properties (as dened by the model)

7 4/33 Highlevel TLC FPS Fingerprint Set (contains a hash of all already explored states) SQ State Queue (contains all unexplored states) WORKER Set of worker nodes (nodes/cores doing heavy calculations) φ Safety properties (as dened by the model)

8 Non-distributed TLC c 1 φ FPS m c... φ SQ c n φ model checker cores 5/33 Figure : Non-distributed TLC

9 Distributed Computation w 1 φ FPS m w... φ SQ w n φ master node worker nodes 6/33 Figure : Compute nodes

10 Distributed Computation & Storage FPS 1 SQ m w 1 w n FPSm master node worker nodes 7/33 Figure : Compute and storage nodes

11 /33 When and why? Compute Use as many cores as possible (to distribute heavy next-state computation) Compute & Storage Additionally use as much memory as available (to speed up FPS lookup)

12 9/33 Scalability Scales (almost) linearly => Hash collisions

13 10/33 Limitations No liveness checking No coverage No distributed calculation of init states No (strict) breadth-rst search

14 1/33 Fault tolerance w 1 FPS n workers (w) 1...(m-1) ngerprint sets (FPS) SQ m w... Compensates neither loosing SQ nor Trace w n FPS m Figure : Broken setup

15 12/33 Outline Introduction Distributed TLC When and why? Ad-hoc distributed TLC Use case Demo Common problems Tuning Conclusion Cloud distributed TLC Use case Demo Conclusion Summary and Outlook Summary Outlook

16 13/33 Use case Experiments block/slow down your desktop

17 14/33 Demo Demo (Ad-hoc network)

18 15/33 Demo notes (slide not part of nal pdf) 1. Create and setup a model for distributed TLC 2. Start model checker (...waiting for workers) 3. Switch to Windows VMs 4. Open IE on Toolbox webserver 5. (Install Java7 via Webstart) 6. Launch Worker in IE in Windows VM 7. Switch to another VM & start more workers

19 16/33 Demo notes (slide not part of nal pdf) 1. Start another model 2. Start Distributed FPSet node in VM1 2.1 Set number of distributed FPSets 3. Start Worker node in VM2

20 17/33 Demo notes (slide not part of nal pdf) 1. Start daemon style Worker node in VM rst 2. Start small model that completes in less than a minute 3. Start small model that completes Mention to use Java8 or OOM due to PermGen

21 18/33 Common problems Broken name resolution Blocked by rewalls Restrictive system security

22 19/33 Fingerprint Set implementations Least Signicant Bit (LSB) halves available FPS memory old default Most Signicant Bit (MSB) exploits all available memory new default O Heap fastest & most ecient Not universally available

23 19/33 Fingerprint Set implementations Least Signicant Bit (LSB) halves available FPS memory old default Most Signicant Bit (MSB) exploits all available memory new default O Heap fastest & most ecient Not universally available

24 19/33 Fingerprint Set implementations Least Signicant Bit (LSB) halves available FPS memory old default Most Signicant Bit (MSB) exploits all available memory new default O Heap fastest & most ecient Not universally available

25 20/33 Distributed Fingerprint Sets Use distributed FPS Performance degrades as soon as TLC goes to disk (Solid state) disks order of magnitude slower compared to RAM Even remote memory still faster

26 21/33 Minor Checkpoints StateQueue buer Worker cache BlockSelector...

27 22/33 Conclusion Only simple if it works Too many screws

28 23/33 Outline Introduction Distributed TLC When and why? Ad-hoc distributed TLC Use case Demo Common problems Tuning Conclusion Cloud distributed TLC Use case Demo Conclusion Summary and Outlook Summary Outlook

29 24/33 Use case Experiments running many hours to days Personal machine not always on

30 25/33 Demo Demo (in the cloud)

31 26/33 Demo notes (slide not part of nal pdf) Set AWS credentials as environment variables Launch toolbox Create and setup model to start on AWS Set drop down to aws-ec2 Set Mention cloud-specic tuning hard-wired into the Toolbox Launch TLC in the cloud Show status in browser (have a backup screenshot available) Wait for response (have a backup result ready!) Open response (result) in Toolbox

32 27/33 Conclusion Don't have to worry about systems idiosyncrasies, but you literally pay the price for it

33 28/33 Outline Introduction Distributed TLC When and why? Ad-hoc distributed TLC Use case Demo Common problems Tuning Conclusion Cloud distributed TLC Use case Demo Conclusion Summary and Outlook Summary Outlook

34 29/33 Summary Exploits remote compute power and storage Scales Ad-hoc deployment can be tricky Cloud deployment despite costs way forward

35 30/33 Outlook Multi-node cloud deployments More pre-sets for cloud instance types Support other cloud providers Auto Scaling based on actual machine load Larger (128 bit) ngerprints?

36 31/33 Q&A Thank you for your attention

37 32/33 Acknowledgment Microsoft Research & INRIA Joint Lab Amazon AWS Experiments presented in this paper were carried out using the Grid'5000 experimental testbed, being developed under the INRIA ALADDIN development action with support from CNRS, RENATER and several Universities as well as other funding bodies -

38 33/33 References Slides: DistributedTLC_MarkusAKuppe.pdf