Karabo. An integrated software framework combining control, data management, and scientific computing tasks. Burkhard Heisen October 11, ICALEPCS 2013

Transcription

1 Karabo An integrated software framework combining control, data management, and scientific computing tasks Burkhard Heisen October 11, ICALEPCS 2013

2 Karabo will be used at the European XFEL 2 Hamburg Germany Data rate (10 GB/s/2d-detector) Bunch pattern (4.5 MHz 10Hz) 100 ms

3 Functional requirements 3 A typical use case: Accelerator Undulator Beam Transport DAQ data readout online processing quality monitoring (vetoing) Control drive hardware and complex experiments monitor variables & trigger alarms allow some control & show hardware status show online data whilst running DAQ Control DM SC Tight integration of applications DM storage of experiment & control data data access, authentication authorization etc. setup computation & show scientific results SC processing pipelines distributed and computing specific algorithms (e.g. reconstruction)

4 What is Karabo? - Lets draw an analogy 4 Karabo devices Karabo framework Karabo device-server running devices

5 What is Karabo? - Lets draw an analogy 5 Karabo devices Karabo framework Karabo device-server running devices

6 What is Karabo? - Lets draw an analogy 6 Karabo devices Karabo central services (archive, database) Karabo framework Karabo device-server running devices

7 Essential components 7 Device Instance Device-Server HV Pump Application Message Broker Diode Camera Store Disk Storage Load Calibrate1 Simulate Calibrate2 Logger Device Sub Control CLI(s) GUI Server RDB GUI(s)

8 Essential components 8 Device Instance Device-Server HV Pump Application Message Broker Diode Camera Store Disk Storage Load Calibrate1 Simulate Calibrate2 Logger Device Sub Control CLI(s) GUI Server RDB GUI(s)

9 Essential components 9 Device Instance Device-Server HV Pump Application Message Broker Diode Camera Store Disk Storage Load Calibrate1 Clustering Simulate Calibrate2 Logger Device Sub Control CLI(s) GUI Server RDB GUI(s)

10 Device Properties and commands 10 Devices are controllable objects managed by a device server Device classes can be loaded at runtime (plugin technology) Can be written in C++ or Python Devices completely describe themselves. Allows automatic GUI creation and auto-completion in IPython Runtime-extension of properties, commands and attributes is possible No need for device developers to validate any parameters. This is internally done taking the expectedparameters as a white-list Properties and commands can be nested, such that hierarchical groupings are possible Class: MotorDevice static expectedparameters( Schema& s ) { FLOAT_ELEMENT(s).key( velocity ).description( Velocity of the motor ).unitsymbol( m/s ).assignmentoptional().defaultvalue(0.3).maxinc(10) }.mininc(0.01).reconfigurable().allowedstates( Idle ).commit(); Attribute INT32_ELEMENT(s).key( currentposition ).description = Current position of the motor.readonly().warnlow(10) [ ] Command SLOT_ELEMENT(s).key( move ).description = Will move motor to target position.allowedstates( Idle ) [ ] // Constructor with initial configuration MotorDevice( const Hash& config ) { [ ] } Property

11 Device Finite state machine (FSM) 11 Any device uses a standardized way to express its possible program flow The state machine calls back device functions (guard, onstateexit, action, onstateentry) The GUI is state-machine aware and enables/disables buttons proactively // Ok Machine FSM_TABLE_BEGIN(OkTransitionTable) // SrcState Event TgtState Action Guard Row< Started, StopEvent, Stopped, StopAction, none >, Row< Stopped, StartEvent, Started, StartAction, none > FSM_TABLE_END FSM_STATE_MACHINE(Ok, OkTransitionTable, Stopped, Self) // Top Machine FSM_TABLE_BEGIN(TransitionTable) Row< Initialization, none, Ok, none, none >, Row< Ok, ErrorFoundEvent, Error, ErrorFoundAction, none >, Row< Error, ResetEvent, Ok, ResetAction, none > FSM_TABLE_END KARABO_FSM_STATE_MACHINE(StateMachine, TransitionTable, Initialization, Self) Start Stop State Machine Initialization stop reset none Stopped Started Error start errorfound OK

12 Device As workflow module 12 Devices can act as modules of a scientific workflow system Configurable generic input/ on devices Memory One channel is specific for one data structure (e.g. Hash, Image, File) -threads New data structures can be registered and are immediately usable Developers just need to code the compute and (optionally) the endofstream method IO system is decoupled from processing system (process whilst transferring data) Broker-based communication transparently establishes point-to-point connection TCP Distributed processing Any workflow device has full access to the live control-system

13 Workflows support load balancing 13

14 Workflows support load balancing 14

15 Workflows support load balancing 15

16 Workflows support load balancing 16

17 Workflows support load balancing 17

18 Workflows support load balancing 18

19 Workflows support load balancing 19

20 Workflows support load balancing 20

21 Workflows support load balancing 21

22 Workflows support load balancing 22

23 Workflows support load balancing 23

24 Workflows support load balancing 24

25 Workflows support load balancing 25

26 Workflows support load balancing 26

27 Workflows support load balancing 27 Pixel parallel processing (one thread per pixel)

28 Workflows support load balancing 28

29 Workflows support load balancing 29

30 Workflows support load balancing 30

31 Workflows support load balancing 31

32 Workflows support load balancing 32 Pixel parallel processing (one thread per pixel)

33 Workflows support load balancing 33 IO whilst computing Pixel parallel processing (one thread per pixel)

34 Workflows support load balancing 34

35 Workflows support load balancing 35

36 Workflows support load balancing 36

37 Workflows support load balancing 37

38 Workflows support load balancing 38 Pixel parallel processing (one thread per pixel)

39 Workflows support load balancing 39 IO whilst computing Pixel parallel processing (one thread per pixel)

40 Workflows support load balancing 40 IO whilst computing Pixel parallel processing (one thread per pixel)

41 Workflows support load balancing 41

42 Workflows support load balancing 42

43 Workflows support load balancing 43

44 Workflows support load balancing 44

45 Workflows support load balancing 45 Pixel parallel processing (one thread per pixel)

46 Workflows support load balancing 46 IO whilst computing Pixel parallel processing (one thread per pixel)

47 Workflows support load balancing 47 IO whilst computing Pixel parallel processing (one thread per pixel)

48 Workflows support load balancing 48 IO whilst computing Notification about new data possible to obtain Pixel parallel processing (one thread per pixel)

49 Workflows support load balancing 49

50 Workflows support load balancing 50

51 Workflows support load balancing 51

52 Workflows support load balancing 52

53 Workflows support load balancing 53 Pixel parallel processing (one thread per pixel)

54 Workflows support load balancing 54 IO whilst computing Pixel parallel processing (one thread per pixel)

55 Workflows support load balancing 55 IO whilst computing Pixel parallel processing (one thread per pixel)

56 Workflows support load balancing 56 IO whilst computing Notification about new data possible to obtain Pixel parallel processing (one thread per pixel)

57 Workflows support load balancing 57 IO whilst computing Notification about new data possible to obtain Pixel parallel processing (one thread per pixel)

58 Workflows support load balancing 58

59 Workflows support load balancing 59

60 Workflows support load balancing 60

61 Workflows support load balancing 61

62 Workflows support load balancing 62 Pixel parallel processing (one thread per pixel)

63 Workflows support load balancing 63 IO whilst computing Pixel parallel processing (one thread per pixel)

64 Workflows support load balancing 64 IO whilst computing Pixel parallel processing (one thread per pixel)

65 Workflows support load balancing 65 IO whilst computing Notification about new data possible to obtain Pixel parallel processing (one thread per pixel)

66 Workflows support load balancing 66 IO whilst computing Notification about new data possible to obtain Pixel parallel processing (one thread per pixel)

67 Workflows support load balancing 67 IO whilst computing Notification about new data possible to obtain Pixel parallel processing (one thread per pixel)

68 Multi-purpose graphical user interface 68 Navigation Custom composition area Configuration Logging / Scripting console Notifications Documentation

69 Before you even start: Login 69 Broker-Message GUI or CLI GUI-Srv Header [ ] uid=42 accesslevel= admin Body [ ] username userid sessiontoken defaultaccesslevel accesslist password provider ownip* brokerhost* brokerport* Locking: if is locked: if is uid == owner then ok Device brokertopic* Access control: if accesslevel >= visibility: Central DB if accesslevel >= param.accesslevel then ok 1. Authorizes 2. Computes context based access levels

70 Property/Command composition 70 Text field Button widget Display widget drag & drop Editable widget Burkhard Hieisen (WP76)

71 Device (workflow) composition 71 Workflow node (device) drag & drop Draw connection

72 The command line interface (CLI) 72 Main functionality: Exploring the distributed system topology (hosts, device-servers, devices, their properties/commands, etc.) getservers, getdevices, getclasses instantiate, kill, set, execute (in wait or nowait fashion), get, monitorproperty, monitordevice Even polled interface will never really poll, but is eventdriven under the hood Remote auto-completion which is access-role and state aware

73 The command line interface (CLI) 73

74 The command line interface (CLI) 74

75 The command line interface (CLI) 75

76 Selected Features Cross-network signals and slot implementation (Qt style) Native function calls providing native argument passing Runtime setup of signals, slots and connect statements High performance dictionary object Fully recursive key to any value mapper (also in C++) Keeps insertion order and supports attributes Serializes to XML, Binary, HDF5, DB, JMS-Message, etc. Event driven system throughout, incl. event driven archiving Archiving happens completely transparent (no extra tooling needed) Archiving policies are configured per property within device code User centric, access controlled system Context based authentication (who, where, when) Kerberos is integrated Full system available in C++ and pythonic Python Devices and workflows can intermix both languages (access to numpy, scipy, etc.) Fully functional also without master or database (good for testing/developing) Easy install (both framework and devices) using software bundle architecture 76

77 Acknowledgments 77 Karabo contributors (internal) K. Wrona S. Esenov K. Weger J. Szuba D. Boukhelef I. Kozlova A. Parenti L. Maia S. Hauf P. Gessler N. Coppola J. Tolkiehn J. Elizondo H. Höhne C. Youngman Karabo contributors (external) C. Yoon (CFEL) N. D. Loh (SLAC) T. Nicholls (STFC) K. Rehlich (DESY / DOOCS)

78 78 Thank you for your kind attention.