ROB IN Performance Measurements

Transcription

1 ROB IN Performance Measurements I. Mandjavidze CEA Saclay, Gif-sur-Yvette CEDEX, France ROB Complex Hardware Organisation Mode of Operation ROB Complex Software Organisation Performance Measurements Discussion CERN, 3 November 1999

2 Saclay ROB Complex Hardware Organisation CPU L2 Cache ROB Controller ROBINs Host Adapter PCI Bridge Host Memory NIC Event Memory 1... Event Memory N Network Link Read-Out Links ROB Controller is based on either PowerPC VME/CompactPCI SBC, or on PC Same network connection serves for L2 / DAQ dataflow, run control and monitoring Host memory and ROBINs event memories are directly accessible to NIC Irakli.Mandjavidze@cea.fr 2 / 12 CERN, 3 November 1999

3 Saclay ROB IN ROB Local Bus Version #1 Version #1 input data port FIFO Event Memory PCI bridge PCI Mezzanine Card I960Jx at 33 MHz PLX9080 PCI bridge Logic Trigger CPU + Memory 32-bit Local Bus 512 kbyte CPU memory 3 PMC-s of Version #1 available Version #2 with 100 MHz I960JT CPU and 8 MByte Event Memory under development Irakli.Mandjavidze@cea.fr 3 / 12 CERN, 3 November 1999

4 Operation of ROB Complex for Data Requests Request Rx Task Receive Process request Data request Data Buffers Header Buffer RoI or event fragment Response Tx Task Send Request ROB IN ROB IN ROB IN Format & Buffers can be allocated in ROB Controller: ROBIN DMAs event data - preprocessing Buffers can be mapped on ROBINs event memory: NIC DMAs event data Capability of sending chained buffers Irakli.Mandjavidze@cea.fr 4 / 12 CERN, 3 November 1999

5 ROB Complex Software Organisation ROB Controller ROBIN-s ROB Complex Application ROB IN API and Application Low Level Communication ROB IN Hardware Abstraction Host Platform Abstraction ROB ROBIN application ROBIN application ROBIN application application Low Low Level Low Level Communication Low Level Communication Level Communication Communication Hardware Hardware Abstraction Hardware Abstraction Hardware Abstraction Abstraction Host Platform and ROB IN hardware are hidden ROB IN hardware abstraction performs basic operation: Load Code, Start, Reset, Halt... Low Level Communication performs message passing and allows for debugging Simplified ROB IN Application: no data input C code for: Saclay ROB IN, LynxOS / PowerPC VME/CompactPCI SBCs, Linux / PC Transtech Sharc PMC, WinNT / PC Irakli.Mandjavidze@cea.fr 5 / 12 CERN, 3 November 1999

6 Low Level Communications Command Descriptor Cmd Type Cmd Size Client Command Table Server Posted Command Queue Command Get Free Command Fill Command Post Command Check Status Free Command Queue Status Queue Get Command Pass it to App or process it Return Status Bi-directional communications Synchronous and asynchronous commands Minimal use of PCI bus for implementation on PLX / 12 CERN, 3 November 1999

7 ROB IN Low Level Communication Performance Dummy asynchronous commands immediately acknowledged by ROB IN 80 Command Servicing Rate 30 Command Servicing Time Rate (khz) Time (µs) Latency 1 / Rate Command Size (32-bit word) Command Size (32-bit word) Data Request command size: 4 32-bit words Event Clear Request command size: bit words (50 events to clear) Irakli.Mandjavidze@cea.fr 7 / 12 CERN, 3 November 1999

8 Event Management in ROB IN Hashing mechanism for incoming events has been added Hash Table Event Desc Ptr Event Desc Ptr Event Desc Ptr Event Desc Ptr Free Event Descriptor Queue Event Descriptor Event Id Event Next Desc Id Ptr Next Status Desc Page Ptr Id Status #_of_pages Page Id #_of_pages Page ID Page Used Id Size Used Size Page ID Page Used Id Size Used Size For data request: -> Find entry in the hash table: event id % hash_table_size -> Find event descriptor in the corresponding chain -> Return event pages For event clear request: -> Find event descriptor in the hash table and remove it -> Free event resources -> Return event descriptor to the free queue Irakli.Mandjavidze@cea.fr 8 / 12 CERN, 3 November 1999

9 Event Memory Page Size: 1 kbyte ROB IN Data Request Performance 60 Max Data Request Rate Data Request Service Latency 30 Rate (khz) k 2k 3k 4k Requested Data Size (Byte) Latency (µs) k 2k 3k 4k Requested Data Size (Byte) Performance at Low Level Communication -> Rate ~68 khz; Latency ~18 µs For 2 kbyte events -> Rate ~51 khz; Latency ~23 µs Irakli.Mandjavidze@cea.fr 9 / 12 CERN, 3 November 1999

10 ROB IN Event Clear Request Performance Event Memory Page Size: 1 kbyte For each cleared event a new event is generated Rate (khz) Clear Request Service Rate 100 Event Rate 10 Latency (µs) Clear Request Service Latency Event Size 1kB 2kB 3kB 4kB Clear Request Rate Number of Decisions Grouped Number of Decisions Grouped Performance at Low Level Communication for event grouping factor of 50 -> Rate ~55 khz; Latency ~25 µs For 2 kbyte events -> Together Event Clear and Generation take ~12 µs per event -> Sustained event rate equals ~83 khz Irakli.Mandjavidze@cea.fr 10 / 12 CERN, 3 November 1999

11 Mixing Data and Clear Event Requests Event Size: 2 kbyte Selection Request: 3 khz No Flow Control on Clear Requests Service Latency (µs) Load (%) Clear Request Data Request Clear Request Rate (khz) Event grouping factor of 50 for Clear Requests Event Filter Request: 2 khz Pipeline only one Clear Request at a time Service Latency (µs) Load (%) Clear Request Data Request Clear Request Rate (khz) Flow Control: Deterministic Data Request Servicing -> Clear Request adds at most 700 µs Irakli.Mandjavidze@cea.fr 11 / 12 CERN, 3 November 1999

12 Discussion ROB IN Application Event management using hash table has been added More realistic behaviour for Data Request and Event Clear commands ROB IN performance has been measured with stand-alone program ROB Complex Platform: 400 MHz Pentium PC running Linux Saclay ROB IN : Version #1 with 33 MHz I960 processor and 512 kbyte system memory Data Request service rate: up to 40 khz and above; Event Clear rate: up to 83 khz For Event Clear requests the necessity of Flow Control has been identified The Flow Control assures deterministic Data Request servicing times ROB IN Application is still under development Waiting for the Version 2 of the Saclay ROB IN More realistic event handling from input port will be added ROB Complex has been integrated in the ATM testbed New series of measurements foreseen at CERN this month Irakli.Mandjavidze@cea.fr 12 / 12 CERN, 3 November 1999