sfcb Small Footprint Cim Broker Introduction

sfcb Small Footprint Cim Broker Introduction Adrian Schuur schuur@de.ibm.com May 2005 LTC Systems Management

2 Contents Simple questions: Why? What? How? A Picture and Terminology used Detailed description of components Indication scenario Preliminary Performance numbers

3 sfcb Simple questions

sfcb Why? WBEM infrastructure is available and becoming well established on servers. In order to become pervasive as part of Systems Management solutions it is required to have WBEM infrastructure available in embedded and/or constrained environments as well. When available, Management Solutions will find a uniform model and instrumentation to work with. IBM and Intel are collaborating on efforts to achieve this by working together in the Small Footprint CIM Broker project. 4

sfcb What is it? Highly configurable CIM Object Manager Capable of supporting all or selected DMTF specifications Standards based: WBEM and CMPI Intended for resource constrained and/or embedded environments Consists of a core that can be extended using providers and input adapters Design for inherent stability survives adapter and provider crashes Main sfcb process very simple should never die Input adapters and providers execute out of process It is a IBM/Intel collaboration Manageability project 2 not yet announced products will use sfcb. Goal is to open source this technology work in progress 5

6 sfcb How? Written in C Main process is essentially not more than a Provider Manager Modeled after Provider Manager of SNIA CIMOM (ServerNamespace) Maintains provider registration data Resolves and loads providers on request from adapters and up calls CIM Data type support based on Remote CMPI experiences Garbage Collection Added hash table, list and string buffer support Uses internally a relocatable CIM object format Objects can be shipped around without marshalling overhead

7 sfcb Picture Please! Client Request Response sfcbroker fork() httpdaemon fork() Request Provider Resolution Returns list of Provider sockets CimXml Request Processor (aka Codec) Handling a Client request sfcbroker forks Daemon (only once) Daemon listens on port Daemon forks Codec Codec parses request Execute Request Returns Result Codec requests provider resolution sfcbbroker resolves requests sfcbbroker forks provider(s) if needed sfcbbroker returns list of sockets fork() Provider Codec calls provider(s) Provider(s) returns result Codec generates response Codec exits

8 Remote CMPI Concept Remote CMPI is a concept the enables an unaltered, ordinary CMPI style provider to execute at a remote location Remote in the sense that it is remote in respect to the central CIMOM Clients field requests to the central CIMOM At the CIMOM side the generic Remote CMPI proxy provider kicks in and pass requests to the Remote CMPI daemon at remote locations The Remote CMPI daemon will load and drive the provider. Proxy and Daemon use a private streaming protocol Client CIMON Proxy Provider Remote Daemon CMPI CMPI Provider Provider Remote system Central CIMOM Remote Daemon CMPI CMPI Provider Provider Remote system

9 sfcb Technical details

10 sfcb Terminology used Terminology used to describe interactions between broker and providers: Driving ff of provider yy: Up call: The broker calling the ff provider function of provider yy Invocation of a broker function by a provider Invoking mm method of zz: A provider calling the extrinsic method mm of provider zz Client GetInstance request broker Driving GetInstance of yy Up call yy zz Invoking mm of zz

sfcb Details: sfcbroker sfcbroker uses configuration file during startup Max number of concurrent Codec processes Max number of concurrent Provider processes Sum of max process numbers used to generate socket pairs and System V semaphores SEM_UNDO option used on counters and locks to observe max numbers automatic throttling of requests arbitration done when reaching max provider processes Initializes usual signal handler including SIGHUP for restart Loads the provider registration file Starts class and InterOp provider Starts input adapter Daemon(s) Enters request for provider resolution loop Input for request is namespace, classname and operation 11

12 sfcb Details: Input Adapter Daemon The httpdaemon supports SSL and basic authentication Authentication exit is offered to customize authentication policies. Opens designated port number (defined in configuration file) Enters processing loop Listen for requests Isolate payload Fork input adapter (codec) Wait for next request (As is obvious, daemon processing is very simple. We might consider folding this into the sfcbroker process as a thread (configuration option)). Input throttling is accomplished automatically with System V semaphores using counting locks and the SEM_UNDO option.

sfcb Details: CIMXML codec CIMXML codec forked once for every client request XML parsed using Bison with custom build XML tag lexer Interfaces with sfcbroker and Providers with providermgr.c to request provider resolution getprovidercontext() providerdrv.c to invoke provider(s) invokeprovider() and invokeproviders() These interfaces are protocol agnostic and can be used by other protocols (WS, CLI, SQL, binary, etc) Generates chunked responses when requested for enumerating type requests. Currently blocks class creation and modifying operations will be changed to send requests to class provider (who will than decide to support this or not) 13

14 sfcb Details: Providers All CMPI provider types will be supported Providers are forked by sfcbroker when needed Idle/Unload monitoring done within each provider process Possible idle/unload policies: idle time never specified in configuration file once for all providers specified during registration after n invocations specified during registration (rejuvenation option) Providers can resist unloading at clean up time by returning either the CMPI_RC_NEVER_UNLOAD or CMPI_RC_DO_NOT_UNLOAD return code. An indication provider becomes a unload candidate when no subscriptions exist any more for this provider. Multiple providers can share a provider process Every provider request is executed as a detached (POSIX) thread (concurrent execution of requests) Up calls are serialized within a provider process

15 sfcb Details: Extension Providers Extension Providers implement function that other CIMOMs often considered part of a CIMOM itself. Examples: Class provider $ClassProvider$ Default provider (static repository) InterOp namespace provider Indication handlers Authorization provider $DefaultProvider$ $InterOpProvider$ using real class names using real class names Message internationalization support $MessageTranslation$ Most of these providers are not class specific because they offer general function. Special pseudo class name are used to register these providers. This enables customization of these extension without altering the core. Extension Providers often have no related cim schema and/or often offer extrinsic method calls that are not defined in a cim schema either (and cannot be used by clients). Extension providers are a key ingredient for enabling customizable CIMOMs

sfcb Details: Class Provider The class provider is an instance and method provider The class provider is only used by sfcbroker for during provider resolution Some of the up call routines that need class information Codecs to hand off class creation/deletion/modification requests to the class provider In general, the class provider knows its class repository and also maintains the inheritance tree. It must respond to very specific method requests issued during provider resolution. Class provider can have very different incarnations depending on the subset of functions are needed in an embedded environment. The current sfcb implementation keeps all classes in memory and has a very intimate relationship with mofc (sfcb s off line mof compiler), 16

sfcb Details: Default Provider Invocation of this provider is instigated solely by provider resolution support. The default provider is the provider of last resorts it no provider is registered for a particular class. Its is an instance, association and method provider. It can respond by accessing a static repository or return an error indication saying that no provider is available for this operation. The sfcb implementation interfaces with sfcb s file based repository support. Other implementation might use a proper database instead, or use a non persistent memory store. 17

sfcb Details: InterOp Provider The InterOp provider is invoked for every operation against the InterOp namespace, unless it is against a registered InterOp class. Normally is an instance, association and method provider. This provider controls the degree of InterOp compliance. Currently used to implement Process Indications, implementing support for CIM_IndicationFilter and CIM_IndicationSubscription. The provider can be invoked as a result of normal Client requests, but also offers a few hidden methods being used by the DeliverIndication up call and Indication Handlers reporting creation or deletion of Indication handler instances. 18

sfcb Details: Indication Handler Indication handlers interface with InterOp provider to report Handler creation/deletion requests using _addhandler and _removehandler methods. In case of a deletion request a check is made to see whether the handler instance is part of an active subscription. If so, the handler is instructed to refuse the deletion request. Indication Handlers must support the _deliver method. The indication instance is part of the _deliver method. CIM_ListenerDestinationCIMXML is supported by sfcb. It requires the Curl package to be installed. 19

sfcb Details: Authorization Provider Meant to serve two purposes: Administrative interface to define authorizations Interface to be used by providers to test for authorized principle Default implementation stores authorizations in side file Other implementations could interface with ldap or other security environments The principle of a request is always passed to a provider via the CMPIContext instance. 20

sfcb Details: Message internationalization support Special provider to manage message catalogues for translation services. Invoked internally by TranslateMessage up call call. Default implementation will return not supported, meaning message will not be translated. 21

22 sfcb Details: Indication Components Client sfcb Listener Indication Provider InterOp Provider Indication Handler

23 sfcb Details: Indication scenario Client sfcb Listener 1. Client creates CIM_IndicationFilter instance Indication 2. InterOp Provider checks query expression Provider 3. Client creates CIM_ListenerDestinationCIMXML instance 4. Handler provider invokes _addhandler method of Interop Provider 5. Client creates CIM_IndicationSubscription association instance 6. Client starts listening for indications 7. InterOp Provider checks whether related Filter and Handler exist 8. InterOp Provider requests provider resolution for this request 9. InterOp Provider drives ActivateFilter of eligible provider(s) 10. Indication provider start monitoring 11. Indication provider makes DeliverIndication up call 12. DeliverIndication evaluates indication against Query expression(s) 13. DeliverIndication invokes _deliver method of InterOp Provider 14. InterOp Provider finds related Subscription(s) 15. InterOp Provider invokes _deliver method of all related handlers instances 16. Handler formats and sends indication 17. Client receives indication InterOp Provider Indication Handler

24 sfcb Preliminary Performance Numbers

25 CIM Server Footprint Comparison Footprint Measurement Problems shared memory usage determination (mainly shared libraries) virtual vs real storage (commit ratio, copy on write) stack heap thread stacks libc libpeg* main()

26 Where and How to Measure Linux /proc filesystem /proc/pid/statm gives virtual, resident and library related page consumption /proc/pid/vmstat gives detailed human readable informations also need to consider other virtual storage consumption factors shared library text and data sizes number of threads (thread stack on process heap)! per process stack and data size

27 Metrics and Measurement Method Total virtual storage consumption (n processes, m threads) Conditions CIM server idle after startup CIM server idle after enumeration don't count libc (~1.1MB) custom thread stack size 512 kb

Footprint sfcb and OpenPegasus SFCB after SFCB after Pegasus Pegasus Startup Enum after Startup after Enum Number of Processes 3 4 1 1 Number of extra Threads 3 5 3 6 Text 8kB 8kB 88kB 88kB Stacks (sum) 1050kB 1400kB 1180kB 1190kB Data + Libdata (sum) 585kB 780kB 900kB 1430kB Lib Text 430kB 430kB 7500kB 7500kB Thread Stacks (sum) 512kB 1536kB 1024kB 2560kB Total 2585kB 4154kB 10692kB 12768kB Total (w/o thread stacks) 2073kB 2618kB 9668kB 10208kB RSS 2060kB 3200kB 6420kB 7964kB Note: these are conservative figures for sfcb don't account for copy on write memory (estimated 1.4MB) don't account for shared RSS 28

Performance sfcb vs. OpenPegasus What's measured response time for enuminstancenames and enuminstances sample file provider (2785 instances) resource access time ~0,3s enuminstancenames enuminstances SFCB CMPI C 0.6s 2.3s SFCB CMPI C++ 0.6s 2.3s SFCB ecute 0.6s 4.0s Pegasus CMPI C 5.3s 15.7s 29