Extensible Transport Framework

Transcription

1 Extensible Transport Framework Revised Submission Borland Software Corp. Objective Interface Systems, Inc. VERTEL Corporation OMG TC Document mars/ March 3, 2003 Joint Messaging Service Submission 1

2 Copyright 2003 by Borland Software Corp. Copyright 2003 by Objective Interface Systems, Inc. Copyright 2003 by Vertel Corp. The submitting companies listsed above have all contributed to this revised submission. These companies recognize that this revised submission is the joint intellectual property of all the submitters, and may be used by any of them in the future, regardless of whether they ultimately participate in a final joint submission. The companies listed above hereby grant a royalty-free license to the Object Management Group, Inc. (OMG) for worldwide distribution of this document or any derivative works thereof, so long as the OMG reproduces the copyright notices and the below paragraphs on all distributed copies. The material in this document is submitted to the OMG for evaluation. Submission of this document does not represent a commitment to implement any portion of this specification in the products of the submitters. WHILE THE INFORMATION IN THIS PUBLICATION IS BELIEVED TO BE ACCURATE,THE COMPANIES LISTED ABOVE MAKE NO WARRANTY OF ANY KIND WITH REGARD TO THIS MATERIAL INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. The companies listed above shall not be liable for errors contained herein or for incidental or consequential damages in connection with the furnishing, performance or use of this material. The information contained in this document is subject to change without notice. This document contains information which is protected by copyright. All Rights Reserved. Except as otherwise provided herein, no part of this work may be reproduced or used in any form or by any means graphic, electronic, or mechanical, including photocopying, recording, taping, or information storage and retrieval systems without the permission of one of the copyright owners. All copies of this document must include the copyright and other information contained on this page. The copyright owners grant member companies of the OMG permission to make a limited number of copies of this document (up to fifty copies) for their internal use as part of the OMG evaluation process. RESTRICTED RIGHTS LEGEND. Use, duplication, or disclosure by government is subject to restrictions as set forth in subdivision (c) (1) (ii) of the Right in Technical Data and Computer Software Clause at DFARS CORBA and Object Request Broker are trademarks of Object Management Group. OMG is a trademark of Object Management Group. 2 mars/ Extensible Transport Framework

3 Introduction Submitting Companies 1.2 Status of this Document 1.3 Guide to the Submission The following companies are pleased to submit this revised specification in response to the OMG Extensible Transport Framework RFP (Document orbos/ ). Borland Software Corp. Objective Interface Systems, Inc. VERTEL Corporation with support from: Mercury Computer Systems, Inc. This document is a revised submission produced for the March 2003, OMG Technical Committee meeting. The relevant IDL file can be found as OMG document mars/ This submission will describe the specification for a framework to enable third parties to design and implement a Transport Plugin that can exchange messages (eg. GIOP) between server and clients over a non-tcp transport layer. The responses to the RFP requirements can be found in Chapter 2. A Design discussion and detailed interface description can be found in Chapter 3. The final Chapter summarizes the conformance issues introduced by this specification. 3/3/03 mars/ : Extensible Transport Framework 3

4 1 1.4 Proof of Concept This submission is derived from the following design and implementation efforts: work performed by Highlander Engineering, Inc. (now part of Borland Software Corp.), for the VisiBroker-RT for Tornado product; and work performed by Objective Interface Systems, Inc. for the ORBexpress product line. 1.5 Conventions IDL appears using this font. Language Mapped code appears using this font. Important Reminders appear using this font. In various places a few issues are highlighted. These are mostly areas where we have discovered that some additional clarification may be needed. A statement expressing a requirement will highlight its verb in bold, eg. shall or may not. 1.6 Submission Contact Points All questions about this submission should be directed to: Dr. Andreas H. Wolf Borland Software Corp. 208 East Pine Street Lakeland, FL USA phone: fax: awolf@borland.com James E. Kulp Mercury Computer Systems, Inc. 199 Riverneck Road Chelmsford, MA USA phone jek@mc.com Bill Beckwith Objective Interface Systems, Inc Park Center Road, Suite 360 Herndon, VA USA phone: fax: bill.beckwith@ois.com 4 mars/ : Extensible Transport Framework 3/3/03

5 1 Shahzad Aslam-Mir, PhD VERTEL Corporation Warner Corporate Center Victory Blvd, Suite 700 Woodland Hills, CA USA 3/3/03 mars/ : Extensible Transport Framework 5

6 1 6 mars/ : Extensible Transport Framework 3/3/03

7 Response to RFP Requirements 2 The following describes the requirements from the Extensible Transport Framework RFP (OMG orbos/ ) and provides a brief description of how this submission is responsive to the RFP. More detail is generally provided within the Extensible Transport Chapter. 2.1 Scope Proposals responding to this RFP shall define concepts behind a separation of the message layer, such as GIOP, from the transport layer, such as the TCP/IP layer of IIOP. This shall include a formal call model between layers. This submission defines the concept of a 1-to-1, bidirectional data pipe, calling it a Connection. This object, as implemented by the plugin author, must comply to certain constraints to be usable for the message layer. (See Connection Interface on page 3-11.) The other important concept introduced in this submission is a server-side Listener object, which takes part in the connection creation, and request dispatch for messages received from any of the open connections. (See Observation 1: If a transport (eg, UDP/IP) does not provide these characteristics by itself, then the author of a plugin must add a code layer on top of the transport, and inside the Connection object, to ensure compliance. on page 3-12.) A Profile interface is provided to handle profile data related to any transport. (See Profile Interface and IORs on page 3-12.) Other interfaces are found in the section Dispatch Functions on page 3-18 and Creation Functions on page Responses shall identify interfaces that make the real-time ORB core, facility, and services layers independent of the transport technology. The interfaces defined in the IDL provided by this specification serve this purpose. 3/3/03 mars/ : Extensible Transport Framework 7

8 2 Issue We intend to use the local interface mechanism to keep calls from being mediated by the ORB. This keyword was introduced to IDL in CORBA version 2.4, and is not known to CORBA 2.3. We therefore propose to spec the Extensible Transport Framework for CORBA 2.4 and higher. Submissions must clearly indicate what kinds of transports are supported through these interfaces, including semantic restrictions. See Connection Interface on page 3-11 for a complete list of requirements for the Connection object presented by the plugin. The use of the abstract Connection concept means that even if the transport per se can not comply with these requirements, the author of a plugin is able to provide the missing functionality in the plugin code. Responses must clearly show how the proposed framework is extensible, permitting use of third party transport solutions. The use of the Factory pattern provides a way for the ORB to instantiate objects for any arbitrary plugged-in transport. The core of the extension mechanism lies in the definition of interfaces (through IDL) that are implemented by the plugin and by the ORB, which, when complying with these specifications, will be sufficient to successfully interact with the message layer of the ORB 2.2 Mandatory Requirements Responses shall specify interfaces and appropriate semantics for the following: Profiles: Responses shall define the IOR profile architecture for the non-tcp transports such that it is possible for the author to create a transport plug-in for two different ORBs that enable application interoperability across that transport This submission describes the general approach to handling IOR profiles and Connection creation in the first sections of Chapter 3, and it gives detailed function descriptions in the relevant sections in the later part of that Chapter. Communication: Responses shall identify the interfaces and call sequence between the real-time ORB and the transport plug-in. Within this proposal, we define IDL describing all interfaces in a language independent manner, and we have documented the meaning of each function as well as the call sequence in the related sections in Chapter 3. Selection: Responses shall identify how the real-time ORB selects a particular transport. This proposal relies on the Real-Time CORBA specifications (OMG Document: ptc/ ), which defines the following protocol selection mechanisms for clients: IOR order: 8 mars/ : Extensible Transport Framework 3/3/03

9 2 The default ordering and selection is derived from the profile order in the IOR used during the bind operation, connecting to a server. RTCORBA::ClientProtocolPolicy The client can set a policy to override the IOR selection method. The user can remove and/or reorder profiles based on their profile id. Both client selection methods provide the ProfileId (as defined in IOP) for the selection of a transport, which is then used to identify the correct plugin factory. For servers Real-Time CORBA defines: RTCORBA::ServerProtocolPolicy: A server can select and order transports on a per POA basis. Again, this policy object contains the ProfileId s of each selected transport, and that value can be used to create the correct server endpoints (Listeners) through the plugin factory. 2.3 Optional Requirements There were no optional requirements in the RFP 2.4 Issues to be discussed. The RFP contained no issues to be discussed. 3/3/03 mars/ : Extensible Transport Framework 9

10 10 mars/ : Extensible Transport Framework 3/3/03 2

11 Extensible Transport Framework Extensible Transport Architecture The Extensible Transport Framework specifies three main interfaces that are used to present an arbitrary transport protcol to the ORB: A Connection, a Profile, and Listener interface. The transport plugin must provide an implementation for all of these interfaces. The client side ORB makes use of the Connection and Profile. It can dynamically create Connection instances. Then the ORB can provide the neccessary address information, using a Profile instance, to successfully connect to a server. One Connection instance may be re-used to multiplex many client requests. The server side ORB makes use of the Listener interface, as well as the Connection and Profile. There is also a factory interface, which is used to create an instance of any of those objects. Again, the plugin must implement that interface to successfully interact with the ORB. Lastly, the ORB provides a handle interface for upcalls from the plugin into the ORB. The following sections describe these interfaces, first their characteristics, and then their individual operations, as defined in IDL Connection Interface This specification separates the message layer (GIOP) from the Extensible Transport Framework layer by employing the concept of a Connection. The Extensible Transport Framework defines the interaction between the Message layer and a Connection instance for both the client and the server side. Any Connection presented to the message layer by a plugged-in transport must satisfy the following characteristics: Reliable: An (in principle) arbitrary length message is reliably sent to the remote endpoint. All transport internal details like packaging, ordering of packets, or re-sending of dropped or corrupted packets shall be hidden. 3/3/03 mars/ Extensible Transport Framework 11

12 3 Bi-directional: Any reply message of a server shall be reliably received examining the local Connection instance after it was used to send that request. (This does not preclude the ORB from multiplexing client requests to the same server through a single connection.) Using such a Connection will enable the GIOP layer to send any kind of message by filling an adequate byte array (ie. message buffer) and pass it to the connection for sending data to the remote peer, without knowing any details of that transport. Secondly, to receive any message the GIOP layer can read standard, fixed-length GIOP message headers, decode them, and subsequently request the message data from a connection. Observation 1: If a transport (eg, UDP/IP) does not provide these characteristics by itself, then the author of a plugin must add a code layer on top of the transport, and inside the Connection object, to ensure compliance. The Connection interface provides all functions needed to send and receive messages, as well as functions to control the connection it represents. The GIOP layer will use the data exchange functions, ie. read and write to communicate with the remote peer, and monitor calls, eg. is_connected to validate the connection on the transport level. This specification describes an interface (through IDL) that shall be implemented by the Connection object in every transport plugin type. See The Connection Interface Definition on page 14. This interface hides the details of the various transports from the GIOP layer. The operations in the "Connection" interface are thread-reentrant such that: ORBs shall only call read() from one single thread at a time on the same connection object; multiple ORB threads may call read() simultaneously on different connection objects at the same time; ORBs shall only call write() from one single thread at a time on the same connection object; and multiple ORB threads may call write() simultaneously on different connection objects at the same time Profile Interface and IORs A single interface is used to manage profiles of any plugin. It contains conversion and matching functions to store profile data in an IOR, and to locate a matching profile read from an IOR. This design hides the details of the specific information stored in a profile. This object holds all data related to an address for a specific transport. It implements the operation to convert that data to a TaggedProfile object defined in IOP, which packages all profile data in an octet sequence. The author has to define all data items in a profile, and implement the insertion code for them. The related extraction operation is used in the Factory object that can create instances of this object. 12 mars/ Extensible Transport Framework 3/3/03

13 Listener Interface There exists an asymmetry between a server and a client in the use of a connection, which is mirrored in this specification. A server provides a transport endpoint that can be connected to on the initiative of a client wanting to send a request to that server. To direct the request to the correct server, IOP defines an IOR that can be interpreted by the client-side ORB. Since it describes a single (unique) endpoint of that transport, all client interested in establishing a connection are able to do this. Consequently, the straightforward use of such an endpoint for GIOP message exchange would create a 1-to-n relationship between the server and the (many) clients that use the same IOR. To recover the bi-directional, 1-to-1 relationship described previously for a connection, the endpoint described by a profile shall be used only to request a new Connection. It handles such requests internally and the only actions visible to the Extensible Transport Framework shall be the registration of a newly created connection through a defined interface. The object holding such a special endpoint is called Listener and shall be an implementation of the interface described in this specification. The author of the plugin shall design the connection request handling inside the transport layer or inside the Listener implementation. The same implementation can then be plugged into any other ORB compliant with this specification, and this way these ORBs can interoperate via the plugin. Observation 2: The author of a plugin must add code on top of the transport layer, if a transport does not contain the concept of a connection. During this work, the connection request mechanism must be designed and implemented as well. Observation 3: No part of this specification can guarantee interoperabilty between plugins that are using the same transport, but were created by two different third-party Plugin vendors, since they could design an incompatible connection request mechanism for the transport. See Observation 1: If a transport (eg, UDP/IP) does not provide these characteristics by itself, then the author of a plugin must add a code layer on top of the transport, and inside the Connection object, to ensure compliance. on page 12. for more details Ancillary Transport Interfaces Factories Two ancillary interfaces are provided by the Extensible Transport Framework: Handle Since the ORB connects to the plugin using objects that implement certain interfaces described in this specification, the factory pattern is used to create an instance of such an object. Consequently, the "Factories" are defined as operations on a single IDL Factories interface, that the author of a plugin must implement. See Section Observation 1: If a transport (eg, UDP/IP) does not provide these characteristics by itself, then the author of a plugin must add a code layer on top of the transport, and inside the Connection object, to ensure compliance. on page 12 3/3/03 mars/ Extensible Transport Framework 13

14 3 Some interaction between the ORB and a plugged-in transport must be performed through "upcalls" from the transport layer into the ORB. The Handle provides the interface for these calls. See Observation 1: If a transport (eg, UDP/IP) does not provide these characteristics by itself, then the author of a plugin must add a code layer on top of the transport, and inside the Connection object, to ensure compliance. on page The Connection Interface Definition The following sections give a detailed description of the IDL interface of each of these classes Data Exchange As described before (See Extensible Transport Framework on page 11), a connection object provides the means to send and receive (in principle) arbitrary length messages. The following operations define the semantics used to perform these actions. write(in boolean is_first, in boolean is_last, in Buffer data, in unsigned long offset, in unsigned long length, in TimeBase::TimeT time_out) The GIOP layer will pass message data stored in the Buffer (octet sequence) to the connection to be sent to the remote peer. The message data memory is described by an array (as defined by the IDL type sequence<octet> ), an offset into that array (where the data start), and the total length of the message data. This function shall send the complete message data to the transport layer or return an exception. If it fails to send the data it shall throw an exception, i.e. COMM_FAILURE, which describes the error. If sending the message data times out, it shall stop trying to send data and throw the TIMEOUT exception. NOTE: Since a connection must guarantee the integrity of the GIOP stream, it might be necessary to close a connection in case of a TIMEOUT exception, since a partial GIOP message might have been delivered to the transport. This shall be done by the plugin without the need of guidance by the GIOP layer. If the time_out value is 0 (zero) the call shall block until all data are sent or a failure in the transport is detected. We also include two boolean flags, that signal to the connection whether the current message data is the first part of a GIOP message for this connection, or the last part of a GIOP message to be sent through the connection. We anticipate the use of these flags in the buffering algorithm inside the transport layer. ORBs are allowed to call write() from only one thread at a time on the same connection object. Multiple ORB threads may call write() simultaneous on different connection objects at the same time. read(inout Buffer data, in unsigned long offset, in unsigned long min_length, in unsigned long max_length, in TimeBase::TimeT time_out) The GIOP layer will pass a memory buffer to this function to read fixed length message data and store them in this byte array. The array description is equivalent to the write call, using an array (defined as an octet sequence ), an offset, and a length parameter. This function shall fill the buffer with at least min_length bytes return an exception and may fill in additional data up to max_length. In the case that it fails to 14 mars/ Extensible Transport Framework 3/3/03

15 3 flush() read the minimum data due to an error in the transport layer it shall throw an exception, i.e. COMM_FAILURE, which describes the error. If reading the message times out, it shall stop trying to read data and throw the TIMEOUT exception. NOTE: In contrast to the write operation, no TIMEOUT situation should cause the plugin to close the connection. Only the GIOP level can check the incoming data for integrity. If the time_out value is 0 (zero) the call shall block until all data are read or a failure in the transport is detected. ORBs are allowed to call read() from only one thread at a time on the same connection object. Multiple ORB threads may call read() simultaneous on different connection objects at the same time. Explicitly flush any possible buffers on the local side of a connection. Data in any buffer shall be sent immediately to the remote peer. This method should be called just before the connection is torn down Connection Handling As described in Section Extensible Transport Framework on page 11, all client side connection operations are part of the Connection interface. These are: connect(in Profile server_profile, in TimeBase::TimeT time_out) close() Attempts to establish a 1-to-1 connection with a server using the Listener endpoint from the given Profile description. It shall throw a COMM_FAILURE exception if it fails (e.g. if the endpoint is unreachable) or a TIMEOUT exception if the given time_out period has expired before a connection is established. If the connection is successfully established it shall store the used Profile data (see further below). Close the connection by using the transport layer s mechanism to disconnect the peer endpoint and then to release all local resources related to this connection. After this call, the ORB shall no longer try to read or write any data on this connection. is_connected() Returns a boolean flag, signalling the state of the connection. It shall return false, if it is not (or no longer) possible to send or receive data through this connection. Otherwise, it shall return true Reading the Server Profile The attempt of a client to connect to a server must be preceded by the initialization of the profile information in that connection instance. In this specification we pass a Profile into the connect method to provide this data. In order to locate an already established connection (by looking at its peer s profile data), we need to have read access to this data. In addition, it is sometimes useful to obtain the Profile for a Connection created by a Listener. The Connection shall store the current profile and the following function shall be used to retrieve it. 3/3/03 mars/ Extensible Transport Framework 15

16 3 Profile get_server_profile() The ORB may use this read method to examine the profile of the remote server that an instance of this object may be connected to. If the Connection object s connect function has never been called, this function shall throw the INITIALIZE exception Request Dispatching Support The data exchange methods described so far are very well suited for a thread per connection/session type model in the request handling (dispatching). To improve the implementation of other dispatching models, that allow a temporary mapping of request execution objects to connections, the connection interface includes a few additional operations. These support the query for data on the local endpoint of a connection, without removing any data from the connection s buffer. is_data_available() Simply return true if calling a read on this instance would find data in the connection. Otherwise, the function shall return false. wait_next_data(in TimeBase::TimeT time_out) Wait for the given time_out period for incoming data on this connection. It shall return false if this call times out and no data is available. It may not throw a TIMEOUT exception. If data can already be read or arrives before the end of the time out, this function shall return true, immediately Connection Attributes A connection has certain attributes that represent some (immutable) characteristics, and may be read by the ORB at any time. supports_callback A boolean flag describing whether this connection supports the Bidirectional GIOP mechanism as described by GIOP-1.2 in CORBA (OMG Document: formal/ ). It shall return true if it does, and false if it does not. use_handle_timeout A flag directing the ORB to use either the Handle class to perform data queries with a time_out, or the transport layer (through this connection). The instance shall return true, if the Handle should signal time outs for read operations. Then the ORB may not call wait_next_data. Otherwise, a false shall be returned, and the function wait_next_message shall be implemented by this class. 16 mars/ Extensible Transport Framework 3/3/03

17 IDL Description module ETF typedef sequence<octet> Buffer;... local interface Connection void write(in boolean is_first, in boolean is_last, in Buffer data, in unsigned long offset, in unsigned long length, in TimeBase::TimeT time_out); void read(inout Buffer data, in unsigned long offset, in unsigned long min_length, in unsigned long max_length, in TimeBase::TimeT time_out); // transport needs to set data.length() to // offset + number of bytes actually read void flush(); void connect(in Profile server_profile, in TimeBase::TimeT time_out); void close(); boolean is_connected(); Profile get_server_profile(); boolean is_data_available(); boolean wait_next_data(in TimeBase::TimeT time_out); readonly attribute boolean supports_callback; readonly attribute boolean use_handle_time_out; The Listener Interface Definition Setup Functions The following operations are used during the creation and deletion of an instance of a Listener object: set_handle(in Handle up) This call establishes the link between the ORB (i.e. the Handle instance) and a server endpoint of the plugged-in transport. All calls upwards into the ORB shall use the given instance. 3/3/03 mars/ Extensible Transport Framework 17

18 3 accept() listen() destroy() This call is an alternative to using set_handle() to initiate the callback-style of accepting new connections. This call blocks until a client connects to the server. Then a new Connection instance is returned. The transport plug-in must ensure that a thread blocked in accept() returns when destroy() is called with a null object reference. The transport plug-in must raise the CORBA::BAD_INV_ORDER with minor code TBD} if the ORB calls this operation and set_handle() has ever been called previously on the same listener instance. It is possible that connection requests arrive after the initial creation of the Listener instance but before the conclusion of the configuration of the specific endpoint in this plugin. In order to provide a clear end of this configuration state, we added the listen() method. It is called by the ORB when it ready for incoming connection and thus signals the Listener instance to start processing the incoming connection requests. Therefore, a Listener instance shall not deliver incoming connections to the ORB before this method was called. The Listener is instructed to close its endpoint. It shall no longer accept any connection requests and shall close all connections opened by it Dispatch Functions An idle connection, not delivering any requests for a long period of time, can be disconnected from the request handling objects in the ORB. In this case, the connection shall be monitored by the Listener object, instead being examined by the ORB. Newly received data shall be signalled to the Handle and thus the connection shall be passed implicitly to the dispatching logic of the ORB and ignored by the Listener until it is returned to it. The following operations shall be used by the ORB or Handle object to comply with theses specifications: completed_data(in Connection conn) The connection instance is returned to the Listener. It now shall signal any incoming data to the Handle Profile Data A Listener object shall create its endpoint during construction and after that it shall contain the correct profile data, describing its address. An attribute in the Listener interface gives the ORB access to it. endpoint Returns a copy of the profile describing the endpoint of this instance. 18 mars/ Extensible Transport Framework 3/3/03

19 IDL Description module ETF... local interface Listener void set_handle(in Handle up); Connection accept(); void listen(); void destroy(); void completed_data(in Connection conn) ;... readonly attribute Profile endpoint; 3.4 Profile Interface Definition Conversion Function The ORB shall manage all Profiles through this base class, and use the marshalling operation provided by this interface to store them in an IOR. marshal(inout IOP::TaggedProfile tagged_profile, inout IOP::TaggedComponentSeq components) This function marshals the appropriate information for this transport into the tagged profile. This information will typically include: an endian octet a GIOP version; the network endpoint information for this Profile instance; and both the components passed to it by the ORB and any additional components as required by the transport plug-in. ORBs will typically need to call the IOR interception points before calling marshal() Match Function The ORB shall use the matching operation during the client-side selection algorithm. This will encapsulate the address information details inside this class. is_match(in Profile prof) This function shall determine if the passed profile, prof, is a match to this profile. The specifics of the match are left to the details of the underlying transport, however profiles shall be considered a match, if they would create connections that share the 3/3/03 mars/ Extensible Transport Framework 19

20 3 same attributes relevant to the transport setup. Among others, this could include address information (eg. host address) and transport layer characteristics (eg. encryption levels). If a match is found, it shall return true, or false otherwise Copy Function This call shall be used to create a copy of a profile. copy() This function shall return an equivalent, deep-copy of the profile on the free store Hash Support hash() To improve the management of a large set of profile instances, the author may provide a hash function using the data in a Profile instance. The Profile shall always implement this function and either return a hash number, or 0 (zero) if no hashing is supported GIOP Related Attributes To allow easy access to profile data needed in the early request message processing by the GIOP layer, we added an attribute to reflect the GIOP version number supported by this type of profile. version This attribute shall contain the GIOP version number that this profile supports. It is initialized each time an instance is created. 20 mars/ Extensible Transport Framework 3/3/03

21 IDL Description module ETF... local interface Profile void marshal(inout IOP::TaggedProfile tagged_profile, inout IOP::TaggedComponentSeq components); // marshal() must set data.profile_data.length() // to the index of the last octet marshalled + 1 unsigned long hash(); Profile copy(); boolean is_match(in Profile prof); readonly attribute GIOP::Version version; Handle Interface Definition Adding new Connections After a plugin was connected to a Handle via the set_handle() function, all new connections requested by a client shall be announced to the server-side ORB via a call to that very same Handle instance. This call also allows the ORB to reject the connection via a boolean return value, if any connection related limits were reached. add_input(in Connection con) Announces a new connection instance to the ORB. The caller shall examine the boolean return value and destroy the connection, if the call returns false. A new connection initially belongs to the plug-in, and it shall signal the connection to the ORB when the first incoming request data was received, using this Handle upcall. The Handle shall accept the connection (and cache information about it if needed), as long as it is allowed to do so by the ORB. In this case it shall return true. If a new connection is currently not allowed, it shall ignore the passed instance and return false Signalling Incoming Data signal_data_available(in Connection conn) The plugged-in transport (e.g. the Listener instance) shall call this function when it owns a server-side Connection and data arrives on the local endpoint. This will start a new request dispatching cycle in the ORB. Subsequently, it shall ignore any other incoming data from this Connection until the Listener s completed_data function is called by the ORB. 3/3/03 mars/ Extensible Transport Framework 21

22 3 The Handle shall initiate the request dispatching in the ORB for this connection. It depends on the design of the request handling mechanism, when and if the connection will be given back to the plugin Client Side Close closed_by_peer(in Connection conn) In some cases, the client side can initiate the closing of a connection. The plugin shall signal this event to the server side ORB via its Handle by calling this function IDL Description module ETF... local interface Handle boolean add_input(in Connection conn); // Tells ORB that a new connection has come in // ORB returns false if it rejects new connection... void signal_data_available(in Connection conn); void closed_by_peer (in Connection conn); 3.6 The Factories Interface Definition This specification uses the factory pattern to allow the ORB to generate instances of implementations provided by any plugged-in transport. To simplify the design, all create operations related to such an implementation are put into a single IDL interface. Additionally, a Factories object needs to have an identifier so that the ORB can select the correct transport type Creation Functions create_connection(in RTCORBA::ProtocolProperties props) This function shall create an unconnected instance of the object implementing the Connection interface, suitable for this transport. The properties related to this transport are passed following the RTCORBA specs. (OMG document orbos/ ) These properties shall be read from client side protocol policies, as defined by RTCORBA, and were obtained from either the object reference, the PolicyCurrent or the ORB. The caller shall locate the correct policy instance as described in RTCORBA, obtain the ProtocolProperties object for the transport and pass it into this function. NOTE: The instance returned by create_connection can either be a Connection or a ConnectionZeroCopy object. The ORB can distinguish between them by trying to narrow to the ConnectionZeroCopy type. 22 mars/ Extensible Transport Framework 3/3/03

23 3 create_listener(in RTCORBA::ProtocolProperties props, in unsigned long stacksize, in RTCORBA::Priority base_priority) This function shall create an instance of the object implementing the Listener interface, suitable for this transport. As described earlier, this instance shall create a new endpoint for receiving connection requests, and the related profile shall be accessible immediately after it is returned from this function. Again, the ProtocolProperties defined by RTCORBA shall be passed to the function, as obtained from the ServerProtocolPolicy that is set on the related POA. Additionally, the specific Plugin is likely to use a thread inside of its listening mechanism (to perform callbacks into the ORB). In order to control the properties of this thread we pass the relevant properties explicitly as two parameters. They were copied from the threadpool specs of RTCORBA. This way of controlling the thread is advantaguous over passing a threadpool id, since we avoid any interaction between threadpool limits imposed on request management and the creation of Listener endpoints by the ORB. If a Plugin does not need an internal thread it shall ignore the relevant parameters. demarshal_profile(inout IOP::TaggedProfile data, out IOP::TaggedComponentSeq components) This function shall create a new instance of the object implementing the Profile interface for this plugin by demarshalling the information in the tagged_profile. This function must demarshal any tagged components in the profile and put them in the components parameter Identification profile_tag Since a Factories instance represents the plugged-in transport as a whole, it shall identify itself using a unique attribute value. Its value shall be the specific Profile Tag number chosen for its transport. 3/3/03 mars/ Extensible Transport Framework 23

24 IDL Description module ETF... local interface Factories Connection create_connection (in RTCORBA::ProtocolProperties props); Listener create_listener (in RTCORBA::ProtocolProperties prop s, in unsigned long stacksize, in RTCORBA::Priority base_priority) ; Profile demarshal_profile (inout IOP::TaggedProfile tagged_profile, out IOP::TaggedComponentSeq components); readonly attribute IOP::ProfileId profile_tag; Optional Interface for Zero Copy Some transport libraries support the notion of a zero copy data transfer into the transport layer. In the Connection interface, we did not use that feature and thus the transport plugin code is forced to eg. make a copy of all data it wants to send before returning from the write call. Similarly, data in transport level buffers need to be copied into the ORB provided buffer in case of a read operation. The following optional interface tries to address this feature, without claiming to support all the various approaches to handle the sharing of memory between the ORB and the transport Memory management BufferList This is local interface that manages multiple buffers that represent the data that either needs to be sent to the remote peer, or the data received by the connection instance. It contains references to octet sequences which can be manipulated by the ORB or the transport layer without the need to copy their contents. add_buffer(in unsigned long size, inout Buffer buf) This is the method how the ORB shall allocate a buffer compatible with the zero copy transport mechanism. It is guaranteed to hold at least as many bytes as was requested in the size parameter. In case new memory is needed, it shall be added to the provided list. The memory is returned in the buf parameter. This function should set the buf.length() to the actual memory allocated (this may be more or less than the amount requested by the size parameter). This function returns the index of the buffer for use by the ORB in future calls to get_buffer(). get_buffer(in unsigned long index, inout Buffer buf) 24 mars/ Extensible Transport Framework 3/3/03

25 3 This is the method how the ORB shall retreive a previously allocated buffer compatible with the zero copy transport mechanism. The index parameter is zero origin so that the first buffer is index zero. In case the given index is out of range (larger or equal num_buffers ) a BAD_PARAM exception shall be thrown. release_buffers() This call explicitely releases all buffers in the BufferList. The ORB can manage the use of Buffer memory without the need to delete the BufferList instance itself. When called, all Buffer instances shall no longer be regarded valid in the get_buffer() method, independent of the actual underlying memory allocation mechanism. This shall be signalled by setting the num_buffers attribute to zero. readonly attribute unsigned long num_buffers This attributes allows the ORB to query the BufferList for the total number of allocated buffers Data Transfer The following function shall be used to transfer data in and out of a connection supporting zero-copy: write_zc(inout BufferList data, in TimeBase::TimeT time_out) Write data in a zero-copy buffer list to the transport. The implementation of this connection object is expected to take ownership of the passed in buffers, so they do not need to be copied. Instead, the data parameter shall have no buffers after the call. The time_out parameter has the same meaning as in the write call of the non-zero-copy connection. read_zc(inout BufferList data, in unsigned long min_length, in TimeBase::TimeT time_out) This call shall read into plug-in supplied buffers and place them into the zero-copy list. The given min_length parameter shall reflect the minimum number of bytes the ORB wants to read. The time_out parameter has the same meaning as in the read call of the non-zero-copy connection. Again, appropriate exceptions shall be generated, if the requested amount of data can not be read from the transport or if the read times out. NOTE: Buffers used in the non-zero-copy interface may not be used in the zero-copy interface and vice versa. 3/3/03 mars/ Extensible Transport Framework 25

26 IDL Description module ETF... local interface BufferList unsigned long add_buffer(in unsigned long size, inout Buffer buf); // Adds an additional buffer to the list. // Returns the zero-origin index of the added buffer. // buf.length() should be set to the actual size of the // memory allocated whether more or less than size. readonly attribute unsigned long num_buffers; void get_buffer(in unsigned long index, inout Buffer buf); // populates the buf argument with the pointer to data void release_buffers(); local interface ConnectionZeroCopy : Connection BufferList create_buffer_list(); void write_zc(inout BufferList data, in TimeBase::TimeT time_out); }... void read_zc(inout BufferList data, in unsigned long min_length, in TimeBase::TimeT time_out); 3.8 Initialization The Factories object is the single interface implementation used for registration and retrieval of objects related to a given transport plug-in. There is one Factories instance per plugged in transport. The ORB shall provide the means to register and locate all Factories instances plugged into the ORB, but the implementation details of this functionality are not in the scope of this specification. NOTE: As a consequence, the registration of newly created Factories with the ORB is vendor specific, but also more flexible in its use of the ORB infrastructure. The drawback of a vendor specific API seems to be small (ie. a single, vendor dependant method call) compared to the restrictions imposed by requiring a fully initialized ORB to be used for such a registration. 26 mars/ Extensible Transport Framework 3/3/03

27 3 3.9 Consolidated IDL // File: ETF.idl #ifndef _ETF_IDL_ #define _ETF_IDL_ #ifdef _PRE_3_0_COMPILER_ # pragma prefix "omg.org" # include "orb.idl" # include "IOP.idl" # include "GIOP.idl" # include "RTCORBA.idl" # include "TimeBase.idl" # ifdef _NO_LOCAL_ # define local # endif #else import ::CORBA; import ::IOP; import ::GIOP; import ::RTCORBA; import ::TimeBase; #endif // _PRE_3_0_COMPILER_ module ETF # ifndef _PRE_3_0_COMPILER_ typeprefix ETF "omg.org"; # endif // _PRE_3_0_COMPILER_ typedef sequence<octet> Buffer; local interface Profile void marshal(inout IOP::TaggedProfile tagged_profile, inout IOP::TaggedComponentSeq components); // marshal() must set data.profile_data.length() // to the index of the last octet marshalled + 1 unsigned long hash(); Profile copy(); boolean is_match(in Profile prof); readonly attribute GIOP::Version version; local interface Connection void write(in boolean is_first, in boolean is_last, in Buffer data, in unsigned long offset, in unsigned long length, in TimeBase::TimeT time_out); void read(inout Buffer data, in unsigned long offset, 3/3/03 mars/ Extensible Transport Framework 27

28 3 in unsigned long min_length, in unsigned long max_length, in TimeBase::TimeT time_out); // transport needs to set data.length() to // offset + number of bytes actually read void flush(); void connect(in Profile server_profile, in TimeBase::TimeT time_out); void close(); boolean is_connected(); Profile get_server_profile(); boolean is_data_available(); boolean wait_next_data(in TimeBase::TimeT time_out); readonly attribute boolean supports_callback; readonly attribute boolean use_handle_time_out; local interface Handle boolean add_input(in Connection conn); // Tells ORB that a new connection has come in // ORB returns false if it rejects new connection void signal_data_available(in Connection conn); void closed_by_peer (in Connection conn); local interface Listener void set_handle(in Handle up); Connection accept(); void listen(); void destroy(); void completed_data(in Connection conn) ; readonly attribute Profile endpoint; local interface Factories Connection create_connection (in RTCORBA::ProtocolProperties props); Listener create_listener (in RTCORBA::ProtocolProperties props, in unsigned long stacksize, in RTCORBA::Priority base_priority) ; Profile demarshal_profile (inout IOP::TaggedProfile tagged_profile, out IOP::TaggedComponentSeq components); readonly attribute IOP::ProfileId profile_tag; // Optional zero copy connection interface 28 mars/ Extensible Transport Framework 3/3/03

29 3 local interface BufferList unsigned long add_buffer(in unsigned long size, inout Buffer buf); // Adds an additional buffer to the list. // Returns the zero-origin index of the added buffer. // buf.length() should be set to the actual size of the // memory allocated whether more or less than size. readonly attribute unsigned long num_buffers; void get_buffer(in unsigned long index, inout Buffer buf); // populates the buf argument with the pointer to data void release_buffers(); local interface ConnectionZeroCopy : Connection BufferList create_buffer_list(); void write_zc(inout BufferList data, in TimeBase::TimeT time_out); void read_zc(inout BufferList data, in unsigned long min_length, in TimeBase::TimeT time_out); #endif // _ETF_IDL_ 3/3/03 mars/ Extensible Transport Framework 29

30 30 mars/ Extensible Transport Framework 3/3/03 3

31 Conformance Issues Introduction This chapter specifies the compliance points for this specification 4.2 Compliance 4.3 Optional Compliance In order to be conformant with this specification, a compliant Plugin shall implement all the non-optional IDL interfaces specified in the list below and act according to the specified semantics. These are: Profile, implemented by the compliant Plugin. Connection, implemented by the compliant Plugin. Listener, implemented by the compliant Plugin. Factories, implemented by the compliant Plugin. In order to be conformant with this specification, a compliant ORB shall implement all the non-optional IDL interfaces specified in the list below and act according to the specified semantics for the non-zero copy connections. These are: Handle, implemented by the compliant ORB. A Plugin conforms to the optional Zero Copy compliance point by additionally implementing the following interfaces: ConnectionZeroCopy, implemented by the compliant Plugin. BufferList, implemented by the compliant Plugin. There are no optional interfaces for the ORB, but a compliant ORB shall additionally use zero-copy connection semantics whenever possible. 3/3/03 mars/ : Extensible Transport Framework 31

32 4 32 mars/ : Extensible Transport Framework 3/3/03

33 Appendix: Call Sequence A A.1 Introduction This section contains the graphical representation of the call sequences between an ORB and a plugin that implement this specification. NOTE: This section does not add to the specification. Do not reference this section! Only the actual text in previous chapters has normative power. A.2 Creation of a server-side Listener This picture shows the sequence during the creation of a Listener instance on the server side. ORB Plugin (1) (2) (3) Factories::create_listener(...) Listener::set_handle(...) Listener::listen() (4) Listener::endpoint() (5) Profile::marshal(...) 3/3/03 mars/ : Extensible Transport Framework 33

34 At step (1) the server-side ORB decides to create a Listener of the specific type (identified by the Profile id). It locates the respective Factory and calls create_listener. Depending on the style of connection handling, it then must call set_handle (2) to allow callbacks from the Listener instance. To finally enable the creation of new connection (3), the ORB must call listen. At the time the ORB needs to read the endpoint s Profile (4) it calls endpoint. Any Profile of this type of transport can be marshalled (5) by calling marshal. A.3 Creating a client-side Connection This picture shows the sequence of calls to create a new connection on the client-side of an ORB. ORB Plugin (1) Factories::demarshal_profile(...) (2) Profile::is_match(...) (3) Factories::create_connection(...) (4) Connection::connect(...) After identifying the matching entry in an IOR, the demarshal_profile call will create a Profile instance that encapsulates the address information of this transport (step (1)). This instance can then be compared to the Profiles of already opened connections in (2) by calling is_match. Should a new connection be needed (3), the same Factory provides a call to create such a Connection instance, called create_connection. This new Connection needs to be connected to the peer in step (4) by calling connect. A.4 Creating a Server-Side Connection This picture shows the call sequence on the server-side ORB when a new connection is being established by another (client) ORB. 34 mars/ : Extensible Transport Framework 3/3/03