A Reliable Multicast Transport Protocol for Communicating Real-time Distributed Objects*

Transcription

1 A Reliable Multicast Transport Protocol for Communicating Real-time Distributed Objects* Jin Sub Ahn, Ilwoo Paik, Baek Dong Seong, Jin Pyo Hong, Sunyoung Han 3, and Wonjun Lee 4 Mobile Telecommunication Research Division, ETRI, Daejeon, Korea jsahn@etri.re.kr Dept. of Information & Communications Engineering, Hankuk University of Foreign Studies, Kyongki-do, Korea {steigensonne, iceboy98, jphong}@hufs.ac.kr 3 Dept. of Computer Science and Engineering, Konkuk University, Seoul, Korea syhan@cclab.konkuk.ac.kr 4 Dept. of Computer Science and Engineering, Korea University, Seoul, Korea wlee@korea.ac.kr Abstract. The TMO (Time-triggered Message-triggered Object) model is a well-known real-time object model for distributed and timeliness-guaranteed computing. A distributed environment of the model may be configured by a number of the TMO nodes as a private network. It requires high reliability due to the feature of a distributed IPC message. The TCP seems to be suitable to this model. However, if a message needs to be broadcasted or multicasted to the other objects, the more the number of the nodes increases, the less efficient the repetitive unicast delivery of the message is. A multicast transport protocol can be considered to overcome this problem. In this paper, we propose a reliable multicast transport protocol suitable for supporting a distributed environment of the TMO model and discuss its performance with respects to the real-time delivery and throughput comparing with the alternative protocols. Results from the extensive performance measurement demonstrate that the proposed protocol outperforms the conventional TCP and existing RMT protocols. Introduction In the past, research on real-time computing focused on the functionality of a kernel, but recent focus moves to the development of real-time system using a realtime object model. The TMO [] model has been greatly issued as a new programming paradigm. It integrates all merits of real-time programming, distributed system programming, concurrent programming, and object oriented programming. Thereupon a TMO model is proposed as a real-time object model for the concept of timeliness-guaranteed computing. The distributed TMOs communicate with each * This research was supported by the MIC (Ministry of Information and Communication), Korea, under the ITRC (Information Technology Research Center) support program supervised by the IITA (Institute of Information Technology Assessment).

2 Jin Sub Ahn, Ilwoo Paik, Baek Dong Seong, Jin Pyo Hong, Sunyoung Han, and Wonjun Lee other by using the TMO methods on a logical unicast or multicast channel []. They may pass messages to the other objects, which require high reliability and very short response time. If the TCP is applied, it is needed to establish connections as many as the number of distributed objects, moreover, the repetitive copy and delivery of a message is required whenever broadcasting or multicasting the message is needed. Therefore, the response time will undoubtedly increase in proportion to the number of distributed objects, and the real-time property might be inherently violated. The IP multicast has been considered as an efficient way to deliver a message to a group of the distributed objects at a time, while it does not guarantee perfect reliability. In this paper, we propose a reliable multicast transport protocol suitable for supporting real-time reliable communication among distributed TMO objects and present the design and implementation of the protocol. Also, we discuss its performance with respects to the real-time delivery and throughput comparing with the alternative transport protocols. Backgrounds. Communicating TMO Objects The TMO channel is identified by a channel ID, which should be assigned to be unique value within a distributed environment of objects. The TMO objects communicating with a channel may define their channel access modes: write, read, or read/write mode. Fig.. TMO IPC Model with Channel The read-mode or read/write-mode objects are allowed to receive all the messages passed through the channel. It means that each receiving object receives a copy of the same message sent by the sending object. A sender with the write or read/write access mode to the channel may send a message through the channel, but this message should be delivered to all the receiving objects waiting for reading the channel. This is why the distributed TMO model inherently requires multicasting capability. Features like above are defined as distributed IPC interface [3] which has functions for channel assignment, synchronization, message transfer, close, etc. TMO engine

3 A Reliable Multicast Transport Protocol for Communicating Real-time Distributed Objects 3 based on Linux, TMO-Linux [4], recently restrict the maximum number of distributed IPC channels to 3 and the length of a message to 56Kbytes.. Reliable Multicast Transport Protocol Standards To support reliability in one-to-many multicast transport, IETF RMT WG processes the standards for error and flow control of reliable multicast transport protocol. There are representative multicast transport protocols such as NORM (Nack-Oriented Reliable Multicast) [5][6] and ALC (Asynchronous Layered Coding) [7] based on LCT (Layered Coding Transport) [8]. The NORM protocol is designed for one-to-many multicast transport of bulk data. The basic behavior is that a receiver request retransmission to a sender polling NACK only when a packet is lost in order to prevent ACK implosion. When a receiver polls NACK, random backoff timer works to prevent NACK implosion. At this time, its duplicated NACK is checked by other receivers, unnecessary NACK is suppressed. A sender transfers data after FEC [9] encoding process. It can enable a receiver to recovery errors with FEC decoding. Consequently it brings the number of retransmission to decrease. The NORM protocol offers relatively high reliability but it is inferior to reliability of TCP. Another weak point is that response time takes long due to complexity of inner structure and congestion control (TFMCC [0]). Another multicast protocol, ALC, is designed for massively scalable multicast distribution on wireless/satellite environment. As compared with NORM, the major difference is that the ALC can support multiple transfer rates for various heterogeneous receivers by using congestion control such as WEBRC []. However, there is no feedback between a sender and receivers, and thus the sender cannot confirm the status of packet reception. The ALC can rely upon FEC codec for reliability. Therefore, existing RMT protocols such as NORM and ALC are not suitable to distributed environment of TMO model in terms of real-time response and perfect reliability. 3 RM-IPC: A RMT Protocol for Communicating Distributed Objects 3. RM-IPC Message Format We can classify RM-IPC messages into three types of CMD, DATA and ACK. DATA message is used for application data delivery, and the other messages are for control and session management. The message types are defined as fixed header and header extension as each message should have different header values depending on its usage. The size of each message is variable.

4 4 Jin Sub Ahn, Ilwoo Paik, Baek Dong Seong, Jin Pyo Hong, Sunyoung Han, and Wonjun Lee Table. RM-IPC Message Types Packet Acronym Transport Type From To Timestamp CMD(TS) Multicast Sender Receiver ACK for Timestamp ACK(TS) Unicast Receiver Sender Receiver List for Advertisement CMD(ADV) Multicast Sender Receiver Data DATA Multicast Sender Receiver Flush CMD(FLUSH) Multicast Sender Receiver ACK for Flush ACK(FLUSH) Unicast Receiver Sender End of Transmission CMD(EOT) Multicast Sender Receiver ACK for EOT ACK(EOT) Unicast Receiver Sender Table summarizes the characters of messages used in RM-IPC. And Fig. presents the major RM-IPC header formats as below ver hlen type chid sequence number checksum (a) Fixed Header Format flavor (= ) reserved acking node list (IP address) window size (b) CMD(ADV) message header format flag lowest sequence number ACK bitmap length reserved payload length payload (c) DATA message header format ACK type (= ) ACK bitmap length lowest sequence number ACK bitmap reserved (d) ACK(FLUSH) message header format Fig.. RM-IPC message header formats

5 A Reliable Multicast Transport Protocol for Communicating Real-time Distributed Objects 5 3. Session Creation and Termination The important role of RM-IPC is that a sender should maintain a receiver list and measure GRTT(Group RTT). The message exchange between a sender and receivers for session creation/termination is similar to 3-way handshake in TCP, but the only difference is to enable to measure GRTT and to manage receivers of the session. Fig. 3 presents the state transition diagram. appl : passive open LISTEN recv : CMD(TS) send : ACK(TS) CMD(TS) RCVD recv : CMD(ADV) appl : active open send : CMD(TS) CMD(TS) SENT recv : All ACK(TS)s send : CMD(ADV) Data Transmission/ Reception recv : CMD(EOT) CMD(EOT) RCVD appl : active close send : CMD(EOT) CMD(EOT) SENT appl : passive close send : ACK(EOT) timer : Retransmission Timeout normal transitions for sender normal transitions for receiver Fig. 3. RM-IPC state transition 3.3 Data Transmission and Reception After session creation, the RM-IPC sender can transfer data, delivered from an application, to receivers by multicast. The sender copies the data to inner buffer by the API function rmipc_send() invoked from an application. And then, it processes segmentation procedure in order to fit the maximum segment size(456 bytes) of RM-IPC. Then the sender configures header of the data, sets sequence number, calculates checksum, and transfer data. The RM-IPC receiver checks the sequence number and evaluates header checksum and. Then, a cumulative feedback is sent to the sender by ACK(FLUSH) bitmap. All data, received from the sender, are assembled in an order that the sender has sent and delivered to the application. Fig. 4 and Fig. 5 present the state transition in Data Transmission/Reception state according to the role of a sender and a receiver.

6 6 Jin Sub Ahn, Ilwoo Paik, Baek Dong Seong, Jin Pyo Hong, Sunyoung Han, and Wonjun Lee Fig. 4. Data Transmission state transition (sender) Fig. 5. Data Reception state transition (receiver) 3.4 Repairing Data In this section, we describe error detection, retransmission by using ACK bitmap and error recovery mechanism related to retransmission of a sender. If the checksum is not valid, the packet is regarded as incorrect and dropped. The sender can detect packet loss by comparing sequence number with the very next one. Fig. 6 shows the sequence of message exchanges from session creation to termination between a sender and receivers. RM-IPC does not provide the status of packet reception by type of range such as selective ACK in TCP or NORM protocol. If the receiver gets a message successfully, it sets ACK bitmap, which corresponds to the sequence number. If not, set to 0. This is to let the sender know the total reception status efficiently by delivering an ACK bitmap once. The sender and receivers maintain the following values to manage an ACK bitmap.

7 A Reliable Multicast Transport Protocol for Communicating Real-time Distributed Objects 7 Receiver Sender Receiver rmipc_open() rmipc_bind() rmipc_setsockopt() passive open rmipc_solicit(block) LISTEN CMD(TS) RCVD Data Reception rmipc_recv(block) CMD(TS) ACK(TS) CMD(ADV) DATA(NEW) DATA(NEW) DATA(NEW) CMD(FLUSH) ACK(FLUSH) rmipc_open() rmipc_bind() rmipc_setsockopt() active open rmipc_advertise(block) CMD(TS) SENT default GRTT timeout Data Transmission rmipc_send() ACK Aggregation Repair Data Transfer ACK Aggregation CMD(TS) ACK(TS) CMD(ADV) DATA(NEW) DATA(NEW) DATA(EOR) CMD(FLUSH) ACK(FLUSH) DATA(REPAIR) DATA(REPAIR) CMD(FLUSH) ACK(FLUSH) rmipc_open() rmipc_bind() rmipc_setsockopt() passive open rmipc_solicit(block) LISTEN CMD(TS) RCVD Data Reception rmipc_recv(block) CMD(EOT) RCVD passive close rmipc_recv() returns 0 rmipc_close() CMD(EOT) ACK(EOT) Data Transfer rmipc_close(block) active close CMD(EOT) SENT All ACK(EOT)s received rmipc_close() returns CMD(EOT) ACK(EOT) CMD(EOT) RCVD passive close rmipc_recv() returns 0 rmipc_close() Fig. 6. Packet exchanges in RM-IPC session Low Sequence Number (LSN): the lowest sequence number of message which a sender sent. This corresponds to the first bit in ACK bitmap that a receiver transfers through ACK(FLUSH) message. Highest Sequence Number (HSN): the highest sequence number of a message which a sender sent. ACK bitmap length: the total number of messages which a sender sent. The valid ACK bitmap length is equal to the expression, HSN-LSN+. Considering the size of a distributed IPC message and MSS (456 bytes) of RM-IPC in TMO-Linux, the maximum length of ACK bitmap is 40. In RM-IPC, a sender makes a receiver poll ACK bitmap by sending CMD(FLUSH) or DATA(EOR). Each receiver transfers the ACK bitmap as type of ACK(FLUSH) message. ACK(FLUSH) is used not only for ACK bitmap deliver, but also for GRTT update. If a sender receives ACK(FLUSH) messages from all receivers, it aggregates them to decide whether it requests retransmission or not. If the result of AND operation of all bitmap is not equal to, the sender retransmits. The sender repeats the cycle from ACK Aggregation to Repair Data transfer until error recovery is finished. Even if ACK(FLUSH) message is received in the retransmission, GRTT is not updated. 3.5 API Table explains API functions of RM-IPC. RM-IPC API is designed at the view of following. Include the general RMT function in distributed IPC interfaces

8 8 Jin Sub Ahn, Ilwoo Paik, Baek Dong Seong, Jin Pyo Hong, Sunyoung Han, and Wonjun Lee Prefix, rmipc is to present RM-IPC protocol Design RM-IPC API on the foundation of the Berkeley Socket API. Table. RM-IPC API Function name rmipc_open() rmipc_bind() rmipc_setsockopt() rmipc_getsockopt() rmipc_advertise() rmipc_solicit() rmipc_send() rmipc_recv() rmipc_close() Description create socket (with setup distributed IPC channel) and decide a role (sender or receive) assign a local protocol address and port to a socket. set socket options (for interface, loopback). get socket option info. transfer CMD(TS) message for RTT (for only sender) transfer ACK(TS) message in response to RTT (for only receiver) send Application data delivery received data to Application terminate session and close a socket. 4 Experiments and Performance Evaluation 4. RM-IPC Experiments with TMO-Linux The RM-IPC has been implemented under Linux environment and has reflected not only requirements of TMO based distributed IPC interface, but also those of TMO-Linux. It has been integrated and tested with the TMO-Linux of TMO engines under the environment as follows: OS : Over the version of Linux Kernel.6 Compiler: g++ Timer resolution: ms Distributed IPC Channel: 3 channels (~3) 4. Performance Evaluation Considering a distributed environment of the TMO model, we evaluate the performance of the TCP, MCL-NORM, UDP/IP multicast and RM-IPC. Fig. 7 shows the results of measuring response time and throughput in UDP/IP multicast, TCP, and RM-IPC. We have measured the NORM protocol using MCLv3 [] open source and MCL-NORM library. The response time and throughput of the MCL-NORM has been shown as abnormally bad performance regardless of the number of receivers, since it excludes congestion control such as TFMCC. In practice, the response time of

9 A Reliable Multicast Transport Protocol for Communicating Real-time Distributed Objects 9 the MCL-NORM has been measured from 3 to 7 seconds while the other protocols could be measured by the unit of millisecond, and thereupon the result is so far and meaningless that it is excluded in those graphs. UDP/IP multicast may not support reliability, but we have evaluated it for comparing the speed of the RM-IPC. The result in comparison between TCP and RM- IPC is meaningful. The TCP should establish connections as many as the number of TMO nodes to support the distributed IPC interface. Total response time of TCP excluding connection establishment phase is also increasing in proportion to the number of receivers because it should transfer the same message repeatedly. Although the number of receivers is increasing, it only takes the time for polling ACK bitmap in the RM-IPC. Compared with TCP, the RM-IPC does not increase the response time too long unlike the case of TCP. In addition, the RM-IPC is able to transfer a message to all receivers by one shot of multicast without the consideration of the number of receivers. (a) Response Time (04 bytes) (b) Response Time (4096 bytes) (c) Response Time (0 receivers) (d) Throughput (MB file transfer) Fig. 7. Performance comparison of RM-IPC, UDP multicast, and TCP

10 0 Jin Sub Ahn, Ilwoo Paik, Baek Dong Seong, Jin Pyo Hong, Sunyoung Han, and Wonjun Lee 5 Conclusions The transport layer protocol, which is the most suitable to a distributed environment of a TMO model, should meet the conditions of reliability and short response time. One of the representative multicast protocols NORM inherently takes a long response time. Therefore, it is unfit to a distributed interface of a TMO model. A reliable unicast protocol such as the TCP meets the requirement for reliability, but the more the number of TMO node increases, the longer response time takes. In order to overcome the existing protocols, we have analyzed the requirements for the distributed IPC interface of the TMO model, designed and implemented a new RMT protocol named RM-IPC. We have also evaluated and compared the performance of the RM-IPC with the TCP and the other protocols. The proposed RM-IPC protocol has been shown to be fit to not only TMO, but also to various distributed environments. References. K. H. Kim and H. Kopetz, A Real-Time Object Model RTO.k and an Experimental Investigation of Its Potentials, in Proc. 8th IEEE Computer Software and Applications Conference, pp.39-40, November K. H. Kim, Realization of Autonomous Decentralized Computing with the RTO.k Object Structuring Scheme and the HU-DF Inter-Process-Group Communication Scheme, in Proc. ISADS 95, April J. G. Kim, M. H. Kim, K. Kim, and S.Heu, TMO-eCOS: An ecos-based Real-time Micro Operating System Supporting Execution of a TMO Structured Program, in Proc. ISORC 05, May Kim. J.G., et al. TMO-Linux: A Linux-based Real-time Operating System Supporting Execution of TMO s, Proc. ISORC 00, April B.Adamson, C.Bormann, M.Handley, and J.Macker, Negative-Acknowledgment (NACK)-Oriented Reliable Multicast (NORM) Building Blocks, IETF RMT WG, RFC 394, November B.Adamson, C.Bormann, M.Handley, and J.Macker, Negative-Acknowledgment (NACK)-Oriented Reliable Multicast (NORM) Protocol, IETF RMT WG, RFC 3940, November M. Luby, J. Gemmell, L. Vicisano, L. Rizzo, and J. Crowcroft, Asynchronous Layered Coding (ALC) Protocol Instantiation, IETF RMT WG, draft-ietf-rmt-pi-alc-revised- 0.txt, October M. Luby, L. Vicisano, J. Gemmell, L. Rizzo, M. Handley, and J. Crowcroft, Layered Coding Transport (LCT) Building Block, IETF RMT WG, draft-ietf-rmt-bb-lct-revised- 0.txt, October M. Luby, L. Vicisano, J. Gemmell, L. Rizzo, M. Handley, and J. Crowcroft, Forward Error Correction (FEC) Building Block, IETF RMT WG, RFC 345, December J. Widmer and M. Handley, TCP-Friendly Multicast Congestion Control (TFMCC): Protocol Specification, IETF RMT WG, RFC 4654, August M. Luby and V. Goyal, Wave and Equation Based Rate Control (WEBRC) Building Block, IETF RMT WG, RFC 3738, April MCLv3(Multicast Library v3),