The Key Technology of Online Service System Based on MQTT. Da-mei CHEN and Ze-hua GAO

Transcription

1 2016 International Conference on Artificial Intelligence: Techniques and Applications (AITA 2016) ISBN: The Key Technology of Online Service System Based on MQTT Da-mei CHEN and Ze-hua GAO Beijing University of Posts and Telecommunications, Beijing, China Keywords: Online customer service system, MQTT, /Subscribe model, Servers. Abstract. The access traffic of websites mainly transfer from PC to Mobile terminal, which is different from the traditional portal webs. MQTT protocol is a lightweight agent-based publish/subscribe messaging transport protocol. Based on TCP, it has small traffic overhead and operates at low bandwidth and unreliable network environment, so it is ideal for use in mobile applications. This paper researches the key technology of online customer service system based on the MQTT protocol. Introduction With the development of the Internet, mobile Internet applications across every corner of people's daily lives. Online customer service system is a web-based instant communication instruments, which creates a real-time communication and exchanges platform for website visitors. The traditional online customer service system could not be met with the concurrency support. This causes the system to respond slowly, messages lost, and frequent problems. Currently, message transmission scheme on the mobile terminal mainly includes SIMPLE protocol, XMPP protocol and MQTT protocol in mobile social network. In the field of telecommunications, SIMPLE protocol as the network protocol of IMS core network has a more mature application base, a more mature audio standard, and supporting a variety of instant messaging. However, when the network environment changes from private network to mobile network and the terminal changes to mobile terminal, there will be more traffic, large packet, the congestion problem, low efficiency and more complex issues in extending new features. This will be a key factor in restricting instant messaging business development. On the traditional Internet, XMPP protocol is a widely used instant messaging. XMPP protocol is an XML-based open instant messaging protocol, integrated XML flexible scalability, but there are difficulties in storing and forwarding, heavy network traffic, large packet, protocol complexity, high cost of deployment, and the lack of stable standards in terms of audio and video as well. XMPP protocol is based on XML, and XMPP protocol itself will carry a lot of redundant information, which will directly lead to heavy network traffic and storage. Meanwhile MQTT protocol supports encrypted channel transmission based on TSL/SSL, authentication support during the connect time to ensure the security of message transmission and terminal access. Today, due to the increased use of smartphones, push notification services[1] are now commonly used. These benefits of MQTT agreements satisfy the needs of the application on mobile systems, which make MQTT can not only be applied to the mobile terminal and also the Internet of things. This means that although the MQTT protocol adds a very small header (i.e., only two bytes) to a message, TCP introduces a considerable overhead to provide reliable, ordered, and error-checked message delivery[2]. MQTT is a lightweight pub-sub messaging protocol for communication in Machine-to-Machine (M2M) or Internet of Things (IoT) contexts[3]. MQTT agreement has differences with SIP and XMPP protocol on the mechanism. The SIP and XMPP protocol use the point to point communication mechanism to determine the forwarding destination address according to the message field. MQTT agreement is based on publish / subscribe model introducing the theme of the proposed concept. If the Terminal subscribes to a topic and the will forward the messages to publish on the topic to the terminal for communication, messaging and offers three Qos quality assurance. One form of data-centric communication is the publish/subscribe messaging system. Compared with other data-centric variants, publish/subscribe 8

2 systems are common and wide-spread in distributed computing[4].the combination of the publish / subscribe model of MQTT and Qos guarantee for real-time communications such as point to point, group chat scenario have a natural advantage. As an example, Facebook Messenger[5] is based on MQTT, and many other open source projects are on the way based on MQTT. The Standards of MQTT Protocol The Overview of MQTT MQTT (Message Queuing Telemetry Transport)[6] is designed for the Internet of Things to deliver messages. MQTT protocol is open, streamlined, lightweight and easy to implement, especially for the low-bandwidth, unstable, high cost network environment and embedded devices and mobile terminals with limited processor and memory resources. MQTT is a lightweight duplexing asynchronous messaging protocol, whose features are fit for certain scenarios of the Internet of Things. Specifically, MQTT can solve intercommunication problems within devices of the Internet of Things, and communication problems between these devices and the back-end application systems. The Message Format of MQTT The signaling format of MQTT protocol includes three parts: fixed header, variable header and payload. Each command message must contain a fixed header. The Features of MQTT Table 1. MQTT message glands. bit Dup QoS RETAI byte 1 Message Type flag level N byte 2 Remaining Length MQTT is a lightweight machine-to-machine messaging protocol[7]. Providing one-to-many message releasing, the publish/subscribe messaging model as shown in Fig.1 can decouple application devices to transmit information by shielding the load contents. Compared to the polling method, push notification method was proved to be more efficient in battery and data consumption[8]. MQTT employs TCP/IP protocol to provide network connection; supporting small transmission whose overhead is small, minimizing agreement exchange to reduce network traffic. In addition, MQTT adopts the Last Will and Testament characteristic to inform concerned clients terminal mechanisms. QoS 0 QoS 1 QoS 2 /Subscribe /Subscribe /Subscribe er (source) subscriber (sink) Figure 1. Topic-based pub/sub communication model. Figure 2. Packet transmission method about QoS Level To ensure the reliability of messaging, MQTT supports 3 levels of Quality of Services (QoS) [9], as shown in Fig.2. Message promulgation totally depends on the underlying TCP / IP network, this level may loss or repeat message, so applying to the case that receiving message is lost once, because there will be a second transmission: "At least once" means the times of received is at least once to ensure that the message reaches, but the situation may occur repeatedly receive message: "only once". It 9

3 refers to the times of arrived message is one and only one, the highest level is suitable for situation that duplicating or missing messages will lead to incorrect results. The Design of Online Customer Service Based on /Subscribe Model From the entire instant communication platform, the online customer service system can be used as an application of a typical instant communication platform, which can ensure that the entire instant communication platform will access more business, so that the whole platform scalability enhancements, more efficient available. The system-architecture of instant communication should ensure the independence of each functional layer in order to facilitate rapid response to a variety of business needs. The reliable message delivery for point to point or for a group is implemented by a push which is based on MQTT protocol. The upload and download of small voice and video file are achieved by small distributed file. The storage and management of online customer service system are implemented by IM. The system architecture based on public/subscribe model as shown in Fig.3. Research of Key Middleware Figure 3. System architecture based on public/subscribe model. The Server of Pushing Message. This system selects an open. Which is called emqttd. The emqttd is implemented by erlang platform. The advantage of emqttd is that it primarily supports publish-subscribe feature, you can ensure reliable and secure transmission of messages. Also it can be easy to integrate with enterprise applications. Emqttd supports to notify news in real-time and push rich content. The flexible pub-sub and the storage and filtration of message can also be complied.it occupies little bandwidth and high bandwidth utilization, low power consumption of mobile applications. The Server of Distributed Small File Storage. The traditional way is to transfer message by text message and file together. But the MQTT message body is only suitable for a small transmission of document. For a file,which occupies a large space, may be interrupted in the process of uploading in the traditional way, so a new type of file transfer that establish a small distributed file storage in the process of implementation and design the interface of small text storage. The interface of uploading and downloading are standard http protocol instead of proprietary protocols which can reduce the work of client SDK integrated for the function of uploading and downloading. There are two approaches for reliable delivery of voice, video and other small files, one is that to regard the contents of the package as MQTT load by MQTT reliable service, the other is only included the package of MQTT link to download and the client will download after receiving the 10

4 message. The former approach is that if people receive a lot of small files messages when people are not online. The offline message will take up much memory of, you need to modify the and send a message to an external database such as additional storage. We chose the second method though the MQTT to download link can reduce the change of MQTT which is convenient to create professional software to store and expend small files. The Business Server. Because of the push is focus on reliable delivery of messages, interactive interfaces between the data of IM service and the terminal of the SDK will be used in the business. The business will package the rest business services and online customer service system. The function of the point to point service messages, the group message, the definition of business-critical messages, and the offline messages, all will be implemented by the business. In addition the file transfer, group management, relationship management, and other friendship will be achieved by the business. The Key Technology When the number of customers increased to a certain number, for example the high concurrent situation, online customer service system should have the ability to deal with the situation. So we increase the ngnix, a high-performance web and reverse proxy, in the actual implementation process. In the case of high concurrent connection, ngnix is a good alternative for Apache service. It can support up to 50,000 concurrent response to high connections. As a load balancing, nginx support rails and php internal procedures in foreign service, the program also supports http proxy service in foreign service. The whole architecture is as shown in Fig.4. ngnix Figure 4. The relation chart of system. Figure 5. The flow chart of messages. In order to make sure reliable message to arrive, the request from the client will send to the business at first, since the push only focus on how to publish/subscribe and push messages. The business performs recording process, then the message will send to MQTT by the business. After processing the message will be sent to the client, then if the message is received by client, the client will send a confirmation to the service, in order to ensure reliable delivery of messages throughout the process. The whole process of implementation as shown in Fig.5. Functional Testing Based on /Subscribe Model Testing environment: The browser as a functional testing terminal; Buidling emqttd based on MQTT protocol; Good network environment. The Scene One As shown in Fig.6, when we subscribe the topic MQTT1, the message can be received by the sender. The Scene Two As shown in Fig.7, when we do not subscribe the topic MQTT1, but we send the message about the topic MQTT2. The receiver will not receive these messages as show in the message list. 11

5 Figure 6. Public and Subscribe the topic and the message list about MQTT1. Figure 7. Public and Subscribe the topic and the message list about MQTT2. Conclusion This Paper introduces the key technology of online service system based on MQTT protocol. Comparing with other instant protocols, the paper analyse the advantage of MQTT which can be implemented in the low-bandwidth environment. The system can be expanded into a highly available instant messaging product. References [1] K. M. Bell, D. N. Bleau, J. T. Davey, Push notification service, U.S. Patent , November, [2] N. De Caro, W. Colitti, K. Steenhaut, G. Mangino and G. Reali, "Comparison of two lightweight protocols for smartphone-based sensing," 2013 IEEE 20th Symposium on Communications and Vehicular Technology in the Benelux (SCVT), Namur, 2013, pp [3] H. C. Jo and H. W. Jin, "Adaptive Periodic Communication over MQTT for Large-Scale Cyber-Physical Systems," Cyber-Physical Systems, Networks, and Applications (CPSNA), 2015 IEEE 3rd International Conference on, Hong Kong, 2015, pp [4] U. Hunkeler, H. L. Truong and A. Stanford-Clark, "MQTT-S A publish/subscribe protocol for Wireless Sensor Networks," Communication Systems Software and Middleware and Workshops, COMSWARE rd International Conference on, Bangalore, 2008, pp [5] Building Facebook Messenger, , cited August, 2012 [6] MQTT V3.1 Protocol Specification [7] Power Profiling: HTTPS Long Polling vs. MQTT with SSL, on Android, cited October, [8] Chen, Whei-Jen and Gupta, Rahul and Lampkin, Valerie and Robertson, Dale M. and Subrahmanyam, Nagesh, Responsive Mobile User Experience. [9] S. Behnel, L. Fiege, G. Muehl, On Quality of Service and Subscribe, Proceedings of the 26th IEEE International Conference on Distributed Computing Systems Workshops, July,