Real-time video over wireless ad-hoc networks

Transcription

1 Real-time video over wireless ad-hoc networks Johannes Karlsson and Haibo Li Digital Media Lab Department of Applied Physics and Electronics Umeå University SE-90187, Umeå Sweden Jerry Eriksson Department of Computing Science Umeå University SE-90187, Umeå Sweden Abstract Sending real-time video over wireless ad-hoc networks is a challenging problem. Video is very sensitive for packet loss and wireless ad-hoc networks are error prone due to node mobility and weak links. In this paper we investigate important issues for real time video over wireless ad-hoc networks on different layers. Multi path routing has been used to increase the robustness for video over ad-hoc networks. This method is useful if the best path is a lossy path and there are alternative paths available. We will show that this rare for typical cases. In this paper we present a new proactive routing protocol for real-time video over wireless ad-hoc networks. We also present an error control method for transporting real-time video over ad-hoc networks using a single path. The performance of these methods is evaluated using simulations. I. INTRODUCTION There are many different types of ad-hoc networks, from very large networks to just a few nodes. The node mobility and node density can also vary a lot. We believe that many applications for real-time video over wireless ad-hoc networks will be for example remote expert help in factories, mines or disaster areas. For these applications the network size will be rather small, not more than nodes. It is likely that system as IEEE a, IEEE b or IEEE g WLAN will be used and the capacity in the network will therefore be rather high compared to what is needed for acceptable video quality. The node mobility will be rather low and the node density will be rather high. This will be a cooperative network and topics like greedy nodes and security will not be an issue. The devices we consider are small handheld or wearable devices so the video resolution will be small, probably QCIF (176x144 pixels), and the required video bit rate rather low. In this type of network, issues like high packet delivery ratio and low delay are much more important than scalability and low routing overhead. The most popular routing protocols today for ad-hoc networks, for example DSR [1] and AODV [2] are reactive routing protocols that are focused on scalability for very large networks and to reduce the routing load [3]. The major problem when sending real-time video over reactive routing protocols is however the long interruption in packet delivery when a route breaks and a new route has to be established. This is a drawback of the reactive design. To achieve high robustness for real-time video over ad-hoc networks issues like the delay and packet delivery ratio are very important. A proactive shortest path routing protocol can provide this, however, at the cost of higher routing load [4]. Many of the popular ad-hoc routing protocols, for example AODV and DSR, only search for the shortest path in the network and do not consider the quality of the link. The wireless links in an ad-hoc network may however vary a lot in quality and it is therefore important to also consider the quality of the link when choosing path. Just using the shortest path may lead to the result that many weak routes are used instead of the best route [5]. In the motion-compensated prediction (MCP) technique used in many video coding standards today a frame is first predicted from the previously coded frame (called reference frame), then the prediction error is coded and transmitted. This makes this type of video codec very sensitive for packet loss; a lost frame will not only cause error within its frame but also introduce error propagation in the following frames. To reduce the effect of lost packets it is important to use efficient error protection and error concealment methods for video transport. One of the most simple error protection methods is to insert periodically intra frames. These frames are not predictively coded from any previous frames and will therefore stop the error propagation. Methods that use retransmission of lost packets are not well suited because of the real-time traffic. Many other error control methods for video over ad-hoc networks use multiple streams and multiple paths in the ad hoc network [6]. Two examples are multiple description motion compensation coding (MDMC) [7] and layered coding with selective ARQ [8]. Ad-hoc routing protocols that make use of multiple paths have been proposed [9] [10]. These error control methods are however only efficient when the best path is a lossy path and there are alternative paths available. The error protection method we use in this paper is dynamic reference picture selection to always use the last possibly correctly decoded frame as reference. We will compare the performance of this error control method with periodic intra frames. To get fast and secure feedback we use explicit notification from the nodes in the network if a packet is lost. This method is used in other work [11] but they use end to end feedback and multiple paths. The video codec we use is the XviD [12] open source mpeg-4 implementation. This codec is modified to add the

2 Fig. 1. A plot of SNR and loss rate. Fig. 2. A plot of signal strength and loss rate. dynamic reference picture selection feature. The aim of this paper is to investigate important issues for unicast real-time video over wireless ad-hoc networks on different layers. In section II we are considering the link layer. In section III and IV we are considering different aspects of the network layer. In section V we propose an error control method for the video codec and compare it to another error control method. Finnaly section VI concludes this paper. II. LINK QUALITY The first experiment was to analyze the quality of the wireless link in IEEE b for different distances. Similar studies has been done in other works, see [5] and [13]. The aim of this experiment was to find out if it is possible to classify the expected link quality into three classes: no loss, lossy and no link. A. Method The test was conducted outdoor in an open field. The hardware used was Orinoco gold (025075/A) IEEE b wireless cards and laptops running Linux. We used two devices. One device was constantly sending packets with 10 ms between each packet. Each packet contains a sequence number that restarts on zero after 100 packets. So every second the sender was starting a new sequence. The other node recorded the received packets. Each time it received a packet having lower sequence number than the previous packet the amount of received packets, the SNR and the signal strength was recorded. For the transmissions, broadcast was used so there were no retransmissions of lost packets. There was therefore no problem with out of order delivery of packets. To measure the link quality wireless extension in Linux was used [14]. The quality was only measured once per second, that is once per 100 packets. The link quality and loss rate were measured for distances from zero meters up to 300 meters. B. Results The loss rate is low when the SNR is above 10 db. If the SNR drops below 10 db, the loss rate increases (Figure 1). It is however hard to find a direct relationship between the measured SNR and the loss rate. This has also noticed by other groups [5]. It is possible to discover the difference between a good and a bad link but not easy to calculate an exact loss rate for a link given the measured SNR. For the signal strength the loss rate starts to increase at about -82 dbm (Figure 2). We also measured the loss rate for different distances. Between 0 and about 250 meters the loss rate was low but after 250 meters the loss rate did increase a lot. III. SINGLE PATH OR MULTI PATH ROUTING Based on the result in our first experiment we classify the quality of the link as no loss, lossy and no link depending on the distance between the sender and the receiver. The next experiment was to randomly distribute 10 nodes and calculate the probability of having a route with different type of route qualities. The aim of this experiment was to see how often multi path transport of the video is a useful method to reduce the risk for packet losses in our type of network. A. Method We distributed 10 nodes in a square of varying sizes. The position of each node was uniformed randomly distributed. The quality of each link between all nodes was then calculated. We classified the quality of the link based on our results in the first experiment, experiments done by other [5], and the theoretical values used in the GloMoSim [15] network simulator. A link having a distance between 0 and 232 meters was classified as no loss, a distance between 232 and 376 as lossy, and for a distance of more than 376 meters as no link. If the distance was more than 376 meters between a pair of nodes we did not add a link between them. The graph is defined as: G = (V, E), (1) where V is the set of all the nodes in the network and E is the set of all the links in the network. A set of all possible routes between two nodes in the network were then calculated and also their loss rate. Each route in the set was a subset of the vertexes in the original graph. We also calculated the loss rate for each route. R i = 1 ((1 E 1 ) (1 E 2 )... (1 E i )), (2) where E i is the lossrate for each edge in the route. We classified the graph in four different classes. If the set of routes was the empty set we classify the graph as no route. If

3 Fig. 3. Average route quality for the best path Fig. 4. Average no loss routes for shortest path and best path the loss rate of the best route was more than zero and if there did not exist any disjoint route to the best route we classified the graph as lossy, no disjoint. Two routes were considered disjoint if the only nodes they had in common was the source and the destination node. (3) U 1 U 2 = {s, d} (3) where U 1 and U 2 are the set of nodes in the routes and s is the source node and d the destination node If the best route was lossy and there existed a disjoint path to the best path we classified the graph as lossy, disjoint. Finally, if the best path had a loss rate of zero we classified the graph as no loss. Thus, R = no route (4) i, R i = 0 no loss (5) i, R i 0 lossy (6) We used 30 different node densities, from 2 nodes per km 2 to 60 nodes per km 2. For each square we generated random graphs and calculated how often the different types of graphs appeared. B. Results A total of 30 different node densities were simulated, from 2 to 60 nodes per km 2 For all simulations we used 10 nodes. Because we used fixed number of nodes, the size of the simulated areas varied for different node densities. For the lowest simulated node density, 2 nodes per km 2, the area was 2236x2236 meters. For the highest node density, 60 nodes per km 2, the area was 408x408 meters. In figure 3 we can see that for low node densities there was most of the time no available path between the source and destination in the network. When the node density was increased it became more likely to find a path. For node density 20 nodes per km 2, an area of 707x707 meters, the amount of no route networks has decreased to 3.7%. It was also at this point there was the highest amount of lossy disjoint routes, 27.5%. The average amount of lossy disjoint routes for all simulated node densities was 13%. In most ad-hoc routing protocols the shortest path is used instead of the best available path. In figure 4 we can see that Fig. 5. The network used in the simulations. the amount of no loss routes was increased much faster when the best path was selected instead of the shortest path. For example, at node density 60 nodes per km 2 the amount of no loss paths was 60% for the shortest path but when the best path was selected the amount of no loss paths was 98% IV. REACTIVE OR PROACTIVE ROUTING The most common routing protocols, for example AODV and DSR, are reactive routing protocols. We believe that this type of routing protocols will not work well for real-time video. We have therefore developed a new link-state routing protocol. We compare the performance of real-time video over the new link-state routing protocol and AODV. Here we use a network where we will force some link breaks and compare the packet loss and delay for the both routing protocols. A. method The network we use in our simulation contains five nodes (Figure 5). The distance between the nodes is 200 meters. The links have no loss between zero and 232 meters then the loss increases up to 376 meters when there will be no link. The destination node starts between the source node and node number two. The node then moves at a constant speed until it is located 200 meters to the right of node four. First there will be a one-hop link between the source node and the destination node. When the destination node is located between node two and node three the one-hop route will be lost and a twohop route has to be established. When the destination node is located between node three and node four the two-hop route will be lost and a three-hop route has to be established. Finally before the destination node reaches its final destination 200 meters to the right of node four the three-hop route will be lost and a four-hop route has to be established.

4 Fig. 6. The delay for each frame when AODV is used. When the delay is zero the frame is lost. Fig. 8. Block based hybrid codec. V. ERROR CONTROL FOR THE VIDEO Fig. 7. used. the delay for each frame when the new link-state routing protocol is The data used in the simulations is the Foreman test sequence in QCIF format (176x144) and 30 frames per second. There are total 400 frames, only the first frame is coded as an intra frame and the rest are coded as predicted frames. The sequence is coded using a constant quality which leads to a variable bit rate (VBR). The average frame size is 775 bytes and this gives an average bit rate of 181 kbit per second. B. results When AODV is used the same route is used until it starts to lose packets. When this happens the routing protocol will initiate a new route discovery. During the time until a new route is found the routing protocol will buffer the packets. In this simulation the route will break three times, at about frame number 60, frame number 180 and frame number 300. When the route breaks first a few packets are lost, but there are also many packets that are lost due to late arrival. For the two first route breaks the interruption is less than one second and for the third route break the interruption lasts more than one second. This is an unacceptable interruption in real time video transmission. When our new routing protocol is used the source node will all the time know about all possible links in the network. This makes it possible for the routing protocol to always choose the best available route in the network. In this example there will always be one route available that has zero packet loss probability. Our link-state routing protocol will therefore not drop any packets in this simulation and the maximum delay is 29 ms. There is an increase in packet delay when the routes become longer. It is important to have efficient error control methods when sending video over wireless ad-hoc networks. In this experiment we develop a new error control method for realtime video called dynamic reference picture selection. For unicast real-time video we believe that this method is very efficient. We compare dynamic reference picture selection with the use of periodic intra frames for different packet loss rates. A. Video codec We use the XviD open source MPEG-4 implementation as the basis for our experiments. To this codec we add a new error control function: dynamic reference picture selection. The block based hybrid codec first divides the image into a number of non overlapping blocks. For MPEG-4 the block size is 8x8 pixels. The image contains one luminance component and two color components. The color component is usually sub sampled 2:1, so one 8x8 block for the color components corresponds to 16x16 pixels. Each block is first motion estimated (ME) against the previous decoded frame, called the reference frame. The motion vector specifies the displacement between the current block and the best matching block in the reference frame. The motion vectors are used to create a motion compensated (MC) block from the reference frame. The difference between the motion compensated block and the original block is then discrete cosine transformed (DCT). This is a transformation to the frequency plane and is used to exploit the correlation between error pixels. The quantized DCT coefficients and the motion vectors are then coded and transmitted. The encoder and the decoder must use the same reference picture. The encoder must therefore include a decoder to create this reference picture. In the decoder part the DCT components are first inverse quantized, and then inverse DCT transformed. Each error block is then added to corresponding motion compensated block to create the decoded picture. This will be the reference picture for the next frame. Video quality is usually measured as peak signal to noise ratio (PSNR). This is basically the normalized average difference between each pixel in the original video and the decoded video:

5 MSE = σe 2 = 1 (ψ 1 (m, n, k) ψ 2 (m, n, k)) 2 N k m,n ψmax 2 P SNR = 10log 10 σe 2, (7) where ψ 1 and ψ 2 are the two video sequences and ψ 2 max is the peak intenisty value of the video B. Method For dynamic reference picture selection we want to always use the last correctly decoded frame as the reference frame. This is however not possible because the only feedback we will have from the network is when something is lost, negative acknowledgment (NACK). We will therefore use the last possible correctly decoded frame as reference. This frame is chosen according to the following algorithm: 1) If a frame is reported to be lost but the frame was already known to be lost (long sequence of frame losses) the last possible correctly decoded frame will remain the same, 2) else if a frame is reported to be lost, the frame before the lost frame is the last possible correctly decoded frame, 3) else the previous encoded frame will be the last possible correctly decoded frame. Because we use interaction between the network layer and the video codec we can not use pre-encoded video. A lost video frame will affect the size of the following video frames. It is therefore necessary to integrate the video codec into the GloMoSim [15] network simulator. We add a new application layer in the simulator that will be the interface to our modified XviD video codec. They are connected using local sockets in Linux. The simulator is event driven and will periodically request a new video frame from the video codec. If a packet is lost the simulator will also inform the video codec about this. The modified XviD codec stores some extra bits in the bitstream and is not standard compliant. We send videos over a network simulator where we drop the packets randomly. Different packet loss rates are tested, from 0% to 15% loss. This is tested for two different intra frame intervals and for dynamic reference picture selection. For the dynamic reference picture selection there is no delay in the feedback. This means that the encoder will know about a loss before it encodes the next frame. If the feedback delay is increased the performance of dynamic reference picture selection will decrease. C. Results Dynamic reference picture selection has the highest average video quality for all loss rates (Figure 9). A short intra frame interval has a better average quality than a long intra frame interval. This is because dynamic reference picture will stop the error propagation immediately and short intra frame Fig. 9. The average video quality as a function of loss rate for different error control methods. Fig. 10. The average frame size as a function of loss rate for different error control methods. interval will stop the error propagation faster than long intra frame intervals. The average frame size is larger for short intra frame intervals then for long intra frame intervals. Dynamic reference picture selection has the smallest average frame size (Figure 10). This is because the intra frames are less efficient coded compared to the predictive coded frames. Dynamic reference picture selection always uses predicted frames and will therefore have the smallest average frame size. For periodic intra frame the average frame size is constant, but for the dynamic reference picture selection the average frame size is increased when the loss rate is increased. This is because dynamic reference picture selection has to use a worse reference picture when the previous frame is lost. Dynamic reference picture selection has both the highest video quality and the lowest average frame size for all loss rates tested. The combination of the effects leads to the result that dynamic reference picture selection has the highest Fig. 11. The coding efficiency as a function of loss rate for different error control methods.

6 coding efficiency measured as video quality per transmitted data (Figure 11). In this figure longer intra frame interval always has a higher efficiency than short intra frame intervals. However for some loss rates and intra frame intervals a shorter interval will be better. The variation in quality is also much lower for dynamic reference picture selection compared to periodic intra frame. For loss rate 5% the variance was 20.3 for intra frame interval 30. For the same loss rate the variance was 3.9 when dynamic reference picture selection was used. This is because dynamic reference picture selection will stop the error propagation immediately and only the lost frame will cause lowered frame quality. A high variation in video quality will be annoying for the user. VI. DISCUSSION It is only useful to use multiple paths in the network to reduce risk for packet losses when the best available path is a lossy path and there are disjoint paths available. In our simulations this was only the case for on average 13% of the time and at maximum 27.5% of the time for all simulated node densities. Our results show that having lossy disjoint routes were rarely the case for our type of network. A multipath transport protocol for video was therefore not useful most of the time. When the shortest path was selected the amount of lossy routes was increased a lot. In our simulations we selected the shortest path having the lowest loss. In most ad-hoc routing protocols a random shortest path is selected and the amount of lossy routes would be even higher. This indicates that it is important to take link quality into account when selecting the route and not just select a random shortest path. In our simulations we only search for a disjoint path from the best path. Even if such path did not exist there can exist another pair of disjoint paths. We also only searched for completely disjoint paths. Paths that were only partly disjoint can still be used to reduce the loss probability. The model used in the simulation is rather simple. In reallife networks the link can be more error prone and fluctuating due to for example user orientation and moving obstacles. A route discovery in AODV can take about one second; this is not an acceptable delay for real-time video. It is necessary to use a more proactive routing protocol. The linkstate routing protocol we have developed can all the time find the best available route in the network and will not cause any interruption in the video traffic as long as there is an route available between the source and the destination. For unicast real-time video the dynamic reference picture selection is very well suited as error control method. Retransmission of video frames are not possible because of the delay constrains. Error control methods that add redundancy to the video stream will have a lower coding efficiency compared to dynamic reference picture selection. The real-time video is not pre encoded which makes it possible to change the reference picture used in the encoding. A fast backchannel is also possible because of the short distance between source and destination in the ad-hoc network. Dynamic reference picture selection has the highest coding efficiency when there are no packet losses at the same time as it can stop error propagation of a lost frame very fast. VII. ACKNOWLEDGEMENTS We would like to thank the VITAL Project for making this research possible. We also thank Mikael Israelsson for the implementation of dynamic reference picture selection in XviD. REFERENCES [1] David B. Johnson, David A. Maltz, and Yih-Chun Hu. The dynamic source routing protocol for mobile ad hoc networks (dsr). Draft 9. [2] C. Perkins, E. Belding-Royer, and S. Das. Ad hoc on-demand distance vector (aodv) routing. RFC 3561, July [3] Chareles E. Perkins, Elizabeth M. Royer, Samir R. Das, and Mahesh K. Marina. Performance comparation of two on-demand routing protocols for ad hoc networks. IEEE Personal Communications Special Issue on Advances in Mobile Ad Hoc Networking, 8(1), February [4] Samir R. Das, Robert Castaeda, and Jiangtao Yan. Simulation based performance evaluation of mobile, ad hoc network routing protocols. ACM/Baltzer Mobile Networks and Applications (MONET) Journal, pages , July [5] Douglas S. J. De Couto, Daniel Aguayo, Benjamin A. Chambers, and Robert Morris. Effects of loss rate on ad hoc wireless routing. Technical Report MIT-LCS-TR-836, MIT Laboratory for Computer Science, March [6] Shiwen Mao, Shunan Lin, Shivendra S. Panwar, Yao Wang, and Embre Celebi. Video transport over ad hoc networks: Multistream coding with multipath transport. IEEE Journal On Selected Areas In Communications, 21(10): , December [7] Yao Wang and Shiwen Lin. Error resilient video coding using multiple description motion compensation. IEEE Trans. Circuits Syst. Video Technol, 12(6): , [8] Shiwen Mao, Shunan Lin, Shivendra S. Panwar, and Yao Wang. Reliable transmission of video over ad hoc networks using automatic repeat request and multipath transport. In Proceedings of IEEE Fall VTC, pages , Atlantic City, NJ, October [9] Mahesh K. Marina and Samir R. Das. Ad hoc on-demand multipath distance vector routing. Technical report, Computer Science Department, Stony Brook University, April [10] Sung-Ju Lee and Mario Gerla. Split multipath routing with maximally disjoint paths in ad hoc networks. In Proceedings of ICC 2002, Helsinki, Finland, June [11] Shunan Lin, Shiwen Mao, Yao Wang, and Shivendra S. Panwar. A reference picture selection scheme for video transmission over ad hoc networks using multiple paths. In Proceedings of IEEE ICME, pages 96 99, Tokyo, Japan, Aug [12] Xvid open source mpeg-4 implementation. [13] Dhananjay Lal, Arati Manjeshwar, Falk Herrmann, Elif Uysal-Biyikoglu, and Abtin Keshavarzian. Measurement and characterization of link quality metrics in energy constrained wireless sensor networks. In Proceedings of Globecom, [14] Tourrilhes/Linux/Tools.html. [15]