A Spatiotemporal Communication Protocol for Wireless Sensor Networks

Transcription

1 IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, TPDS R2 1 A Spatotemporal Communcaton Protocol for Wreless Sensor Networks Tan He, Member, IEEE, John A. Stankovc, Fellow, IEEE, Chenyang Lu, Member, IEEE and Tarek F. Abdelzaher, Member, IEEE Abstract In ths paper, we present a spatotemporal communcaton protocol for sensor networks, called. s specfcally talored to be a localzed algorthm wth mnmal control overhead. End-to-end soft real-tme communcaton s acheved by mantanng a desred delvery speed across the sensor network through a novel combnaton of feedback control and nondetermnstc geographc forward-ng. s a hghly effcent and scalable protocol for sensor networks where the resources of each node are scarce. Theoretcal analyss, smulaton experments and a real mplementaton on Berkeley motes are provded to valdate the clams. Index Terms Wreless sensor networks, routng, real-tme, spatotemporal, networkng. 1 INTRODUCTION M ANY sensor network applcatons, such as battlefeld survellance and earthquake response systems, are desgned to nteract wth fast changng events n the real world. It s often necessary for the underlyng communcaton nfrastructure to meet real-tme constrants [9][1][4]. In survellance systems [8], for example, communcaton delays wthn sensng and actuatng loops drectly affect the qualty of trackng. Our spatotemporal communcaton protocol, called, s nspred by followng observaton: In wred networks, the end-to-end delay s ndependent of the physcal dstance between the source and destnaton. Whle n mult-hop wreless sensor networks, snce communcaton s physcally bounded, the end-to-end delay depends not only on sngle hop delay (tme-constrants), but also on the dstance a packet travels (spatal-constrants). In vew of ths, the key desgn goal of ths work s to support a spatotemporal communcaton servce wth a desred delvery speed across the sensor network, so that end-to-end delay s proportonal to the dstance between the source and destnaton. We deem ths servce as one type of soft real-tme communcaton, because t acheves a predcable end-to-end communcaton delay under gven dstance (spatal) constrants. The key contrbuton of s achevng the spatotemporal requrements through a novel combnaton of a tme-aware feedback control mechansm and a spatalaware non-determnstc geographc forwardng scheme. We evaluate usng GloMoSm [22]. The performance results show that 1) reduces the number of Tan He s wth Department of Computer Scence, Unversty of Vrgna,Charlottesvlle, E-mal: tanhe@cs.vrgna.edu John A. Stankovc s wth Department of Computer Scence, Unversty of Vrgna, Charlottesvlle, E-mal: stankovc@cs.vrgna.edu Chenyang Lu s wth Department of Computer Scence & Engneerng, Washngton Unversty n St Lous. E-mal: lu@cs.wustl.edu Tarek F. Adelzaher s wth Department of Computer Scence, Unversty of Vrgna, Charlottesvlle, E-mal: zaher@cs.vrgna.edu packets that mss ther end-to-end deadlnes, 2) reacts to transent congeston n the most stable manner, and 3) effcently handles vods [12] wth mnmal control overhead. To demonstrate ts applcablty, we also mplement on the Berkeley motes [3]. The results show that helps balance the traffc load to ncrease the system lfetme. 2 STATE OF THE ART Several routng protocols have been developed for ad hoc wreless networks. Sensor networks can be regarded as a sub-category of such networks, but wth a number of dfferent requrements. In sensor networks, locaton s more mportant than a specfc node s ID. For example, trackng applcatons only care where a target s located, not the ID of the reportng node. In sensor networks, such spatal-awareness [7] s necessary to make the sensor data meanngful. Therefore, t s natural to utlze spatal-aware routng. A set of locaton based routng algorthms have been proposed. Fnn [5] proposed a greedy geographc forwardng protocol wth lmted floodng to crcumvent the vods nsde the network. GPSR [12] by Karp and Kung use permeter forwardng to get around vods. Smlarly, GOAFR [14] combnes the greedy routng wth the adaptve face routng to provde an asymptotcally optmal path to the destnaton. Geographc dstance routng (GEDIR) [20] guarantees loopfree delvery n a collson-free network. LAR [13] by Young-Bae Ko mproves the effcency of the on-demand routng algorthms by restrctng packet floodng n a specfed request zone. Basagn, et. al. propose a dstance routng algorthm [2] for moblty (DREAM), n whch each node perodcally updates ts locaton nformaton to other nodes. An updatng rate s set accordng to a dstance effect n order to reduce the number of control packets. Recently, Huang [4] et. al. proposed Mobcast protocol extended geocast by provdng just-n-tme nformaton dssemnaton to nodes n a moble delvery zone. Manuscrpt receved Oct.29, 2003;, revsed Sept 2,2004; accepted Jan, /05/$ IEEE

2 2 IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, TPDS R2 also utlzes geographc locaton to make localzed routng decsons. The key dfference s that takes tmely delvery nto account and s desgned to be the frst spatotemporal-aware communcaton protocol for sensor networks. Moreover, provdes an alternatve soluton to handle vods other than approaches based on planar graph traversal [12] [14] and lmted floodng [5]. Reactve routng algorthms such as AODV [17] and DSR [10] mantan routng nformaton for a small subset of possble destnatons, namely those currently n use. If no route s avalable for a new destnaton, a route dscovery process s nvoked. Route dscovery broadcasts can lead to sgnfcant delays n a sensor network wth a large network dameter. Ths lmtaton makes on-demand algorthms less sutable for real-tme applcatons. Several real-tme protocols have been proposed for ad hoc and sensor networks. SWAN [1] uses feedback nformaton from the MAC layer to regulate the transmsson rate of non-real-tme TCP traffc n order to sustan real-tme UDP traffc. RAP [1] uses velocty monotonc schedulng to prortze real-tme traffc and enforces such prortzaton through a dfferentated MAC Layer. V. Kanoda etc. [11] proposed a servce dfferentaton for delay-senstve traffc by prortzng Woo and Culler [21] proposed an adaptve MAC layer rate control to acheve farness among nodes wth dfferent dstances to the base staton. All of these algorthms work well by locally degradng a certan porton of the traffc. However, ths knd of local MAC layer adaptaton cannot handle long-term congeston where routng assstance s necessary to dvert traffc away from any hotspot. provdes a combnaton of MAC layer and network layer adaptaton that effectvely deals wth such ssues. To the best of our knowledge, no routng algorthm has been specfcally desgned to provde soft realtme guarantees under spatotemporal constrants for sensor networks. 3 DESIGN GOALS The key desgn goal of the algorthm s to support a spatotemporal communcaton servce wth a desred delvery speed across the sensor network, so that end-to-end delay s proportonal to the dstance between the source and destnaton. It should be noted that delvery speed refers to the approachng rate along a straght lne from the source toward the destnaton. Unless the packet s routed exactly along that straght lne, delvery speed s smaller than the actual speed of the packet n the network. For example, f the packet s routed n the opposte drecton from the destnaton, ts speed s negatve. Our algorthm ensures that ths condton never occurs. More specfcally, satsfes the followng desgn objectves. Soft Real-Tme: We defne the soft real-tme guarantee provded by as delay guarantee per unt delvery dstance (speed guarantee). Under ths guarantee, we can obtan a predctable end-to-end communcaton delay under gven spatal (dstance) constrants before hand. Consequently, applcatons can make admsson control to delver packets that are able to meet end-to-end deadlnes. Mnmal State Archtecture: The physcal lmtatons of sensor networks, such as large scale, hgh falure rate, and constraned memory capacty necesstate a mnmal state approach. only mantans mmedate neghbor nformaton. It does not requre a routng table as n DSDV [18] nor per-destnaton states as n AODV [17]. Thus, ts memory requrements are mnmal. Mnmum MAC Layer Support: does not requre real-tme MAC support. The feedback control scheme employed n allows all exstng best effort MAC layers. QoS Routng and Congeston Management: Most reactve routng protocols can fnd routes that avod network hot spots durng the route acquston phase. Such protocols work well when traffc patterns do not fluctuate durng a sesson. However, these protocols (e.g. [10]) are less successful when congeston patterns change rapdly compared to the sesson lfetme. When a route becomes congested, such protocols ether suffer a delay or ntate another round of route dscovery. As a soluton, uses a novel backpressure re-routng scheme to re-route packets around large-delay lnks wth mnmum control overhead. Traffc Load Balancng: In sensor networks, the bandwdth and energy are scarce resources compared to a wred network. Because of ths, t s valuable to utlze several smultaneous paths to carry packets from the source to the destnaton. uses non-determnstc forwardng to balance each flow among multple concurrent routes. Localzed Behavor: Pure localzed algorthms are those n whch any acton nvoked by a node should not affect the system as a whole. In algorthms such as AODV, DSR and TORA, ths s not the case. In these protocols, a node uses floodng to dscover new paths. In sensor networks where thousands of nodes communcate wth each other, broadcast storms may result n sgnfcant power consumpton and possbly a network meltdown. To avod that, all dstrbuted operatons n are localzed to acheve hgh scalablty. Vod Avodance: In some scenaros, pure greedy geographc forwardng may fal to fnd a greedy path to the destnaton, even when one actually exsts. handles the vod the same way as t handles congested areas and guarantees that f there s a greedy route between the source and destnaton, t wll dscover t. Note, whle does not use routng tables, does utlze locaton nformaton to carry out routng. Thus, we assume that each node s locaton-aware [7]. 4 PROTOCOL mantans a desred delvery speed across sensor networks wth a two-ter adaptaton ncluded for dvertng traffc at the networkng layer and locally regulatng packets sent to the MAC layer. It conssts of the followng components: An API A delay estmaton scheme A neghbor beacon exchange scheme A Non-determnstc Geographc Forwardng algorthm (NGF)

3 TIAN HE, JOHN A. STANKOVIC, CHENYANG LU AND TAREK F. ABDELZAHER: A SPATIOTEMOPORAL COMMUNICATION PROTOCOL FOR WIRELESS SEN- SOR NETWORKS 3 A Neghborhood Feedback Loop (NFL) Backpressure Reroutng Last mle processng Backpressure Reroutng Beacon Exchange Fgure 1: Protocol API UnCast MultCast AnyCast Last Mle Process SNGF Neghbor Table MAC NFL Delay Estmaton As shown n Fgure 1, NGF s the routng module responsble for choosng the next hop canddate to support the desred delvery speed. NFL and Backpressure Reroutng are two modules to reduce or dvert traffc when congeston occurs, so that NGF has avalable canddates from whch to choose. The last mle process s provded to support three types of real-tme communcaton servces, namely, realtme uncast, real-tme area-multcast and real-tme areaanycast, for sensor networks. Delay estmaton s the mechansm by whch a node determnes whether congeston has occurred. And beacon exchange provdes geographc locaton of the neghbors so that NGF can perform geographc based routng. The detals of these components are dscussed n the subsequent sectons, respectvely. 4.1 Applcaton API and Packet Format The protocol provdes four applcaton-level API calls: AreaMultcastSend (poston, radus, packet): Ths servce dentfes a destnaton area by ts center poston and radus. It sends a copy of the packet to every node nsde the specfed area wth a speed above a certan desred value. AreaAnyCastSend (poston, radus, packet): Ths servce sends a copy of the packet to at least one node nsde the specfed area wth a speed above a certan desred value. UncastSend(Global_ID, packet): In ths servce the node dentfed by Global_ID receves the packet wth a speed above a certan desred value. SpeedReceve(): ths prmtve permts nodes to accept packets targeted to them. There s a sngle data packet format n. It contans the followng major felds: PacketType: the type of communcaton -- Area Multcast, AreaAnyCast or Uncast. Global_ID: only used n Uncast communcaton to dentfy a destnaton node. Destnaton Area: Descrbes a three-dmensonal space wth a center pont and radus n whch the packets are destned. TTL: Tme To Lve feld s the hop lmt used for last mle processng. Payload. 4.2 Delay Estmaton We use sngle hop delay as the metrc to approxmate the load of a node. We notce that the delays experenced by broadcast packets and uncast packets are qute dfferent due to dfferent handlng nsde the MAC layer. Uncast packet delay s more approprate for makng routng decsons. In a scarce bandwdth envronment, we cannot afford to use probng packets to estmate the sngle hop delay. Instead, we use the data packets passng ths node to perform ths measurement. Delay estmaton s done at the sender, whch tmestamps a packet (T arrvng ) enterng the tal of network output queue and tme-stamps the packet (T departure) when the last bt of ths packet s sent out. Sngle trp delay equals the nterval between T arrvng and T departure. Propagaton delay s gnored. In case of transmsson falures, T departure s set and used for calculaton only at a successful transmsson. We compute the current delay estmaton by combnng the newly measured delay wth prevous delays va the exponental weghted movng average (EWMA) [15] as followng: Delay(k) = α * Delay new + ( 1-α)*Delay(k-1) We argue that ths delay estmaton s a better metrc than average queue sze for representng the congeston level of the wreless network, because the shared meda nature of the wreless network allows the network to be congested even f queue szes are small. 4.3 Neghbor Beacon Exchange Smlar to other geographc routng algorthms, every node n perodcally broadcasts a beacon packet to ts neghbors. Ths perodc beaconng s only used for exchangng locaton nformaton between neghbors. We argue that the beaconng rate can be very low when nodes nsde the sensor network are statonary or slow movng. Moreover, pggybackng [12] methods can also be exploted to reduce ths beacon overhead. In addton to perodc beaconng, uses an ondemand backpressure beacon, to quckly notfy the upstream nodes of traffc changes nsde the network. As shown n the evaluaton (secton 0), our on-demand beacon scheme ntroduces only a small overhead n exchange for a fast response to congeston. In, each node keeps a neghbor table to store nformaton passed by the beaconng. Each entry nsde the table has the followng felds: (NeghborID, Poston, Send- ToDelay, ExpreTme). The ExpreTme s used to tmeout ths entry. If a neghbor entry s not refreshed after a certan tmeout, t s removed from the neghbor table. SendToDelay s the delay estmaton to the neghbor node dentfed by the NeghborID feld. The detals of obtanng ths value have been dscussed n prevous secton Non-determnstc Geographc Forwardng Before elaboratng on Non-determnstc Geographc Forwardng (NGF), we frst ntroduce three defntons: The Neghbor Set of Node : NS s the set of nodes wthn

4 4 IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, TPDS R2 the rado range of node. Note, we do not assume that the rado s a perfect crcle. works wth rregular rado patterns. The Forwardng Canddate Set of Node : A set of nodes that belong to NS and are closer to the destnaton. Formally, FS (Destnaton) = {node NS L L_next > 0} where L s the dstance from node to the destnaton and L_next s the dstance from the next hop forwardng canddate to the destnaton. These nodes are nsde the cross-hatched shaded area as shown n Fgure 2. We can easly obtan FS (Destnaton) by scannng the NS set of nodes once. It s worth notcng that the membershp of the neghbor set only depends on the rado range, but the membershp of the forwardng set also depends on destnaton area. Fgure 2: NS and FS defntons Relay Speed. Relay speed s calculated by dvdng the advance n dstance from the next hop node j by the estmated delay to forward a packet to node j. Formally, j L L _ next Speed ( Destnaton) = j HopDelay Snce n nodes keep the Neghbor Set (NS), but do not keep a routng table or flow nformaton, the memory requrements are only proportonal to the number of neghbors. Based on the destnaton of the packet and the current FS, the Non-determnstc Geographc Forwardng (NGF) porton of our protocol routes the packets accordng to the followng rules: Packets are forwarded only to the nodes that belong to the FS (Destnaton). If there s no node nsde the FS (Destnaton), packets are dropped and a backpressure beacon s ssued to upstream nodes to prevent further drops (for more detals see 0). To reduce the chance of such drops, we provde a lower bound of node densty that can vrtually elmnate these drops. The theoretcal analyss on ths ssue s provded n secton 0. dvdes the neghbor nodes nsde FS (Destnaton) nto two groups. One group contans the nodes that have relay speeds larger than a certan desred speed S setpont, the other contans the nodes that cannot sustan such desred speed. The S setpont s a system parameter that depends on the communcaton capablty of the nodes and desred traffc workload a sensor network should support. For gven bandwdth T, packet sze L and Rado Range R, followng nequalty should hold: 0 S setpont RT L The forwardng canddate s chosen from the frst group, and the neghbor node wth hghest relay speed has a hgher probablty to be chosen as the forwardng node. To trade off between load balancng and optmal delvery delay, we use followng dscrete exponental dstrbuton functon: K ( Speed ) P ( x = ) = 1 N N K ( Speed ) = 0 In ths dstrbuton functon, N s the number of forwardng canddates nsde the frst group. K s used to trade off between load balance and optmal delvery delay. A larger K value leads to a shorter end-to-end delay; whle a smaller K value acheves a better load balance. If there are no nodes belongng to the frst group, a relay rato s calculated based on the Neghborhood Feedback Loop (NFL), whch s dscussed n more detal n secton 0. Whether a packet drop wll really happen depends on whether a randomly generated number between (0,1) s bgger than the relay rato. In a packet s dropped only when no downstream node can guarantee the sngle hop speed set pont S setpont and droppng packets must be performed to reduce the congeston. Though one can consder bufferng packets as an alternatve to the droppng, however, we argue that under real-tme and small memory constrants, droppng s often a better choce. NGF provdes two nce propertes to help meet our desgn goals. Frst, snce NGF sends packets to the downstream node capable of mantanng the desred delvery speed, soft real-tme end-to-end delvery s acheved wth a theoretcal delay bound: Delay Bound = L e2e /S setpont, where L e2e s the dstance between the source and destnaton. S setpont s the unform speed to be mantaned across the sensor network. Second, NGF can balance traffc and reduce congeston by dspersng packets nto a large relay area. Ths load balancng s valuable n a sensor network where the densty of nodes s hgh and the communcaton bandwdth s scarce and shared. Load balancng also balances the power consumpton nsde the sensor networks to prevent some nodes from dyng faster than others. NGF provdes MAC layer adaptaton and reduces the congeston by locally droppng (or optonally bufferng) packets. Ths adaptaton s good enough to deal wth transent overshoot nsde the sensor networks. But f such congeston remans for a relatvely long tme, network layer adaptaton s desred to redrect traffc to a less congested area, whch s dscuss further n secton Neghborhood Feedback Loop (NFL) The Neghborhood Feedback Loop (NFL) s the key component n mantanng the sngle hop relay speed. The NFL s an effectve approach to mantanng system performance at a desred value. Ths has been shown n [19], where a low mss rato of real-tme tasks and a hgh utlzaton of the computatonal nodes are smultaneously acheved. Here we want to mantan a sngle hop relay speed above a certan value S setpont, a performance goal desred by the desgner.

5 TIAN HE, JOHN A. STANKOVIC, CHENYANG LU AND TAREK F. ABDELZAHER: A SPATIOTEMOPORAL COMMUNICATION PROTOCOL FOR WIRELESS SEN- SOR NETWORKS 5 (/)1HLJKERUV MAC Feedback,''HOD\ 1RGHV PLVV UDWLR Neghborhood Table Back Pressure Beacon HWSRLQW RQRII - Relay Rato Relay Controller Rato SNGF Neghbor Nodes 2 3 'HOD\ 10 Boo Fgure 3: Neghborhood Feedback Loop (NFL) We deem t a mss when a packet delvered to a certan neghbor node has a relay speed less than S setpont, or f there s a loss due to collson. The percentage of such msses s called ths neghbor s mss rato. The responsblty of the NFL s to force the mss ratos of the neghbors to converge to a set pont, namely zero. As shown n Fgure 3, the MAC layer collects mss nformaton and feeds t back to the Relay Rato controller. The Relay Rato controller calculates the relay rato and feeds that nto the NGF where a drop or relay acton s made. The Relay Rato controller currently mplemented s a multple nputs sngle output (MISO) proportonal controller that takes the mss ratos of ts neghbors as nputs and proportonally calculates the relay rato as the output to the NGF. Formally t s descrbed by the followng formulas. e u = 1 K f e > 0 N u = 1 f e = 0 where e s the mss rato of the neghbor nsde the FS set, N s sze of the FS set. u s the output (relay rato) to NGF. And K s the proportonal gan. It should be noted that the Relay Rato controller s actvated only when all nodes nsde the forwardng set (FS) cannot mantan the desred sngle hop relay speed S setpont and a drop s absolutely necessary to mantan the sngle hop delay. Such a scheme ensures that re-routng has a hgher prorty than droppng. In other words, does not drop a packet as long as there s another path that can meet the delay requrements. By reducng the sendng rate to the downstream nodes, the neghborhood feedback loop can mantan a sngle hop relay speed. However, ths MAC layer adaptaton can t solve the hotspot problem, f the upstream nodes, whch are unaware of the congeston, keep sendng packets nto ths area. In ths case, backpressure reroutng (network layer adaptaton) s necessary to reduce the traffc njected nto the congested area. 4. Back-Pressure Reroutng Backpressure re-routng s naturally generated from the collaboraton of neghbor feedback loop (NFL) routnes as well as the non-determnstc geographc forwardng (NGF). To be more explct, we ntroduce ths scheme wth an example (Fgure 4). Fgure 4: Backpressure reroutng case one 2,''HOD\ 1RGHV17 3,''HOD\ 1RGHV 'HOD\ Fgure 5: Backpressure reroutng case two Suppose n the lower-rght area, heavy traffc appears, whch leads to a low relay speed n nodes 9 and 10. Through the MAC layer feedback, node 5 detects that nodes 9 and 10 are congested. Snce NGF reduces the chance of selectng nodes 9 and 10 as forwardng canddates and routes more packets to node 7, t reduces the congeston around nodes 9 and 10. Snce all neghbors of 9 and 10 react the same way as node 5, eventually nodes 9 and 10 are able to relay packets above the desred speed. A more severe case could occur when all the forwardng neghbors of node 5 are also congested as shown n Fgure 5. In ths case, the neghborhood feedback loop s actvated to assst backpressure re-routng. In node 5, a certan percent of packets are dropped n order to reduce the traffc njected nto the congested area. At the same tme, an ondemand backpressure beacon s ssued by node 5 wth the followng felds: (ID, Destnaton, AvgSendToDelay) AvgSendToDelay s the average SendToDelay of all nodes nsde FS ID (Destnaton). In our example, when the destnaton s node 13, AvgSendToDelay s the average delay from node 5 to nodes 7, 9 and 10. When a neghbor receves the back-pressure beacon from node 5, t determnes whether node 5 belongs to ts FS(Destnaton). If node 5 does, ths neghbor modfes the SendToDelay for node 5 accordng to the AvgSendToDelay. For example only node 3 consders node 5 as a next hop forwardng canddate to the destnaton where node 13 resdes. If node 5 s not n the FS(Destnaton), then ths neghbor gnores the backpressure beacon. Ths backpressure mechansm can reduce the chance of false congeston ndcaton, to ensure that traffc from node 4 to node s not affected by the backpressure beacon. If, unfortunately, node 3 s n the same stuaton as node 5, further backpressure s mposed on node 2. In the extreme case, the whole network s congested and the backpressure Boo 13 12

6 IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, TPDS R2 proceeds upstream untl t reaches the source, where the source quenches the traffc flow to that destnaton. Backpressure reroutng s a network layer adaptaton used by to reduce the congeston nsde the network. In ths case no packet needs to be sacrfced. Network layer adaptaton has a hgher prorty than MAC layer adaptaton used by NGF and NFL. A drop va the feedback loop s only necessary when the stuaton becomes so congested and there s no alternatve to mantanng a sngle hop speed other than droppng packets. 4.7 Vod Avodance Greedy geographc based algorthms have many advantages over the tradtonal MANET routng algorthms for real-tme sensor network applcatons. They do not suffer route dscovery delay and tend to choose the shortest path to the destnaton. Moreover wthout floodng, they have relatvely low control packet overhead. Unfortunately, they also have a serous drawback. In many cases, they may fal to fnd a path even though one does exst. To overcome ths, deals wth a vod the same way t deals wth congeston. As shown n the Fgure, f there s no downstream node to relay packets from node 2 to node 5, node 2 sends out a backpressure beacon contanng felds: (ID, Destnaton, ). The upstream node 1 that needs node 2 to relay the packets to that destnaton sets the SendToDelay for node 2 to nfnty and stop sendng packets to node 2. If node 3 does not exst, further backpressure occurs untl a new route s found. It should be admtted that our scheme of vod avodance sn t guaranteed to fnd a path f there s one as n GPSR[12], but t s guaranteed to fnd a greedy path f one exsts. To mantan real-tme propertes, we do not allow backtrackng to volate our desred speed setpont. However, as we can see from the evaluaton secton 0, such a smple scheme can sgnfcantly reduce packet loss due to vods n hgh-densty sensor networks %DFN3UHVVXUH Fgure : Vod avodance scheme 92,' 4 5 'HVWL 4.8 Last Mle Process Snce s targeted at sensor networks where the ID of a sensor node s not mportant, only cares about the locaton where sensor data s generated. The last mle process s so called because only when the packet enters nto the destnaton area wll such a functon be actvated. The NGF module mentoned above controls all prevous packet relays. The last mle process provdes two novel servces that ft the scenaro of sensor networks: Area-multcast and Areaanycast. The area n ths case s defned by a center-pont (x,y,z) and a radus, n essence a sphere. More complex area defntons can be made wthout jeopardzng the desgn of ths last mle process. Nodes can dfferentate the packet type by the PacketType feld mentoned n secton 4.1. If t s an anycast packet, the nodes nsde the destnaton area delver the packet to the transport layer wthout relayng t onward. If t s a multcast packet, the nodes nsde the destnaton area whch frst receve the packet comng from the outsde of the destnaton area set a TTL. Ths allows the packet to survve wthn the dameter of the destnaton area and be broadcast wthn a specfed radus. Other nodes nsde ths destnaton area keep a copy of the packet and re-broadcast t. The nodes that are outsde the destnaton area just gnore t. The last mle process for uncast s nearly the same as multcast, except that only the node wth a specfed global_id nsde the destnaton area delvers the packet to the transport layer. If the locaton servce s precse, the estmated destnaton area for a gven node wll be much smaller than sngle rado coverage. As a result, addtonal floodng overhead for the uncast packets s neglgble (sometmes zero). We note that the current mplementaton of the last mle process s relatvely straghtforward. More effcent and robust technques are desred for future research. 5 PROTOCOL ANALYSIS Ths secton provdes a protocol analyss of several practcal ssues related to the protocol. 5.1 Impact of Node Densty One basc assumpton of sensor networks s ther relatvely hgh node densty. It s an nterestng research ssue to determne the mpact of node densty on routng performance. Specfcally, n the geographc based algorthm, we want to fnd the lower bound of node densty that can probablstcally guarantee no vod that can prevent a greedy geographc forwardng step from happenng. In GF based algorthms, a node forwards packets to next hop nodes that are nearer to the destnaton. The area where such qualfed nodes resde s called the forwardng area (FA). Assume the nodes are randomly dstrbuted nsde the system, the larger the sze of the forwardng area, the hgher s the probablty that there s a canddate to be chosen. In fact, the forwardng area sze s not constant; t s depends on how far away the sendng node s from the destnaton node. Fgure 7: Forwardng Areas As shown n Fgure 7, when the destnaton node s nfntely far away from the sendng node, the forwardng area s the largest (Best Case Forwardng Sze) and when the destnaton node s exactly R away from the sendng node,

7 TIAN HE, JOHN A. STANKOVIC, CHENYANG LU AND TAREK F. ABDELZAHER: A SPATIOTEMOPORAL COMMUNICATION PROTOCOL FOR WIRELESS SEN- SOR NETWORKS 7 the avalable forwardng sze s the worst case forwardng sze (WCFS). For guaranteeng purposes, we only consder the worst case, even though most of the tme the forwardng sze s nearer to the best case. In the worst case, the forwardng sze s calculated by formula (1). For the purpose of analyss, here we use R as a nomnal rado radus. (1) 1 3 WCFS = 1 2 2cos R 2 2 Lower Bound of Node Densty Hop Route 10-Hop Route 100-Hop Route Now, we consder the worst case forwardng area. We desre to know the lower bound of node densty that satsfes the followng condton: P (At least one node nsde the FA) > 1-ε (2) Equvalently, P (No nodes nsde the FA) <= ε (3) Assumng a unform dstrbuton, accordng to (3) the followng condton must hold: (the sze of the area covered by the sensor network s denoted by AreaSze >> WCFS) WCFS (4) ( 1 AreaSze Densty ) ε AreaSze Snce the left hand sde of the nequalty s a monotoncally ncreasng functon when the AreaSze ncreases and monotoncally decreasng when node densty ncreases, the lower bound of the node densty s acheved when AreaSze s nfnte: lm( 1 WCFS ) AreaSze Hence: = e AreaSze Densty WCFS Densty ln ε Densty WCFS ε As for the greedy geographc based routng algorthm wthout backpressure, we must guarantee that for every hop they can fnd a forwardng canddate. More formally, to guarantee: P(successfully delver packets to a destnaton () through #hop greedy forwardng) >= 1- ε Assume vods follow an dentcal ndependent dstrbuton (d), equvalently: [P (At least one node nsde the FA)] #hop > 1- ε (7) Follows the same dervaton from (2) to (5), we get the lower bound of node densty: #hop ( ) (8) ln 1 1 ε Densty WCFS Fgure 8 shows the lower bound of node denstes that can probablstcally guarantee that there s no vod that can prevent greedy routng under dfferent ε values and lengths of the routes. For example, for a 10-hop route, t s statstcally guaranteed that 99% delvery rato n worst case f the node densty nsde networks s above 1 node/ nomnal range. (5) epslon Fgure 8: Lower bound of node denstes Estmate Delvery Rato GF-lke Fgure 9: Estmate Delver rato Node Densty (nodes per rado radus) On the other hand, we need to guarantee there exst a greedy path to the destnaton for the protocol. We observe that can t enforce backpressure at the source node, where no upstream node exsts. After the frst hop relay, the backpressure effectvely reduces packet lost due to the vod at subsequent hopes as mentoned n secton 0. To smplfy the analyss, we only consder frst hop loss due to the vod. Ths approxmaton slghtly overestmates the delvery rato of and serves as an upper bound. Accordng to nequalty 7, Fgure 9 plots the estmate delvery rate under dfferent node denstes. We note that the result we obtaned from the formal analyss (Fgure 9) s qute smlar to the results obtaned through smulaton (secton 0); For example, both smulaton and formal analyss have about 95% delvery rato for at the densty of 8 nodes per rado crcle. 5.2 Analyss of Localzaton Impact Theoretcally, t s desrable for locaton-based routng algorthms to have a perfect localzaton servce; however, n practcal, n order to obtan hgher locaton accuracy, systems have to ncrease the cost of the localzaton va sophstcated devces or addtonal communcaton overhead. Although more accurate locaton nformaton s preferable, the desred level of granularty should depend on a cost/beneft analyss of the protocols that utlze ths nformaton. In ths secton, we nvestgate the mpact of localzaton errors on the protocol. Specfcally, we nvestgate the pseudo vod problem caused by localzaton errors, whch leads to routng falures n and other locaton-based routng algorthms

8 8 IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, TPDS R2 Locaton-based routngs normally follow a greedy forwardng rule, as long as there s at least a node nsde forwardng area. Due to the localzaton error, some nodes, whch are actually located nsde the forwardng area of a sender, mght be mstaken by the sender to be outsde (Fgure 10A). More specfcally, the pseudo vod problem happens when all nodes nsde the forwardng area get localzaton results that are outsde of the forwardng area of the sender. We note that the forwardng area can be shfted by the localzaton error of the sender. Assumng that the localzaton error of each node follows an dentcal ndependent dstrbuton (d), M (9) P (pseudo vod) = P (Localzaton (N ) = 1 FA (Localzaton (sender)) Locaton (N ) FA(Locaton(sender)) ) Where FA s the forwardng area, N s an arbtrary node nsde forwardng area and M s the number of nodes nsde forwardng area. 2 R Densty π 3 = 7.7 nodes / crcle WCFS (11) Fgure 11 also demonstrates that when node densty s suffcently hgh (M = 5 & correspondng node densty > 12.78), statstcally the pseudo problem rarely happens (<0.2%) when the localzaton error below half rado range. Ths theoretcal analyss s consstent wth our smulaton result n [7]. P(Pseudo Vod) M = 1 M = 2 M = 3 M = 4 M = 5 M = Estmaton Error (R) Fgure 11: Probablty of Pseudo Vod Problem Fgure 10: Pseudo Vod Problem Assume localzaton error s omn-drectonal and maxmum dstance between the estmated locaton and the real locaton s e. Accordng to Fgure 10B, P (pseudo vod) equals: M 1 πe 2 = m + n e FA( m, n) Sze dashedarea 2 πe dxdy dmdn (10) Where FA (m,n) denotes the forwardng area of the sender when the localzaton result of the sender s (m, n). We obtan the value of equaton (8) through numercal ntegraton. We only consder the worst case n whch the destnaton s exactly R away from sendng node. The results are shown n Fgure 11: Fgure 11 gves us nsght on the relatonshp between node densty (M denotes the number of nodes nsde forwardng area), estmaton error (n the unt of rado range) and the probablty of the pseudo vod problem occurrng. For example n order to reduce chance of drop due to the pseudo vod problem below 5%, the number of nodes nsde the forwardng area should be equal to or larger than 3 when localzaton error s as much as half rado range. Based on the worse case forwardng area sze gven n equaton (1), the correspondng node densty should be 7.7 node/ nomnal rado crcle EXPERIMENTATION AND EVALUATION We smulate on GloMoSm [22], a scalable dscreteevent smulator developed by UCLA. Ths software provdes a hgh fdelty smulaton for wreless communcaton wth detaled propagaton, rado and MAC layers. Table 1 descrbes the detaled setup for our smulator. The communcaton parameters are mostly chosen n reference to the Berkeley Telos mote specfcaton. TABLE 1: SIMULATION SETTINGS Routng AODV, DSR, GF, GPSR,, -S, -T MAC Layer ( Smplfed DCF) Rado Layer RADIO-ACCNOISE Propagaton model TWO-RAY Bandwdth 200Kb/s Payload sze 32 Byte TERRAIN (200m, 200m) Node 100 Nodes, Unform placement Rado Range 40m In our evaluaton, we compare the performance of seven dfferent routng algorthms: AODV [17], DSR [10], GF [20], GPSR [12],, -S, -T. We adopt both ad hoc routng protocols (AODV and DSR) and sensor network protocols (GF, GPSR). GF forwards a packet to the node that makes the most progress toward the destnaton. GPSR has dentcal performance as GF when network densty s relatvely hgh, however t acheves better delvery ratos n sparse networks. -S and -T are reduced versons of. -S replaces the NGF wth a MAX- routng algorthm that geographcally forwards the packets to nodes that can provde a max sngle hop relay speed. -T replaces the NGF wth a MIN-DELAY routng

9 TIAN HE, JOHN A. STANKOVIC, CHENYANG LU AND TAREK F. ABDELZAHER: A SPATIOTEMOPORAL COMMUNICATION PROTOCOL FOR WIRELESS SEN- SOR NETWORKS 9 algorthm that geographcally forwards packets to nodes that have a mnmum sngle hop delay. Both reduced versons have no backpressure reroutng mechansms. In our evaluaton, we present the followng set of results: 1) end-to-end delay under dfferent congeston levels, 2) mss rato, 3) control overhead, 4) communcaton energy consumpton, and 5) packet delvery rato under dfferent node denstes. All experments are repeated 1 tmes wth dfferent random seeds and dfferent random node topologes. We also mplement on the Berkeley. The results obtaned from ths testbed show a load balance feature of protocol (see secton 0)..1 Sensor Network Traffc Pattern There are two typcal traffc patterns n sensor networks: a base staton pattern and a peer-to-peer pattern. The base staton pattern s the most representatve one nsde sensor networks. For example, n survellance systems, multple sensors detect and report the locaton of an ntruder to the control center. In trackng systems, a base staton ssues multple trackng commands to a group of pursuers. In a dfferent respect, the peer-to-peer pattern s usually used for data aggregaton and consensus n a small area where a team of nearby motes nteract wth each other. The end-toend delay n the base staton pattern s the major part of delay for the sensng-actuaton loop, and s therefore, the focus of our evaluaton..2 Congeston Avodance In a sensor network, where node densty s hgh and bandwdth s scarce, traffc hot spots are easly created. In turn, such hot spots may nterfere wth real-tme guarantees of crtcal traffc n the network. In, we apply a combned network and MAC layer congeston control scheme to allevate ths problem. To test the congeston avodance capabltes, we use a base staton scenaro, where nodes, randomly chosen from the left sde of the terran, send perodc data to the base staton at the mddle of the rght sde of the terran. The average hop count between the node and base staton s about 8~9 hops. Each node generates 1 CBR flow wth a rate of 1 packet/second. To create congeston, at tme 80 seconds, we create a flow between two randomly chosen nodes n the mddle of the terran. Ths flow then dsappears at tme 150 seconds nto the run. Ths flow ntroduces a step change nto the system, whch s an abrupt change that stress-tests s adaptaton capabltes to reveal ts transent-state response. In order to evaluate the congeston avodance capablty under dfferent congeston levels, we ncrease the rate of ths flow step by step from 0 to 100 packets/second over several smulatons Fgure 12 and Fgure 13 plot the end-to-end (E2E) delay for the sx dfferent routng algorthms. At each pont, we average the E2E delays of all the packets from the 9 flows (1 runs wth flows each). The 90% confdence nterval s wthn 2~15% of the mean, whch s not plotted for the sake of legblty. Under the no or lght congestons, Fgure 12 and Fgure 13 show that all geographc based routng algorthms have short average end-to-end delay n comparson to AODV and DSR. There are several factors accountng for ths outcome. Frst, the route acquston phase n AODV and DSR leads to sgnfcant delays for the frst few packets, whle geographc based routng doesn t suffer from ths. We argue that wthout an ntal delay cost, geographc based routng s more sutable for real-tme applcatons lke target trackng where the base staton sends the actuaton commands to the sensor group, whch s dynamcally changng as the target moves. In such a scenaro, DSR and AODV need to perform route acquston repeatedly n order to track the target. Second, the route dscovered through floodng and path reversal has relatvely more hops than greedy geographc forwardng. The reason for even hgher delay n AODV than DSR s that DSR mplementaton ntensvely uses a route cache to reduce route dscovery and mantenance cost. As shown n Fgure 13, -T has hgher delay than GF, -S and, because -T only uses hop delay to make routng decson and dsregards the progress each hop makes, whch leads to more hops to the destnaton n wreless mult-hop networks. Instead, under lghtly congested stuaton, GF, -S and tend to forward a packet at each step as close to the destnaton as possble, thereby reducng the number of hops and the end-to-end delay. Under the heavy congested stuatons (Fgure 12 and Fgure 13), each routng algorthm responds dfferently. performs best. For example, reduces the average end-to-end delay by 30%~40% n the face of heavy congeston n comparson to the other algorthms consdered. The key reasons for s better performance are 1) DSR, AODV and GF only respond to severe congeston, whch leads to lnk falures (.e., when multple retransmssons fal at the MAC layer). They are nsenstve to long delays as long as no lnk falures occur. 2) DSR, AODV and GF routng decsons are not based on the lnk delays, and therefore may cause congeston at a partcular recever even though t has long delays. 3) DSR and AODV flood the network to redscover a new route when the network s already congested. 4) -T and -S do not provde traffc adaptaton. When all downstream nodes are congested, -T and -S cannot reduce or redrect the traffc to uncongested routes. 5) not only locally reduces the traffc through a combnaton of NGF and Neghborhood Feedback loops n order to mantan the desred speed, but also dverts the traffc nto a large area through ts backpressure reroutng mechansm. Ths combnaton leads to lower end-to-end delay. Delay (MS) AODV DSR Rate (P/S) Fgure 12: E2E Delay Vs. Congeston

10 10 IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, TPDS R2 Delay (MS) GF -S -T Mss Rato 50% 40% 30% 20% GF -S -T 40 Rate(P/S) Fgure 13: E2E Delay Vs. Congeston.3 E2E Deadlne Mss Rato The deadlne mss rato s the most mportant metrc n soft real-tme systems. We set the desred delvery speed S setpont to 1km/s, whch leads to an end-to-end deadlne of 200 mllseconds. In the smulaton, some packets are lost due to congeston or forced-drops. We also consder ths stuaton as a deadlne mss. The results shown n Fgure 14 and Fgure 15 are the summary of 1 randomzed runs. AODV and DSR don t perform well n the face of congeston because both algorthms flood the network n order to dscover a new path when congeston leads to lnk falure. Ths floodng just serves to ncrease the congeston. GF only swtches the route when there are lnk falures caused by heavy congeston. The routng decson s based solely on dstance and does not consder delay. -T only consders the sngle hop delay and doesn t take dstance (progress) nto account, whch leads to a longer route. -S provdes no adaptaton to the congeston and cannot prevent packets from enterng the congeston area. Only tres to mantan a desred delvery speed through MAC and network layer adaptatons, and therefore has a much less mss rato than other algorthms. Due to ts transent behavor, stll has about a 20% mss rato when the network s heavly congested. Future work s needed to reduce the convergence tme n order to mprove the performance. Comparng Fgure 14 and Fgure 15, we argue that purely localzed algorthms wthout floodng outperform other algorthms when traffc congeston ncreases. Generally, the less state nformaton a routng algorthm depends on, the more robust t s n the face of packet loss and congeston. 10% 0% Rate (P/S) Fgure 15: MssRato Vs. Congeston.4 Control Packet Comparson Except for AODV, all other routng algorthms studed use a relatvely low number of control packets. Most control packets n DSR and AODV are used n route acquston. Because AODV ntates route dscovery (floodng) whenever a lnk breaks due to congeston, t requres a large number of control packets. DSR uses a route cache extensvely, so t can do route dscovery and mantenance wth a much lower cost than AODV. The only control packets used n GF, -S and -T (Fgure 1) are perodc beacons, whose number s constant at 750 under dfferent congeston levels. In addton to perodc beacons, uses two types of on-demand beacons to notfy neghbors of the congeston. Ths costs more control packets than the other three geographc based routng algorthms (Fgure 1). #Packets DSR GF -S -T Rate (P/S) Fgure 1: Control overhead comparson 40 Mss Rato 50% 40% 30% 20% 10% 0% AODV DSR Rate(P/S) Energy consumpton (mwhr) Rate (P/S) AODV DSR GF -S -T Fgure 14: MssRato Vs. Congeston Fgure 17: Transmsson Energy Consumpton.5 Energy Consumpton Under energy constrants, t s vtal for sensor nodes to

11 TIAN HE, JOHN A. STANKOVIC, CHENYANG LU AND TAREK F. ABDELZAHER: A SPATIOTEMOPORAL COMMUNICATION PROTOCOL FOR WIRELESS SEN- SOR NETWORKS 11 mnmze energy consumpton n rado communcaton to extend the lfetme of sensor networks. From the results shown n Fgure 17, we argue that geographc based routng tends to reduce the number of hops n the route, thus reducng the energy consumed for transmsson. AODV performs the worst as a consequence of sendng out many control packets durng congeston. DSR has larger average hop counts and more control packets than other geographc base routng algorthms. -T only takes delay nto account, whch leads to longer routes. Fgure 17 shows that has nearly the same power consumpton as GF and -S when the network s not congested. Under such stuatons, tends to choose the shortest route and does not requre any on-demand beacons. Under heavy congeston, has slghtly hgher energy consumpton than GF and -S, manly because delvers more packets to the destnaton than the other protocols when heavly congested.. Node Densty Impact The typcal densty of a sensng-covered sensor network system [8] s about 20~30 nodes/rado range n order to provde hgh fdelty n localzaton, detecton and trackng. In the prevous evaluatons, we use a 12 nodes/rado range as a typcal settng. However, t s mportant to understand how performs under very low-densty settngs. Ths experment evaluates the end-to-end delvery rato of all routng algorthms under dfferent node denstes. To elmnate packet loss due to congeston, we only use four flows wth a rate of 0.5 packets/second, these flows go from the left sde of the terran to the base staton at the rght sde of the terran. To change the densty of the network, nstead of ncreasng the number of nodes n the terran, we keep the number of nodes constant at 100, and ncrease the sde length of the square terran from 180 to 250 n steps of 5 meters. It s no surprse that DSR performs best n the delvery rato, snce t uses floodng to dscover the route. Theoretcally, DSR should have 100% delvery rato (Fgure 18) as long as the network s not parttoned. All other geographc based algorthms have 100% delvery rato when the network has hgh densty (>12 nodes / per rado range). However, when the network densty s reduced below 9 nodes/ per rado range, GF, -S and -T degrade performance rapdly. Only manages to delver 95% of ts packets to the destnaton. It should be ponted out that as shown n Fgure 18, GPSR [12], another well-known geographc based routng algorthm, permts backtrackng and can acheve 100% delvery rate as long as the network s not parttoned. However, drops 5% of ts packets, because these packets need backtrackng n order reach the destnaton. If these packets were to backtrack, these packets would have a negatve delvery speed. Ths s not allowed by for the sake of mantanng the real-tme propertes, whch s not supported by GPSR. We note that GPSR defaults to the GF protocol when node densty s hgh. The E2E deadlne mss rato of GF shown n Fgure 15 s sgnfcantly hgher than when the network becomes congested. Delvery Rato 100% 95% 90% 85% 80% 75% 70% DSR GF -S -T GPSR Densty (nodes per rado crcle) Fgure 18: Delver rato Vs. denstes 100% 90% 80% 70% GPSR 0% 50% 40% 30% 20% 10% 0% Delvery Rato 0% 10% 20% 30% 40% Normalzed Avg Localzaton Error ( % Range) Fgure 19: Delvery Rato Vs. Loc Errors.7 Locaton Error Impact Whle most work n locaton-based routng assumes perfect locaton nformaton, the fact s that erroneous locaton estmates are vrtually mpossble to avod. In ths experment, we nvestgate the tolerable localzaton error bound for the protocol. To prevent congeston, and therefore solate the effects of localzaton error, the traffc loads are set to the rate of 1 packet/second. We ncrease the localzaton error from 0% to 50% of the rado range n steps of 5% to measure the end-to-end delvery rato. Fgure 19 shows that when the localzaton error s below 20% of the rado range, can acheve almost 100% delvery. We note that because the GPSR protocol allows backtrackng, GPSR s more flexble n dealng wth locaton error mpact; hence, t acheves a slghtly better performance n ths case, as shown n Fgure Implementaton on Motes We have mplemented the protocol on the Berkeley motes platform wth a code sze of 03 bytes (code s avalable at []). Three applcatons ncludng data placement, target trackng and CBR are bult on top of. Due to the physcal lmtatons of the motes, t s extremely dffcult to perform as extensve evaluaton as we dd n the wreless smulator. Consderng the space lmtaton, we only present partal results here as a study n developng a more complete soluton on a mote testbed. In the experment, we use 25 motes to form a 5 by 5 grd. To evaluate