HORNET: A Packet-Over-WDM Multiple Access Metropolitan Area Ring Network

Transcription

1 2004 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 18, NO. 10, OCTOBER 2000 HORNET: A Packet-Over-WDM Multiple Access Metropolitan Area Ring Network Kapil V. Shrikhande, Ian M. White, Duang-rudee Wonglumsom, Steven M. Gemelos, Matthew S. Rogge, Yasuyuki Fukashiro, Moritz Avenarius, and Leonid G. Kazovsky, Fellow, IEEE Abstract Current metropolitan area networks (MANs) based on the SONET transport are not developing at the rate required to support the phenomenal increase in data traffic. To address the needs of future MANs, the Optical Communications Research Laboratory at Stanford University and Sprint Advanced Technology Laboratories are building HORNET (Hybrid Optoelectronic Ring NETwork). HORNET has a multiple access architecture, in which nodes access any WDM channel using a novel media access control protocol and fast tunable laser transmitters. HORNET transports data packets directly over the WDM ring, eliminating the SONET transport. This paper presents the HORNET architecture, the node design consisting of novel packet-over-wdm components, and the experimental testbed with recent results. Index Terms Clock recovery, IP-over-WDM, media access control, metropolitan area networks, optical networking, subcarrier signaling, tunable lasers. I. INTRODUCTION AND MOTIVATION AS THE Internet continues to evolve, the traffic carried by LANs, MANs, and the WAN is not only increasing in volume, but is also experiencing a change in pattern. The old traffic model consisted of client requests traversing the WAN to a worldwide-web server, followed by the retrieval of data (or content). In this model, the role of the MAN was mainly to connect the clients on the LANs to the WAN. This model is being quickly replaced by one where clients access content like streaming media stored at local servers on ISP MANs, and/or directly exchange content among each other using distributed content-sharing applications. This has already caused a dramatic and controversial increase in interclient communication on LANs and internode communication on MANs [1]; and this rapid growth is expected to continue in the future. Current MANs based on SONET WDM ring architecture [2] are not flexible enough to adapt to this sudden increase in MAN Manuscript received October 15, 1999; revised May 16, The HORNET project is supported by Sprint Advanced Technology Laboratories under Contract CK K. V. Shrikhande, I. M. White, D. Wonglumsom, M. S. Rogge, and L. G. Kazovsky are with Stanford University, Optical Communications Research Laboratory, Stanford, CA USA ( ocrl@erika.stanford.edu). S. M. Gemelos was with Stanford University, Optical Communications Research Laboratory, Stanford, CA. He is now with Lucent Technologies, Holmdel, NJ xxxxx USA. Y. Fukashiro was with Stanford University, Optical Communications Research Laboratory, Stanford, CA. He is now with Telecommunications Division, Hitachi, Ltd., Yokohama, Japan. M. Avenarius was with Stanford University, Optical Communications Research Laboratory, Stanford, CA. He is now with the Optical Institute, Physics Department, Technical University Berlin, Berlin, Germany. Publisher Item Identifier S (00)09003-X. traffic. SONET nodes encapsulate IP/ATM traffic received from LANs into frames and transport them to the point of presence (POP). The POP houses a high capacity SONET switch and an IP router or ATM switch to route the traffic either to the WAN, or back to the MAN to connect nodes on different add/drop wavelengths. As internode traffic increases on the MAN, this architecture scales poorly. The POP, being the central switch gets heavily loaded, increasing the packet latency. Second, network resources are not used efficiently: in the worst case, adjacent nodes on the ring communicate through the POP, using network resources along the path. A more efficient solution is to allow nodes to communicate directly, over the WDM layer, using wavelength conversion at each node [3]. Moreover, Internet data traffic grows exponentially as opposed to the slow growth in voice traffic. With the emergence of voice over IP, an increasing percentage of voice traffic will be carried in packet form. But SONET is not optimized for packet data transport. SONET is a circuit switched protocol, in which a node maintains a fixed circuit with its downstream node on the same add/drop wavelength for synchronization purposes. This circuit occupies a fixed bandwidth, regardless of whether or not there are packets to be transmitted. In a bursty traffic environment, channel bandwidth that could have been used by other nodes through multiple access schemes is wasted. Additionally, current MANs run IP over ATM over SONET over WDM or IP over SONET over WDM. This extended network stack results in high overhead, complicates the system infrastructure, and increases cost. HORNET (Hybrid Optoelectronic Ring NETwork) [3] [5] a joint effort between the Optical Communications Research Laboratory at Stanford University and Sprint Advanced Technology Laboratories is designed to address the inefficiencies of SONET and meet the needs of future MANs. HORNET transports IP packets/atm cells directly over WDM (no SONET). HORNET nodes, called access points (APs), drop a fixed WDM channel and use a fast tunable laser transmitter to transmit packets on any wavelength. The network uses a novel media access control (MAC) protocol, that is implemented at each AP independently (without any centralized control), to govern access to all network wavelengths. The APs are not synchronized with each other, unlike a SONET network, and are equipped to transmit/receive back-to-back packets to/from different APs. To test the functionality of our design, an experimental testbed consisting of three nodes running of independent, unsynchronized clocks is being built. We have demonstrated that the AP can unobtrusively listen to the network wavelengths /00$ IEEE

2 SHRIKHANDE et al.: HORNET: PACKET-OVER-WDM MULTIPLE ACCESS METROPOLITAN AREA RING NETWORK 2005 Fig. 1. HORNET physical topology for multiple access points (APs) and a point of presence (POP). and detect the presence/absence of optical packets in real time [6]. The AP can fast tune its laser transmitter within 4 ns; over a fine range (0.8 nm) [7] and a wide range (15 nm); and transmit packets in empty time slots on a free wavelength [6]. Clock recovery can be performed within 5 ns (12 bits at 2.5 Gb/s) [3]. This enables a packet-over-wdm (no SONET) multiple access architecture, capable of internode packet communication via fast wavelength tuning. The rest of the paper is organized as follows. Section II describes the HORNET architecture. Section III describes the node design and novel technologies therein. Section IV describes the current testbed and experimental results. Section V summarizes the current status of the project and discusses future work. II. HORNET ARCHITECTURE HORNET is designed as a WDM ring network (see Fig. 1) to serve a metropolitan area. The access points (APs) connects local and campus area networks to the ring, while the POP connects the MAN to the WAN. The network is designed to scale to 100 APs and a circumference of around 100 km. The APs use a bit rate of 2.5 Gb/s per wavelength, although this is just a specification. HORNET has a tunable transmitter, fixed receiver (TTFR) design. The TTFR architecture has been extensively researched in [8]. The source AP has the ability to tune its transmitter to the receiving APs drop wavelength, transmit a packet(s), and then hop to another wavelength and send packets to another AP. This forms a logical mesh topology on the underlying physical ring. This architecture is flexible and scales well since only traffic intended for the WAN gets switched at the POP. To minimize overhead incurred when the transmitter hops between wavelengths on a packet-by-packet basis, tuning time should be of the order of nanoseconds (negligible compared to the packet duration at 2.5 Gb/s). To avoid the use of expensive wide-band WDM equipment and improve scalability, multiple APs share the same drop wavelength, instead of dropping a unique wavelength. HORNET packets carry the destination AP address on a subcarrier-multiplexed header, a common technique [9], [10]. When an AP drops its designated wavelength, it checks the packet s destination. If the packet is destined for a downstream AP on the same drop wavelength, it is retransmitted on the same wavelength (multihopping), till it reaches its destination. Multihopping not only allows the MAN to scale to a larger number of nodes, but also regenerates the signal. Fig. 2 shows the principle of operation. As per the cost of wide band WDM components, the network can be incrementally scaled to a larger number of wavelengths to minimize the number of hops. HORNET is a multiple access network. The APs can access any wavelength, and the bandwidth of each wavelength is statistically shared by the APs. A media access control (MAC) protocol is therefore necessary to govern access to the wavelengths and avoid collisions between APs. HORNET uses a novel carrier sense multiple access with collision avoidance (CSMA/CA) protocol [11], based on subcarrier signaling [12], [13]. Each network wavelength is associated with a unique subcarrier (, etc.). When an AP transmits a packet (on ), it multiplexes the corresponding subcarrier frequency. The APs determine the occupancy of all wavelengths (in parallel) by monitoring the subcarriers in the RF domain a novel technique described in the next section. To minimize queueing delays at the AP, the packets that are to be multihopped are given priority for transmission. Additionally, delay due to head of line (HOL) blocking is avoided, by providing virtual output queues (VOQs), with one queue per wavelength. The benefits of VOQs in the HORNET architecture are quantified in [14]. The HORNET TTFR architecture with multihopping is compared to a traditional SONET WDM ring architecture, consisting of a centralized switch with wavelength conversion in [3]. Additionally, the HORNET architecture is compared to a packet-over-sonet (POS) MAN, consisting of point-to-point SONET links, in [5]. In both cases, HORNET performs better, in terms of packet latency and queue depth. III. HORNET ACCESS POINT AND NOVEL TECHNOLOGIES The AP acts as an interface between the edge IP router/atm switch of the campus network and the MAN. The AP consists of three subsections: slot manager, smart drop, and smart add, as shown in Fig. 3. The slot manager taps off some optical power

3 2006 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 18, NO. 10, OCTOBER 2000 Fig. 2. Principle of operation. Direct internode transmission via fast tuning. Multihopping on the same wavelength. for subcarrier recovery to perform two functions in parallel. It monitors the subcarriers (carrier sense) and relays the wavelength occupancy information to the smart add. It also demodulates the subcarrier (FSK demodulation) that corresponds to the APs drop wavelength, recovers the address, and informs the smart drop whether the incoming packet is destined for itself or for a downstream AP. The smart drop drops a fixed wavelength using a circulator and a fiber Bragg grating. The dropped wavelength is detected inside a burst mode receiver that recovers the packet bit clock, using an ultrafast clock recovery scheme. It then uses the address information provided by the slot manager to switch the received packet either to the LAN or to a retransmit queue, where it waits to be multihopped to a downstream AP. The smart add services the LANs transmit queue and the retransmit queue. It chooses a transmission wavelength depending on the destination AP of the queued packet and the wavelength availability information from the slot manager (collision avoidance). It then tunes the fast tunable laser transmitter to the target wavelength and modulates the packet on to the optical carrier. A broadband coupler is used to couple the packets from the smart add on to the MAN. The three novel technologies implemented in the AP, namely, CSMA/CA media access control, burst mode clock recovery, and fast tunable transmitters, are described in detail below. A. CSMA/CA MAC Protocols The slot manager performs the carrier sensing part of HORNET s novel CSMA/CA protocol. Fig. 4 shows the structure of the slot manager. A 10/90 coupler is used to tap off a small portion of optical power from the ring. The optical signal on the 10% output branch is detected using a single photodiode. Since one photodiode is used to detect all the wavelengths, the data from the wavelengths collide at baseband, but the subcarriers will appear at their original frequency. Fig. 4 shows the resulting spectrum at the photodiode output. If a particular wavelength is not occupied (e.g.,, the corresponding subcarrier will be absent. Once the optical signal is detected, it is sent through an array of bandpass filters, which separate the subcarrier signals from each other. An FSK demodulator is used on the subcarrier corresponding to the APs drop wavelength to recover the header of the next incoming packet (the header is FSK modulated on the subcarrier). Once the destination AP address is recovered from the subcarrier header, it is sent to the smart drop. All of the other subcarriers are individually sent to amplitude-shift-key (ASK) demodulators, which convey the presence or absence of the subcarrier signals to the slot detector. The slot detector monitors the absence of subcarriers and determines whether the wavelength is free for a time duration (slot) that is long enough to fit a packet. This availability information is then given to the smart add to aid in the decision making process concerning when to transmit and which wavelength to use. Meanwhile, the optical signal is delayed (using a fiber delay between slot manager and smart add) for a duration equal to one packet length guard time to assure that the head of the empty slot arrives at the smart add when it is ready to transmit the packet. The delay is very small since an ATM cell at 2.5 Gb/s is only 170 ns long. If the network is slotted with fixed size slots (suitable for an ATM network), the AP has to monitor only the beginning of slot to determine its occupancy. In such a case, the fiber delay can be minimized to processing guard time delay. Fig. 5 shows the graphical representation of the CSMA/CA scheme: the slot manager detects an opening on, and the tunable transmitter transmits its packet into the open time slot. The carrier-sensing scheme using subcarriers is far cheaper than demultiplexing the wavelengths and monitoring them separately. This approach requires a single photodiode and hence scales better as the channel count increases. This is a very powerful technique for media access control and is generic enough to be applied to networks with a different size or topology, like LANs and the WAN. 1) Handling Variable-Size IP Packets: The above solution works for fixed-size packets (ATM) only, because of the fixed fiber delay. Traditionally, optical MAC protocols have been designed for fixed-size cells. This allows a slotted network design with fixed-size slots for MAC and fixed-size optical delay lines for contention resolution. The variable nature of the IP packet size makes the design of a MAC protocol nontrivial. The use of variable length optical delays is impractical. A possible alternative is to use a large fixed-size supercell to carry multiple IP packets. Small packets can be aggregated into the supercell, while packets larger than the supercell can be fragmented. This effectively forces the network to have a fixed slot size equal to the duration of the supercell. This makes MAC functions simpler, but adds complexity to the protocol because IP packets need to be multiplexed into and demultiplexed out of the supercell, delineated from each other inside the supercell and fragmented. Clearly, there is a need for simple MAC schemes

4 SHRIKHANDE et al.: HORNET: PACKET-OVER-WDM MULTIPLE ACCESS METROPOLITAN AREA RING NETWORK 2007 Fig. 3. Structure of the HORNET AP. Fig. 4. Structure of the slot manager. Fig. 5. Graphical representation of HORNET s CSMA/CA MAC protocol. that are specifically designed to handle variable size IP packets rather than enforcing traditional fixed slot-size solutions. Although the IP standard allows a packet length between 40 bytes and 64 kbytes, a measurement trace from one of MCIs backbone OC-3 links shows a discrete packet-size distribution, from 40 bytes to 1500 bytes (see Fig. 6) [15]. The smallest packet of 40 bytes corresponds to Internet control message protocol (ICMP) messages (TCP header IP header) and the

5 2008 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 18, NO. 10, OCTOBER 2000 Fig. 6. Cumulative distribution function (CDF) of IP packet sizes on an Internet backbone link. Fig. 7. Slotted CSMA/CA with multiple slot sizes byte packet is Ethernet s maximum transfer unit (MTU). Fig. 6 shows that almost 50% of the packets are 40 bytes long while only 10% (but 50% of the byte-volume) are 1500 bytes long. Since the MTU of Ethernet (the most popular LAN standard) is 1500 bytes, a negligible percentage of packets are above 1500 bytes. This packet size distribution is not expected to change much, even though traffic keeps on increasing [15]. This information is used to design CSMA/CA protocols for variable size IP packets. Two different IP-MAC schemes are described below. a) Slotted CSMA/CA with Multiple Slot Sizes: The network is slotted by the POP or a slot governor, with slots of multiple sizes, such that the slot sizes match the packet size distribution of Fig. 6 or the typical distribution of the network under consideration. For example, 50% of the slots are made 40 bytes long, 30% are 552 bytes long, while 20% are 1500 bytes long, as a specific case. The POP fills the slots when it has packets to be transmitted to the downstream APs. The slots that remain empty are used by the APs. Fig. 7 depicts this scenario. The slot boundaries and current slot size can be carried on a special control wavelength, which is tapped by all APs. The APs carrier sense the network wavelengths by monitoring the subcarriers, as before, and also determine the slot size by detecting the control wavelength. The APs choose the appropriate wavelength and transmit the appropriate sized packet from its queues. A small fixed fiber delay inside the AP accounts for processing time and guard time. Additional flexibility can be introduced if slot size information is modulated onto the subcarrier of each wavelength separately. Then, different wavelengths can carry different sized slots and the AP can choose the correct queued packet for transmission depending upon wavelength and slot size availability. This scheme handles variable size packets with a small fixed-size optical delay (for processing time), without altering the IP packets. The slot sizes can be modified as the packet size distribution on the MAN changes. But it requires centralized control for slotting the network and additional synchronization. b) Unslotted CSMA/CA with Backoff: The protocol described below allows the network to remain unslotted and is

6 SHRIKHANDE et al.: HORNET: PACKET-OVER-WDM MULTIPLE ACCESS METROPOLITAN AREA RING NETWORK 2009 comparable to Ethernet in terms of its simplicity and robustness. The AP listens to all wavelengths by monitoring subcarriers. When a packet is ready for transmission, the AP checks the occupancy of the target wavelength. If it is free at that instant, the AP begins to transmit the packet. However, since the AP cannot know if the opening is long enough to accommodate the packet, it continues to monitor the network. If it detects a packet arriving on the same wavelength at its input, it immediately ends the packet transmission and sends a jamming signal. The jamming signal (like in Ethernet 10/100 BaseT) could be a unique bit pattern, either at baseband or on the subcarrier. A small fixed optical delay line is placed between the point at which the node listens for incoming packets and the point at which the node inserts new packets. This allows the node to terminate its transmission before the added packet interferes with the packet already on the ring. Fig. 8 shows the operation of this MAC scheme. The downstream AP recognizes the incomplete packet by the presence of the jamming signal and pulls it off the ring. The AP can reschedule the transmission of the packet for a later time. This scheme is very simple and scalable since it does not need centralized control and the network remains unslotted. It also uses a small fixed-size optical delay independent of packet size. But the presence of incomplete packets on the ring constitutes wasted bandwidth, and hence it is important to clean the incomplete packets at the first opportunity. Fig. 8. Unslotted CSMA/CA with backoff. Incoming packet is sensed. Backoff for collision avoidance. C. Fast Tunable Transmitter B. Burst Mode Receiver HORNET nodes do not maintain synchronization with each other. Since an IP packet can be as small as 40 bytes (128 ns long at 2.5 Gb/s), a burst mode receiver should ideally recover the packet clock within a few nanoseconds. HORNET uses the embedded clock transport (ECT) technique in which the transmitter frequency multiplexes its bit-clock with the baseband data and modulates the optical carrier with the composite signal. The data and clock travel along the fiber with negligible dispersion, maintaining their original phase difference. Fig. 9 shows the schematic of the burst-mode packet receiver. A photodiode detects the optical signal. Its RF output is amplified and split. The data and clock are first separated using a low-pass filter (LPF) with cutoff, the bit-rate of the data, and a narrow bandpass filter (BPF), centered at and then fed to a deserializing analog receiver. If the delay from the output of the photodiode to the input of the receiver is matched, the clock and data will be in bit-synchronization for all incoming packets. The deserializer receives the signals, samples the data, and deserializes it to slow speed bus, which can then be switched depending on subcarrier address information. The ECT technique is simple yet effective. It does not need ASICs, unlike other clock recovery techniques [16], [17]. It scales very well with bit rate and is independent of data encoding. The recovery time is inversely proportional to the filter bandwidth. A 60 MHz filter was able to recover a 2.5 Gb/s clock within a few nanoseconds (see Section IV). The only disadvantage of the ECT technique is that the modulation depth on the laser or external modulator at the transmitter is shared between clock and data. The goal of the fast tunable transmitter is to achieve nanosecond tuning between any two wavelengths to reduce the tuning overhead. HORNET uses a commercially available grating coupler sampled reflector (GCSR) tunable laser [18], fast-tuned using a novel digitally controlled driver [7]. The GCSR laser is a 4-section DBR laser with one gain section and three tuning sections: coupler, phase, and reflector. The cavity modes of the laser are spaced approximately 0.27 nm apart. The reflector section consists of a sampled Bragg grating that acts like a comb filter, capable of selecting laser cavity modes that are approximately 4 nm apart. The coupler section filters one such mode, which becomes the lasing mode of the laser. The phase section provides fine-tuning of the laser cavity modes. Fig. 10 and show the current-wavelength characteristics of the reflector and coupler sections. It can be seen that the reflector can be used for fine wavelength tuning while the coupler can be used for coarse tuning. To achieve fast tuning, it is essential to have a rapid change in the carrier density inside the tuning sections. Hence, the rise time of the tuning current needs to be as small as possible. Furthermore, the tuning current pulse needs to be controlled such that it does not cross the threshold current values for adjacent modes and accidentally excite them. For example, in Fig. 10: while tuning to = nm, if the current pulse at the reflector crosses approximately 4.5 ma, the adjacent mode nm) is excited, resulting in longer tuning time. Hence, accuracy of tuning current is also important. To fast tune the laser, while retaining accuracy, ultrafast digital-to-analog converters (DACs) with current source outputs are used. Moreover, the DACs can be digitally controlled with programmable logic devices (PLDs). Fig. 11 shows the

7 2010 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 18, NO. 10, OCTOBER 2000 Fig. 9. Burst-mode packet receiver. With the use of digitally controlled drivers and lasers with integrated electroabsorption (EA) modulators, robust, compact and cost-effective wavelength tunable transmitters seem feasible. Fig. 10. Current-wavelength characteristics of the GCSR laser. Reflector section. Coupler section. schematic of the fast tunable transmitter. The PLD controller provides the three DACs with 10-bit digital data. The DACs output current tunes the laser to the target wavelength. By using two DACs per tuning section or by changing the input data to a DAC, current waveforms with short-width overdriving pulses can be generated (see Fig. 12). The overdriving pulses reduced the tuning time considerably to within 4 ns for 0.8 nm (reflector section) tuning and 1.8 ns for 15 nm (coupler section) tuning. IV. HORNET TESTBED AND EXPERIMENTAL RESULTS A. Generic Testbed Description The OCRL is constructing a HORNET testbed to test the functionality of the experimental AP. Fig. 13 shows the current setup. The POP and the AP function on completely independent clocks, as in a real network. The testbed has two wavelengths on the ITU grid, = nm and nm, with corresponding subcarrier signals at 3.0 GHz and 3.5 GHz. For experimental purposes, the packet size is kept fixed at 250 ns, which corresponds to around 80 bytes at Gb/s. The bit rate for the testbed was chosen to be Gb/s (and not 2.5 Gb/s) simply because components like crystal oscillators, deserializers, etc., are easily available at multiples of SONET frequencies. B. POP Node and the Sink Node A POP traffic generator, shown in Fig. 14, generates optical packets on both wavelengths, by turning the distributed feedback (DFB) laser on for the duration of the packet (250 ns in this case) using the direct modulation port of the laser. While the laser is on, a combination of Gb/s payload data, GHz clock tone, and subcarrier header (either at 3 or 3.5 GHz) modulates the optical carrier via the integrated electroabsorption modulation (EA) port. Fig. 15 shows an optical packet generated at the POP, in the time and frequency domain. The sink node, shown in Fig. 16, demultiplexes the two wavelengths exiting the AP, and detects them using PIN receivers. The detected signal is amplified, and a combination of 2 GHz low-pass and 2.5 GHz bandpass filters is used to separate the baseband data and clock (similar to the packet receiver using ECT). C. Experimental AP The experimental AP is shown in Fig. 17. To test the functionality of the smart drop, one of the two testbed wavelengths in this case is dropped. In the slot manager, a 10/90 splitter taps off 10% of the optical power for subcarrier address recovery (carrier sensing is not required for the smart drop). Since nm is the APs drop wavelength, the subcarrier

8 SHRIKHANDE et al.: HORNET: PACKET-OVER-WDM MULTIPLE ACCESS METROPOLITAN AREA RING NETWORK 2011 Fig. 11. Fast-tunable packet transmitter. Fig. 12. Over-driving current pulses generated at output of the DAC. Fig. 13. Current HORNET testbed setup. GHz is bandpass filtered. An FSK demodulator followed by the header clock recovery unit retrieves the header from the subcarrier. The remaining optical power enters the smart drop. A combination of an optical circulator and a fiber Bragg grating is used to drop. The dropped optical packets are detected by a PIN photodiode. The detected signal enters the packet receiver for clock recovery. The packet receiver uses a 2.0 GHz LPF to filter the data, and a 60 MHz BPF centered at GHz to extract the clock. The data and clock are input to a SONET 1 : 16 deserializer chip, an AMCC product, which has an analog receiver front-end with subsequent digital deserialization stages. The incoming Gb/s data is deserialized into a 16-bit lower speed 155 Mb/s bus. The deserialized packet is then switched depending upon the detected subcarrier address, Fig. 14. Experimental POP transmitter. either to the local network or back to the queuing system at the smart add for multihopping. Fig. 18 shows an oscilloscope trace of the 250 ns long packet data and clock seen at the output of the LPF and BPF in the

9 2012 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 18, NO. 10, OCTOBER 2000 Fig. 15. HORNET packet at the POP with Gb/s data, GHz clock tone, and a 3.0 GHz subcarrier. Time domain. Frequency domain. Fig. 16. Experimental sink node. packet receiver; the scope being triggered by the packet generator. Fig. 19 shows the rise time of the clock at the output of the 60 MHz wide bandpass filter. The clock output reaches a 300 mv voltage swing, the threshold voltage for AMCCs chip, in 5 ns, which corresponds to 12 bits of recovery time at GHz. The deserialized packet is switched using a 16-bit packet switch, depending upon the subcarrier address recovered by the slot manager (details in [4]). Bit error rate (BER) measurements are performed to test HORNET s transmission scheme involving frequency multiplexed data, clock, and subcarrier, using a Gb/s PRBS data stream instead of packets. The BER is measured using the recovered data and clock, at the output of the LPF and BPF, inside the packet receiver in the smart drop. Fig. 20 shows the BER plots. The line on the far left is the result of connecting the PIN detector directly to the laser that is modulated only with the payload data, bypassing the optical drop components. The second curve was obtained when the optical drop was added to the setup. Then, the subcarrier was added, then the clock tone, and then the combination of the two. The line titled with clock, VCO is the result for recovered data from the drop wavelength. The addition of a second wavelength with equal power, 0.8 nm from the drop wavelength, shows almost the same BER performance. This shows the ability to recover the Gb/s baseband data with less than 10 BER with 8 dbm input power, including GHz embedded clock tone and subcarrier header. The fast tunable transmitter in the testbed uses Altitun s GCSR laser [18]. It should be noted that the laser used is a commercially purchased component in a 14-pin butterfly package that is not optimized for fast packet switching, but rather for dc and slow switching. To test the fine and wide tuning of the laser, the reflector and coupler section are tuned separately (the steady-state currents are selected from Fig. 10), using combination of a PLD and a DAC, to generate the tuning current with overdriving pulses. The overdriving pulses compensate for the impedance mismatches incurred due to the laser package and force a fast rise current into the laser chip. The gain current, phase current, and temperature are kept fixed: ma, ma, temperature 21 C. The output of the laser (point A in Fig. 17) is connected either directly to the optical spectrum analyzer (OSA) or to the input of the sink node, for measurement purposes. The laser output is not being modulated in this experiment. Fig. 21 shows the tuning results when the reflector is tuned. In Fig. 21, the two strong peaks are the target modes and, while the two weak peaks are cavity modes that the laser sweeps through when tuning from to [the middle modes are seen in Fig. 10 as well]. It is seen that overdriving pulses on the tuning current result in a larger suppression of the middle modes with respect to the target modes, resulting in faster tuning. Fig. 21 shows the switching waveforms observed in the time domain, at point A-A in the sink node. The tuning time from to is reduced from 25 ns, without overdriving pulses, to around 4 ns by adding overdriving pulses (details in [7]). Similar results are observed from to. Fig. 22 and shows similar plots for wide tuning, using the coupler section. The OSA plot shows a good 20 db of suppression between the target modes ( nm and nm) and the middle modes of the laser indicating a good tuning time. Fig. 22 shows only 1.8 ns switching time for 15 nm spacing. Since two separate sections are used for fine and wide tuning, the tuning time is not proportional to the tuning range. This is the big advantage of DBR lasers with multiple tuning sections, with independent tuning capabilities. To test the CSMA/CA functionality of the experimental AP, both the testbed wavelengths are allowed to enter the smart add (the optical drop is removed from the setup in Fig. 17). The slot manager monitors both the subcarriers (3G Hz and 3.5 GHz) and informs the PLD transmitter controller of open slots. The transmitter controller controls the entire smart add. When it receives an open slot signal from the slot detector, it sends the tune signal to DAC1, which tunes the reflector section of the laser (in this experiment, the coupler is kept fixed). While the laser is tuning, the transmitter controller shuts the Mach Zehnder modulator gate by applying a voltage through the RF switch. After the laser is tuned, the transmitter controller opens

10 SHRIKHANDE et al.: HORNET: PACKET-OVER-WDM MULTIPLE ACCESS METROPOLITAN AREA RING NETWORK 2013 Fig. 17. Experimental AP. Fig. 18. Packet data and clock at the output of 2.0 GHz LPF and GHz BPF, respectively. Fig. 20. HORNET BER plots. Fig. 19. filter. Payload clock recovery within 5 ns at the output of 60 MHz bandpass the MZ gate for a packet-duration and allows the tuned carrier to be modulated by the data signal. The packets from the smart add are coupled with the packets from the POP, which are delayed the appropriate amount (packet length processing time) using the fiber delay. Fig. 23 shows the results of the CSMA/CA implementation, obtained by connecting the sink node at different points in the smart add and observing the plots at A-A in the sink node. Fig. 23 shows the output of the POP, obtained by connecting point B in Fig. 17 to the sink node. Packets of 250 ns duration can be observed on both wavelengths, with empty slots in the middle. Fig. 23 shows the output from the MZ modulator in the smart add, obtained by connecting point C to the sink node. The figure shows that the tunable transmitter transmitted a packet on, then immediately hopped to and transmitted a packet on. Fig. 23(c) shows the multiplexed output, consisting of packets from the POP and the smart add, obtained by connecting point D to the sink node. This verifies that the AP

11 2014 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 18, NO. 10, OCTOBER 2000 Fig. 21. Reflector section fast tuning. OSA waveform before WDM de-mux. Switching waveforms after detection. found the open slots on and and inserted packets without collisions. At the output of the AP, the two wavelengths therefore have packets from both the POP and the AP, generated off separate bit clocks. It is important that the clock recovery technique be able to recover the bit clock and sample the data correctly, even though it receives two back-to-back packets from different sources. This function is tested at point B-B in the sink node, by detecting packets on one wavelength, separating clock and data and viewing the eye diagram of the data, using the recovered bit clock as a trigger. Fig. 24 shows the eye diagram on (a similar result was obtained on, which shows an excellent opening, proving that the packet receiver in any AP can receive back-to-back packets from different sources, using the embedded clock technique. V. SUMMARY AND FUTURE WORK Metropolitan area networks based on the SONET transport are not equipped to handle the large amount of data traffic expected to be carried by future networks. There is a need for a new MAN architecture optimized to carry data and capable of allowing efficient intra-man communication, without the redundancy of SONET. HORNET is a packet-over-wdm multiple access ring network designed to serve the need of future MANs. HORNET switches packets directly over the WDM ring using a novel CSMA/CA access protocol to monitor all wavelengths for empty time slots and a fast-tunable packet transmitter to transmit on the available wavelength. Ultrafast clock recovery Fig. 22. Coupler section fast tuning. OSA waveform before WDM de-mux. Switching waveforms after detection. is enabled by transporting the transmitter s clock along with the data, eliminating any synchronization issues. Subcarrier headers carry packet destination address to enable switching at the APs. Next on the agenda is to complete building the AP. This includes building standalone packet generators (we are presently gating bit error rate testers to generate data packets) and queues to queue packets from the local traffic source and packets intended for multihopping. The MAC schemes designed for IP over WDM systems, discussed in Section III-A, need to be implemented and tested. The fast-tunable transmitter will be characterized for simultaneous coupler and reflector section tuning to provide fast tuning capabilities between ITU wavelengths, spaced at 50 or 100 GHz, spanning nm. At the moment, we have not looked into the interface between the IP router/atm switch and the POP or AP. We have assumed that the AP/POP receives unframed packets/cells to be transported over HORNET. A simple method needs to be established to map the destination IP address/atm virtual circuit number of the packet/cell to the subcarrier address carried by the HORNET packet. A good survivability scheme needs to be developed to supersede SONET survivability. The final task is to completely construct the HORNET testbed, investigate its performance for a variety of traffic conditions, and provide further insight into the performance of a packet-over-wdm optical network.

12 SHRIKHANDE et al.: HORNET: PACKET-OVER-WDM MULTIPLE ACCESS METROPOLITAN AREA RING NETWORK 2015 Fig. 23. Demonstration of CSMA/CA MAC. Upstream packets from POP. Packets to be added by smart add. (c) Combination of upstream traffic and inserted traffic. [2] O. Gerstel and R. Ramaswami, Upgrading SONET rings with WDM instead of TDM: An economic analysis, presented at OFC 1999, ThE4. [3] T. Ono, S. M. Gemelos, I. M. White, and L. G. Kazovsky, Latency characteristics of wavelength switched packets in WDM multihop ring networks, in Proc. OFC 99, vol. 2, Paper ThM8. [4] S. M. Gemelos, K. Shrikhande, D. Wonglumsom, I. M. White, T. Ono, and L. G. Kazovsky, HORNET: A packet switched WDM metropolitan area network, presented at Consorzio Nazionale Interuniversitario per le Telecommunicazioni (CNIT), Invited Paper, Sept [5] I. M. White, D. Wonglumsom, K. Shrikhande, S. M. Gemelos, M. S. Rogge, and L. G. Kazovsky, The architecture of HORNET: A packet-over WDM multiple-access optical metropolitan area ring network, Computer Networks, Feb [6] I. M. White, Y. Fukashiro, K. Shrikhande, M. Avenarius, M. S. Rogge, D. Wonglumsom, and L. G. Kazovsky, Experimental demonstration of a media access protocol for HORNET: A WDM multiple access metropolitan area ring network, in OFC 2000, submitted for publication. [7] Y. Fukashiro, K. Shrikhande, M. Avenarius, M. S. Rogge, I. M. White, D. Wonglumsom, and L. G. Kazovsky, Fast and fine wavelength tuning of GCSR laser by using a digitally controlled driver, in OFC 2000, submitted for publication. [8] K. Bogineni, K. M. Sivalingam, and P. W. Dowd, Low complexity multiple access protocols for wavelength division multiplexed photonic networks, IEEE J. Select. Areas Commun., vol. 11, pp , May [9] A. Carena, M. D. Vaughn, R. Gaudino, M. Shell, and D. J. Blumenthal, OPERA: An optical packet experimental routing architecture with label swapping capability, J. Lightwave Technol., vol. 16, pp , Dec [10] A. Budman et al., Multigigabit optical packet switch for self routing networks with subcarrier addressing, in Proc. OFC 92, San Jose, CA, Feb. 4 7, 1992, pp [11] S. M. Gemelos, I. M. White, D. Wonglumsom, K. Shrikhande, T. Ono, and L. G. Kazovsky, WDM metropolitan area network based on CSMA/CA packet switching, IEEE Photon. Technol. Lett., Nov [12] S. F. Su and R. Olshansky, Performance of multiple access WDM networks with subcarrier multiplexed control channels, IEEE/OSA J. Lightwave Technol., vol. 11, pp , May June [13] C. L. Lu, D. J. M. Sabido, P. Poggiolini, R. T. Hofmeister, and L. G. Kazovsky, CORD A WDMA optical network: Subcarrier-based signaling and control scheme, IEEE Photon. Technol. Lett., vol. 7, pp , May [14] D. Wonglumsom, I. M. White, S. M. Gemelos, K. Shrikhande, and L. G. Kazovsky, HORNET A packet-switched WDM metropolitan area ring network: Optical packet transmission and recovery, queue depth, and packet latency, presented at LEOS 99, Paper WEE0004. [15] WAN packet size distribution. [Online]. Available: net/na/learn/packetsizes.html [16] A. Tajima, A 10-Gbit/s optical asynchronous cell/packet receiver with a fast bit synchronization circuit, in OFC 99 Tech. Dig., San Diego, CA, TuI6, pp [17] Y. Yamada, S. Mino, and K. Habara, Ultrafast clock recovery for burst-mode optical packet communication, in OFC/IOOC 99 Tech. Dig., San Diego, CA, TuI7, pp [18] B. Broberg et al., Widely tunable semiconductor lasers, in Proc. OFC 99, WH4, p Kapil V. Shrikhande received the B.S. degree in electrical engineering from the University of Pune, India in He is currently studying at Stanford University for both the M.S. and Ph.D. degrees in electrical engineering. Fig. 24. Eye diagram of packets multiplexed from POP and the smart add on triggered using recovered bit clock. REFERENCES [1] n Ian M. White received the B.S. degree in electrical engineering from the University of Missouri-Columbia in 1997, and the M.S. degree in electrical engineering from Stanford University in January He is currently studying in the Ph.D. program in electrical engineering at Stanford.

13 2016 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 18, NO. 10, OCTOBER 2000 Duang-rudee Wonglumsom received the B.S. degree in electrical engineering from Lehigh University in 1995, and the M.S. degree in electrical engineering from Stanford University. She will complete the Ph.D. degree in electrical engineering at Stanford in Yasuyuki Fukashiro received the B.S. degree in physics and the M.S. degree in nuclear physics from the Tohoku University, Sendai, Japan, in 1990, 1992, respectively. He joined Hitachi, Ltd. in 1992, where he was engaged in R&D on optical communications. In 1999, he spent one year at OCRL, Stanford University. Since April 2000, he has been with Telecommunications Division, Hitachi, Ltd. Steven M. Gemelos received the B.S. degree in electrical engineering from Purdue University in 1994, the M.S. degree from Stanford University, and the Ph.D. degree from Stanford University in He is currently at Lucent Technologies, Holmdel, NJ. Matthew S. Rogge received the B.S. degree in electrical engineering from the University of Missouri-Columbia in He is currently studying at Stanford University for both the M.S. and Ph.D. degrees in electrical engineering. Moritz Avenarius spent one year at the Optical Communications Research Lab at Stanford University as a visiting scholar. Currently, he is at the Optical Institute of the Physics Department at the Technical Institute, Berlin, Germany. Leonid G. Kazovsky (M 80 SM 83 F 91) received the M.S. and Ph.D. degrees in electrical engineering from the Leningrad Electrotechnical Institute of Communications in 1969 and 1972, respectively. From 1974 to 1984 he taught and was engaged in research at Israeli and U.S. universities. From 1984 to 1990, he was with Bellcore, Red Bank, NJ, researching coherent and WDM optical fiber communications systems. In 1990, he joined Stanford University as Professor of Electrical Engineering and founded the Optical Communications Research Laboratory. In 1998, he founded an optical networking company which is now called Alidian Networks in Mountain View, CA.