Plexxi LightRail White Paper

Transcription

1 White Paper CWDM and Limited Fiber Plant Installations Introduction This document contains information about using the CWDM capabilities of the Plexxi Switch hardware & Control software components within a limited fiber plant. Wavelength Division Multiplexing (WDM) has enabled networking vendors the ability to push more data through fiber optics. There are 2 major variations of WDM: (1) Dense and (2) Coarse Wavelength Division Multiplexing, respectively abbreviated as DWDM and CWDM. CWDM uses fewer wavelengths with more space between the wavelengths, whereas DWDM uses more wavelengths with very small spaces between each, but it comes with a significantly higher price point. Plexxi Switches come with 2 different types of internal LightRail optical options: (1) CWDM or (2) standard 1310nm optics 1. This paper focuses on the CWDM versions of the Plexxi Switch (Switch 2 and Switch 2s) only and both Plexxi Switch and Control must be running version at a minimum. A discussion about Plexxi s CWDM solution wouldn t be complete without talking about technology. interfaces are ports used to create a highly meshed fabric. A Plexxi Switch 2 has 4 LightRail interfaces (2 in the east direction labeled in red, and 2 in the west direction labeled in blue) and a Plexxi Switch 2s has 2 LightRail interfaces (1 in the east direction labeled in red, and 1 in the west direction labeled in blue), as shown in Figure 1. Figure 1: Left: Plexxi Switch 2. Right: Plexxi Switch 2s. In the CWDM versions of the Plexxi Switches each LightRail interface uses an industry-standard MTP cable with 12 individual fiber strands embedded, corresponding to 6 Tx/Rx pairs for each LightRail. Overlaid on top of these fiber pairs are the CWDM wavelengths. Each 1 Though there is no native DWDM optics built into a Plexxi Switch, DWDM optics can be outfitted in the front-panel ports. Page 1 of 6

2 data path wavelength corresponds to 10 Gbps 2 and there is a single 1 Gbps control path wavelength for inter-switch communications. From a data path perspective there are different combinations of wavelengths on these various Tx/Rx fiber pairs. As shown in Figure 2 the Local Channels are the wavelengths that get terminated on the next adjacent Plexxi Switch (the switch directly cabled to the east and the switch directly cabled to the west). Figure 2: LightRail channel termination points The Express Channels are the wavelengths that skip over the adjacent Plexxi Switches. Depending on which Express Channel a particular wave is on will determine where that wave terminates on the other end. A good analogy to understand this would be to think about a metro subway system that has local trains and express trains. The local train terminates at each station. An express train terminates at, say, every 3rd station. The LightRail cabling works in a similar fashion. Four of the 12 LightRail waves terminate at the adjacent switch, while two LightRail waves terminate on each of the 2nd, 3rd, 4th & 5th switches ( =12). A mux/demux located in every switch directs the Local Channels to the local switch and passively passes the Express Channels to the 2nd, 3rd, 4th and 5th switches. The default bandwidth to your immediately adjacent Plexxi Switch neighbor via the Local Channel is 40 Gbps going to the east and another 40 Gbps going to the west (from the perspective of 1 LightRail pair) 3. Each Express Channel will have 20 Gbps also going to their east and 20 Gbps going to their west. The Plexxi Switch 2 has 2 (two) LightRail pairs and the Plexxi Switch 2s has 1 (one) LightRail pair. If we now add up all the available fabric bandwidth per LightRail port pair we have: In the east direction: 40 Gbps on the local channel 20 Gbps on the 1st express channel 2 All data path rates are full-duplex. So, 10 Gbps means 10 Gbps on the Tx strand and 10 Gbps on the Rx strand. 3 The bandwidth will be doubled for the Plexxi Switch 2 as there are 2 LightRail ports going east and another 2 going west. Page 2 of 6

3 20 Gbps on the 2nd express channel 20 Gbps on the 3rd express channel 20 Gbps on the 4th express channel Total (on 1 east LightRail port): = 120 Gbps Total (on 2 east LightRail ports): 2 * ( ) = 240 Gbps In the west direction: 40 Gbps on the local channel 20 Gbps on the 1st express channel 20 Gbps on the 2nd express channel 20 Gbps on the 3rd express channel 20 Gbps on the 4th express channel Total (on 1 west LightRail port): = 120 Gbps Total (on 2 west LightRail ports): 2 * ( ) = 240 Gbps With respect to the Plexxi Switch 2 (which has 4 LightRail ports) this is how we get 480 Gbps of fabric bandwidth full-duplex when you aggregate both the east & west directions. Plexxi s Highly Meshed Fabric Once you understand how the LightRail paths work on a Plexxi Switch you can then see how Plexxi switches can create a highly meshed and highly resilient network between switches via the Express Channels. If we can use another big-city analogy and consider the Manhattan borough of New York City the streets are set up in a mesh fashion. You can get from Times Square to Wall Street in many different ways because there are many paths through that interconnected mesh of streets. In reference to Figure 3 you can see that there are also many different paths from one Plexxi Switch to another because of all the different combinations of Local Channels and Express Channels on the mesh. This allows Plexxi Control a lot of flexibility in order to calculate paths for packets through the system as well as take advantage of all those 10 Gbps paths available for high bandwidth applications. Figure 3: Six Plexxi Switch full mesh Page 3 of 6

4 Within Limited Fiber Plants In order to fully utilize the bandwidth capabilities of the LightRail between datacenters one would really need 6 Tx/Rx fiber pairs towards the datacenter / POP to the east and 6 Tx/Rx fiber pairs towards the datacenter / POP to the west. This may not be feasible as the number of available fiber links may not be enough or may be cost prohibitive. Fortunately one can use the native CWDM optics in the Plexxi Switch 2 and Switch 2s to take advantage of the centralized control-based features in the Plexxi ecosystem even in a limited fiber plant. By default fiber strands #1 and #7 of the 12-fiber MTP LightRail cable carry 4 (four) 10Gbps wavelengths and also the 1 Gbps control channel. With a simple configuration change available in Plexxi software v2.2.0 a network administrator can redirect 3 (three) additional 10 Gbps wavelengths onto fiber pair #1 and #7 for a total of 7 (seven) 10 Gbps wavelengths. In addition, a 1U Plexxi breakout cartridge 4 is needed to extract fibers #1 and #7. (See Figure 4 below) This means that one can put 70 Gbps of data traffic per LightRail on a single fiber pair. Note that the Switch 2 has 4 LightRail ports so if you had 4 fiber pairs you can then use all 4 LightRail ports for 140 Gbps of data traffic going east and 140 Gbps of data traffic going west for a total of 280 Gbps leaving the Switch over just 4 pair of fibers. If you only had 2 datacenters then both east & west channels go to the same location for a total of 280 Gbps between those 2 datacenters using Plexxi Switch 2 devices. Natively the CWDM optics in the Plexxi Switch 2 and 2s can support distances of up to 10 km therefore the Plexxi solution is very flexible for installations located within the same campus or metro area. For distances greater than 10 km you can configure the Plexxi Flexx ports as fabric connections and populate the Flexx ports with DWDM optics for greater distances. Figure 4: Three building limited fiber plant wavelength breakout 4 Three (3) of these standard LGX cartridges can fit into a single 1U mounting bracket Page 4 of 6

5 Single Fiber Pair Connectivity With two-way point-to-point communications there is a transmit channel (Tx) and a receive channel (Rx). When this occurs within fiber optics you typically have a separate fiber strand for Tx and a fiber strand for Rx. So on any one particular fiber strand data is only travelling in one direction at a particular wavelength and the return channel is travelling in the opposite direction on the other fiber strand at the same wavelength. With Plexxi s optional optical 3-port circulator you can run both Tx and Rx channels to a single fiber strand, running the same wavelengths in opposite directions. This eliminates the need for the other fiber strand, providing the same data throughput as before using half as many fiber strands. See Figure 5 for a diagram of the equipment. Therefore, if you have only one pair of fiber (ie. there are 2 strands of fibers in the pair) you could use one strand of the pair for the east direction (with its own Tx/Rx wavelength pairs) and the other strand of the pair for the west direction (also with its own Tx/Rx wavelength pairs). Figure 5: Three building limited fiber plant wavelength breakout with optical circulators This is amazing! It is amazing! But it can be confusing. It seems that everywhere you turn another factor of 2 in throughput is achieved. The important thing to remember is to look at it from the perspective of each LightRail port. Each LightRail Tx/Rx pair (fibers #1 & #7) can support 70 Gbps of data traffic full-duplex. So if you have a Plexxi Switch 2s (with 2 LightRail ports one east, one west) then you have 140 Gbps (70 * 2 LightRail ports) of data traffic leaving the Switch (aggregate of both east & west LightRail ports) and 140 Gbps of data traffic coming into the Switch (aggregate of both east & west LightRail ports). Page 5 of 6

6 If you have a Plexxi Switch 2 (with 4 LightRail ports two east, two west) then you have twice the throughput available as the Plexxi Switch 2s 280 Gbps in, 280 Gbps out. When the availability of fiber pairs is limited, you can use the Plexxi optical circulators to get the same throughputs as stated in the last 2 paragraphs but with half the fiber strands needed. Conclusion The Plexxi CWDM solutions are versatile enough to be deployed in either greenfield and/or brownfield installations where the number of fibers between datacenters / POPs is limited. Regardless of your underlying cabling infrastructure the Plexxi solution can adapt to your environment Innovative Way, Suite 3322 Nashua, NH PLEX (7539) info@plexxi.com Page 6 of 6