Using PEX 8648 SMA based (SI) Card

Size: px

Start display at page:

Download "Using PEX 8648 SMA based (SI) Card"

Calvin Cole
5 years ago
Views:

1 Introduction Using a standard PC with PCIe slots, the PLX SMA based riser card allows quick connection and signal observation into non-standard configurations (i.e. backplanes) thru use of SMA connectivity.

2 1 Introduction Using a standard PC with PCIe slots, the PLX SMA based riser card allows quick connection and signal observation into non-standard configurations (i.e. backplanes) thru use of SMA connectivity. With the PLX SDK, the card allows programming either via I 2 C or PCIe in-band methods, to observe signal output, adjust transmitter and receiver settings, check Gen 1 / Gen 2 link-up thru a channel, operate at a max x4 width, check system errors, run loopback and card to card signal integrity (SI) testing. 2 Minimum Requirements Figure 1. PLX SMA based Riser Card A PC system with a x4 PCIe slot to provide power and system clock if separate clocking required i.e. two PCs - PC must allow SSC clocking to be disabled. o PCI-SIG baseboard with x4 connector for power and clock - optional substitute o Ardvaark I 2 C controller PLX SDK (version 5.22 or higher) 2.1 Additional References PLX PEX 8648 Quick Start Design Guide, by PLX Technology, Inc. All Rights Reserved 2

3 3 Configuration The PEX 8648 is a three station Gen 2 switch device with 48 maximum SerDes lanes. The SI RDK uses only stations 0 and 1. The first station Station 0 contains both the default upstream port (Port 0 feeding a standard PCIe connector) and also four SMA output lanes on an additional port (Port 3 Downstream) used for direct data observation and reception without the degradation of the PCIe connector. The transmitter and receiver pairs of each lane of the port are clearly marked with lane and polarity (however, it is not necessary to track lane polarity as this is automatically resolved by the PCIe specification). The PCIe connector at the top of the PCB is an alternate PCIe connection, located off of station1 of the switch (Port 6). It is not typically needed for SI use, but if desired a second card (including another SI RDK) can be installed and operated off the same base reference. Below are block diagrams of the port connections (Figure 1) and clocking (Figure 2). Figure 2. PEX 8648 SMA Card Port Configuration 2010 by PLX Technology, Inc. All Rights Reserved 3

Figure 3. PEX 8648 SMA Card Clocking The default card configuration accepts the 100 MHz PCIe system clock from the bottom PCIe connector, similar to a standard add-in card.

4 Figure 3. PEX 8648 SMA Card Clocking The default card configuration accepts the 100 MHz PCIe system clock from the bottom PCIe connector, similar to a standard add-in card. Care should be used to ensure that any system clock used is of good quality reference clock jitter is a primary source of degraded signal performance. If a clock other than the PCIe connector system clock is desired, the PCB can be modified to accept an external, differential clock via SMA J17/18 and relocation of zero ohm resistors R133 and R134 to location R135 and R136. (Schematic picture shown below) Any clock feeding the PCB is AC coupled onboard and internally terminated within the chip. Signal swing should be typically 400mV, differential, pk-pk (range 250mV min 1600mV max). Frequency value should be 100 MHz +/- 300ppm max. The reference clock feeding the chip is also made available for output via J24/ by PLX Technology, Inc. All Rights Reserved 4

5 Figure 4. PEX 8648 SMA Card Ref Clock Input Options The default PCB configuration has Port 0 the PCIe connector port as the upstream connection. This means it must be the port that points towards the PCIe root complex. This configuration allows a user to directly connect the PCB into a PCIe system chassis, observe signals and begin using PLX software for register manipulation without an I 2 C controller. To move from signal observation to an adjustable PCIe link,.bin files are supplied, which demonstrate how to convert the SMA port to the upstream link and operate as a standard PCIe link progressing thru link negotiation, and yet hold the desired channel tuning values. The various EEPROM.bin files contain key SerDes registers used for: Upstream Port configuration Signal Swing De-emphasis RX equalization Electrical Idle threshold adjustment With this method of operation (using EEPROM to set values), the EEPROM is programmed or adjusted for the desired settings and the system is then reset to verify operation by PLX Technology, Inc. All Rights Reserved 5

6 4 Appendix 4.1 Obtaining a Gen 2 Compliance pattern for Observation This can be done with I 2 C or In-band command (In-Band provided Port 3 is not upstream Port) 1. Connect the desired SMA TX lane to the high speed scope the internal 50Ω termination of the scope is required for the circuit to operate. 2. With the PLX SDK active, open the memory mapped register set. (left side of screen) 3. Go to register 0234h bits 19:16 (Disable Port X). Set bits 19:16 to 8. (disables Port 3) 4. Go to register 3098h (Link Status and Control). Set bit 4=1. (Enter Compliance). Note: bits 3:0 should already have a register value of 2 5. Go to register 0234h. Return bits 19:16 to 0. Terminated link is now in Gen 2 Compliance mode. If it is desired to change the TX swing and emphasis settings, it can be done as below without resetting the chip. 1. Go to register 0B90 (SerDes Drive Level 3) Depress a register read. Value returned should be the default Gen 2 value of 0E. 2. Go to register 0BA0 (SerDes Post Cursor Emp) - Depress a register read. Value returned should be the default Gen 2 value of With the Gen 2 values confirmed, you can now change these settings in accordance with the formula outlined in the PEX 8648 Data Book for Swing and Pre-emphasis (Table thru 19-12) Swing setting is loosely approximated as 20log ((Drive + De-emp)/(Drive De-emp)) As a point of note, we have observed some bit flips when using I 2 C for changing these registers. To confirm final setting, be sure to read back value after adjustment. Need to confirm against SDK SW revisions by PLX Technology, Inc. All Rights Reserved 6

5 Obtaining PCIe Linkup with Alternate Transmitter Settings from Default This is most easily done by using the EEPROM to configure the four links of the SMA Port Port 3. An example.

This bin was used to modify the configuration and establish channel linkup along a 30 Tyco backplane, 6 of paddled card and Hz-Md connectors, and ~ 2 feet of coax cable.

7 5 Obtaining PCIe Linkup with Alternate Transmitter Settings from Default This is most easily done by using the EEPROM to configure the four links of the SMA Port Port 3. An example.bin image is below for descriptive purposes. This bin was used to modify the configuration and establish channel linkup along a 30 Tyco backplane, 6 of paddled card and Hz-Md connectors, and ~ 2 feet of coax cable. In this file for Port 3, o SerDes Drive level is set to the PEX 8648 Data Book (Table 19-10) setting of 0Ch. o SerDes de-emphasis set to the PEX 8648 Data Book (Table 19-11) setting of 19h o Receiver equalization left at default 0 o Signal Detection Threshold (EIDLE) set to minimum (0) for Port 3 only For RX equalization, Port 3 (SMA outputs) is controlled via bits 31 thru 19. By clicking on the + to the far left of the actual bin file, the individual control bits are expanded. An expanded view is below, looking at the SerDes Drive Level. Note that in addition to the SerDes Drive level set to 0C, the Auto Load Disable bit is set. This is required to prevent the PCIe link from overwriting the desired signal swing and emphasis settings with the default Gen 2 values when the link is allowed to progress thru the normal PCIe linkup and Gen 2 negotiation process. If connecting directly to a scope to observe and change the settings for the compliance pattern output, setting this bit is not required by PLX Technology, Inc. All Rights Reserved 7

8 6 Bring Up of I 2 C Following steps describe how to bring up the I 2 C Arrdvark controller. 1) Install 4 pin connect on HD1 header as shown below. HD1 Red Ribbon wire HD2 Ardvaark Ribbon Connector 2) Power up the system with SI card and I2C header in place and USB port connected to system. With SDK active, look under Tools. Select Find I2C Device 3) Locate and click I2C Scan button. Type 68 for device address. Next, locate Select I2C Device Type and pull-down 8648AB. Next depress Find Devices. You should see AB appear in lower right window after a few seconds. Next, depress Refresh Device List it is okay that the 8648 goes away on the screen. 4) To the left of the screen, below Device Selector, locate pull-down selector and select item PLX(I2C) Devices. The PEX 8648 register and EEPROM menu are now visible on the left side of the page by PLX Technology, Inc. All Rights Reserved 8

7 Using EEPROM Bin Files 7.1 How to configure upstream Port other than default Port 0? The software needed is the PEX Device Editor included in the PLX SDK.

9 7 Using EEPROM Bin Files 7.1 How to configure upstream Port other than default Port 0? The software needed is the PEX Device Editor included in the PLX SDK. The PLX SDK can be downloaded from Instructions 1. Plug the SI riser card in the PC with the SDK installed. 2. Launch the PEX Device Editor. You will see 8648 in {Device Selector} box. a. Click [Tools] [Program EEPROM] b. Click [Browse ] button in Browse Bin File box, select.bin file * Open c. Click [PROGRAM EEPROM] button. 3. After confirmation of programming, shut down the PC. Remove the SI riser card. 4. Connect programmed SI riser card (#1 card) Port 3 to another SI riser card (or second device nearer the root complex) by SMA coax cables (note cables are equal length). Connect TX to RX and RX to TX of the two devices. Match lane numbers. For x1, use Lane 0. For x2, use Lane 0 to Lane 0 and Lane 1 to Lane 1 and so on... * Two.bin files are provided. SI_Port3_Upstream.bin and SI_Port3_Adjust.bin. Shown below is the exact SI_Port3_Upstream.bin file that will execute this action. Configuring the.bin file exactly as above will result in Port 3 of the card to be configured as the upstream port, and as such, should point toward the root complex. Note the register load is done in two steps. If editing an alternate.bin file to include a change of upstream ports, use the same two step sequence. Below, SI_Port3_Adjust.bin use file to operate the link in PCIe mode, while adjusting the SerDes settings and configuring Port 3 as upstream. Here, drive is set to the PEX 8648 Data Book value of 600mV and resultant de-emphasis is set for ~ 10dB by PLX Technology, Inc. All Rights Reserved 9

10 8 Using Two SI Cards for Link Testing Another means of testing employs the use of two SI cards. This approach can be used in three connectivity scenarios: 1) One root complex (PC) to control the PCIe link and one base card (i.e. PCIe compliance load board) to provide power to the downstream card. The default clocking for this configuration is asynchronous (hence PC clock must not set SSC active) or an out of band reference clock is fed into the downstream card. The downstream card is configured to have the SMA port as the Upstream. (Note: it is necessary for both cards to see at least one reset in order to initiate link-up negotiation. If needed, manually depress card reset switch) 2) Two root complex systems each SI card is placed in a PC and the SMA ports connected. Typically, in this configuration, NT is employed to allow Host-to-Host connectivity. Standard clocking is again asynchronous. (Note: it is necessary for both cards to see at least one reset in order to initiate link-up negotiation. If needed, manually depress card reset switch) 3) One root complex, stacked connectivity as shown below. In this configuration, coherent clocking is maintained between cards. Consequently, the PC can have SSC enabled. If alternate reference clock injection is desired, it is done so by changing the RefClk input configuration on the bottom SI Card unit. Examples for this configuration are discussed. Figure 5. Single PC connectivity 2010 by PLX Technology, Inc. All Rights Reserved 10

11 8.1 Digital Loopback (Two Cards) Step 1: Configuring port 3 on card #2 as the upstream port; o Plug card #2 in MB and boot the system. o Open PEX Device Editor (PDE) o Click on EEPROM editor. o Add register 0x1dc (default value should be 0x ) o Add register 1dc again and program the value 0x (port 3 is upstream port). o Shut down system. Step 2 -Install Cards o Remove card #2 from MB. o Plug card #1 in MB. o Plug card #2 in slot provided on card #1. This will provide power and clock to card #2. o Connect card #1 Port 3 to card #2 Port 3 by SMA coax cables (note cables are equal length). Connect TX of card #1 to RX of card #2. Match Lane numbers. For x1, use Lane 0, for x2 use Lane 0 and Lane 1. (You can add backplane in the path once you have verified that the initial setup works). Digital Loopback Test o Reboot system. o Link between both cards should be up. You will see port 3 LED on the cards blink. o Launch PDE and using memory map put port 3 in loopback. The following registers are programmed on card #1 using memory map. Program registers 0x210-0x21C with any pattern. Set 0x230[12] to enable loopback on lanes Check to see if loopback is established, 0x230[15] = 1. Set 0x228[31] to enable user pattern on lanes Check 0x244 SerDes quad 3 diagnostic data for error count. This should be zero. Change pattern in register 0x210. Check 0x244. You should see error count. Reset bits 0x228[31], 0x254[19] and 0x230[12]. Reboot system Repeat test with backplane in path. Change SerDes settings and check for error counts by PLX Technology, Inc. All Rights Reserved 11

12 8.2 Analog Loopback (Single Card) Configuration: Upstream Port = Port 0 This test loops the SMA output from Port 3 transmitter(s) to Port 3 receiver(s). Analog Loopback (Cable) with UTP (on Port 3) o Enable External Loopback by EEPROM: Write PHY Additional Status Register 254h[19:16]=8h in EEPROM. Power down system. o Wire the corresponding transmitter to receiver connections (either x1, x2 or x4) o Reboot the system. o Set PHY User Pattern Register 210h through 21Ch with desired data pattern. o Enable Loopback Master: Set Physical Layer Port Command Register 230h[12]. Verify bit [15] is set. o Enable User Test Pattern: Set Physical Layer Test Register 228h[31:28]=8h. Now to check link status / errors; o Read SerDes Quad 3 Diagnostic Data Register 244h. Verify User Test Pattern mode active via bit [30]=0 (UTP checker) o Select desired SerDes (0 thru 3) for error checking via register 244h, bits [24:25] where 00 = SerDes channel 0. o Check error count for selected SerDes bits [23:16] for UTP/PRBS error count. If link is operating properly, value should be zero. o If more than one lane is connected, select alternate SerDes and check corresponding lane errors per above. Figure 6. Error Register 2010 by PLX Technology, Inc. All Rights Reserved 12

13 9 Using Packet Generator in Two Card Set In this example, the PLX PEX 8648 Packet Generator and Monitor features are used to generate PCI Express heavy traffic across the link under test. Step 1) Program Port 3 as upstream port on card # 2 as outlined in Loopback test. Step 2) Install cards as outlined in Loopback test. Step 3) Programming packet generator and monitor features o Under programs bring up PLX GENMON o Go to Performance Monitor box at top of screen. SI cards are identified by bus number. The device nearest the root complex will have the lower bus number. Select this device as monitor (either will work). Now depress Open Monitor button. o With Monitor screen now open, select Port 3 Ingress and Port 3 Egress. You can optionally deselect the Port 0 monitors. Depress Start. Minimize screen. o Starting Traffic Generator: Go back to main GENMON screen. Select the PEX 8648 chip with the higher bus number this is the required traffic generator. Go to Load File and load the SI Card supplied script Upstream_Exerciser.PLE. Depress Start. o Looking at the Monitor screen traffic should now be evident. Step 4) Check bad DLLP and bad TLP counters in 8648 on card #1 and card #2 o Start standard SDK software. o In either chip read Port 3 memory mapped registers for TLP/DLP error count. (Registers x31e8h / x31ech) These registers keep a running tab of DLLP and TLP errors on Port 3. To clear write all 0 to each register. Step 5) Change SerDes settings and check for errors by PLX Technology, Inc. All Rights Reserved 13

14 10 Using External BERT (Analog Slave Loopback) This mode allows the user to pre-configure the SI Card (Port 3) in slave analog loopback such that non- PCIe compliant patterns (such as PRBS) can be injected via external means and then externally checked. BERT and System Offset within +/-300ppm Ext CLK Ext CLK System Box CLK/PWR For Synchronous Operation, can reconfigure PCB for external clocking. PCB CLKOUT feeds BERT Detector. Figure 7. Configuration Drawing Example BERT application Configuration of this mode can be done via EEPROM (example bin file included). Setup requires several writes to the base register in Station 0 (Port 0) and a write to the Port 3 specific register. As a point of clarity, note that Port 3 consists of SerDes numbers (Port 3 Lane 0-3). These are the channels that will be placed into loopback. Programming Steps; BERT (slave analog loopback) o Register 0204 set SerDes to Mask EIDLE (Fh) o Register 0234 set Port 3 bits 23:20 to Port Quiet (8h) o Register 0234 set Port 3 bits 27:24 for rate (8h) o Register 0BE4 set value bits 31:0 to ( ) SerDes overrides 2010 by PLX Technology, Inc. All Rights Reserved 14

15 To see the representative programming via EEPROM, the appropriate bin file is below; Figure 8. Example of Programming for Slave Loopback via EEPROM Note: the above bin file is provided as part of the SI card documentation. Using the PEX SDK, this bin can be directly programmed into the card. File name; SI_Card_BERT_Loopback.bin. 11 Revision History Date Version Revision September Initial Release October Added Analog Slave Loopback for BERT operability by PLX Technology, Inc. All Rights Reserved 15

2. Software Generation of Advanced Error Reporting Messages

2. Software Generation of Advanced Error Reporting Messages 1. Introduction The PEX 8612 provides two mechanisms for error injection: Carter Buck, Sr. Applications Engineer, PLX Technology PCI Express Advanced Error Reporting Status register bits (which normally