FC-LS-3 RDP ROUND-UP Roger Hathorn (IBM) Howard L. Johnson (Brocade) Krishna Babu Puttagunta (HP) T11/15-034v1 T11 FC-LS-3 1
Table of Contents Agreements Login Required FEC Descriptor Contemplations FEC Active The trouble with E_Ports Ponderings Buffer Credit Descriptor Optical Menagerie 2
Agreements Login Required Login Required Yep Loop Ports if any FEC Descriptor Decision Leave FEC Descriptor as-is Descriptor Corrected blocks Uncorrectable blocks Bits 31 24 23 16 15 8 7 0 Word 0 FEC Diagnostics descriptor Tag 1 FEC Diagnostics descriptor Length (8 bytes) 2 Number of corrected blocks 3 Number of uncorrectable blocks 3
Contemplations SFP FEC Active Flag Considering 15-050v1 An indication FEC is active would be nice to know Utilize a Reserved bit How about bit-15 in the SFP Flags field? Ponderings? Bit Description 15 8 Reserved 7 6 Connector Type: 00b = unknown 01b = SFP+ 10b - 11b = reserved 5 SFP Diagnostic Parameters not valid: If set to 1, the response does not include valid values for; a) Temperature; b) Vcc; c) Tx Bias; d) Tx Power; and e) Rx Power. 4 Optical Port: If set to 1, the FC_Port is an optical port. If set to 0, the FC_Port is not an optical port. 3-0 Port Tx Type: 0000b = Not optical or other type 0001b = Short Wave Laser 0010b = Long Wave Laser LC 1310nm 0011b = Long Wave Laser LL 1550nm 0100b - 1111b = reserved. 4
Contemplations E_Ports The trouble with E_Ports Determining existence Obtaining an N_Port_ID Knowing which one to query Observations RDP covers getting in and out of the fabric Diagnostics implicates the gross parts 5
Contemplations The Trouble with E_Ports V 20 W My N_Port A B U 10 30 C X Y Z N_Port D 40 F E Your N_Port 6
Pondering Buffer Credit Descriptor Define descriptor My advertised credit My attached port s advertised credit What else? Optical Menagerie Still pondering 7
What s left, partner? The road ahead New Optional Descriptors Buffer Credit Descriptor Other Optics Descriptors Optical Menagerie 8
End of Frame Thank You T11 FC-LS-3 9
References Notes from February Meeting 10
New Items Optical Menagerie 11
FEC Descriptor Descriptor or Status Block Descriptor Counter Values Corrected blocks Uncorrectable blocks Status Block FEC Corrected blocks count FEC Uncorrectable blocks count Reserved Should we change the descriptor? Bits 31 24 23 16 15 8 7 0 Word 0 FEC Diagnostics descriptor Tag 1 FEC Diagnostics descriptor Length (8 bytes) 2 Number of corrected blocks 3 Number of uncorrectable blocks Bits 31 24 23 16 15 8 7 0 Word 0 FEC Status Block descriptor Tag 1 FEC Status Block descriptor Length (16 bytes) 2-5 FEC Status Block 12
Optics Diagnostics Product Data, Warnings, and Alarms Information from Optics Product Data Vendor Name Model Number Serial Number Warnings Exceed warning level on temperature, voltage, bias, or power Alarms Exceeded alarm level on temperature, voltage, bias, or power New descriptors When triggered? Always returned? Bits Word 31.. 24 23.. 16 15.. 08 0 Threshold descriptor tag = 0001 0007h 1 Threshold descriptor length (40 bytes) 07.. 00 2 Low Alarm Temperature High Alarm Temperature 3 Low Warning Temperature High Warning Temperature 4 Low Alarm Voltage High Alarm Voltage 5 Low Warning Voltage High Warning Voltage 6 Low Alarm Bias Current High Alarm Bias Current 7 Low Warning Bias Current High Warning Bias Current 8 Low Alarm TX Power High Alarm TX Power 9 Low Warning TX Power High Warning TX Power 10 Low Alarm Rx Power Low Alarm Rx Power 11 Low Warning Rx Power High Warning Rx Power 13
Optics Diagnostics Comments from Babu Recommendations Always send product, threshold, and flags descriptors Waiting until "triggered" event may too late Unable to read the data when event is triggered due to the failure Bits Word 31.. 24 23.. 16 15.. 08 0 Warning flags descriptor tag = 0001 0008h 1 Warning flags descriptor length (20 bytes) 2 Module Alarm flags Module Warning flags 3 Channel-1 Alarm flags * Channel-1 Warning flags 4 Channel-2 Alarm flags Channel-2 Warning flags 5 Channel-3 Alarm flags Channel-3 Warning flags 6 Channel-4 Alarm flags Channel-4 Warning flags 07.. 00 Optional descriptors Should we add a flag in request to limit descriptors sent? 14
Text Changes Changes from December Meeting for Reference 15
RDP Diagnostic Descriptors Tag Values and Descriptions Required Link Service Request Information descriptor Port Speed descriptor Link Error Status Block descriptor Port Names descriptor Required either SFP Diagnostics descriptor QSFP Diagnostics descriptor Optional FEC Status descriptor Tag value Descriptor 0000 0001h Link Service Request Information 0000 0002h Reserved 0000 0003h N_Port_ID Descriptor 0001 0000h SFP Diagnostics descriptor 0001 0001h Port Speed descriptor 0001 0002h Link Error Status Block descriptor 0001 0003h Port Names descriptor 0001 0004h QSFP Diagnostics descriptor 0001 0005h FEC Status descriptor All other values Reserved 16
QSFP Flags Definition Not Valid One bit for component wide readings and all lane readings Port Tx Type Assumed the Long Wave laser types will also be LC and LL for single mode. Bit Description 15 8 (see next slide) 7 6 Connector Type: 00b = Other or unknown 01b = QSFP 10b - 11b = reserved 5 QSFP Diagnostic Parameters not valid: If set to 1, the response does not include valid values for; a) Temperature; b) Vcc; c) Tx Bias for any lane; d) Tx Power for any lane; and e) Rx Power for any lane. 4 Optical Port: If set to 1, the FC_Port is an optical port. If set to 0, the FC_Port is not an optical port. 3-0 Port Tx Type: 0000b = Other or unknown 0001b = Short Wave Laser 0010b = Long Wave Laser LC 1310nm 0011b = Long Wave Laser LL 1550nm 0100b - 1111b = reserved. 17
QSFP Flags Definition (continued) Lane Identifier Indicates lane corresponding to this response Lane Identifier Valid Indicates if the QSFP is operating in parallel lane mode Bit Description 15 11 Reserved 10 9 Lane Identifier: 00b = Lane 0 01b = Lane 1 10b = Lane 2 11b = Lane 3 8 Lane Identifier Valid: If set to 1, the response is for the single lane indicated by the Lane Identifier. If set to 0, the Lane Identifier is not meaningful. 18
What s left, partner? The road ahead Just E_Ports And an expanding horizon 19
References Notes from December Meeting 20
RDP Diagnostic Descriptors Tag Values and Descriptions Required Link Service Request Information descriptor Port Speed descriptor Link Error Status Block descriptor Port Names descriptor Required either SFP Diagnostics descriptor QSFP Diagnostics descriptor Optional FEC Status descriptor Tag value Descriptor 0000 0001h Link Service Request Information 0000 0002h 0000 0003h N_Port_ID Descriptor 0001 0000h SFP Diagnostics descriptor 0001 0001h Port Speed descriptor 0001 0002h Link Error Status Block descriptor 0001 0003h Port Names descriptor 0001 0004h QSFP Diagnostics descriptor 0001 0005h FEC Status descriptor All other values Reserved 21
QSFP Descriptor Definition Common Statistics Temperature Votage Flags Per Lane Statistics Tx Bias and Power Rx Power Bits 31 24 23 16 15 8 7 0 Word 0 QSFP Diagnostics descriptor Tag 1 QSFP Diagnostics descriptor Length (40-bytes) 2 Temperature Vcc 3 Reserved QSFP Flags 4 Lane 0 - Tx Bias Lane 0 - Tx Power 5 Lane 0 - Rx Power Reserved 6 Lane 1 - Tx Bias Lane 1 - Tx Power 7 Lane 1 - Rx Power Reserved 8 Lane 2 - Tx Bias Lane 2 - Tx Power 9 Lane 2 - Rx Power Reserved 10 Lane 3 - Tx Bias Lane 3 - Tx Power 11 Lane 3 - Rx Power Reserved 22
QSFP Flags Definition Not Valid One bit for component wide readings and all lane readings Port Tx Type Assumed the Long Wave laser types will also be LC and LL for single mode. Bit Description 15 8 (see next slide) 7 6 Connector Type: 00b = unknown 01b = QSFP 10b - 11b = reserved 5 QSFP Diagnostic Parameters not valid: If set to 1, the response does not include valid values for; a) Temperature; b) Vcc; c) Tx Bias for any lane; d) Tx Power for any lane; and e) Rx Power for any lane. 4 Optical Port: If set to 1, the FC_Port is an optical port. If set to 0, the FC_Port is not an optical port. 3-0 Port Tx Type: 0000b = Not optical or other type 0001b = Short Wave Laser 0010b = Long Wave Laser LC 1310nm 0011b = Long Wave Laser LL 1550nm 0100b - 1111b = reserved. 23
QSFP Flags Definition (continued) Lane Identifier Indicates lane corresponding to this response QSFP Lane Diagnostic Parameters validity indicator Set if QSFP diagnostic parameters are invalid for this lane Bit Description 15 11 Reserved 10 9 Lane Identifier: 00b = Lane 0 01b = Lane 1 10b = Lane 2 11b = Lane 3 8 Breakout Operation: If set to 1, the response is for the lane indicated by the Lane Identifier. If set to 0, the Lane Identifier is not meaningful. Breakout Operation Indicates QSFP is operating in breakout mode 24
FEC Descriptor Definition Counter Values Corrected blocks Uncorrectable blocks Bits 31 24 23 16 15 8 7 0 Word 0 FEC Diagnostics descriptor Tag 1 FEC Diagnostics descriptor Length (8 bytes) 2 Corrected blocks 3 Uncorrectable blocks 25
Coming Attractions See you in St. Petersburg Login Required if the N_Port_ID is not logged in with an F_Port within the domain, the Domain Controller should reply with an LS_RJT specifying a reason code of Logical error (03h) and reason code explanation Invalid N_Port_ID (1Fh). Loop Ports Remind me again what s the issue with sending an RDP to a loop port? E_Ports I m waiting on getting QSFPs done and in the text before we tackle this one. 26
References Notes from October Meeting 27
SFP Descriptor Reference Fundamental Statistics Temperature Votage Tx Bias and Power Rx Power Flags Bits 31 24 23 16 15 8 7 0 Word 0 SFP Diagnostics descriptor Tag 1 SFP Diagnostics descriptor Length (12-bytes) 2 Temperature Vcc 3 Tx Bias Tx Power 4 Rx Power SFP Flags No change to current proposal 28
SFP Flags Per SFP Connector Type Changed from 4-bits to 2-bits Return to original proposal (minus reference to QSFP ;-) Future connector types Represented by new descriptors Bit Description 15 8 Reserved 7 6 Connector Type: 00b = unknown 01b = SFP+ 10b - 11b = reserved 5 SFP Diagnostic Parameters not valid: If set to 1, the response does not include valid values for; a) Temperature; b) Vcc; c) Tx Bias; d) Tx Power; and e) Rx Power. 4 Optical Port: If set to 1, the FC_Port is an optical port. If set to 0, the FC_Port is not an optical port. 3-0 Port Tx Type: 0000b = Not optical or other type 0001b = Short Wave Laser 0010b = Long Wave Laser LC 1310nm 0011b = Long Wave Laser LL 1550nm 0100b - 1111b = reserved. 29
QSFP Descriptor Proposed Common Statistics Temperature Votage Flags Per Lane Statistics Tx Bias and Power Rx Power Removed lane flags Bits 31 24 23 16 15 8 7 0 Word 0 QSFP Diagnostics descriptor Tag 1 QSFP Diagnostics descriptor Length (40-bytes) 2 Temperature Vcc 3 Reserved QSFP Flags 4 Lane 0 - Tx Bias Lane 0 - Tx Power 5 Lane 0 - Rx Power Reserved 6 Lane 1 - Tx Bias Lane 1 - Tx Power 7 Lane 1 - Rx Power Reserved 8 Lane 2 - Tx Bias Lane 2 - Tx Power 9 Lane 2 - Rx Power Reserved 10 Lane 3 - Tx Bias Lane 3 - Tx Power 11 Lane 3 - Rx Power Reserved 30
QSFP Flags Proposed per QSFP Connector Type Changed from 4-bits to 2-bits Consistent with SFP Flags field Port Tx Type What are the long wave options? 1310nm/1550nm/? Bit Description 15 8 (see next slide) 7 6 Connector Type: 00b = unknown 01b = QSFP 10b - 11b = reserved 5 QSFP Diagnostic Parameters not valid: If set to 1, the response does not include valid values for; a) Temperature; and, b) Vcc. 4 Optical Port: If set to 1, the FC_Port is an optical port. If set to 0, the FC_Port is not an optical port. 3-0 Port Tx Type: 0000b = Not optical or other type 0001b = Short Wave Laser 0010b = Long Wave Laser 0011b - 1111b = reserved. 31
QSFP Flags Proposed per QSFP (continued) Lane Identifier Indicates lane corresponding to this response QSFP Lane Diagnostic Parameters validity indicator Set if QSFP diagnostic parameters are invalid for this lane Bit Description 15 12 Reserved 11 10 Lane Identifier: 00b = Lane 0 01b = Lane 1 10b = Lane 2 11b = Lane 3 9 QSFP Lane Diagnostic Parameters not valid: If set to 1, the response does not include valid values for; a) Tx Bias; b) Tx Power; and c) Rx Power. 8 Breakout Operation: If set to 1, the response is for the lane indicated by the Lane Identifier. Breakout Operation Indicates QSFP is operating in breakout mode 32
FEC Descriptor Same descriptor, different description Counter Values Corrected blocks Uncorrectable blocks Bits 31 24 23 16 15 8 7 0 Word 0 FEC Diagnostics descriptor Tag 1 FEC Diagnostics descriptor Length (8 bytes) 2 Number of corrected blocks 3 Number of uncorrectable blocks 33
Coming Attractions See you in St. Petersburg Login Required Same as RLS? Loop Ports Do we have too? E_Ports Let the Fabric do it? 34
References Notes from August Meeting 35
RDP Summary The new language The diagnostic parameter descriptors (see 4.3.48.5) in an RDP LS_ACC shall include; a) a Link Service Request Information descriptor; b) a Port Speed descriptor; c) a Link Error Status Block descriptor; d) a Port Names descriptor for the Fx_Port or Nx_Port for which diagnostic parameters are being provided; and e) a Port Names descriptor for the directly attached Fx_Port or Nx_Port. The diagnostic parameter descriptors in an RDP LS_ACC shall include either; a) an SFP Diagnostics descriptor; or b) a QSFP Diagnostics descriptor. The diagnostic parameter descriptors in an RDP LS_ACC may include; a) a FEC Status descriptor. 36
QSFP Descriptor Provided instead of an SFP Descriptor Bits Word 31 24 23 16 15 8 7 0 0 QSFP Diagnostics descriptor Tag 1 QSFP Diagnostics descriptor Length (40-bytes) 2 Temperature Vcc 3 reserved QSFP Flags 4 Lane 0 - Tx Bias Lane 0 - Tx Power 5 Lane 0 - Rx Power Lane 0 Flags 6 Lane 1 - Tx Bias Lane 1 - Tx Power 7 Lane 1 - Rx Power Lane 1 Flags 8 Lane 2 - Tx Bias Lane 2 - Tx Power 9 Lane 2 - Rx Power Lane 2 Flags 10 Lane 3 - Tx Bias Lane 3 - Tx Power 11 Lane 3 - Rx Power Lane 3 Flags 37
QSFP Flags Per QSFP Bit Description 15 10 Reserved 9 6 Connector Type: 0000b = unknown 0001b = QSFP 0010b - 1111b = reserved 5 QSFP Diagnostic Parameters not valid: If set to 1, the response does not include valid values for; a) Temperature; and, b) Vcc. 4 Optical Port: If set to 1, the FC_Port is an optical port. If set to 0, the FC_Port is not an optical port. 3-0 Port Tx Type: 0000b = Not optical or other type 0001b = Short Wave Laser 0010b = Long Wave Laser 0011b - 1111b = reserved. 38
Lane Flags Per Lane Bit Description 15 1 Reserved 0 QSFP Lane Diagnostic Parameters not valid: If set to 1, the response does not include valid values for; a) Tx Bias; b) Tx Power; and c) Rx Power. 39
FEC Descriptor Optionally provided Bits Word 31 24 23 16 15 8 7 0 0 FEC Diagnostics descriptor Tag 1 FEC Diagnostics descriptor Length (8 bytes) 2 Number of corrected blocks 3 Number of uncorrected blocks 40
E_Ports Anyone? 41
Addressing Rules Works great for N_Ports and F_Ports My F_Port If the D_ID is the F_Port Controller Well-known address (FFFFFEh), the N_Port_ID in the N_Port_ID descriptor shall be set to the S_ID of the request. The diagnostic parameters requested are for the Fx_Port through which the N_Port_ID is logged in My F_Port D_ID == F_Port Controller S_ID == My N_Port_ID N_Port_ID == My N_Port_ID Your F_Port If the D_ID is a Domain Controller well known address (FFFCxxh), the N_Port_ID in the N_Port_ID descriptor shall be set to an N_Port_ID within the associated domain. The diagnostic parameters requested are for the Fx_Port through which the N_Port_ID is logged in Your N_Port For all other D_IDs, the N_Port_ID in the N_Port_ID descriptor shall be set to the D_ID of the request. The diagnostic parameters requested are for Nx_Port to which the D_ID is assigned Your F_Port D_ID == Your Domain Controller S_ID == My N_Port_ID N_Port_ID == Your N_Port_ID Your N_Port D_ID == Your N_Port_ID SID == My N_Port_ID N_Port_ID == Your N_Port_ID 42
Read Diagnostic Parameters What s missing? V 20 W My N_Port A B U 10 30 C X Y Z N_Port D 40 F E Your N_Port 43
Read Diagnostic Parameters What s missing? My F_Port (F_Port Controller) V 20 W My N_Port A B U 10 30 C X Y Z N_Port D 40 F E Your N_Port 44
Read Diagnostic Parameters What s missing? V 20 W My N_Port A B U 10 30 C X Y Z N_Port D 40 F E Your N_Port Your F_Port (Domain Controller) 45
Read Diagnostic Parameters What s missing? V 20 W My N_Port A B U 10 30 C X Y Z N_Port Your N_Port (N_Port_ID) D 40 F E Your N_Port 46
Read Diagnostic Parameters What s missing? My F_Port (F_Port Controller) V 20 W My N_Port A B U 10 30 C We can t get the E_Ports! X Y Z N_Port Your N_Port (N_Port_ID) D 40 E Your F_Port (Domain Controller) F Your N_Port 47
Let s Add a Rule D_ID is the Domain Controller (but not for Your N_Port_ID) Our Flow Definition If the D_ID is a Domain Controller well known address (FFFCxxh) and the N_Port_ID in the N_Port_ID descriptor is set to an N_Port_ID not on the associated domain, the diagnostic parameters requested are for Ex_Port(s) through which the N_Port_ID is accessed Our Flow Description The fabric returns an RDP LS_ACC for each possible Ex_Port traversed to reach the N_Port_ID. The fabric forwards the request to each hop in the fabric and floods the responses back. The S_ID is updated to reflect the Domain Controller of the forwarding switch. The exchange remains open until the last response is processed. Our Flow Addressing D_ID == Domain Controller S_ID == My N_Port_ID then Domain Controller for My Switch then Domain Controller for Next Switch then Domain Controller for Your Switch. N_Port_ID == Your N_Port_ID 48
Read Diagnostic Parameters What s missing? Our Flow (Domain Controller == 10 && Your N_Port_ID) My N_Port A B Our Flow (Domain Controller == 20 && Your N_Port_ID) U 20 10 30 C V D 40 W Our Flow (Domain Controller == 30 && Your N_Port_ID) 1. Domain 10 sends back diagnostics for U. 2. Domain 20 sends back diagnostics for V and W. 3. Domain 30 sends back diagnostics for X and closes the exchange. E X F Y Z Your N_Port 49
References Notes from June Meeting 50
RDP Descriptors Let s Change the Language Descriptors can appear in any order Except, the Link Service Request Information descriptor must come first The rest shall be: Port Speed descriptor Link Error Status Block descriptor Port Names descriptors The Fx_Port or Nx_Port for which diagnostic parameters are being provided The directly attached Fx_Port or Nx_Port And, they may all come in any order Except, the Link Service Request Information descriptor, which must come first 51
RDP Descriptors Optical Descriptors for Everyone Thou shalt supply either An SFP Descriptor Temperature, Voltage Tx Bias, Tx Power, Rx Power, Flags Or, a QSFP Descriptor Temperature, Voltage, Flags Appears Once per descriptor Tx Bias, Tx Power, Rx Power Appears per fiber pair (4 times) Thou shalt not supply neither nor both, but either (and not neighther) 52
RDP Descriptors Just a FECing Descriptor Optionally, and in any order (except not first, for that is reserved for the Link Service Request Information descriptor, which must come first) A FEC Status Descriptor Count of Uncorrected Errors Count of Corrected Errors One word per counter 53
RDP Summary Descriptors And the descriptors are: Link Service Request Information Port Speed LESB Port Name of diagnosed port Port Name of directly attached port Optical Characteristics of your choice And sometimes, the FEC Counters 54
RDP Summary Suggested Text The diagnostic parameter descriptors (see 4.3.48.5) in an RDP LS_ACC shall include; a) a Link Service Request Information descriptor; b) a Port Speed descriptor; c) a Link Error Status Block descriptor; d) a Port Names descriptor for the Fx_Port or Nx_Port for which diagnostic parameters are being provided; and e) a Port Names descriptor for the directly attached Fx_Port or Nx_Port. The diagnostic parameter descriptors in an RDP LS_ACC shall include either; a) an SFP Diagnostics descriptor; or b) a QSFP Diagnostics descriptor. The diagnostic parameter descriptors in an RDP LS_ACC may include; a) a FEC Status descriptor. 55
References Notes from April Meeting 56
Suggestion for 128GFC QSFP Diagnostic Parameters descriptor New Descriptor Type Unique for QSFP Used instead of SFP Diagnostic Parameters descriptor in reply Fields QSFP wide (case) Temperature Voltage Per lane (fiber pair) Tx Bias Tx Power Rx Power Flags 57
Questions QSFP unique things General Should there be a count of lanes? Descriptor for each lane (use SFP descriptor format)? Should there be separate validity flags for the component (i.e. temperature and voltage)? Or, just replicate the validity within the flags for each lane? Parallel Operation How do we distinguish the lanes in the descriptor? Does it matter? Break-out Operation Can the SFP descriptor be used? Need flag indicating QSFP running in break-out 58
RDP Descriptor FC-LS-3 4.3.48.4 Reply Sequence The diagnostic parameter descriptors (see 4.3.48.5) in an RDP LS_ACC shall include; a) a Link Service Request Information descriptor; b) an SFP Diagnostics descriptor; c) a Port Speed descriptor; d) a Link Error Status Block descriptor; e) a Port Names descriptor for the Fx_Port or Nx_Port for which diagnostic parameters are being provided; and f) a Port Names descriptor for the directly attached Fx_Port or Nx_Port. 59
RDP Descriptor FC-LS-3 4.3.48.5 Diagnostic parameter descriptors LESB expands to FEC corrections? SFP descriptor becomes Module descriptor Lane descriptor(s) 60
LESB FC-FS-4 22.4.8 Link Error Status Block The errors shown in table 94 are accumulated over time within a PN_Port. The format shown is the format in which the LESB information shall be supplied in response to an RLS ELS. It does not require any specific hardware or software implementation. The errors accumulated provide a coarse measure of the integrity of the link over time. No means are provided to reset a counter in the LESB; however, on overflow it shall be set to zero and then continue counting. The counts shall be 32 bit values. 61
This slide intentionally left blank Thank You T11 FC-LS-3 62