Space Plug-and-Play Architecture Standard

Transcription

1 BSR/ AIAA S Space Plug-and-Play Architecture Standard Networking Sponsored by American Institute of Aeronautics and Astronautics Approved Month 201 Abstract This document specifies the overall SPA network methodology, the approach to abstraction of unique transport details, and methods of communicating across multiple similar and dissimilar networks. This document does not discuss details about messaging protocol families, message structure, or the format of specific SPA messages. Those specifications are expressed in the SPA Interface document.

2 BSR/AIAA S Published by American Institute of Aeronautics and Astronautics 1801 Alexander Bell Drive, Reston, VA Copyright 201 American Institute of Aeronautics and Astronautics All rights reserved No part of this publication may be reproduced in any form, in an electronic retrieval system or otherwise, without prior written permission of the publisher. Printed in the United States of America ii

3 BSR/AIAA S Contents Contents... iii Foreword... v Introduction Scope Applicable Documents Vocabulary Acronyms and Abbreviated Terms Terms and Definitions Networking Requirements for a SPA System Overview Topology Discovery and Routing Topology Discovery Dynamic Topology Reconfiguration Routing During Component Registration Requesting an Address from the CAS for an Item not in the SM-x Table After Initial Discovery Component-to-Component Routing Requirement to Support Packet Fragmentation Specific Topology Considerations Multiple SM-x on the Same Subnet Multiple Available Paths Within a Subnet SPA Network Requirements SPA Local Bus Requirements Central Addressing Service (CAS) Requirements General Requirements Redundancy Logical address Assignment Component Information Stored and Delivered SPA Lookup Service Requirements General Requirements Component Discovery Subscription SPA subnet Manager (SM-x) Requirements Topology Discovery Network Contracts iii

4 BSR/AIAA S Routing Generic Router Requirements Generic SPA Endpoint Requirements SPA Checksum Generation Annex A Compliance Matrix for a SPA Network Figures Figure 1 Example SPA network implementation... 4 Figure 2 SPA network phases of operation... 5 Figure 3 A sample SPA network topology for discovery... 6 Figure 4 Network topology discovery... 7 Figure 5 SPA lookup Service to SPA component intercommunication Figure 6 Component registration data flow sequence Figure 7 Address request sequencing diagram Figure 8 Subnet to subnet communication via SPA-L Figure 9 Subnet to subnet communication without SPA-L Figure 10 A subnet with two SM-x Figure 11 A subnet with two routes to every component Tables Table 1 SPA request address block format... 9 Table 2 SPA assign address block message format... 9 Table 3 SPA request CAS route message format Table 4 SPA reply CAS route message format Table 5 SPA distribute route message format Table 6 CAS routing table example Table 7 SM-x routing table example Table A1 Minimum requirements for a compliant SPA network iv

5 BSR/AIAA S Foreword This document was developed by the Space Plug and Play Avionics (SPA) Standards Working Group as one of a series of 10 documents describing the various components of the standard. The SPA standards were initially recorded in earlier documentation. This document set separates content along logical boundaries to better organize the volumes (so that developers or domain experts need only reference the documents applicable to their needs) and to avoid duplication of content between documents in the standard series. This 201 AIAA standard supersedes all previous documentation of the SPA standards. This particular volume of the SPA Networking standard contains information not recorded in previous documentation. It is part of a set of 10 volumes describing other components of the standard: SPA Guidebook SPA Networking Standard SPA Physical Interface Standard Ontology Standard SPA 28V Power Service Standard SPA System Timing Standard SPA Ontology Standard SPA Test Bypass Extension Standard SPA SpaceWire (SPA-S) Subnet Adaptation Standard SPA System Capability Standard At the time of approval, the members of the AIAA SPA Standards Committee were: Fred Slane, Chair Jeanette Arrigo Scott Cannon Ken Center Don Fronterhouse* Rod Green Jane Hansen Doug Harris Paul Jaffe Stanley Kennedy* Ronald Kohl Bill Kramer Ramon Krosley Denise Lanza James Lyke Space Infrastructure Foundation Sierra Nevada Corporation Utah State University PnP Innovations PnP Innovations Design Group HRP Systems Operationally Responsive Space Office Naval Research Laboratory Comtech Aero-Astro R.J. Kohl & Associates Independent Independent SAIC Air Force Research Laboratory v

6 BSR/AIAA S Joseph Marshall Gerald Murphy* Gary Rodriguez Steven Schenk Robert Vick* BAE Systems Design Group sysrand Comtech Aero-Astro SAIC The above consensus body approved this document in Month 201. The AIAA Standards Executive Council (VP-Standards Name, Chairman) accepted the document for publication in Month 201. The AIAA Standards Procedures dictates that all approved Standards, Recommended Practices, and guides are advisory only. Their use by anyone engaged in industry or trade is entirely voluntary. There is no agreement to adhere to any AIAA standards publication and no commitment to conform to or be guided by standards reports. In formulating, revising, and approving standards publications, the committees on standards will not consider patents that may apply to the subject matter. Prospective users of the publications are responsible for protecting themselves against liability for infringement of patents or copyright or both. *Alternate CoS participant. vi

7 Introduction SPA is a collection of standards designed to facilitate rapid construction of spacecraft systems using modular components. The SPA Networking standard is intended to specify the overall SPA network methodology. To this end it discusses how packets are routed and how component discovery occurs, with attention given to topology concerns. The Networking standard does not discuss the specific fields of individual packet formats, nor does it attempt to limit the types of networks that might connect to a SPA network; those are considerations addressed in the SPA Logical Interface Standard and the various subnet standards (of which SpaceWire is the only currently approved subnet standard). This standard describes the minimum requirements for the components in a SPA network and for the functions of network topology discovery, routing table construction and distribution, packet routing, and dynamic reconfiguration. 1

8 1 Scope This document specifies the overall SPA network methodology, the approach to abstraction of unique transport details, and methods of communicating across multiple similar and dissimilar networks. This document does not discuss details about messaging protocol families, message structure, or the format of specific SPA messages. Those specifications are expressed in the SPA Logical Interface Standard (AIAA S ). 2 Applicable Documents The following standards documents contain provisions which, through reference in this text, constitute provisions of this standard. For dated references, subsequent amendments to, or revisions of, any of these publications do not apply. However, parties to agreements based on this standard are encouraged to investigate the possibility of applying the most recent editions of the normative documents indicated below. For undated references, the latest edition of the normative document referred to applies. AIAA G , SPA Guidebook AIAA S , SPA Logical Interface Standard AIAA S , SPA SpaceWire Subnet Adaptation Standard 3 Vocabulary 3.1 Acronyms and Abbreviated Terms AIAA ASME ASTM CAS EOP MTU SM-L SM-x SM-s SPA SPA-L xteds UUID American Institute of Aeronautics and Astronautics American Society of Mechanical Engineers American Society of Testing and Materials Central Addressing Service End of Packet SPA Endpoint Maximum Transmission Unit SPA Manager for the SPA Local interconnects SPA Subnet Manager, where x represents a given technology protocol SPA Manager for SpaceWire Protocol subnet Space Plug-and-Play Architecture SPA Local Interconnect Extensible Transducer Electronic Data Sheet Universally Unique Identifier 2

9 3.2 Terms and Definitions For the purposes of this Standard, the following terms and definitions shall apply. Central Addressing Service responsible for providing logical address blocks to be assigned to each hardware or software component registered with the system during component registration. It stores the logical address block, UUID, and physical address for each core component. Component an endpoint whose interface conforms to the SPA Interface standard and does not connect to another SPA object via a different port. Note that a SPA hardware component is indistinguishable from a SPA software component for the purposes of this document. Plug and play ability to connect a device to the larger system and have the two work together with little or no set-up required (e.g., automated system recognition and data exchange) Rrouter a device with multiple ports that may be attached to another SPA router, a SPA manager, gateway, or SPA component endpoint SPA lookup service responsible for accepting component registration and providing data source route information for components requesting a particular type of service (or returning a negative acknowledgment if the service is not available) SPA manager responsible for performing discovery for a particular subnet. It maps incoming packets to the correct SPA endpoint on the subnet, encapsulating the SPA packet with the correct protocol header. In the reverse direction it removes the protocol header and possibly adds a new header conforming to the subnet the packet is about to enter. It is also responsible for topology discovery and reporting within the subnet. 4 Networking Requirements for a SPA System 4.1 Overview An example network topology implemented with SPA components is shown in Figure 1 below. The major components of the network are as follows: a. Component Addressing Service (CAS). A process that provides a fixed-size logical address block (65536 bytes or 2 16 addresses) unique to the SPA network upon request from a subnet manager. The addresses in that block shall only be assigned to components (not SPA Managers, e.g. SM-L, SM-x) residing on that subnet. Only one CAS shall be active in a particular SPA network at any time; redundancy is encouraged but not required. The CAS process shall exist on a SPA-L interconnect, and shall be directly accessible by all SM-x in the SPA network after the SM-L has delivered its address. b. SPA Lookup Service: A process that accepts component registrations of SPA components providing services and provides the ID for that service to other components which request them. Only one SPA Lookup Service shall be active in a particular SPA network at any time; redundancy is encouraged but not required. The SPA Lookup Service process must be instantiated on a SPA-L interconnect, and can be directly accessed by SM-x nodes after it has been discovered and its address has been distributed. 3

10 c. SPA Manager (SM-x): SM-x is a protocol specific SPA process on a SPA-L that acts as a translation device for a particular protocol. A SPA-S Manager (SM-s, for the specific SpaceWire protocol set), for example, accepts packets received from SPA-L, and removes the protocol specific header. Then it performs a lookup of the SPA destination address in its local lookup table, and encapsulates the packet in a SpaceWire header/end of Packet (EOP), with the correct destination or logical SpaceWire route to reach that endpoint, or adjacent manager if the endpoint is distant. In the outgoing direction it removes the SpaceWire header from the outgoing packet and adds a protocol header as indicated in its outgoing lookup table. d. In a hardware implementation, an SM-x might act as a simple bridging device to move packets from one protocol subnet to another. In this case the SM-x shall be required to request addresses from the CAS for items in both subnet, and translate packet moving from one domain to another. e. It is important to note that it is normal practice for multiple SM-x to coexist on the same subnet, independently performing topology discovery. Each SM-x shall construct a table of physical routes to each component on the subnet. An SM-x shall subsequently request and distribute an address block from the CAS for SM-x existing on its sub-net. Those SM-x shall then request an address block in turn from the CAS for SM-x on their subnet. Note that the CAS shall always distribute the same block for a SM-x with a particular SM-x if it has been previously assigned. f. SPA-L: SPA-L is the interconnect protocol used to connect applications on a processing node of the SPA network. A SPA-L interconnect is controlled by a SM-L process, just as a physical subnet would be controlled by a SM-x process or device. The SM-L process receives an address block for each process on the node, just as the SM-s receives an address block for each endpoint on a SpaceWire network. g. The CAS must exist on a SPA-L interconnect, but it will exist on only one SPA-L in a single SPA network. There would be one SPA-L interconnect for each processing node in the system. h. SPA Gateway: A SPA Gateway is a bridge to another, non-spa network on the spacecraft. As such, a SPA Gateway shall provide a fully compliant SPA interface (e.g., a SPA Endpoint) on the SPA side of the Gateway recognized, identified, and registered during the topology discovery process. SPA Lookup Service SM-L CAS SM-s Spacewire Subnet SM-s SPA-L SM-s Spacewire Subnet Gateway SM-L SPA-L SM-s Spacewire Subnet SM-s SPA-L SM-L SM-i I2C Subnet Figure 1 Example SPA network implementation 4

11 4.2 Topology Discovery and Routing This section discusses the method by which the topology of the SPA network with associated subnets is discovered and how packets are routed during and following topology discovery. The four main stages of operation are shown below in Figure 2. Figure 2 SPA network phases of operation Topology Discovery Topology discovery should assign to each component a logical address in the SPA network. A logical address is unique across the SPA network, whereas the Universally Unique ID (UUID) is unique for each specific SPA component (UUIDs are further described in the SPA Logical Interface Standard (AIAA S ). The smaller address space created by using SPA logical addresses makes message headers smaller, while maintaining the identity provided by the UUID s. The mapping from UUID to logical address shall be one-to-one. The logical addresses assigned to each component shall remain constant from the time of first assembly of the network (e.g., initial system integration). The CAS and SPA managers (SM-L, SM-x) shall assign the same logical addresses to components with the same UUID requesting addresses multiple times, as might occur in the event of a power cycle or reset of a subnetwork or component. The routing information shall allow for any component to establish a consumer-provider relationship with any other component. Such relationships are described in Section and Section below. The discovery process shall notify the single active SPA Lookup Service function of each SPA component using a SPARqstDmProbe message. 5

12 During this phase the CAS assigns a logical address block to each SPA Manager in the SPA network. Each SM-x is informed of the route to the other mangers, and the CAS and SPA Lookup Service processes. This step may optionally be skipped on power-up if the network has a stored configuration. However the network shall be capable of repeating topology discovery in operation to detect newly added components Initial Topology Discovery Sequence Consider the example network shown in Figure 3. When the system is first configured, none of the SPA components have been assigned addresses, and none of the managers have any knowledge of the components residing on their subnet. The detailed sequence for the required discovery and address assignment procedure is shown below in Figure 4. CAS SPA (1) DM SPA-L SM-x (1) Router SM-x (2) Router SM-L SUBNET 1 SUBNET 2 Figure 3 A sample SPA network topology for discovery 6

13 Figure 4 Network topology discovery 1. While performing component discovery for SPA processing node (local) components (applications, CAS, SPA Lookup Service, SM-x,, etc.) the SM-L discovers CAS. 2. While performing component discovery for SPA local components the SM-L discovers SM-x s on its local interconnect. 3. Each SM-x probes its subnet this happens concurrently with steps above. 4. SM-x s which reside on a subnet are probed as well. 5. Each SM-x probes its subnet this happens concurrently with steps above. 6. SM-L requests an address block for itself from the CAS using the SPARqstAddrBlock message. 7

14 7. CAS returns an address block of addresses to the SM-L using the SPAAssignAddrBlock message. 8. SM-L assigns logical addresses to all known components except other SM-x s on its subnet. 9. SM-L requests an address block on behalf of the SM-x (1) using the SPARqstAddrBlock message. 10. CAS returns an address block of addresses to the SM-L using the SPAAssignAddrBlock message. 11. SM-L sends the newly requested address block to SM-x (1) using the SPAAssignAddrBlock message. 12. SM-L sends the routes for CAS, DM, and all known SM-x s to SM-x (1) using multiple SPADistributeRoute messages. 13. SM-x (1) assigns logical addresses to all known components except other SM-x s on its subnet. 14. SM-x (1) requests an address block on behalf of the SM-x (2) using the SPARqstAddrBlock message. 15. CAS returns an address block of addresses to the SM-x (1) using the SPAAssignAddrBlock message. 16. SM-x (1) sends the newly requested address block to SM-x (2) using the SPAAssignAddrBlock message. 17. SM-x (1) sends the routes for CAS, SPA Lookup Service, and for all known SM-x s to SM-x (2) using multiple SPADistributeRoute messages. 18. SM-x (1) sends the new route to SM-x (2) to SM-L using a SPADistributeRoute message. 19. SM-x (2) assigns logical addresses to all known components except other SM-x s on its subnet. This process continues throughout the network. Each attached subnet may contain its own processor nodes and other attached subnets. Each shall receive an address block assignment from a SM-x closer to the CAS. Each attached subnet shall also receive routing information for the other managers and core components, and shall pass these routes along to other managers. The detailed description of each of these related message formats is found below in Sections through SPA Request Address Block Message A SPARqstAddrBlock message shall be sent to the CAS to request a block of logical addresses for use in assignment. This may be sent during the topology discovery process, or later when a previously assigned block is exhausted. The format for the SPA Request Address Block message is shown in Table 1. 8

15 Table 1 SPA request address block format Message Name Message Opcode SPA Version Summary Description 0x64 Field Name Size Offset Value 1 (Note: 1 indicates the initial version) Request a block of logical addresses SPARqstAddrBlock (net) Description UUID Universally Unique SPA Identifier of requesting device Description Name Full Description Size 16 Offset 16 DataType Units DefaultValue Range UUID Universally Unique SPA Identifier of requesting device The UUID is a unique identifier assigned to all SPA compliant components. It is unique not only for a particular SPA network, but for all SPA networks. uint SPA Assign Address Block Message This message shall be provided in response to the SM-x or CAS requesting an address block for component assignments. The format for the SPA Request Address Block message is shown below in Table 2. Table 2 SPA assign address block message format Message Name Message Opcode SPA Version Summary Description 0x65 1 (Note: 1 indicates the initial version) Response to SPARqstAddrBlock SPAAssignAddrBlock (net) Field Name Size Offset Value Description RespType 1 16 Type of address response AssignedBaseAddress 4 17 The base of the address block UUID Universally Unique SPA Identifier Description Name Full Description Size 1 Offset 16 DataType Units DefaultValue Type of address response RespType Indicates if the response contains a valid block, or indicates the requested block size is not available. uint8 9

16 Range Description Name Full Description Size 4 Offset 17 DataType Units DefaultValue Range Description Name Full Description Name Description Value Valid Block assigned is valid. 0 Invalid Request denied, ignore block fields. 1 The base of the address block AssignedBaseAddress The lowest address in the allocated block. uint32 Size 16 Offset 21 DataType Units DefaultValue Range Universally Unique SPA Identifier UUID The UUID is a unique identifier assigned to all SPA compliant components. It is unique not only for a particular SPA network, but for all SPA networks. This UUID matches the one sent in the original request. uint SPA Request CAS Route Message If a manager receives a packet that is leaving its subnet for which the logical address has no assigned route, this command shall be sent to the CAS to request the route information. This will commonly occur the first time a component attempts to negotiate or deliver services from a component outside of its own subnet. CAS should reply with a manager which can receive the packet (SPADistributeRoute), or an error message (SPAReplyCasRouteUnknown) The format for the SPA Request CAS Route message is shown below in Table 3. Table 3 SPA request CAS route message format Message Name SPARqstCasRoute (net) Message Opcode 0x66 SPA Version 1 (Note: 1 indicates the initial version) Summary Description Reqest to CAS for unknown route Field Name Size Offset Value Description UnknownAddress 4 16 Logical Address Name UnknownAddress Description Logical Address Full Description The logical address received at the manager which is not present in its routing 10

17 tables. Size 4 Offset 16 DataType uint32 Units DefaultValue Range SPA Reply CAS Route Message When the CAS receives a request for a route that it cannot fulfill (the address is unknown), this message shall be returned to indicate that the packet cannot be delivered. The returned address distinguishes the request, and allows the manager to check if its request was garbled. The format for the SPA Reply CAS Route message is shown below in Table 4. Table 4 SPA reply CAS route message format Message Name SPAReplyCasRouteUnknown (net) Message Opcode 103 SPA Version 1 (Note: 1 indicates the initial version) Summary Description Error response for SPARqstCasRoute Field Name Size Offset Value Description UnknownAddress 4 16 Logical Address Name UnknownAddress Description Logical Address Full Description The Logical address received at the CAS, for which it cannot find a route. Size 4 Offset 16 DataType uint32 Units DefaultValue Range SPA Distribute Route Message This message shall be used by the CAS or any SPA Manager to advertise the route for any SPA component. SPA Managers must use this command to report any other SPA Managers they discover during topology discovery. The CAS uses this message to respond to a route request. The format for the SPA Distribute Route message is shown in Table 5. 11

18 Table 5 SPA distribute route message format Message Name Message Opcode SPA Version 0x68 1 (Note: 1 indicates the initial version) Summary Description Advertises a route to a SPA component Message Description SPADistributeRoute (net) Field Name Size Offset Value Description ComponentAddress 4 16 Logical Address ComponentType 1 20 Description Name Full Description Size 4 Offset 16 DataType Units DefaultValue Range Description Name Full Description Size 1 Logical Address Type of SPA component for which the route is being advertised. ComponentAddress Component for which the route is being advertised. uint32 Offset 20 DataType Units DefaultValue Range ComponentType Type of SPA component for which the route is being advertised. The description of the device being advertised uint8 Name Description Value CAS Central Addressing Service 0 DM Data Manager 1 SM-L SPA Local Interconnect Managet 2 SM-x SPA Subnet Manager (not local interconnect) 3 SPA Gateway Gateway device bridging two SPA networks with different address spaces Other Other SPA component Mapping the SPA Lookup Service The SPA Lookup Service is a special case in the discovery process. When the SPA Lookup Service is discovered in the SPA system and an address is requested for it from the CAS, its address shall be distributed to all of the SM-L and SM-x manager processes in the system. The SPA Lookup Service address shall be stored statically in their routing tables, along with the CAS, so that all components have access to those processes. 12

19 This completes topology discovery. Each component can now negotiate with the SPA Lookup Service to perform registration and request subscriptions Routing table state following topology discovery This standard does not require a particular implementation for the routing tables present in the CAS or the SM-x. However, for a component to be considered compatible, it shall store certain information as addressed in the following sections CAS Tables Table 6 shows an example of a CAS routing table as defined after topology discovery. The CAS shall record the object type, its UUID, its assigned address block, and the logical address of the SM-x requesting the address. Recording the requesting SM-x logical address allows another SM-x to locate a route to the component via the SPA-L. The logical addresses can be viewed as a single 4-byte address or two 16-bit fields as shown in the Logical Address column. The UUID ensures that if a SM- comes back on line and requests a logical address block from the CAS, it shall receive the same address block it previously received. The CAS shall not assign duplicate or overlapping address blocks. Table 6 CAS routing table example Logical Address of Requesting SM-x Type UUID Assigned Address Block 1 (0,1) CAS N/A (1,0) SM-L (2,0) SM (3,0) SM (4,0) SM (5,0) SM SM-x Tables The SM-x routing table shall be subnet specific, and is discussed in detail in the specific subnet standard (e.g. SPA-S SpaceWire Subnet Standard, AIAA S ). However, a generic SM-x routing table might look like the example provided in Table 7. 13

20 Table 7 SM-x routing table example Logical Address Component Type UUID Address Type Physical Address (subnet route) 1 (0,1) CAS L Port # (1, 1) SPA Lookup Service 7354 L Port # (1,0) SM-L 6243 L Port # (2,0) SM L Port # (3,0) SM SpW SpW Path (3,1) COMP SpW SpW Path (3,2) COMP SpW SpW Path (3,3) COMP SpW SpW Path A SM-x routing table shall contain the type of item, the UUID, and the logical address, just like the CAS table. However, it shall also contain the additional fields pertaining to the associated physical address. For subnet addresses, this shall include the subnet address needed to construct the packet required to reach the component Routing Table Entry Deletion Although an implementer will likely ensure that there is enough space in all routing tables during integration to accommodate all known hardware and software SPA components in the worst case, the conditions of the network may change leading to more connections than can possibly be entered into the table. A routing table shall not delete a connection to the CAS, SPA Lookup Service, or subnet manager, unless those services have been replaced by redundant processes. Beyond that, this standard makes no requirement on the algorithm used to select an entry for deletion Dynamic Topology Reconfiguration After discovery, registration, and subscription processing has been completed, the network settles down into a static period of operation during which data sources deliver data to their subscribers and the routes and topology of the network do not change. It might, however, be necessary to repeat the topology discovery in part or full after the system is running. This could occur for several reasons: A component or entire subnet might be powered on after the rest of the system is running A SPA software process component might terminate or become active A switch to a redundant SPA Lookup Service or CAS might cause topology information to be lost. In the last case, a full system reset would probably be needed with a full topology discovery process as previously described. However, in the event that a single component or subnet were added or removed from the system, affected routes shall be adjusted while the remainder of the system remains in operation. 14

21 Each SM-x shall be responsible for monitoring their own subnet for added or removed components. This means, for example, that a SM-s must detect SpaceWire components changing, and a SM-L must detect processes (including full SM-x present on the interconnect). As new components are detected, a logical address shall be allocated from its assigned block. If there are no more addresses available, the SM-x shall request a new block from the SM-x which assigned it a block originally Routing During Component Registration Following topology discovery each component shall register its Extensible Transducer Electronic Data Sheet (xteds) with the SPA Lookup Service to allow the SPA Lookup Service to process subscription requests. The full details are discussed in the SPA Logical Interface Standard (AIAA S ), but it is instructive to show an example of how registration requests are routed (see Figure 5). CAS DM SPA subnet SPA-L SM-x Protocol Specific Fabric Figure 5 SPA lookup service to SPA component intercommunication To move the request to the component from the SPA Lookup Service and return the xteds information to the SPA Lookup Service, data must move from the SPA Lookup Service on the SPA-L network, through the subnet via the correct SM-x, and finally reach the correct endpoint. The sequence is shown in Figure 6 below: 15

22 1. SM-x receives the SPA Lookup Service address during topology discovery SPA Lookup Service SM-x SPA Component 3. The SPA Lookup Service stores the requesting address until ready to register 2. SM-x uses the SPA Lookup Service address to inform it that components are available. 4. When it is ready, the SPA Lookup Service uses address to query component 5. SM-x passes request to component 8. The SPA Lookup Service stores xteds info, Moves to next component 7. SM-x uses DM address to forward xteds over SPA-L 6. Component returns xteds to SM-x Figure 6 Component registration data flow sequence 1. First, during topology discovery, the SPA Lookup Service is located in the network and its location is distributed to each SM-x. 2. For each component on its subnet, the SM-x informs the SPA Lookup Service that the component exists, provides its address, and tells the SPA Lookup Service it can be queried for its xteds. This behavior is also performed by SM-L s for SPA processes. 3. The SPA Lookup Service stores this information until it is ready to register that item. 4. The SPA Lookup Service selects a component to query for registration. The packet has the endpoint s logical address as the destination address, and the SPA Lookup Service s logical address as the source address. The SPA Lookup Service encapsulates the packet in the appropriate SPA-L protocol header, and the packet is delivered to the recorded port on the SPA-L for the SM-x. 5. The SM-x uses its routing table to encapsulate the packet in the correct protocol specific packet to reach the endpoint. The SPA frame as generated by the SPA Lookup Service is encapsulated in the subnet packet without modification, but the SPA packet destination address has been used to look up the correct protocol packet header information. The packet is sent to the component. 6. The SPA component receives the packet, and prepares its xteds information to return to the SPA Lookup Service. To reach the SPA Lookup Service, it reverses the SPA address fields in the request packet so that the SPA Lookup Service logical address is the destination address, 16

23 and its own logical address is the source. This SPA packet is encapsulated in a protocol specific frame allowing it to route to the SM-x. This route is known to the component either from a configuration loaded at power-on, or configured as the SM-x is performing topology discovery on its subnet. 7. The SM-x removes the packet encapsulation and uses the SPA destination address as a lookup into its routing table. From this it finds the correct port number for the SPA Lookup Service on the SPA-L interconnect and transmits the packet. 8. The SPA Lookup Service stores the xteds information and moves to the next component on the list Requesting an Address From the CAS for an Item not in the SM-x Table After Initial Discovery Because the tables in the SM-x can be created in non-linear manner during the discovery process, it may occur that one of its subnet components attempts to address a component in a different subnet which is not in the SM-x routing table. This can occur because the SM-x has not previously routed to the component, or because the route was removed from the table. In this case the SM-x requests the route to the other SM-x (the logical address over SPA-L) for the endpoint it is attempting to route to. The CAS responds with the address to route the packet, or a response that the address is not known (see Figure 7). CAS SM-x Subnet 1. Any SPA Message, Dest (A) 1. SPARqstCasRoute (A) 2. SPADistributeRoute (A,SM-x) or 2. SPAReplyCasRouteUnknown(A) Figure 7 Address request sequencing diagram 1. The SM-x shall recognize that it does not know the route to component in another subnet and request the route information from the CAS 2. The CAS shall respond with the correct SM-x to access over the SPA-L, or an indication that no route exists for that endpoint. 3. Behavior when a route is not known is beyond the scope of this standard Component-to-Component Routing After components have been registered, the SPA Lookup Service shall respond to subscription requests by returning the address of an acceptable component to fulfill a service. At this point the subscriber shall 17

24 negotiate with the source component directly to setup the subscription lease, after which the source periodically sends the subscriber its data. Within a SPA network, there are three possible routing cases, as described in the following paragraphs Routes Within a Single SPA Subnet Component to component communication between two components on the same subnet shall be governed by the relevant SPA Subnet Standard. In general, the implementation may either allow direct communication, implying that the components have a copy of the routing tables for the local subnet, or traffic is routed to the SM-x, which looks up the address and sends the packet back out onto the subnet with a modified protocol header to reach the destination Routes Between Components on Different Subnets Using the SPA-L Interconnect If a source and subscriber are on two subnets, the packet shall be routed to the local SM-x for routing (see Figure 8). To route the packet, the SM-x needs to know the proper SM-x to access across the SPA- L to reach the next hop on the route. The logical address of all SM-x for the local SPA-L reside in its routing table, however the association of the target endpoint logical address to the correct SM-x may not be present. For a small system all SM-x might know the addresses for all components, but for a large system the space for this might be prohibitive. This requires that the SM-x check its table, and if there is a miss on the SPA destination address, it shall request the CAS to deliver its full routing table entry for that component. This shall include the correct SM-x. The packet will then be transmitted to the target SM-x via the SPA-L. The procedure for the target SM-x to deliver the packet to the target component shall be the same as discussed above for registration. It is possible that the target SM-x may need to perform a similar lookup to return the response packet. SPA subnet SM-x Protocol Specific Fabric SPA subnet SPA-L SM-x CAS DM Protocol Specific Fabric Figure 8 Subnet to subnet communication via SPA-L 18

25 Routes Between Components on Different Subnets Without Using the SPA-L Interconnect Normally a SPA-L subnet would be the connection between two SM-x. However, bridge-like SM- combinations might be used for some implementations that do not want to support the SPA-L protocol or do not want data to move through that domain. In this case the SM-x shall support both subnets directly, performing topology discovery on each side and translating packets moving from one domain to the other (see Figure 9). SPA subnet SPA subnet SM-x Protocol Specific Fabric Protocol Specific Fabric Figure 9 Subnet to subnet communication without SPA-L 5 Requirement to Support Packet Fragmentation A component at the boundary of a SPA subnet may be required to perform segmentation to meet the requirements of a smaller maximum transmission unit (MTU). This is considered a possible role for the SM-x entry point to that subnet, and any fragmentation and reassembly would be discussed in the appropriate SPA Subnet Adaptation Standard. 6 Specific Topology Considerations A SPA network is fairly flexible in terms of the different topologies which may be implemented. However, certain topologies for a particular subnet or collection of subnets have implications on topology discovery and routing, which are addressed in the following sections. 6.1 Multiple SM-x on the Same Subnet This is actually a very common configuration, because multiple managers are usually needed to bridge different processor nodes (see Figure 10). As shown below the effects are very small: SM-x A shall be informed of the CAS location and shall have an address block requested for it by the SM-L on its attached network. SM-x A shall do its own discovery and find SM-x B. SM-x A shall assign addresses to all components on its subnet, including SM-x B (only if SM-x B is the same type of SM-x A). SM-x A shall also inform SM-x B of the location of any other SPA core components it was aware of. SM-x B should attempt to locate the SM-L on its off-side network and check if it has been assigned an address block; if not, it shall request one in its behalf. 19

26 SM-x B shall perform its own discovery of the network and record the routes to subnet components from its own perspective. It shall also inform SM-x A of any core components on its offside network, as well as the address it has assigned them from its address block. SPA SPA SPA CAS SM-x A R R R SM-x B SPA-L SPA-L Figure 10 A subnet with two SM-x 6.2 Multiple Available Paths Within a Subnet There may be multiple paths to a component within a subnet, for purposes of redundancy or some other consideration (see Figure 11). The additional paths shall be invisible to managers outside of the subnet (the SM-x would perform translation on incoming packets to the local subnet protocol, choosing the physical route). Any special considerations for building the local route table are addressed in the relevant SPA Subnet Adaptation Standard. SPA SPA SPA CAS SPA-L SM-x A R R R Figure 11 A subnet with two routes to every component 20

27 7 SPA Network Requirements This section lists the formal requirements for a SPA network component. 7.1 SPA Local Bus Requirements a. For each processor node, a SPA network shall implement a SPA-Local interconnect for inter process communication. b. Each SPA-L network shall contain a SM-L. c. One SPA-L shall contain the active CAS process. d. One SPA-L network shall contain the SPA Lookup Service process. e. All SM-x on the same SPA-L interconnect shall be able to communicate with each other on the SPA-L interconnect without moving traffic through the SM-L. Data movement through the SM-L is permitted but may not be a requirement of the implementation. 7.2 Central Addressing Service (CAS) Requirements General Requirements a. A SPA network shall implement a CAS to distribute unique fixed size logical address blocks for SPA components as required. b. The CAS process shall be instantiated on a SPA-L interconnect Redundancy a. Only a single CAS shall be operational within a SPA network at one time. b. One or more redundant CAS may optionally be in the system, but shall not respond to any request for address blocks while the primary CAS is operational Logical address Assignment a. The logical address block provided by the CAS for any SPA Manager shall be unique within the SPA network and be of size 65,536 (2 16 ) bytes. The first block, , shall be reserved for system use. b. If a request for an address block arrives from a manager with a UUID which was previously assigned a logical address, and the message indicates a first request, the CAS shall assign the same ID number as previously. This can occur if a manager is powered off and subsequently returned to service Component Information Stored and Delivered a. The CAS shall store the UUID, assigned logical address block(s), and requesting SM-x of all SPA managers which have been registered with it. b. The CAS shall deliver the logical address, UUID, and entry SM-x of all currently known core components and managers to a SPA-Manager (SM-x or SM-L) which requests it. For a component address request, it must return the manager in charge of the block of addresses including the component. 21

28 7.3 SPA Lookup Service Requirements General Requirements a. A SPA network shall implement a single SPA Lookup Service process. b. One or more redundant SPA Lookup Services may optionally be in the system but shall not respond to any requests for services while the primary SPA Lookup Service is operational Component Discovery a. The Data Manager shall store a component s request for component registration, until it is ready to register that component. b. The SPA Lookup Service shall store the xteds registered from each component, as well as the assigned logical address for each item. c. The SPA Lookup Service shall store the correct route (SM-x address) for each item, but is not required to do so. If it does not, it will forward the message to its local SM-x to select a route. d. The SPA Lookup Service may optionally load the route and xteds information from a storage device rather than performing component discovery, but is not required to do so. e. The SPA Lookup Service shall be capable of identifying and accepting new component information after initial component discovery has completed Subscription a. The SPA Lookup Service shall accept requests for subscription from endpoints after component registration has completed. b. The SPA Lookup Service shall respond to a request for services with a message indicating the address of an acceptable service. c. The SPA Lookup Service shall respond to a request for services with a message indicating that no service is available if an appropriate service is not preset in its storage. 7.4 SPA subnet Manager (SM-x) Requirements The following requirements apply to all SM-x managers in a SPA system. Additional specific requirements may be found in the relevant SPA Subnet Adaptation Standard (SAIAA S-133-x-201) Topology Discovery a. A SM-x shall be capable of communicating with all of the components on its subnet to query them for their UUID, assigned logical address, and type, according to the topology discovery procedure outlined in the subnet specification. b. A SM-x shall assign a logical address for any component on its subnet from the block granted by the CAS if that component has not already been assigned a valid logical address. c. A SM-x shall be able to perform topology discovery on its local subnet if requested during operation (dynamic reconfiguration). d. A SM-x shall be capable of storing received route information from the CAS for SPA components existing in other subnets. e. The SM-x shall be capable of dynamically accepting changed routes and adjusting its local tables. 22

29 f. The SM-x shall be capable of detecting a failed or recently added component, and assigning an address, or removing the associated service from the SPA Lookup Service. g. A SM-x shall remember the logical address assignment it has made for each component. If a newly detected component has the same UUID, it shall be assigned the same logical address as previously. This can occur if a component shuts down and subsequently restarts after topology discovery has occurred. h. If hierarchical subnets are implemented as part of the topology, then each SM-x shall be able to access the CAS only via a single path; no looping through multiple subnet paths is allowed Network Contracts a. A SM-x may optionally load a stored configuration rather than performing discovery. b. A SM-x may optionally have the ability to store its current configuration upon request, so that it can be reloaded later Routing a. A SM-x shall be able to use an incoming SPA packet s destination address to identify a destination target or target(s) on its subnet. b. A SM-x shall be capable of translating an incoming packet in a manner appropriate to the subnet s protocol, so that the packet will correctly route on the subnet to the selected target or set of targets. c. A SM-x shall be capable of fragmenting a SPA packet entering its subnet if required by the protocol, and reassembling fragmented SPA packets at its output. d. A SM-x may identify itself as the intended target, and delete the incoming packet after processing. e. A SM-x may use the SPA Quality of Service (QoS) field to select the order in which incoming or outgoing packets are processed, but is not required to do so. If the QoS field is used, the priority assigned shall be in accordance with the method described in the SPA Logical Interface Standard (AIAA S ). f. A SM-x may drop incoming or outgoing packets due to congestion or error without notifying the source or the intended target. g. A SM-x shall be capable of translating a packet on its subnet such that it can be properly routed to reach a destination outside of its subnet. The means by which this occurs (encapsulation, strip the local header and add a new one, address translation, etc.) is implementation specific. h. A SM-x shall recognize a SPA message address of 0x as invalid, and process the message without using it. This condition may occur during topology discovery when addresses have not been assigned. i. If a SM-x must construct a SPA message without a valid source or destination address, it shall use the address 0x Generic Router Requirements The following requirements apply to all routers in a SPA system subnet; more specific requirements may be found in the relevant SPA subnet standard. 23

30 a. A router in a SPA network may decode the SPA message encapsulated within the protocol specific header for the subnet in order to differentiate traffic for QoS or security concerns, but is not required to do so. b. A router in a SPA network may drop an incoming packet due to congestion or error without notifying the source or target of the occurrence. c. A router in a SPA network is not required to be assigned a UUID or a logical address if the SPA manager for the subnet can route to endpoints correctly without this being performed. 7.6 Generic SPA Endpoint Requirements a. A SPA endpoint shall be able to communicate with a SPA Subnet Manager on its subnet in accordance with the SPA Logical Interface Standard (AIAA S ), as well as its specific SPA Subnet Adaptation Standard. b. A SPA endpoint shall be capable of being installed at any position within a subnet with a matching protocol. c. If a SPA endpoint construct a SPA message without a valid source or destination address, it shall use the address 0x It shall also recognize the use of this address in a received packet as an invalid address. 7.7 SPA Checksum Generation Any SPA message sent over a physical link shall have a 16 bit checksum generated and appended in the message footer before it is sent over the first physical link (see SPA Logical Interface Standard, AIAA S , Section for details) 24

31 Annex A Compliance Matrix for a SPA Network This Annex provides a compliance matrix (Table A.1) to implement a SPA network and a place to identify the action by which the requirement is verified. The full requirement descriptions can be found in Section 7 of this Standard. Table A.1 Minimum requirements for a compliant SPA network Requirement Number Notes Required Optional Verified by (action) 7.1 SPA Local Bus Requirements 7.1, a SPA-L shall be present on each processor node in a SPA network 7.1, b Each SPA-L shall contain a SM-L 7.1, c SPA-L shall contain the CAS 7.1, d One SPA-L interconnect shall contain the DM process 7.1, e All SM-x on a SPA-L interconnect shall be able to communicate with each other after topology discovery without moving data through the SM-L 7.2 Central Addressing Service (CAS) Requirements 7.2.1, a Each SPA network shall implement a CAS to distribute unique fixed size logical address blocks for components as required , b The CAS shall be implemented on the same node of the SPA-L as SM-L 7.2.2, a Only a single CAS shall be active at once , b A system may have redundant CAS, as long as only a single CAS is active at once , a A CAS assigned logical address block shall be unique within the boundaries of the SPA network and of size bytes , b CAS shall provide the same logical address to components which was applied before, if more than one requests arrive , a CAS shall store UUID, assigned address, type, and requesting SM-x for registered components 7.2.4, b CAS shall return routing information at the request of a manager 25