StorageGRID Webscale 10.0

Transcription

1 StorageGRID Webscale 10.0 Cloud Data Management Interface Implementation Guide NetApp, Inc. 495 East Java Drive Sunnyvale, CA U.S. Telephone: +1 (408) Fax: +1 (408) Support telephone: +1 (888) Web: Feedback: Part number: _A0 September 2014

2

3 Table of Contents 3 Contents Introduction to CDMI implementation... 6 Who should read this guide... 6 Revision history... 6 Supported versions of CDMI and HTTP... 6 How the StorageGRID Webscale system implements CDMI... 7 CDMI specification sections supported by the StorageGRID Webscale system... 7 How client applications store data objects through a CLB service How client applications retrieve data objects (CLB service) How client applications retrieve data objects through an LDR service How the StorageGRID Webscale system resolves conflicts How the system's ILM rules and metadata manage data objects How the StorageGRID Webscale system implements immediate redundancy CDMI namespace permissions you can specify Client application access permissions and HTTP methods What the Read access type is Use of GET method to read data objects and data object metadata Use of GET method to retrieve data and container object metadata Predefined metadata Modify or write access How applications use the HTTP PUT and POST methods to create and store objects How applications use HTTP PUT to update metadata Delete access Query access Last access time metadata Connecting client applications using CDMI Configuring the StorageGRID Webscale system to accept client connections Associating client application IP addresses with link-cost groups Creating HTTP profiles to set namespace permissions Associating HTTP profiles with client applications How client application authentication works Finding IP addresses for grid nodes... 24

4 4 StorageGRID Webscale CDMI Implementation Guide Finding CDMI port numbers for the LDR and CLB services How the StorageGRID Webscale system implements security in CDMI Configuring a custom server certificate Reverting a custom server certificate to the default certificate Copying the StorageGRID Webscale system's CA certificate How client applications use certificates for security in CDMI Supported hashing and encryption algorithms for TLS libraries Choosing hash algorithms for data objects How security partitions are used Testing client application connections to the system CDMI root URI Testing HTTP connections by using Telnet Testing HTTP connections by using OpenSSL Retrieving CDMI capabilities with curl Testing nameless data object storage and retrieval by using curl Testing named data object storage and retrieval by using curl Using client applications and the StorageGRID Webscale system Managing HTTP connections Viewing HTTP transactions for CDMI objects Accessing audit logs Viewing information about data objects Time synchronization between client applications and the system How CDMI retrieves data objects stored in client applications How the StorageGRID Webscale system uses UUIDs How the StorageGRID Webscale system uses UUIDs and CDMI object IDs Ruby code examples for deriving a CDMI data object ID from a UUID Benefits of active, idle, and concurrent HTTP sessions Benefits of different types of HTTP sessions Benefits of keeping idle HTTP sessions open Benefits of active HTTP sessions Benefits of concurrent HTTP sessions Separation of HTTP session pools for read and write operations How client applications affect the HTTP transaction load Copyright information Trademark information... 52

5 Table of Contents 5 How to send your comments Index... 54

6 6 StorageGRID Webscale CDMI Implementation Guide Introduction to CDMI implementation The StorageGRID Webscale system supports the storage and retrieval of objects by applications that interface to the StorageGRID Webscale system through the Cloud Data Management Interface (CDMI). For basic information about the StorageGRID Webscale system, see the StorageGRID Webscale 10.0 Grid Primer and the StorageGRID Webscale 10.0 Administrator Guide. Who should read this guide Your should use the Cloud Data Management Interface Implementation Guide if you are creating applications that interface with the StorageGRID Webscale system through CDMI. You can also use the guide to gain a basic understanding of how the StorageGRID Webscale system supports CDMI. Revision history The following table lists changes to the StorageGRID Webscale system's support for CDMI along with the accompanying changes to the documentation. You might find it helpful to identify those changes in preparing for CDMI implementation. Date Release Comments September Added support for named objects and multipart MIME. Supported versions of CDMI and HTTP The StorageGRID Webscale system supports specific versions of CDMI and HTTP. When you are developing the interface with your client application, it is helpful to know the supported versions. The following versions are supported: Item Version CDMI specification Published by the Storage Networking Industry Association (SNIA) HTTP 1.1 For more information about HTTP, see HTTP/1.1 (RFC 2616).

7 7 How the StorageGRID Webscale system implements CDMI A client application can connect to the CLB service or LDR service, create object containers, and store and retrieve data objects. The StorageGRID Webscale system uses information lifecycle management (ILM) rules to manage these data objects ingested into the StorageGRID Webscale system by a client application that interfaces to the system through CDMI. When you are designing interfaces to the system, It might be helpful to know how the system processes your data objects. For more information about ILM rules, see the Administrator Guide. CDMI specification sections supported by the StorageGRID Webscale system The StorageGRID Webscale system supports the Cloud Data Management Interface (CDMI) specification published by the Storage Networking Industry Association (SNIA). You might find it helpful to understand how the system implements various sections of the CDMI specification. Data object resource operations The StorageGRID Webscale system supports a single CDMI domain. Data objects ingested through the StorageGRID Application Programming Interface (SGAPI) can be accessed through CDMI. Likewise, data objects ingested through CDMI can be accessed through SGAPI. To access through CDMI a data object that was ingested through SGAPI, you must convert the data object's UUID (returned in an SGAPI ingest response) to a CDMI Object ID. The following table lists the supported Data Resource Operations sections of the Cloud Data Management Interface (CDMI) specification. Section number Section description 8.4 Read a data object (CDMI Content Type) 8.5 Read a data object (Non-CDMI Content Type) 8.6 Update a data object (CDMI Content Type) 8.7 Update a data object (Non-CDMI Content Type) 8.8 Delete a data object (CDMI Content Type) 8.9 Delete a data object (Non-CDMI Content Type)

8 8 StorageGRID Webscale CDMI Implementation Guide Byte range read operations The StorageGRID Webscale system supports byte range read operations using both CDMI and non- CDMI content types. For byte range read using non-cdmi content type, the following byte range is returned by the StorageGRID Webscale system: If a single contiguous byte range is requested, the system returns the byte range. If multiple byte ranges are requested that can be coalesced without holes, the system returns a single coalesced range. If multiple byte ranges are requested that cannot be coalesced without holes, the system returns the entire data object bytes. The length of time the system takes to return requested portions of a data object is impacted by the following: If you enable compression (by using the Grid Management > Grid Configuration > Configuration > Stored Object Compression option) or if the data object is retrieved from tape, the StorageGRID Webscale system locates and returns the requested portion of the data object by reading the data object, starting at the beginning of the segment containing the first byte of the requested range. If you disable compression and the data object is retrieved from disk, the StorageGRID Webscale system is able to begin reading the segment from the start of the requested byte range and not the beginning of the segment. Thus, if compression is enabled or the data object is retrieved from tape, it takes the system longer to return the requested portion of a data object. Container object resource operations The StorageGRID Webscale system supports container objects, named data objects, and the Container Object Resource Operations section of the CDMI specification. For named data objects, your StorageGRID Webscale system always includes a public administrative container object. For each StorageGRID Webscale system, there is only one public container object, which cannot be deleted. Client applications can create more than one container object under the public container object (public/<container> ). All named data objects must be created in containers under public. To delete a container object, it must be empty. You must use the HTTP POST method to store nameless objects in the StorageGRID Webscale domain. Note: Container object metadata is not supported by the StorageGRID Webscale system. The following table lists the supported "Container Object Resource Operations" sections of the Cloud Data Management Interface (CDMI) specification.

9 How the StorageGRID Webscale system implements CDMI 9 Section number Section description 9.2 Create a container object (CDMI content type 9.3 Create a container object (non-cdmi content type) 9.4 Read a container object (CDMI content type) 9.6 Delete a container object (CDMI content type) 9.7 Delete a container object (non-cdmi content type) 9.8 Create (POST) a new data object (CDMI content type) 9.9 Create (POST) a new data object (non-cdmi content type) Domain object resource operations The StorageGRID Webscale system supports a single CDMI domain. The creation of additional domains is not supported. Capability object resource operations The following table lists the supported "Capability object Resource Operations" sections of the Cloud Data Management Interface (CDMI) specification. Section Section description Notes 12.2 Read a capabilities object (CDMI Content Type) The StorageGRID Webscale system supports the following: System-wide capabilities Storage system metadata capabilities Data system metadata capabilities Data object capabilities Metadata The following table lists the supported "Metadata" sections of the Cloud Data Management Interface (CDMI) specification.

10 10 StorageGRID Webscale CDMI Implementation Guide Section Section description Notes 16.3 Storage system metadata The StorageGRID Webscale system supports the following: cdmi_size cdmi_ctime cdmi_atime cdmi_hash 16.4 Data system metadata The StorageGRID Webscale system supports the following: cdmi_data_redundancy cdmi_immediate_redundancy cdmi_value_hash 16.5 Provided data system metadata The StorageGRID Webscale system supports the following: cdmi_value_hash_provided The StorageGRID Webscale system converts the following StorageGRID Webscale metadata to populate the values of some CDMI storage system metadata. The following table identifies which StorageGRID Webscale metadata is used to populate CDMI system metadata. For more information, see the StorageGRID Administrator Guide. CDMI storage system metadata cdmi_size cdmi_ctime cdmi_atime cdmi_hash StorageGRID Webscale metadata CSIZ CTME LATM The StorageGRID Webscale system uses the value from cdmi_ctime when a data object lacks LATM (last access time) metadata. The StorageGRID Webscale system returns the hash for the data object.

11 How the StorageGRID Webscale system implements CDMI 11 CDMI storage system metadata cdmi_value_hash_provided StorageGRID Webscale metadata The StorageGRID Webscale system returns the name of the hash algorithm selected in the NMS Management Interface (NMS MI) when the system stored the data object. Data objects stored in the StorageGRID Webscale system by SGAPI client applications can include non-cdmi metadata. For example, when SGAPI client applications store objects in the system, these client applications can use predefined metadata, which is a type of metadata for SGAPI client applications and the StorageGRID Webscale system. When you use CDMI client applications to retrieve these data objects, the response includes the predefined metadata. Extensions The StorageGRID Webscale system supports the CDMI Multi-part MIME Extension 1.0g for the creation and retrieval of both named and nameless data objects, otherwise subject to the same limitations for data object operations within the StorageGRID Webscale system's standard CDMI implementation. This extension allows clients to read and write the CDMI value as binary in a multipart body part, which avoids incurring encoding (for example, base64) overhead on both the client and the server. Related concepts How CDMI retrieves data objects stored in client applications on page 43 Related tasks Retrieving CDMI capabilities with curl on page 34 Related information SNIA: SNIA Cloud Data Management Interface (CDMI) Version 1.0.2

12 12 StorageGRID Webscale CDMI Implementation Guide How client applications store data objects through a CLB service Client applications can use HTTP to connect directly with a CLB service and store data objects. The CLB service identifies the optimal LDR service to satisfy client requests and forwards requests to the LDR service. 1. The client application opens an HTTPS connection to the configured HTTP port for a CLB service. The CLB service acts as a proxy for the LDR services. 2. The client application issues an HTTP PUT or HTTP POST request that includes the data object and any metadata. 3. The CLB service identifies the optimal LDR service to satisfy client requests and forwards requests to the LDR service. Note: The CLB service uses ranking criteria to identify which LDR service to use. As a result, the client application does not have to identify which LDR service to use. 4. After the LDR service stores a copy of the data object, the LDR service returns a valid CDMI response to the client application through the CLB service.

13 How the StorageGRID Webscale system implements CDMI 13 How client applications retrieve data objects (CLB service) Client applications can use HTTP to connect directly with a CLB service and retrieve data objects. The CLB service identifies the optimal LDR service to satisfy client requests and forwards these requests to the LDR service. 1. The client application opens an HTTPS connection to the configured HTTP port for a CLB service. The CLB service acts as a proxy for the LDR services. 2. The client application issues an HTTP GET request that includes the object ID or name for the data object that it wants to retrieve. 3. The CLB service identifies the optimal LDR service to satisfy the request and forwards the request to the LDR service. Note: The CLB service uses ranking criteria to identify which LDR service to use. As a result, the client application does not have to identify which LDR service to use. 4. The LDR service returns the data object and any requested metadata.

14 14 StorageGRID Webscale CDMI Implementation Guide How client applications retrieve data objects through an LDR service Client applications can use HTTP to connect directly with an LDR service and retrieve data objects. A connection with one or more LDR services enables high-performance parallel transfers and eliminates the single point of failure that is associated with connecting to a CLB service. 1. The client application opens an HTTPS connection to the configured HTTP port for an LDR service. 2. The client application issues an HTTP GET request that includes the object ID or name. 3. The LDR service returns the data object and any requested metadata. A client application can connect directly with multiple LDR services. How the StorageGRID Webscale system resolves conflicts If the StorageGRID Webscale system detects that two or more objects with same name exist in the same container, the system allows these objects to remain, keeping the objects unique through the use of each object's UUID. However, conflicts might arise when a client application attempts to access an object by name and the action is not performed on the expected object. The StorageGRID Webscale system resolves this conflict by always performing actions on the most recently created object. How the system's ILM rules and metadata manage data objects The StorageGRID Webscale system's information lifecycle management (ILM) rules enable you to use metadata in rules to manage data objects automatically. You can use the following metadata: You can use object size, last access time, and CDMI user-defined metadata in ILM rules for data objects stored in the StorageGRID Webscale system by client applications interfacing to the system through CDMI. You can use object size to ensure that small objects are stored to disk and not tape, which avoids the poor retrieval performance of tape. You can use last access time metadata to identify content that has not been retrieved for a set amount of time and have this content moved to a cheaper grade of storage.

15 How the StorageGRID Webscale system implements CDMI 15 You can set the filter criteria to evaluate data objects against CDMI user defined metadata. The CDMI protocol handler version and last access time are StorageGRID Webscale system metadata. For more information about ILM rules, see the StorageGRID Webscale Administration Guide. How the StorageGRID Webscale system implements immediate redundancy The StorageGRID Webscale system supports the CDMI Data System Metadata Capabilities functionality cdmi_data_redundancy and cdmi_immediate_redundancy to save up to two copies of a data object (dual commit) to two Storage Nodes at object creation. This functionality provides protection against data loss should a Storage Node fail. A client application specifies redundancy with the data system metadata cdmi_data_redundancy and cdmi_immediate_redundancy in a CDMI POST request when creating a data object. If this metadata is not included in a CDMI content type request, the following defaults are assumed: "cdmi_data_redundancy" : "2" "cdmi_immediate_redundancy" : true These defaults also apply to non-cdmi content type POST data object create requests. For CDMI content type requests, you can display redundancy by setting cdmi_data_redundancy to false. Internally, the StorageGRID Webscale system achieves redundancy by using dual commit, which creates two copies of a data object. Thus, a cdmi_data_redundancy request with a value greater than 2 creates two copies and not the requested value. Note: Dual commit creates two copies of a data object before ILM rules are evaluated, which might create additional copies. The following table summarizes the responses for successful redundancy requests: Client application request cdmi_data_red undancy cdmi_immediat e_redundancy Does the system use dual commit? CDMI response for success cdmi_data_redunda ncy_provided not present not present Yes 2 true 0 true No 1 true 1 true No 1 true 2 true Yes 2 true cdmi_immediat e_redundancy_ provided

16 16 StorageGRID Webscale CDMI Implementation Guide Client application request cdmi_data_red undancy cdmi_immediat e_redundancy Does the system use dual commit? CDMI response for success cdmi_data_redunda ncy_provided cdmi_immediat e_redundancy_ provided not present false No 1 true because one data object was stored a number greater than 2 true Yes 2 true The following table summarizes the responses for failed redundancy requests: Client application request cdmi_data_red undancy cdmi_immediat e_redundancy Does the system use dual commit? CDMI response for failure cdmi_data_redunda ncy_provided not present not present No 1 false 0 true No 1 true 1 true No 1 true 2 true No 1 false not present false No 1 true a number greater than 2 true No 1 false Note: The system stores a maximum of two copies. cdmi_immediat e_redundancy_ provided After the StorageGRID Webscale system creates a copy of the data object, it evaluates the ILM rules for the copy and the data object. The rules in the ILM policy determine the actions the system takes with the copy and the data object. For example, the ILM rules might instruct the system to make additional copies of the data object in different locations and delete the original copy. For more information about dual commit and ILM rules, see the Administration Guide.

17 17 CDMI namespace permissions you can specify You can specify permissions (such as read, modify, delete, and query permissions) for client applications in the CDMI namespace in the StorageGRID Webscale system. For each permission, it is helpful to know its implementation and which HTTP methods you should use. Client application access permissions and HTTP methods In the NMS MI, you can specify whether a client application has permission to read, write, modify, or delete data objects in the CDMI namespace. You can also specify whether to enable last access time metadata. For each permission, it is helpful to know which HTTP methods you should use. You can grant or deny the following access to client applications through the StorageGRID Webscale system's NMS MI. The table also indicates the HTTP method used for each permission: Permission type Read Modify/Write Delete Query Last Access Time HTTP method GET PUT for modify access POST for write access DELETE The StorageGRID Webscale system does not support the Query option for CDMI client applications. GET You must enable both Last Access Time and Read. When a CDMI client application uses GET to retrieve a data object, the StorageGRID Webscale system stores the time that the CDMI application client retrieved the data object in internal object metadata called last access time metadata. What the Read access type is The Read access type determines whether a client application has permission to read and retrieve data objects and data object metadata from the CDMI namespace. Note: For data objects stored to the StorageGRID Webscale system by SGAPI client applications, you must derive the CDMI object ID from the StorageGRID Webscale UUID before you can read or retrieve the data object.

18 18 StorageGRID Webscale CDMI Implementation Guide Use of GET method to read data objects and data object metadata Client applications use the HTTP GET method and an object ID or name to read data objects and data object metadata in the CDMI namespace. Use of GET method to retrieve data and container object metadata Client applications use the HTTP GET method and an object ID or name to retrieve data object and container object metadata from the CDMI namespace. The response might include predefined metadata or custom metadata when an SGAPI client application created the object in the StorageGRID Webscale system. For GET, when the requested field childrenrange is specified and the requested field children is not included, the complete range of objects within the container is returned. For example, GET / CDMI/foo/?childrenrange returns "children range" : " ". If no requested fields are specified or the requested field children is specified without a range or a range that includes more than 10, 000 objects, GET is limited by a maximum range of 10,000 objects (0 through 9999). If a container includes more than 10,000 objects, multiple GET operations must be run. Predefined metadata Only SGAPI client applications can use predefined metadata with the StorageGRID Webscale system. Unless otherwise noted, predefined metadata becomes read only after the object is ingested into the StorageGRID Webscale system, and you cannot delete the metadata. Predefined metadata use the X- BYC-XXXX format, in which XXXX is one of the following values: Metadata Set by Status Description XVER Client application Read-only Indicates the version of the metadata Defined by the client application MCLS Client application Read-only Indicates the TSM management class Defined by the client application STR0-STR9 Client application Read-write Can be deleted Identifies a string value Value defined by the client application NUM0- NUM9 Client application Read-write Can be deleted Identifies a numerical value Value defined by the client application

19 CDMI namespace permissions you can specify 19 Modify or write access The Modify/Write access type determines whether a client application has permission to store data objects and update data object metadata in the namespace. How applications use the HTTP PUT and POST methods to create and store objects Client applications can create container objects, and store (create) both named and unnamed data objects in the StorageGRID Webscale system. Use the HTTP PUT and POST methods to create and store objects. Client applications use the following methods in the CDMI namespace: HTTP POST method to store unnamed data objects HTTP PUT method to create container objects and store named data objects By default, the StorageGRID Webscale system enables immediate redundancy for all POST requests for the CDMI content type and the non-cdmi content type. You can disable immediate redundancy for the CDMI content type, but not the non-cdmi content type. However, you should include cdmi_immediate_redundancy metadata set to true in all PUT and POST requests to enable immediate redundancy and protect against data loss. The StorageGRID Webscale system supports user metadata for CDMI data objects; however, the system does not support user metadata for CDMI container objects. Related concepts How the StorageGRID Webscale system implements immediate redundancy on page 15 How applications use HTTP PUT to update metadata Client applications use the HTTP PUT method to update data object user metadata in the CDMI namespace. The following considerations apply with the PUT method: The StorageGRID Webscale system supports adding or updating only all user metadata for a data object. It does not support adding or updating an individual user metadata item (using the URI syntax?metadata:<metadataname>). Updating other fields (for example, value or mimetype) using the PUT method is not supported.

20 20 StorageGRID Webscale CDMI Implementation Guide Delete access The Delete access type determines whether a client application has permission to delete data objects and container objects from the namespace. Query access The Query access type determines whether a client application has permission to perform queries in the namespace. Note: The StorageGRID Webscale system does not support the Query option for client applications. Last access time metadata The last access time permission determines whether the StorageGRID Webscale system updates last access time metadata for a data object when a client application retrieves the object. You can create information lifecycle management (ILM) rules to take action on data objects based on the last time that a client application retrieved the object. When a client application that is assigned an HTTP profile with last access time enabled uses GET to retrieve a data object, the StorageGRID Webscale system saves the retrieval time in internal object metadata called last access time metadata. Only the StorageGRID Webscale system can use internal metadata. For example, ILM policies can use last access time metadata to identify when an object was last retrieved. Note: Because last access time metadata updates each time that a client application retrieves a data object, it can affect system performance. It is recommended that you disable Last Access Time in the NMS MI when no ILM policies use last access time metadata. For more information about last access time metadata and ILM policies, see the Administration Guide.

21 21 Connecting client applications using CDMI You must configure the StorageGRID Webscale system to accept HTTP connections from client applications. Client applications use HTTP connections to access and communicate with the StorageGRID Webscale system. Note: IPv6 is only supported for client application connections through the CLB service. For more information about support for IPv6, see the Administrator Guide. Connecting client applications to the StorageGRID Webscale system involves the following tasks: Configuring client connections to accept HTTP connections Identifying IP addresses for grid nodes Identifying port numbers for CLB and LDR services Copying the system's certificate authority (CA) certificate for client applications that require server validation Configuring the StorageGRID Webscale system to accept client connections Configuring the StorageGRID Webscale system to accept HTTP connections from client applications requires you to complete several steps. Steps 1. Associating client application IP addresses with link-cost groups on page 22 You can associate a link-cost group with the IP addresses that client applications use to connect with the StorageGRID Webscale system. The link-cost group enables the StorageGRID Webscale system to route client applications to the appropriate servers. You can improve system performance when you associate a link-cost group with client applications. 2. Creating HTTP profiles to set namespace permissions on page 23 You can create HTTP profiles that identify whether read, write, modify, query, or delete permissions are enabled or disabled in a namespace. You can create multiple HTTP profiles. 3. Associating HTTP profiles with client applications on page 23 HTTP profiles identify whether read, write, modify, query, or delete permissions are enabled in a namespace. You can associate HTTP profiles with individual client applications or with groups of client applications, based on IP addresses. The association gives client applications access to the StorageGRID Webscale namespace and identifies the HTTP permissions for the client application in the namespace.

22 22 StorageGRID Webscale CDMI Implementation Guide Associating client application IP addresses with link-cost groups You can associate a link-cost group with the IP addresses that client applications use to connect with the StorageGRID Webscale system. The link-cost group enables the StorageGRID Webscale system to route client applications to the appropriate servers. You can improve system performance when you associate a link-cost group with client applications. About this task Servers for the StorageGRID Webscale system are organized into link-cost groups. Link-cost groups identify the cost of operating the group of servers. The system uses the IP address and the link-cost group to route clients to the LDR service or CLB service on the appropriate servers. Steps 1. In the NMS MI, select Grid Management > Grid Configuration > Link Cost Groups > Configuration > Main. 2. In the Client Group IP Ranges table, perform one of the following actions: When... No entries exist One or more entries exist Then... Click Edit. Click Insert. 3. In the IP Range Name box, type a name for the IP address or the range of IP addresses. You can use any name. The configuration does not reference the name elsewhere. 4. In the IP Range box, type the IP address or the range of IP addresses that the client uses to contact the StorageGRID Webscale system. Use a hyphen or slash to indicate an inclusive range of IP addresses, as shown in the following examples: /24 (CIDR format) (dotted decimal format) You can use an abbreviated format for masks in eight bit steps. For example, is equivalent to the CIDR notation /24, and you can extend it as follows: n.n.0.0 is equivalent to n.n.0.0/ In the Group ID list, select an ID. The ID number identifies the group of servers to which the client application with the specified IP address should connect. 6. Click Apply Changes.

23 7. Repeat this procedure for each range of IP addresses that client applications use to access the StorageGRID Webscale system. Creating HTTP profiles to set namespace permissions You can create HTTP profiles that identify whether read, write, modify, query, or delete permissions are enabled or disabled in a namespace. You can create multiple HTTP profiles. Steps Connecting client applications using CDMI In the NMS MI, select Grid Management > HTTP Management > Permissions > Configuration > Main. 2. In the HTTP /CDMI and /UUID Namespace table, perform one of the following actions: When... No entries exist One or more entries exist Then... Click Edit. Click Insert. 3. Check the boxes for the HTTP operations that you want to enable in the profile. The StorageGRID Webscale system does not support the Query operation in the HTTP /CDMI and /UUID Namespace for CDMI client applications. 4. Click Apply Changes. 5. Create additional profiles as needed. Associating HTTP profiles with client applications HTTP profiles identify whether read, write, modify, query, or delete permissions are enabled in a namespace. You can associate HTTP profiles with individual client applications or with groups of client applications, based on IP addresses. The association gives client applications access to the StorageGRID Webscale namespace and identifies the HTTP permissions for the client application in the namespace. Steps 1. In the NMS MI, select Grid Management > HTTP Management > Clients > Configuration > Main. 2. In the HTTP Entities table, perform one of the following actions: When... No HTTP entities exist One or more HTTP entities exist Then... Click Edit. Click Insert.

24 24 StorageGRID Webscale CDMI Implementation Guide 3. In the Description box, enter a description of the client. 4. In the IP Range box, enter the range of IP addresses that the client application can use to connect to the LDR service or the CLB service. 5. In the Profile Name list, select the name of the HTTP profile that you created. 6. Click Apply Changes. How client application authentication works The StorageGRID Webscale system uses its HTTP management settings to authenticate client application requests for access to the StorageGRID Webscale system. Understanding client application authentication helps when in establishing successful connections. When a client application requests access to the StorageGRID Webscale system, the system authenticates the request against the HTTP management settings that you created for the client application and completes the following steps: 1. The StorageGRID Webscale system checks that the client application is using the same IP address or range of IP addresses that are defined in the HTTP management settings. 2. When the client application passes the authentication process, the StorageGRID Webscale system opens a TCP/IP connection. Finding IP addresses for grid nodes You can find the IP address in the Network Management System (NMS) Management Interface (MI) for grid nodes that host the CLB or LDR services. You need the IP address to connect client applications to the CLB or LDR services. About this task You can also find IP addresses for grid nodes by using the SAID package. For details, see the StorageGRID Webscale 10.0 Installation Guide. Steps 1. In the NMS MI, expand Grid Topology. 2. In the Grid Topology tree, locate and expand the grid node to which you want to connect. The services for the selected grid node appear. 3. Expand the SSM service, click Resources, and scroll to the Network Addresses table. You can establish HTTP connections from client applications to any of the listed IP addresses.

25 Connecting client applications using CDMI 25 Related tasks Finding CDMI port numbers for the LDR and CLB services on page 25 Finding CDMI port numbers for the LDR and CLB services You can find port numbers for the LDR service and the CLB service by using the Network Management Service (NMS) Management Interface (MI). You need these port numbers to create an HTTP connection from client applications to the LDR service on storage nodes or to the CLB service on API Gateway Nodes. About this task To maintain system efficiency, you should use the default HTTP ports for their intended purposes. An ingest request sent to a retrieve port might fail if the StorageGRID Webscale system directs the query to an LDR service that is read only. As a StorageGRID Webscale system matures, LDR services fill up and become read only. When the StorageGRID Webscale system directs queries to an ingest port, the CLB service directs queries to resources that support both read and write operations, not resources that support read-only operations. The following ports are used for CDMI: Service Purpose CDMI port number CLB Query and retrieve 8080 Ingest 8081 LDR Query and retrieve Ingest Steps 1. In the NMS MI, select Grid Management > Grid Configuration > Storage > Main. 2. Scroll to the Ports table and locate the port numbers for the LDR and CLB services. How the StorageGRID Webscale system implements security in CDMI The StorageGRID Webscale system employs the use of Transport Layer Security (TLS) connection security, server authentication, client authentication, and client authorization. When considering

26 26 StorageGRID Webscale CDMI Implementation Guide security issues, you might find it helpful to understand how the StorageGRID Webscale system implements security, authentication, and authorization. The StorageGRID Webscale system accepts only HTTP commands submitted over a network connection that uses TLS to provide connection security, application authentication and, optionally, transport encryption. TLS enables the exchange of certificates as entity credentials and allows a negotiation that can use hashing and encryption algorithms. Server authentication uses server certificates signed by the grid's certificate authority (CA) certificate. The administrator might replace the system's certificate with a single, common server certificate applicable to all the API ports within the system. For details about configuring server certificates, see the Administration Guide. The following table lists security issues when using CDMI: Security issue Connection security Server authentication Client authentication Client authorization Client origin permissions CDMI TLS X.509 server certificate signed by grid CA or server certificate supplied by administrator Anonymous or client certificate (security partition) For CDMI, a client certificate (needed for security partitions) is supported only on nameless objects. Client profile permissions and object ownership A grid-wide command to disable the client's ability to delete can override client authorization. IP range Related information StorageGRID Webscale 10.0 Administrator Guide Configuring a custom server certificate When a client application establishes a TLS session to the StorageGRID Webscale system, the target LDR service sends a server certificate to the client application. By default, each LDR service identifies itself by using a separate certificate that is signed by the system Certificate Authority (CA). Rather than use separate server certificates, you can choose to use a single, server certificate supplied

27 Connecting client applications using CDMI 27 by you for all LDR services. This provides flexibility in enabling support for certificate hostname verification. About this task This configuration applies to SGAPI, CDMI, and S3 APIs. Only RSA custom server certificates are supported. The certificate and private key should be entered in PEM format. You can choose to use the wildcard certificate format: for example, *.storagegrid.mycompany.com. In this case, CLB services and LDR services must have DNS entries that map their IP addresses to host names that match the wildcards: for example, dc1- gw1.storagegrid.mycompany.com and dc2-s3.storagegrid.mycompany.com. Client applications are then configured to connect to the system using these DNS names, which enables host name verification. Steps 1. In the NMS MI, select Grid Management > Grid Configuration > Configuration > Main: 2. In the Custom Server Certificate box inside the API Server Certificates section, copy and paste the server certificate, including the

28 28 StorageGRID Webscale CDMI Implementation Guide -----BEGIN CERTIFICATE----- and the -----END CERTIFICATE----- encapsulation boundaries. 3. In the Custom Private Key box, copy and paste the corresponding private key, including the -----BEGIN RSA PRIVATE KEY----- and the -----END RSA PRIVATE KEY Ensure that this is an unencrypted private key. 4. Click Apply Changes. The private key becomes obscured. 5. Click Overview to see the custom certificate on the Overview page. Note: The CA Certificate box on the Overview page displays the default generated server certificate. 6. If a custom server certificate is issued by one or more intermediate CAs, you must also enter the certificates of all intermediate CAs within Grid Management > HTTP Management > Certificates > Certificate Authorities. For details about HTTP management, see the Administrator Guide. After you finish If you configured a custom server certificate, then clients should verify using the root CA certificate that issued the custom server certificate. However, if you use the default certificate, then client applications should verify connections using the system certificate. Reverting a custom server certificate to the default certificate You can change from using your custom server certificate back to using the default, automatically generated certificate. You might want to revert to the default certificate if, for example, the custom certificate has expired. About this task This configuration applies to SGAPI, CDMI, and S3 APIs. Steps 1. In the NMS MI, select Grid Management > Grid Configuration > Configuration > Main. 2. In the API Server Certificates section, delete the text from the Custom Server Certificate box.

29 Connecting client applications using CDMI Delete the text from the Custom Private Key box. 4. Click Apply Changes. 5. Click Overview to see the default certificate on the Overview page. After you finish You must reconfigure your client applications to use the default system CA certificate. Copying the StorageGRID Webscale system's CA certificate You can copy the StorageGRID Webscale system's certificate authority (CA) certificate from the Network Management System (NMS) Management Interface (MI) for client applications that require server verification. If a custom server certificate has been configured, then client applications should verify the server using the root CA certificate that issues the custom server certificate, rather than copy the CA certificate from the NMS MI. Steps 1. In the NMS MI, select Grid Management > Grid Configuration > Overview > Main. 2. Under API Server Certificates, expand CA Certificate. 3. Select the CA certificate. Include the -----BEGIN CERTIFICATE----- and the -----END CERTIFICATE----- in your selection.

30 30 StorageGRID Webscale CDMI Implementation Guide 4. Right-click the selected certificate, and then select Copy. How client applications use certificates for security in CDMI When a client application establishes a TLS session to the StorageGRID Webscale system, the system sends a server certificate to the client application for verification to ensure that the HTTP connection is secure. Understanding certificate use is important for system security. You can verify the server certificate by using the StorageGRID Webscale system's CA certificate. The client application should load the grid CA certificate and use it to verify that the client

31 application is communicating with the expected StorageGRID Webscale system. This process protects against man in the middle and impersonation attacks. Client applications can send client certificates to the StorageGRID Webscale system as part of session establishment. Related tasks Copying the StorageGRID Webscale system's CA certificate on page 29 Supported hashing and encryption algorithms for TLS libraries Client applications use the HTTP protocol to communicate with the StorageGRID Webscale system over a network connection that uses Transport Layer Security (TLS). The StorageGRID Webscale supports a limited set of hashing and encryption algorithms from the TLS libraries that client applications can use when establishing a TLS session. When you are setting up the communication processes, it is important for you to know which security algorithms the system uses. The StorageGRID Webscale system supports the following cipher suite security algorithms: AES128 SHA AES256 SHA NULL SHA NULL MD5 Based on system measurements and general security domain knowledge, AES128 SHA and AES256 SHA provide reasonable security without requiring inordinate amounts of computational resources. The choice between AES128 SHA and AES256 SHA depends on the client application requirements that balance performance with encryption security. Note: You should use one of the NULL ciphers if encryption is not required and you want to eliminate the overhead associated with encryption. The client application must explicitly request the NULL cipher. Choosing hash algorithms for data objects The StorageGRID Webscale system can use either SHA-1 or SHA secure hash algorithms to generate a hash for each data object stored in the StorageGRID Webscale system. You can choose the algorithm that best meets your security needs. About this task Connecting client applications using CDMI 31 The following table maps the choices in the NMS MI to the names of the hash algorithms: Algorithm choice in NMS MI SHA-256 Name of algorithm SHA bits

32 32 StorageGRID Webscale CDMI Implementation Guide Algorithm choice in NMS MI SHA-1 Name of algorithm SHA-1 Because you can change the algorithm, you might have data objects in the system with hashes generated by different algorithms. As a result, metadata for different data objects might include different algorithm names. The algorithm name associated with the data object depends on which algorithm was selected in the NMS MI when the data object was stored in the system. Steps 1. In the NMS MI, go to Grid Management > Grid Configuration > Main. 2. In the Stored Object Hashing option, select the hash algorithm for the StorageGRID Webscale system. 3. Click Apply Changes. How security partitions are used Security partitions are system-wide settings that provide you with a means to restrict access to ingested content. For instance, two client applications ingesting objects to the same system can be denied access each other's objects. If you use security partitions, consider how they are supported and how the StorageGRID Webscale system uses a certificate. Security partitions are supported only for nameless data objects, not for named data objects. If the client application is assigned to a security partition or if the client application's assigned HTTP profile requires certificate authentication, a certificate is required. This certificate must be loaded to the StorageGRID Webscale system as part of the configuration process. For more information and procedures to enable and configure security partitions, see the StorageGRID API Reference.

33 33 Testing client application connections to the system Using either Telnet or OpenSSL, you can test the HTTP connection between the client application and the StorageGRID Webscale system to ensure that the connection works. You can also test that the client application can store objects to and retrieve objects from the StorageGRID Webscale system. If you copy a command from this section and paste the command into another application, the copyand-paste process might remove dashes that appear between words near a line break. Ensure that the pasted command includes all dashes before you run the command. CDMI root URI The root URI for CDMI access to the StorageGRID Webscale system is IP_address:port/CDMI. You need the root URI when testing HTTP connections. For IP_address and port, you must use the IP address and port for the API Gateway Node that hosts the CLB service or the Storage Node that hosts the LDR service. Testing HTTP connections by using Telnet You can use a utility such as Telnet to test the HTTP connection between client applications and the StorageGRID Webscale system to ensure that the HTTP connection is correctly configured. Before you begin You have configured an IP address for the client application in the NMS MI. You know the IP address and port number for the API Gateway Node that hosts the CLB service or the Storage Node that hosts the LDR service. About this task You can connect the client application to the API Gateway Node that hosts the CLB service or the Storage Node that hosts the LDR service. Step 1. From a client application, use Telnet to connect to the CLB service or the LDR service: telnet IP_address port

34 34 StorageGRID Webscale CDMI Implementation Guide For IP_address and port, use the IP address and port for the API Gateway Node that hosts the CLB service or the Storage Node that hosts the LDR service. If you correctly configured the IP address for the client application in the NMS MI, a delay of several seconds occurs, and then the CLB service or the LDR service drops the connection. If you incorrectly configured the IP address for the client application in the NMS MI, the connection closes immediately. If the CLB service or the LDR service is not running or if a network error occurs, Telnet is unable to connect to the CLB service or the LDR service. Testing HTTP connections by using OpenSSL You can use the openssl command to test the HTTP connection between client applications and the StorageGRID Webscale system to ensure that the HTTP connection is correctly configured. Before you begin You have configured an IP address for the client application in the NMS MI. You know the IP address and port number for the API Gateway Node that hosts the CLB service or the Storage Node that hosts the LDR service. About this task You can connect the client application to the API Gateway Node that hosts the CLB service or the Storage Node that hosts the LDR service. Step 1. From a client application, establish an HTTP connection to the CLB service or the LDR service: openssl s_client -tls1 -connect IP_address:port For IP_address and port, you must use the IP address and port for the API Gateway Node that hosts the CLB service or the Storage Node that hosts the LDR service. If you correctly configured the IP address for the client application in the NMS MI, a connected response appears. If you incorrectly configured the IP address for the client application in the NMS MI, an error response appears. Retrieving CDMI capabilities with curl You can retrieve the CDMI capabilities of the StorageGRID Webscale system to determine which CDMI functions the StorageGRID Webscale system supports. Knowing the CDMI capabilities helps