Issue 04 Date 2017-12-20 HUAWEI TECHNOLOGIES CO., LTD.
2017. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd. Trademarks and Permissions and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd. All other trademarks and trade names mentioned in this document are the property of their respective holders. Notice The purchased products, services and features are stipulated by the contract made between Huawei and the customer. All or part of the products, services and features described in this document may not be within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information, and recommendations in this document are provided "AS IS" without warranties, guarantees or representations of any kind, either express or implied. The information in this document is subject to change without notice. Every effort has been made in the preparation of this document to ensure accuracy of the contents, but all statements, information, and recommendations in this document do not constitute a warranty of any kind, express or implied. Huawei Technologies Co., Ltd. Address: Website: Email: Huawei Industrial Base Bantian, Longgang Shenzhen 518129 People's Republic of China http://www.huawei.com support@huawei.com i
Contents Contents 1 OBS...1 2 Advantages...2 3 Basic Concepts...3 3.1 Object...3 3.2 Bucket...3 3.3 AK and SK...4 3.4 Region...5 3.5 User Permissions... 5 4 Functions... 6 5 Application Scenarios...8 6 Related Services... 10 7 Documentation Guide... 11 8 Change History... 13 ii
1 OBS 1 OBS Object Storage Service (OBS) is an object-based storage service that provides customers with massive, secure, reliable, and cost-effective data storage capabilities, such as bucket creation, modification, and deletion, as well as object upload, download, and deletion. OBS provides super large storage capacity that can store any type of file and is suitable for websites, enterprises, developers, and common subscribers. As an Internet-based service, OBS provides web service interfaces over Hypertext Transfer Protocol (HTTP) and Hypertext Transfer Protocol Secure (HTTPS). You can use OBS Console or OBS Browser to access and manage data stored on OBS from any Internet-connected computer anytime and anywhere. In addition, OBS supports SDKs of programming languages including Java, Python,.NET, ios, C++, Android, Ruby, and PHP, and Representational State Transfer (REST) application programming interfaces (APIs) that are compatible with Amazon S3 APIs. This way, you can easily manage your data stored on OBS and develop a wide range of upper-layer applications. Cloud services allow infrastructures to be deployed in multiple regions and deliver flexible expansion and enhanced reliability. Users can deploy OBS in specific regions to obtain fast access at an affordable price. 1
2 Advantages 2 Advantages OBS has the following advantages: Brilliant data durability and service continuity: OBS is used by the Huawei Cloud Album for mobile phones, able to support a vast number of users' access. Device and data redundancy within each availability zone (AZ) and data disaster recovery between AZs help deliver a data durability of 99.999999999% and a service continuity of 99.99%, which are far higher than those of traditional storage. Multi-level Protection and Authorization Management: OBS uses trusted cloud authentication to secure data. The multiple data protection mechanisms, including versioning, server-side encryption, uniform resource locator (URL) validation, virtual private cloud (VPC)-based network isolation, access log audit, and fine-grained permission control, promote the data security to a new level. Intelligent scheduling for smooth data access: With intelligent scheduling and response, OBS optimizes data access paths. Transfer acceleration, event notification, and big data oriented optimization further improve data access experience on the basis of massive concurrency, large bandwidth, and minimized latencies. Easy to use and on-demand capacity resizing: OBS supports REST APIs, multi-version SDKs, and data migration tools, facilitating transition to the cloud. It is needless to plan a storage capacity beforehand and worry about storage capacity expansion or reduction because storage resources are available for linear and nearly infinite expansion. Two charging options: Pay per use or monthly/yearly subscription. Data in each of the Standard, Infrequent Access, and Archive storage classes can be metered and charged independently, reducing storage costs. 2
3 Basic Concepts 3 Basic Concepts This section describes concepts related to OBS, including object, bucket, Access Key ID (AK), Secret Access Key (SK), permission, and region. 3.1 Object An object is a basic data unit of OBS. It contains both data and the metadata that describes data attributes. Data uploaded to OBS is stored into buckets as objects. An object consists of data, metadata, and a key. A key specifies the name of an object. An object key is a string ranging from 1 to 1024 characters in UTF-8 format. The object key for each object in a bucket must be unique. Metadata provides a description of objects. Metadata contains system metadata and user metadata. All metadata is uploaded to OBS as key-value pairs. System metadata is automatically generated by OBS and is used for processing object data. System metadata includes Date, Content-length, Last-modify, and Content-MD5. User metadata is specified when users upload objects and is used to describe objects. Data is the information contained by an object. Generally, objects are managed as files. However, as an object-based storage service, OBS does not involve real files or folders. For easy data management, OBS provides a method to simulate folders. By adding a slash (/) in an object name, for example, test/123.jpg, you can simulate test as a folder and 123.jpg as the name of a file under the test folder. However, the key remains test/123.jpg. In OBS Console or OBS Browser, you can use folders directly. 3.2 Bucket On OBS, buckets are containers that store objects. OBS provides flat storage methods based on buckets and objects. All objects in a bucket are at the same logical layer, eliminating the traditional multi-layer directory structure of file systems. OBS has the following storage classes: Standard, Infrequent Access, and Archive. In such a manner, OBS meets customers' requirements on storage performance and costs. When 3
3 Basic Concepts creating a bucket, you can set a storage class for the bucket. On OBS, each object has a storage class that is the same as its owning bucket. On OBS, each bucket name must be globally unique and cannot be changed. When a bucket is created, its access control list (ACL) is generated by default. The ACL records authorized users' permissions such as the read and write permissions. Only authorized users can perform bucket operations, such as creating, deleting, viewing, and setting ACLs for buckets. A user can create a maximum of 100 buckets. However, the number and total size of objects in a bucket are not restricted. Users do not need to worry about system scalability. As OBS is based on RESTful HTTP and HTTPS, you can use URLs to locate resources. Figure 3-1 illustrates the relationship between objects and buckets. Figure 3-1 Relationship between objects and buckets 3.3 AK and SK A user's account provided by OBS contains an AK and an SK. The AK and SK are used for user authentication. If you use a client to send a request to OBS, the request header must contain a signature. The signature is generated based on the SK, request time, and request type. AKs and SKs are key pairs used to access OBS. When OBS APIs are used to access stored data, AKs and SKs are used to generate authentication information. After subscribing to OBS, you can log in to My Credential and create AKs and SKs based on site requirements. The system identifies users who access the system by AKs, and SKs are used for key authentication. One AK maps to only one user but one user can have multiple AKs. One SK maps to one AK, forming a key pair for accessing OBS and thereby ensuring access security. 4
3 Basic Concepts 3.4 Region When creating a bucket, you can specify a region for the bucket based on your requirements on the response latency, price, and request source. The specified region cannot be changed after the bucket is created. OBS is available in multiple regions. However, functions available on OBS may vary in these regions. The actual graphical user interface (GUI) prevails. After a bucket is created, all objects uploaded to the bucket will be stored in the data center of the region. 3.5 User Permissions The public cloud system provides two types of user permissions by default: user management and resource management. User management refers to the management of users, user groups, and user group rights. Resource management refers to the control operations that can be performed by users on cloud service resources. For details, see Permissions. 5
4 Functions 4 Functions OBS provides you with a variety of platforms, such as OBS Console and OBS Browser, as well as SDKs (Java,.NET, Python, PHP, Android, C++, and Ruby), and REST APIs. Table 4-1 lists functions provided by OBS Console. Table 4-1 OBS functions Function Basic bucket operations Basic object operations Server-side encryption Event Bucket permissions Access logging Lifecycle management Fragment management Object URL Versioning Description Creates and deletes buckets of different storage classes in specific regions (service areas). Manages objects by uploading (or multipart uploading), downloading, and deleting objects. OBS allows users to encrypt objects using server-side encryption so that the objects can be securely stored on OBS. Allows you to receive notification messages of OBS. Manages bucket permissions, including bucket policies, ACLs, and cross-origin resource sharing (CORS). Records bucket access requests in logs for request analysis and log audit. Supports the ability to set lifecycle rules for buckets to automatically delete expired objects. Fragments are incomplete data in buckets generated due to data upload failures. Fragment management enables you to clear fragments that have been generated. Shares data using object URLs. Manages bucket versioning, allowing multiple versions of objects to co-exist in a bucket. 6
4 Functions Function Static website hosting URL validation Object ACL setting Object metadata Description Supports the ability to set website attributes for buckets, achieving static website hosting, and to set web page redirection to redirect bucket requests to specific hosts. Provides URL validation to prevent object links of OBS from being stolen by other websites. Enables you to set permissions for objects. Allows you to set properties for objects on demand. 7
5 Application Scenarios 5 Application Scenarios OBS applies to the following scenarios: Table 5-1 Application scenarios Storage Class Standard Infrequent Access Application Scenario Big data analysis Cloud-native application program data Static website hosting Backup/Active archive Other service scenarios Description Data, including files, images, and videos, of big data applications (such as gene, financing, meteorology, and advertising) can be directly stored to OBS through the Internet. OBS provides high-performance and high-availability storage services. In this manner, users can easily use affordable and efficient storage services for their mobile and Internet applications. In addition, OBS enables users to access data of any size on OBS from anywhere. Website operators or personal website publishers employ OBS as static website hosting resource pools and set website functions to host entire static websites after uploading static web pages into OBS and to quickly deliver data through a Content Delivery Network (CDN). OBS is a durable, expandable, and secure solution for backing up and archiving users' key data. Its versioning function further protects data. Its high-durability and secure infrastructure aims to provide an advanced data protection and disaster recovery solution. Additionally, OBS supports third-party backup and archive software, such as Commvault and NetBackup. OBS can be used as storage resource pools or backup storage of other services, including Image Management Service (IMS), Volume Backup Service (VBS), and Relational Database Service (RDS). 8
5 Application Scenarios Storage Class Archive Application Scenario Deep/Cold archive Description Data that requires long-term retention for backup (months to years) and is insensitive to restoration latency ( 12 hours) can use OBS Archive as its archive storage. 9
6 Related Services 6 Related Services OBS provides an ultra-large storage space. You can use storage resources provided by OBS in conjunction with other cloud service products that you have purchased. OBS can be used as the storage resource pool or snapshot backup storage for the following services: Data Express Service (DES) Relational Database Service (RDS) Image Management Service (IMS) Cloud Trace Service (CTS) Depending on the following service, OBS can perform server-side encryption on objects: Key Management Service (KMS) Depending on the following service, OBS can authenticate users and set permissions for specific users: Identity and Access Management (IAM) Depending on the following service, OBS can notify users that OBS is being used by them through a message: Simple Message Notification (SMN) The following service can be used to analyze data stored on OBS: MapReduce Service (MRS) OBS provides data sources of machine learning for the following service: Machine Learning Service (MLS) 10
7 Documentation Guide 7 Documentation Guide Table 7-1 OBS support documentation Document Abstract Format Glossary Purchase Guide Console Operation Guide API Reference Developer Guide FAQ Browser Operation Guide OBS Third-Party Client User Guide Lists OBS related terms and acronyms such as AK, SK, and object. Describes the OBS concepts, functions, features, application scenarios, and product advantages. Describes the OBS pricing and how to purchase OBS. Describes all functions of OBS Console, guides you through operations on the Console, and provides use cases to help you better use OBS. Describes the OBS APIs, as well as the function and parameters of each API. Describes how to conduct secondary developments based on the APIs and SDKs provided by OBS, which can be used as a reference for your secondary developments. Lists the frequently asked questions, covering OBS concepts, security, functions, and features. Describes the OBS Browser functions, features, and restrictions, and provides the operation guidance and use cases. Describes how to use third-party clients such as s3fs and S3cmd to access and operate OBS. HTML HTML PDF HTML PDF HTML PDF HTML PDF HTML PDF HTML PDF PDF 11
7 Documentation Guide NOTE You can download PDFs from Document Download. 12
8 Change History 8 Change History Release Date What's New 2017-12-20 This issue is the fourth official release. This issue incorporates the following changes: Added the following content in OBS Console Operation Guide. Added the description of projects to "Configuring Event Notification". Added the section of "Time Difference Is Longer Than 15 Minutes Between the Client and Server". Added the function of one-click bucket policy setting for a new bucket. 2017-10-30 This issue is the third official release. This issue incorporates the following changes: Added the information about the endpoint and access domain name in the "Viewing Basic Information About a Bucket" section in OBS Console Operation Guide. Modified the URL format in the "Accessing an Object Using its URL" section in OBS Console Operation Guide. Added the configuration of universal policy in the "Configuring a Bucket Policy" section in OBS Console Operation Guide. Added the "Failed to Configure Event Notification" section under "Troubleshooting" in OBS Console Operation Guide. In OBS Purchase Guide, deleted the restriction on the account whose balance is less than CNY 100. Deleted the description about privacy protection from the "Registering a Public Cloud Service Account" section in OBS Quick Start. 13
8 Change History Release Date What's New 2017-09-28 This issue is the second official release. This issue incorporates the following changes: Added the "Documentation Guide" section in OBS Product Introduction. Added the "Related Operations" section in OBS Quick Start. Modified the "Configuring a Bucket Policy" section and added the "Details About Actions and Conditions" section in OBS Console Operation Guide. 2017-08-20 This issue is the first official release. 14