FunctionGraph. Best Practices. Issue 05 Date HUAWEI TECHNOLOGIES CO., LTD.

Transcription

1 Issue 05 Date HUAWEI TECHNOLOGIES CO., LTD.

2 Copyright Huawei Technologies Co., Ltd All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd. Trademarks and Permissions and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd. All other trademarks and trade names mentioned in this document are the property of their respective holders. Notice The purchased products, services and features are stipulated by the contract made between Huawei and the customer. All or part of the products, services and features described in this document may not be within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information, and recommendations in this document are provided "AS IS" without warranties, guarantees or representations of any kind, either express or implied. The information in this document is subject to change without notice. Every effort has been made in the preparation of this document to ensure accuracy of the contents, but all statements, information, and recommendations in this document do not constitute a warranty of any kind, express or implied. Huawei Technologies Co., Ltd. Address: Website: Huawei Industrial Base Bantian, Longgang Shenzhen People's Republic of China Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. i

3 Contents Contents 1 Compressing Images Introduction Preparation Building a Program Adding an Event Source Compressing Images Watermarking Images Introduction Preparation Building a Program Adding an Event Source Watermarking Images Processing DIS Data Introduction Preparation Building a Program Adding an Event Source Processing Data Building an AI Application for Identifying Pornographic Images Introduction Preparation Building a Backend Function Building a Frontend Function Image Identification Analyzing Logs in Real Time While Integrating with LTS Introduction Preparation Building a Program Adding an Event Source Processing Log Data Other Application Scenarios...56 Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. ii

4 Contents 6 Analyzing Login/Logout Security While Integrating with CTS Introduction Preparation Building a Program Adding an Event Source Processing Operation Records Other Application Scenarios Constructing a Video Processing Service Introduction Preparation Building Functions Building a Function for Obtaining Video Formats Building a Function for Converting Video Formats Function for Handling Video Conversion Failures Function for Capturing Video Screenshots Orchestrating Functions Adding an Event Source Processing Videos...88 Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. iii

5 1 Compressing Images 1 Compressing Images 1.1 Introduction The best practice for HUAWEI CLOUD FunctionGraph guides you through image compressing based on a function. Scenario Upload images to a specified Object Storage Service (OBS) bucket. Compress each uploaded image. Upload the processed images to another specified OBS bucket. NOTE 1. This tutorial uses two different OBS buckets. 2. The function you create must be in the same region (default region) as the OBS buckets. Procedure Create two buckets on the OBS console. Create a function with an OBS trigger. Upload an image to one of the buckets. The function is triggered to compress the image. The function uploads the processed image to the other bucket. NOTE After you complete this tutorial, your HUAWEI CLOUD account will have the following resources: 1. Two OBS buckets (respectively used for storing uploaded and processed images) 2. A thumbnail image creation function (fss_examples_image_thumbnail) 3. An OBS trigger used for associating the function with the OBS buckets 1.2 Preparation Before creating a function and adding an event source, you need to create two OBS buckets to respectively store uploaded and compressed images. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 1

6 1 Compressing Images Creating OBS Buckets After creating the OBS buckets, you must create an agency to delegate FunctionGraph to access OBS resources. Precautions Ensure that the source bucket for storing uploaded images, the target bucket for storing processed images, and the function you will create are in the same region. In this tutorial, all of them are in CN North-Beijing1, the default region where FunctionGraph resides. Use two different OBS buckets. If only one bucket is used, the function will be executed infinitely. (When an image is uploaded to the bucket, the function is triggered to process the image and store the processed image into the bucket again. In this way, the function executes endlessly.) Procedure Step 1 Log in to the HUAWEI CLOUD management console, and choose Object Storage Service. Click Create Bucket. Step 2 On the Create Bucket page, set the bucket information, as shown in Figure 1-1. For Region, select CN North-Beijing1. For Bucket Name, enter your-bucket-input. For Storage Class, select Standard. For Bucket Policy, select Private. Click Create Now. Figure 1-1 Creating the source bucket Step 3 Repeat Step 2 to create the target bucket. Name the target bucket as your-bucket-output, and select the same region and storage class as the source bucket. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 2

7 1 Compressing Images Step 4 After creation, check whether the two buckets are displayed in the bucket list, as shown in Figure 1-2. Figure 1-2 Bucket list ----End Creating an Agency Step 1 Log in to the HUAWEI CLOUD management console, and choose Identity and Access Management. Choose Agency, and click Create Agency. Step 2 On the Create Agency page, set the agency information, as shown in Figure 1-3. For Agency Name, enter serverless_trust. For Agency Type, select Cloud service. For Cloud Service, select FunctionGraph. For Validity Period, select Permanent. For Permissions, click Modify on the right of the Global service > OBS row, and choose Base > Tenant Administrator, as shown in Figure 1-4. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 3

8 1 Compressing Images Figure 1-3 Creating an agency Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 4

9 1 Compressing Images Figure 1-4 Modifying the permission set NOTE Users with the Tenant Administrator permission can perform any operations on all cloud resources of the enterprise. Step 3 Click OK. ----End 1.3 Building a Program You can download and use the image compressing program package provided in this example. Creating a Deployment Package This example uses a Python function to compress images. For details about function development, see Developing Functions in Python. Figure 1-5 shows the sample code directory. The service code is not described. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 5

10 1 Compressing Images Figure 1-5 Sample code directory Under the directory, index.py is a handler file. The following code is a snippet of the handler file. Parameter obs_output_bucket is the address for storing compressed images and must be configured when you create a function. def handler (event, context): srcbucket, srcobjname = getobsobjinfo(event) obs_address = context.getuserdata('obs_address') outputbucket = context.getuserdata('obs_output_bucket') if obs_address is None: obs_address = ' ' if outputbucket is None: outputbucket = 'casebucket-out' ak = context.getaccesskey() sk = context.getsecretkey() print "*** srcbucketname: " + srcbucket print "*** srcobjname:" + srcobjname print "*** obs_address: " + obs_address print "*** output bucket: " + outputbucket # download file uploaded by user from obs GetObject(obs_address, srcbucket, srcobjname, ak, sk) outfile = image_thumbnail(srcobjname) # Uploads converted files to a new OBS bucket. PostObject (obs_address, outputbucket, outfile, ak, sk) return 'OK' Creating a Function When creating a function, specify an agency with OBS access permissions so that FunctionGraph can invoke the OBS service. Step 1 Log in to the HUAWEI CLOUD management console, and choose FunctionGraph. Choose Functions, and click Create Function. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 6

11 1 Compressing Images Step 2 Set the function information. 1. Set the basic information, as shown in Figure 1-6. For Function Name, enter fss_examples_image_thumbnail. For Description, enter Image compressing. For Agency, select serverless_trust created in Creating an Agency. Figure 1-6 Basic information 2. Set the code information, as shown in Figure 1-7. For Runtime, select Python 2.7. For Handler, enter index.handler. For Code Entry Mode, select Upload ZIP file, and upload the sample code package fss_examples_image_thumbnail.zip. Figure 1-7 Code information 3. Click Create Function. Step 3 On the fss_examples_image_thumbnail page, select the Configuration tab and set the environment information, as shown in Figure 1-8. For Memory, select 256. For Timeout, enter 40. For Environment Variables, define parameter obs_output_bucket in the index.py file as the key and bucket your-bucket-output created in Creating OBS Buckets as the value. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 7

12 1 Compressing Images Figure 1-8 Environment information Step 4 Click Save. ----End 1.4 Adding an Event Source After creating the OBS buckets and function, you can add an event source to the function by creating an OBS trigger. Perform the following procedure: Step 1 On the fss_examples_image_thumbnail page, click the Triggers tab and click Create Trigger. Step 2 Select OBS for Trigger Type, and set the trigger information, as shown in Figure 1-9. For Bucket Name, select your-bucket-input created in Creating OBS Buckets. For Events, select Put and Post. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 8

13 1 Compressing Images Figure 1-9 Creating a trigger Step 3 Click OK. The trigger is created, as shown in Figure Figure 1-10 OBS trigger NOTE ----End After the OBS trigger is created, when an image is uploaded or updated to bucket your-bucket-input, an event is generated to trigger the function. 1.5 Compressing Images When an image is uploaded or updated to bucket your-bucket-input, an event is generated to trigger the function. The function compresses the image and stores the compressed one into bucket your-bucket-output. Uploading an Image to Generate an Event Perform the following steps to upload an image to a specified bucket: 1. Log in to the OBS console. 2. Click your-bucket-input in the bucket list. 3. Choose Object in the navigation pane, and click the Objects tab. 4. Click Upload File and upload the image.jpg file. 5. Click OK. The image.jpg file is successfully uploaded, as shown in Figure Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 9

14 1 Compressing Images Figure 1-11 Uploading an image NOTE The size of the original image.jpg file exceeds 28 KB. Triggering the Function After the image is uploaded to bucket your-bucket-input, OBS generates an event to trigger the image compressing function. The function compresses the image and stores the compressed one into bucket your-bucket-output. You can view running logs of the function on the Logs tab page, as shown in Figure Figure 1-12 Logs On the your-bucket-output page on the OBS console, you can see the compressed image image-thumbnail.jpg, whose size is less than 8 KB after compression, as shown in Figure Figure 1-13 Compressed image Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 10

15 2 Watermarking Images 2 Watermarking Images 2.1 Introduction The best practice for HUAWEI CLOUD FunctionGraph guides you through image watermarking based on a function. Scenario Upload images to a specified OBS bucket. Watermark each uploaded image. Upload the processed images to another specified OBS bucket. NOTE 1. This tutorial uses two different OBS buckets. 2. The function you create must be in the same region (default region) as the OBS buckets. Procedure Create two buckets on the OBS console. Create a function with an OBS trigger. Upload an image to one of the buckets. The function is triggered to watermark the image. The function uploads the processed image to the other bucket. NOTE After you complete this tutorial, your HUAWEI CLOUD account will have the following resources: 1. Two OBS buckets (respectively used for storing uploaded and processed images) 2. An image watermarking function 3. An OBS trigger used for associating the function with the OBS buckets 2.2 Preparation Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 11

16 2 Watermarking Images Creating OBS Buckets Before creating a function and adding an event source, you need to create two OBS buckets to respectively store uploaded and watermarked images. After creating the OBS buckets, you must create an agency to delegate FunctionGraph to access OBS resources. Precautions Ensure that the source bucket for storing uploaded images, the target bucket for storing processed images, and the function you will create are in the same region. In this tutorial, all of them are in CN North-Beijing1, the default region where FunctionGraph resides. Use two different OBS buckets. If only one bucket is used, the function will be executed infinitely. (When an image is uploaded to the bucket, the function is triggered to process the image and store the processed image into the bucket again. In this way, the function executes endlessly.) Procedure Step 1 Log in to the HUAWEI CLOUD management console, and choose Object Storage Service. Click Create Bucket. Step 2 On the Create Bucket page, set the bucket information, as shown in Figure 2-1. For Region, select CN North-Beijing1. For Bucket Name, enter hugb-bucket-input. For Storage Class, select Standard. For Bucket Policy, select Private. Click Create Now. Figure 2-1 Creating an OBS bucket Step 3 Repeat Step 2 to create the target bucket. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 12

17 2 Watermarking Images Name the target bucket as hugb-bucket-output, and select the same region and storage class as the source bucket. Step 4 After creation, check whether the two buckets are displayed in the bucket list, as shown in Figure 2-2. Figure 2-2 Bucket list ----End Creating an Agency Step 1 Log in to the HUAWEI CLOUD management console, and choose Identity and Access Management. Choose Agency, and click Create Agency. Step 2 On the Create Agency page, set the agency information, as shown in Figure 2-3. For Agency Name, enter serverless_trust. For Agency Type, select Cloud service. For Cloud Service, select FunctionGraph. For Validity Period, select Permanent. For Permissions, click Modify on the right of the Global service > OBS row, and choose Base > Tenant Administrator, as shown in Figure 2-4. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 13

18 2 Watermarking Images Figure 2-3 Creating an agency Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 14

19 2 Watermarking Images Figure 2-4 Modifying the permission set NOTE Users with the Tenant Administrator permission can perform any operations on all cloud resources of the enterprise. Step 3 Click OK. ----End 2.3 Building a Program You can download and use the image watermarking program package provided in this example. Creating a Deployment Package This example uses a Python function to watermark images. For details about function development, see Developing Functions in Python. Figure 2-5 shows the sample code directory. The service code is not described. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 15

20 2 Watermarking Images Figure 2-5 Sample code directory Under the directory, index.py is a handler file. The following code is a snippet of the handler file. Parameter obs_output_bucket is the address for storing watermarked images and must be configured when you create a function. def handler(event, context): srcbucket, srcobjname = getobsobjinfo4obstrigger(event) obs_address = context.getuserdata('obs_address') outputbucket = context.getuserdata('obs_output_bucket') if obs_address is None: obs_address = ' ' if outputbucket is None: outputbucket = 'casebucket-out' ak = context.getaccesskey() sk = context.getsecretkey() # download file uploaded by user from obs GetObject(obs_address, srcbucket, srcobjname, ak, sk) outfile = watermark_image(srcobjname) # Uploads converted files to a new OBS bucket. PostObject (obs_address, outputbucket, outfile, ak, sk) return 'OK' Creating a Function When creating a function, specify an agency with OBS access permissions so that FunctionGraph can invoke the OBS service. Step 1 Step 2 Log in to the HUAWEI CLOUD management console, and choose FunctionGraph. Choose Functions, and click Create Function. Set the function information. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 16

21 2 Watermarking Images 1. Set the basic information, as shown in Figure 2-6. For Function Name, enter fss_examples_image_watermark. For Description, enter Image watermarking. For Agency, select serverless_trust created in Creating an Agency. Figure 2-6 Basic information 2. Set the code information, as shown in Figure 2-7. For Runtime, select Python 2.7. For Handler, enter index.handler. For Code Entry Mode, select Upload ZIP file, and upload the sample code package fss_examples_image_watermark.zip. Figure 2-7 Code information 3. Click Create Function. Step 3 On the fss_examples_image_watermark page, select the Configuration tab and set the environment information, as shown in Figure 2-8. For Memory, select 128. For Timeout, enter 3. For Environment Variables, define parameter obs_output_bucket in the index.py file as the key and bucket hugb-bucket-output created in Creating OBS Buckets as the value. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 17

22 2 Watermarking Images Figure 2-8 Environment information Step 4 Click Save. ----End 2.4 Adding an Event Source After creating the OBS buckets and function, you can add an event source to the function by creating an OBS trigger. Perform the following procedure: Step 1 On the fss_examples_image_watermark page, click the Triggers tab and click Create Trigger. Step 2 Select OBS for Trigger Type, and set the trigger information, as shown in Figure 2-9. For Bucket Name, select hugb-bucket-input created in Creating OBS Buckets. For Events, select Put and Post. Figure 2-9 Creating an OBS trigger Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 18

23 2 Watermarking Images Step 3 Click OK. The trigger is created, as shown in Figure Figure 2-10 OBS trigger NOTE ----End After the OBS trigger is created, when an image is uploaded or updated to bucket hugb-bucket-input, an event is generated to trigger the function. 2.5 Watermarking Images When an image is uploaded or updated to bucket hugb-bucket-input, an event is generated to trigger the function. The function watermarks the image and stores the watermarked one into bucket hugb-bucket-output. Uploading an Image to Generate an Event Perform the following steps to upload an image to a specified bucket: 1. Log in to the OBS console. 2. Click hugb-bucket-input in the bucket list. 3. Choose Object in the navigation pane, and click the Objects tab. 4. Click Upload File and upload the image.jpg file. 5. Click OK. The image.jpg file is successfully uploaded, as shown in Figure Figure 2-11 Uploading an image Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 19

24 2 Watermarking Images Triggering the Function After the image is uploaded to bucket hugb-bucket-input, OBS generates an event to trigger the image watermarking function. The function watermarks the image and stores the watermarked one into bucket hugb-bucket-output. You can view running logs of the function on the Logs tab page, as shown in Figure Figure 2-12 Logs On the hugb-bucket-output page in the OBS console, you can see the watermarked image image-watermark.jpg, as shown in Figure In the Operation column, click Download to download the image and view the watermarking effect, as shown in Figure Figure 2-13 Output image Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 20

25 2 Watermarking Images Figure 2-14 Watermarked image Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 21

26 3 Processing DIS Data 3 Processing DIS Data 3.1 Introduction Scenario The best practice for HUAWEI CLOUD FunctionGraph guides you through DIS data processing based on a function. When using the Data Ingestion Service (DIS) to collect real-time Internet of Things (IoT) data streams, process the collected data, for example, convert its format, and then store the processed data into the CloudTable Service (CloudTable). Procedure Create a Virtual Private Cloud (VPC) and cluster. Build a data processing program and package the code. Create a function on the FunctionGraph console. Configure a DIS event to test the data processing function. 3.2 Preparation Creating a VPC This tutorial demonstrates how to convert the format of DIS data and store the converted data into CloudTable. To achieve this purpose, you need to create a VPC and then create a cluster on the CloudTable console. Before creating a function, you must create an agency to delegate FunctionGraph to access DIS and CloudTable resources. Step 1 Step 2 Log in to the VPC console. Choose Dashboard or Virtual Private Cloud, and click Create VPC. Set the private cloud information. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 22

27 3 Processing DIS Data For Basic Information, enter name vpc-cloudtable, and use the default settings of other parameters, as shown in Figure 3-1. For Subnet Settings, use the default settings, as shown in Figure 3-2. Figure 3-1 Basic information Figure 3-2 Subnet settings Step 3 Confirm the configuration information and click Submit. ----End Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 23

28 3 Processing DIS Data Creating a Cluster Step 1 Step 2 Log in to the CloudTable console. In the navigation pane, choose Overview, and then click Create Cluster. Set the cluster information. For Region, use the default region, as shown in Figure 3-3. For Cluster Configuration, enter name cloudtable-dis, and use the default settings of other parameters, as shown in Figure 3-4. For Computing Unit, use the default quantity, as shown in Figure 3-5. For Network Configuration, select vpc-cloudtable created in Creating a VPC, and use the default settings of other parameters, as shown in Figure 3-6. For Software Configuration, use the default settings, as shown in Figure 3-7. Figure 3-3 Region Figure 3-4 Cluster configurations Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 24

29 3 Processing DIS Data Figure 3-5 Computing unit Figure 3-6 Network configurations Figure 3-7 Software configurations Step 3 Confirm the configuration information and click Apply Now. Select the I have read and agree to the Free Open Beta Service Agreement check box, and click Submit Application. Figure 3-8 Creating a cluster Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 25

30 3 Processing DIS Data NOTE ----End Creating an Agency Step 1 Creating a cluster takes a long time. You can check the creation progress according to Figure 3-8. Log in to the HUAWEI CLOUD management console, and choose Identity and Access Management. Choose Agency, and click Create Agency. Step 2 On the Create Agency page, set the agency information, as shown in Figure 3-9. For Agency Name, enter DISDemo. For Agency Type, select Cloud service. For Cloud Service, select FunctionGraph. For Validity Period, select Permanent. For Permissions, click Modify on the right of CN North-Beijing1 > cn-north-1 row, and choose Base > Server Administrator, Base > Tenant Administrator, and coludtable > coludtable Administrator, as shown in Figure Figure 3-9 Creating an agency Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 26

31 3 Processing DIS Data Figure 3-10 Modifying permissions Step 3 Click OK. ----End 3.3 Building a Program Creating a Project You can download and use the source code and program package (including function dependencies) provided in this example for format conversion of DIS stream data. This example uses a Java function to convert the format of DIS stream data. For details about function development, see Developing Functions in Java. The service code is not described. Download the sample source code package fss_examples_dis_cloudtable_src.zip, decompress the file, and import it to Eclipse, as shown in Figure Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 27

32 3 Processing DIS Data Figure 3-11 Sample code In the sample code, modify proid (project ID) and clusid (cluster ID), and save the modification. To obtain the project ID, perform the following steps: 1. Under the current login account in the upper right corner, choose My Credential, as shown in Figure Find the project ID in the project list, as shown in Figure Figure 3-12 My Credential Figure 3-13 Project ID To obtain the cluster ID, perform the following steps: 1. Log in to the CloudTable console. 2. In the navigation pane, choose Cluster Management. Click cluster cloudtable-dis created in Creating a Cluster. 3. On the cloudtable-dis page that is displayed, find the cluster ID, as shown in Figure Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 28

33 3 Processing DIS Data Figure 3-14 Cluster ID Packaging the Code When creating a function on the FunctionGraph console, set a handler in the format of [package name].[file name].[function name], for example, com.huawei.cff.tabletester.myhandler for the preceding code. Use Eclipse to package the code into a JAR file named Table Tester.jar according to the following figures. Figure 3-15 Exporting the code Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 29

34 3 Processing DIS Data Figure 3-16 Selecting a file type Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 30

35 3 Processing DIS Data Figure 3-17 Publishing the code file Package the function dependencies by performing the following steps: 1. Download program package fss_examples_dis_cloudtable.zip, and decompress it, as shown in Figure Use Table Tester.jar to replace DIS Test.jar, as shown in Figure Package all of the files into disdemo.zip, as shown in Figure Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 31

36 3 Processing DIS Data Figure 3-18 File directory before replacement Figure 3-19 File directory after replacement Figure 3-20 Packaging the files Creating a Function When creating a function, specify an agency to delegate FunctionGraph to access DIS and CloudTable resources. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 32

37 3 Processing DIS Data Step 1 Log in to the HUAWEI CLOUD management console, and choose FunctionGraph. Choose Functions, and click Create Function. Step 2 Set the function information, as shown in Figure For Template, select Create from scratch. For Function Name, enter DISDemo. For Agency, select DISDemo created in Preparation. For Runtime, select Java 8. For Handler, enter com.huawei.cff.tabletester.myhandler. For Code Entry Mode, select Upload ZIP file. Upload code package disdemo.zip created in Packaging the Code. Figure 3-21 Creating a function Step 3 Click Create Function. ----End Modifying Function Configurations After the function is created, the default memory is 128 MB, and the default timeout is 3s, which are insufficient for the data processing. Perform the following steps to modify the configurations. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 33

38 3 Processing DIS Data Step 1 On the DISDemo page, click the Configuration tab and modify the configuration information, as shown in Figure For Memory, select 512. For Timeout, enter 15. Keep other parameters unchanged. Figure 3-22 Function configurations Step 2 Click Save. ----End 3.4 Adding an Event Source After creating the function, you can add an event source by creating a DIS trigger. Perform the following procedure: Step 1 On the DISDemo page, select Configure test event, as shown in Figure Figure 3-23 Configuring a test event Step 2 In the Configure Test Event dialog box, set the test event information, as shown in Figure For Configure Test Event, select Create new test event. For Event Template, select dis-event-template. For Event Name, enter dis-test. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 34

39 3 Processing DIS Data Figure 3-24 Test event Step 3 Click Save. ----End 3.5 Processing Data Perform the following procedure to process simulated stream data: Step 1 On the DISDemo page, select test event dis-test, and click Test to test the function, as shown in Figure Figure 3-25 Selecting a test event Step 2 After the function is executed successfully, check the logs shown in Figure For all logs of the function, go to the Logs tab page. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 35

40 3 Processing DIS Data Figure 3-26 Function execution result ----End Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 36

41 4 Building an AI Application for Identifying Pornographic Images 4 Building an AI Application for Identifying Pornographic Images 4.1 Introduction Objective Scenarios The best practice for HUAWEI CLOUD FunctionGraph guides you through pornographic image identification based on functions. In this example, FunctionGraph interconnects with the Artificial Intelligence (AI) Service to identify pornographic images. You can create an APIG trigger to provide external API services for pornographic image identification. To visually demonstrate how to identify pornographic images, this example provides a frontend page for you to upload images and view the identification results. Using FunctionGraph to build AI applications has the following advantages: Simplifies the development process and focuses on service implementation. Flexibly interconnects with Huawei cloud services to improve development efficiency. Scales automatically and bills per use to prevent waste of resources. Requires no maintenance and provides high availability at low costs. This example uses the following Huawei cloud services: AI: provides image moderation. For details, see Content Moderation. FunctionGraph: builds a pornographic image identification function. For details, see FunctionGraph. API Gateway: provides an API for pornographic image identification. For details, see API Gateway. OBS: hosts frontend code to provide a static website. For details, see Object Storage Service. Figure 4-1 shows the simulated image identification page. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 37

42 4 Building an AI Application for Identifying Pornographic Images Figure 4-1 Image identification Figure 4-2 illustrates the principle of image identification. Figure 4-2 Principle of image identification OBS hosts frontend code to provide a static website. A frontend page is provided for uploading images. After an image is uploaded, the frontend code calls the API provided by API Gateway to trigger the backend function. During execution, the function calls the image moderation API. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 38

43 4 Building an AI Application for Identifying Pornographic Images Resource Preparation The Image Moderation service determines whether the uploaded image is acceptable and returns the identification result. Apply for the Image Moderation service on the Artificial Intelligence Service console. Figure 4-3 shows the Content Moderation option, and Figure 4-4 shows the Content Moderation page where you can apply for the Image Moderation service. Figure 4-3 Content Moderation Figure 4-4 Applying for OBT of the Image Moderation service Procedure Apply for the API Gateway service. Apply for the FunctionGraph service. Apply for the OBS service. 1. Create the following resources: One agency: enables FunctionGraph to access API Gateway, Image Moderation, and OBS resources. One API group: a collection of APIs. API providers classify APIs into different API groups and manage APIs by API group. One OBS bucket: hosts the frontend program to provide a static website. 2. Build a backend function. Create a backend function to implement image identification. Create an APIG trigger for the backend function and provide an API for the frontend to invoke the backend function. 3. Build a frontend function. Create a frontend function to provide a frontend page. Create an APIG trigger for the frontend function and provide an API to invoke the frontend function. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 39

44 4 Building an AI Application for Identifying Pornographic Images Price Description 4. Upload an image and check whether the image is acceptable. The Image Moderation, API Gateway, and FunctionGraph services used in this example are under an OBT, and can be used for free during this period. You will be charged only after the OBT is ended. For details, see the Product Introduction of each service. For OBS, you will be charged based on the following rules. For details, see OBS pricing details. Storage space: 0.148/GB-month Request: 0.01/10,000 requests Outbound traffic of data transfer over the Internet: 0.5/GB 4.2 Preparation Creating an Agency An agency is an entrusting relationship created by an administrator to grant specific permissions to other accounts. After an entrusting relationship is established, the administrator of the entrusted enterprise can manage cloud resources on behalf of the entrusting enterprise by switching the agency. This ensures secure and efficient service management. You must create an agency to enable FunctionGraph to access API Gateway, Image Moderation, and OBS resources. Step 1 Step 2 Step 3 Log in to the IAM console, and choose Agency in the navigation pane. On the Agency page, click Create Agency. In the Create Agency dialog box, set the agency information according to Table 4-1. The parameters marked with an asterisk (*) are mandatory. For details, see Creating an Agency. Table 4-1 Parameters for creating an agency Parameter *Agency Name *Agency Type *Cloud Service *Validity Period Description Description Enter serverless-trust. Select Cloud service. Perform the following steps to select a cloud service: 1. Click Select. 2. In the Select Cloud Service dialog box, select FunctionGraph. 3. Click OK. Select Permanent. This example does not require a description. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 40

45 4 Building an AI Application for Identifying Pornographic Images Parameter *Permissions Description Perform the following steps to set permissions: 1. Select the region where FunctionGraph resides, for example, CN North- Beijing1. Then click Modify. 2. In the Policy dialog box, select Tenant Administrator, as shown in Figure Click OK. Figure 4-5 Modifying permissions NOTE Users with the Tenant Administrator permission can perform any operations on all cloud resources of the enterprise. Step 4 Click OK. ----End Creating an API group An API group is a collection of APIs used for the same type of services. One API group can be considered as a service. You can manage APIs by API group. When creating an APIG trigger, select an API group in the same region as FunctionGraph. Create an API group by following the procedure described in this section. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 41

46 4 Building an AI Application for Identifying Pornographic Images Step 1 Log in to the API Gateway console. In the navigation pane, choose API Publishing > API Groups. Step 2 Click Create API Group. Step 3 In the Create API Group dialog box, set the API group information according to Table 4-2. The parameter marked with an asterisk (*) is mandatory. Table 4-2 Information required for creating an API group Parameter *Group Name Description Description Enter 3 24 characters, starting with a letter. Only letters, digits, and underscores (_) are allowed. Enter function. This example does not require a description. Step 4 Click OK. ----End Creating an OBS bucket Create an OBS bucket to host the frontend program and provide a static website. Step 1 Log in to the OBS console, and click Create Bucket. Step 2 Set the bucket information according to Figure 4-6. For Region, select CN North-Beijing1. For Bucket Name, enter obs-aidemo. For Storage Class, select Standard. For Bucket Policy, select Public Read. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 42

47 4 Building an AI Application for Identifying Pornographic Images Figure 4-6 Creating a bucket ----End 4.3 Building a Backend Function Creating a Backend Function Step 1 Log in to the HUAWEI CLOUD management console, and choose FunctionGraph. Choose Functions, and click Create Function. Step 2 Set the function information, as shown in Figure For Template, select Create from scratch. 2. For Function Name, enter AIDemo. 3. For Agency, select serverless-trust. The selected agency must have permissions to access API Gateway, Image Moderation, and OBS resources. For details on how to create an agency, see Creating an Agency. 4. For Runtime, select Node.js For Handler, enter index.handler. 6. For Code Entry Mode, select Edit code inline. 7. Enter the following code in the code box: const https = require('https'); var constructpayload = function(event) { if (event.httpmethod!= "POST") { return { "image": "", "url": " } } var bodystr = new Buffer(event.body, 'base64').tostring('ascii') var body = JSON.parse(bodyStr) if (body.url!= null && body.url.trim()!= "") { return { Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 43

48 4 Building an AI Application for Identifying Pornographic Images } "image": "", "url": body.url }; } else { return { "image": body.image, "url": "" }; } exports.handler = function (event, context, callback) { new Promise(function(resolve, reject) { var token = context.gettoken(); const options = { hostname: 'ais.cn-north-1.myhwclouds.com', port: 443, path: '/v1.0/moderation/image/anti-porn', method: 'POST', headers: { 'Content-Type': 'application/json;charset=utf8', 'X-Auth-Token': token } }; var payload = constructpayload(event) console.log("sending request to AIS...") const req = https.request(options, (res) => { //console.log('headers:', res.headers); var status = res.statuscode console.log("status:", status) res.on('data', (d) => { resolve(d.tostring()); }); }); req.on('error', (e) => { console.error(e); }); req.write(json.stringify(payload)); req.end(); }).then(function(value) { var result = { body: value, headers: { "Content-Type":"application/json", "Access-Control-Allow-Origin": "*", "Access-Control-Allow-Headers": "Content- Type,Accept", "Access-Control-Allow-Methods": "GET,POST,PUT,DELETE" }, statuscode: 200, isbase64encoded: false }; callback(null, result); }); } Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 44

49 4 Building an AI Application for Identifying Pornographic Images Figure 4-7 Function configurations Step 3 In the right pane of the page, review the function configuration and billing information, and click Create Function. ----End Creating an APIG Trigger for the Backend Function Step 1 Log in to the FunctionGraph console. Choose Functions, and then click AIDemo. Step 2 Click the Triggers tab and click Create Trigger, as shown in Figure 4-8. Figure 4-8 Creating a trigger Step 3 Select APIG for Trigger Type, and set the trigger information, as shown in Figure For API Name, enter AIDemo. 2. For API Group, select function. For details on how to create an API group, see Creating an API group. 3. For Environment, select RELEASE. 4. For Security Authentication, select NONE. 5. For Request Protocol, select HTTP. 6. For Backend Timeout, enter Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 45

50 4 Building an AI Application for Identifying Pornographic Images Figure 4-9 Creating an APIG trigger NOTE After you create an APIG trigger for the RELEASE environment, an API is generated and published on the API Gateway console. The API can be called by the function. Step 4 Click OK. NOTE ----End The invocation address of the APIG trigger for the backend function is e812-42b3-85bf-a7aac3cd6cba.apigw.cn-north-1.huaweicloud.com/aidemo. You will use this address when building a frontend function. 4.4 Building a Frontend Function You can deploy a frontend page on OBS as a static resource, and use OBS to host your static website with a customized domain name. You can download and use the frontend program package provided in this example. Downloading the Program Package Download the frontend program package functiongraph-demo.zip, and decompress it, as shown in Figure Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 46

51 4 Building an AI Application for Identifying Pornographic Images Figure 4-10 Program package directory In the /functiongraph/assets/config/apis.json file, change the value of checkimage into the URL ( of the APIG trigger created in Creating an APIG Trigger for the Backend Function, as shown in Figure Figure 4-11 Changing the value of checkimage Uploading the Code Files to an OBS Bucket Log in to the OBS console, click bucket obs-aidemo created in Creating an OBS bucket, and upload the code files to the bucket, as shown in Figure Figure 4-12 Frontend program package directory Setting Up Static Website Hosting 1. On the obs-aidemo page, choose Static Website Hosting, and click Disabled, as shown in Figure Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 47

52 4 Building an AI Application for Identifying Pornographic Images Figure 4-13 Static website hosting 2. In the Static Website Hosting dialog box, set the hosting information according to Figure Click Use this bucket to host a website. For Default Home Page, enter index.html. Leave other parameters blank. Record the static website address, for example: Figure 4-14 Static website hosting configurations Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 48

53 4 Building an AI Application for Identifying Pornographic Images FunctionGraph 3. Click OK. 4.5 Image Identification 1. In the address bar of the browser, enter the website address obtained in Building a Frontend Function, as shown in Figure Figure 4-15 Image identification page 2. Upload an image to start identification, as shown in Figure Figure 4-16 Uploading an image 3. Issue 05 ( ) Check the identification result, as shown in Figure The uploaded image is unacceptable. Copyright Huawei Technologies Co., Ltd. 49

54 4 Building an AI Application for Identifying Pornographic Images Figure 4-17 Identification result Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 50

55 5 Analyzing Logs in Real Time While Integrating with LTS 5 Analyzing Logs in Real Time While Integrating with LTS 5.1 Introduction Scenario Quickly collect, process, and convert task logs of servers, such as ECSs, through Log Tank Service (LTS). Obtain log data based on an LTS trigger created on FunctionGraph, analyze and process key information in the logs by using a customized function, and then filter alarm logs. Use SMN to push alarm messages to service personnel by SMS message or . Store processed log data in a specified OBS bucket for subsequent processing. The processing workflow is shown in Figure 5-1. Figure 5-1 Processing workflow Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 51

56 Values Quickly collects and converts logs through LTS. Processes and analyzes data in response to log events in a serverless architecture, which features automatic scaling, no operation and maintenance, and pay-per-use billing. Sends alarm notifications through SMN. 5 Analyzing Logs in Real Time While Integrating with LTS 5.2 Preparation Collecting and Storing Logs Create a log group, for example, polo.guoying, on the LTS console. For details, see Creating a Log Group. Create a log topic, for example, lts-topic-gfz3 on the LTS console. For details, see Creating a Log Topic. Configure an agent to collect logs from servers, such as ECSs, to a specified log group, as shown in Figure 5-2. For details, see Installing and Configuring Telescope. Figure 5-2 Configuring an agent Pushing Alarm Messages Create a topic named fss_test on the SMN console. For details, see Creating a Topic. Add subscriptions to the fss_test topic to push alarm messages to specified endpoints. For details, see Adding Subscriptions. Define an environment variable named SMN_Topic with value fss_test to push alarm messages to the subscription endpoints under the fss_test topic. NOTE Alarm messages of a subscribed topic can be pushed through , SMS messages, and HTTP/ HTTPS. In this example, when log events trigger the specified function through an LTS trigger, the function filters alarm logs and pushes alarm message to the subscription endpoints. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 52

57 5 Analyzing Logs in Real Time While Integrating with LTS Processing Cloud Data Create an OBS bucket and object, and configure event notifications. 1. Create a bucket and an object on the OBS console, as shown in Figure 5-3. For details, see Creating a Bucket. Figure 5-3 Creating a bucket NOTE Name the bucket as logstore and the object as log.txt to store log data. 5.3 Building a Program Creating a Function You can download and use the alarm log extracting program package provided in this example. Create a function by uploading the sample code package to extract logs, as shown in Figure 5-4. For details, see Creating a Function. Figure 5-4 Creating a function This function performs Base64 decoding on received log event data, extracts alarm logs containing keyword WRN, WARN, ERR, or ERROR, and then stores the extracted logs in Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 53

58 5 Analyzing Logs in Real Time While Integrating with LTS the specified OBS bucket. Set log extraction conditions based on the content of your service logs. Setting Environment Variables On the Configuration tab page of the preceding function, set environment variables to pass the bucket address, bucket name, and object name, as shown in Table 5-1. Table 5-1 Environment variables Environment Variable obs_address obs_store_bucket obs_store_objname SMN_Topic Description OBS endpoint. To obtain the OBS endpoint, see Regions and Endpoints. For example, the OBS endpoint in North China is obs.cnnorth-1.myhwclouds.com. Name of the target bucket for storing logs. Name of the target file for storing logs. SMN topic. For details, see Environment Variables. The environment variables are shown in Figure 5-5. Figure 5-5 Setting environment variables 5.4 Adding an Event Source Create an LTS trigger by using the log group and log topic created in Preparation, and configure the trigger information according to Figure 5-6. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 54

59 5 Analyzing Logs in Real Time While Integrating with LTS Figure 5-6 Creating an LTS trigger When the accumulated log size or log retention period meets a specified threshold, LTS log data is consumed, which triggers the function associated with the log group. 5.5 Processing Log Data notifications will be received from SMN if alarm logs containing keyword WRN, WARN, ERR, or ERROR are generated, as shown in Figure 5-7. You can also view details of the alarm logs by opening the log.txt file in the specified bucket, as shown in Figure 5-8. Figure 5-7 notification Figure 5-8 Alarm log details On the Monitoring tab page of the function, check the number of invocations, as shown in Figure 5-9. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 55

60 5 Analyzing Logs in Real Time While Integrating with LTS Figure 5-9 Function metrics 5.6 Other Application Scenarios This example demonstrates how to process cloud logs and push alarm messages by using FunctionGraph and LTS, and how to store filtered alarm logs in a specified OBS bucket. The combination of FunctionGraph and LTS also applies to the following scenarios: 1. Leverage the function orchestration capability of FunctionGrph to analyze and process log data in a bucket based on an OBS trigger for more complex purposes. 2. Periodically analyze and process log data in a bucket as required based on a timer trigger. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 56

61 6 Analyzing Login/Logout Security While Integrating with CTS 6 Analyzing Login/Logout Security While Integrating with CTS 6.1 Introduction Scenario Collect real-time records of operations performed by HUAWEI CLOUD users on cloud resources. Create a Cloud Trace Service (CTS) trigger to obtain records of subscribed cloud resource operations; analyze and process the operation records, and report alarms. Use SMN to push alarm messages to service personnel by SMS message or . The processing workflow is shown in Figure 6-1. Figure 6-1 Processing workflow Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 57

62 Values Quickly analyzes operation records collected by CTS and filters out operations from specified IP addresses. Processes and analyzes data in response to log events in a serverless architecture, which features automatic scaling, no operation and maintenance, and pay-per-use billing. Sends alarm notifications through SMN. 6 Analyzing Login/Logout Security While Integrating with CTS 6.2 Preparation Enabling CTS Configure the tracker on CTS, as shown in Figure 6-2. For details, see Configuring a Tracker. Figure 6-2 Configuring the tracker Creating an Agency Step 1 Log in to the HUAWEI CLOUD management console, and choose Identity and Access Management. Choose Agency, and click Create Agency. Step 2 Set the agency information, as shown in Figure 6-3. For Agency Name, enter serverless_trust. For Agency Type, select Cloud service. For Cloud Service, select FunctionGraph. For Validity Period, select Permanent. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 58

63 6 Analyzing Login/Logout Security While Integrating with CTS For Permissions, click Modify on the right of the Global service > Global, Global service > OBS, and CN North-Beijing1 > cn-north-1 rows, and choose Base. Figure 6-3 Creating an agency Step 3 Click OK. ----End Pushing Alarm Messages Create a topic named cts_test on the SMN console. For details, see Creating a Topic. Add subscriptions to the cts_test topic to push alarm messages to specified endpoints. For details, see Adding Subscriptions. NOTE Alarm messages of a subscribed topic can be pushed through , SMS messages, and HTTP/ HTTPS. In this example, when operation log events trigger the specified function, the function filters operations not from an IP address whitelist and pushes alarm message to the subscription endpoints. 6.3 Building a Program You can download and use the alarm log analysis program package provided in this example. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 59

64 6 Analyzing Login/Logout Security While Integrating with CTS Creating a Function Create a function by uploading the sample code package to analyze and process operation logs, as shown in Figure 6-4. For details, see Creating a Function. Figure 6-4 Creating a function This function analyzes received operation records, filters logins or logouts from unauthorized IP addresses using a whitelist, and sends alarms under a specified SMN topic. This function can be used to build an account security monitoring service. Setting Environment Variables On the Configuration tab page of the function details page, set the environment variables listed in Table 6-1. Table 6-1 Environment variables Environment Variable SMN_Topic RegionName IP Description SMN topic. Region name. IP address whitelist. For details, see Environment Variables. Set the environment variables, as shown in Figure 6-5. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 60

65 6 Analyzing Login/Logout Security While Integrating with CTS Figure 6-5 Setting environment variables 6.4 Adding an Event Source Create a CTS trigger, as shown in Figure 6-6. Figure 6-6 Creating a CTS trigger CTS records the logins and logouts of users on IAM. 6.5 Processing Operation Records The function runs in response to account logins and logouts to filter those not from the IP address whitelist, and sends a message or through SMN, as shown in Figure 6-7. Figure 6-7 notification Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 61

66 6 Analyzing Login/Logout Security While Integrating with CTS The contains the unauthorized IP address and user operation (login or logout). On the Monitoring tab page of the function, check the number of invocations, as shown in Figure 6-8. Figure 6-8 Function metrics 6.6 Other Application Scenarios This example demonstrates how to analyze and process cloud logs and push alarm messages by using FunctionGraph and CTS. The combination of FunctionGraph and CTS also applies to the following scenario: Leverage the function orchestration capability of FunctionGrph to collect resource operations of users, and send operation logs to users by , forming a secure logging system. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 62

67 7 Constructing a Video Processing Service 7 Constructing a Video Processing Service 7.1 Introduction Background The best practice for HUAWEI CLOUD FunctionGraph guides you through video processing based on a workflow. Many videos are uploaded to a website in various formats, and the website backend needs a unified format to store these videos. Therefore, video format conversion is required. Problems and Requirements Current implementation Virtual machines (VMs) are purchased to build a video processing service, which must be maintained regularly. Figure 7-1 shows the video processing service that has been orchestrated using code. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 63

68 7 Constructing a Video Processing Service Figure 7-1 Current implementation Requirements FunctionGraph-based Solution Flexible process orchestration: The current video processing service is orchestrated using code, making maintenance difficult because incorporating service changes is complex. Easy fault locating: When video conversion fails, it is difficult to locate the faulty step in the relevant video. Lower O&M workloads and costs: The VMs are idle for most of the time. The enterprise is not willing to increase the O&M labor, increasing the workloads of the existing O&M personnel. Figure 7-2 shows a FunctionGraph-based serverless solution. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 64

69 7 Constructing a Video Processing Service Figure 7-2 Serverless solution The entire process is based on serverless cloud services, and requires no VMs or O&M. The video processing service is no longer maintained through code but hosted on a FunctionGraph workflow. After a few changes, the existing code can be converted to a small video processing function and hosted on a FunctionGraph workflow. To add more functions, you only need to compile independent small functions and simply orchestrate them in the workflow. Definition of the Video Processing Workflow Figure 7-3 shows the video processing workflow. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 65

70 7 Constructing a Video Processing Service Figure 7-3 Video processing workflow 7.2 Preparation Creating an OBS Bucket gettype: obtains video format types. checktype: identifies video format types. avi2mp4: invokes a small function to convert AVI videos to MP4 videos. flv2mp4: invokes a small function to convert FLV videos to MP4 videos. notifyuser: sends a notification to the user if video conversion fails. screenshots: invokes a small function to capture a screenshot from each processed video. Before building and orchestrating programs, create an OBS bucket and agency. The OBS bucket is used to store program code, video files, and video screenshots. The agency is used to delegate FunctionGraph to access required cloud sources. The bucket information is shown in the following table. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 66

71 7 Constructing a Video Processing Service Table 7-1 Bucket information Parameter Description Value Region Set to the region where FunctionGraph resides. CN North-Beijng1 Bucket Name - serverless Folder 1 Stores videos to be processed. original Folder 2 Stores converted videos. transfer Folder 3 Stores video screenshots. screenshot Folder 4 Stores function code packages. code Step 1 Log in to the OBS console, and click Create Bucket. Step 2 Set the bucket information according to Figure 7-4. For Region, select CN North-Beijing1. For Bucket Name, enter serverless. For Storage Class, select Standard. For Bucket Policy, select Private. Figure 7-4 Creating a bucket Step 3 Creating folders. On the object details page of the serverless bucket, click Create Folder. Create four folders named as original, transfer, screenshot, and code, as shown in Figure 7-5. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 67

72 7 Constructing a Video Processing Service Figure 7-5 Bucket folders ----End Creating an Agency Step 1 Log in to the HUAWEI CLOUD management console, and choose Identity and Access Management. Choose Agency, and click Create Agency. Step 2 On the Create Agency page, set the agency information, as shown in Figure 7-6. For Agency Name, enter serverless_trust. For Agency Type, select Cloud service. For Cloud Service, select FunctionGraph. For Validity Period, select Permanent. For Permissions, click Modify on the right of the Global service > Global, Global service > OBS, and CN North-Beijing1 > cn-north-1 rows, and choose Base, as shown in Figure 7-7. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 68

73 7 Constructing a Video Processing Service Figure 7-6 Creating an agency Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 69

74 7 Constructing a Video Processing Service Figure 7-7 Modifying permissions Step 3 Click OK. ----End 7.3 Building Functions Building a Function for Obtaining Video Formats Function Parameters Create a function named gettype for obtaining video format types. Set the parameters listed in Table 7-2 for this function. Table 7-2 Parameters to be set for the gettype function Parameter Function Name Description gettype Runtime Python 2.7 Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 70

75 7 Constructing a Video Processing Service Parameter Code Description Code in the gettype.py file from the program package Memory 256 Timeout 30s Procedure Step 1 Step 2 Log in to the FunctionGraph console, choose Functions, and click Create Function. Set the function information by performing the following steps: 1. Set the basic information. For Function Name, enter gettype. For Agency, select serverless-trust created in Creating an Agency, as shown in Figure 7-8. Figure 7-8 Basic information 2. Set the code information. For Runtime, select Python 2.7. For Handler, retain the default value. For Code Entry Mode, select Edit code inline, and paste the code in the gettype.py file to the code box, as shown in Figure 7-9. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 71

76 7 Constructing a Video Processing Service Figure 7-9 Code information 3. Click Create Function. Step 3 On the gettype function details page, click the Configuration tab, and set the environment information. Select 256 for Memory and enter 30 for Timeout, as shown in Figure Figure 7-10 Environment information Step 4 Click Save. ----End Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 72

77 7 Constructing a Video Processing Service Building a Function for Converting Video Formats Function Parameters Create a function named convertvideo for converting video formats to MP4. Set the parameters listed in Table 7-3 for this function. Table 7-3 Parameters to be set for the convertvideo function Parameter Function Name Description convertvideo Runtime Java 8 Handler Code com.huawei.function.app.myhandle Package convertvideo.zip Memory 512 Timeout 300 Environment Variables dstpath:transfer/ Procedure Step 1 Step 2 Log in to the FunctionGraph console, choose Functions, and click Create Function. Set the function information by performing the following steps: 1. Set the basic information. For Function Name, enter convertvideo. For Agency, select serverless-trust created in Creating an Agency, as shown in Figure Figure 7-11 Basic information 2. Set the code information. For Runtime, select Java 8. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 73

78 7 Constructing a Video Processing Service For Handler, enter com.huawei.function.app.myhandle. For Code Entry Mode, select Upload file from OBS, paste the link of convertvideo.zip obtained in functiongraph_05_0120.xml#functiongraph_05_0120/ section , as shown in Figure Figure 7-12 Code information 3. Click Create Function. Step 3 On the convertvideo function details page, click the Configuration tab, and set the environment information. Select 512 for Memory and enter 300 for Timeout. For Environment Variables, enter dstpath:transfer/, as shown in Figure Figure 7-13 Environment information Step 4 Click Save. ----End Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 74

79 7 Constructing a Video Processing Service Function for Handling Video Conversion Failures Function Parameters Create a function named notifyuser for notifying users through SMN when format conversion fails. A null function is used to simulate messages because an individual account cannot send messages. Set the parameters listed in Table 7-4 for this function. Table 7-4 Parameters to be set for the notifyuser function Parameter Function Name Description notifyuser Runtime Python 2.7 Code Edit code inline Memory 128 Timeout 30 Environment Variables dstpath:transfer/ Procedure Step 1 Step 2 Log in to the FunctionGraph console, choose Functions, and click Create Function. Set the function information by performing the following steps: 1. Set the basic information. For Function Name, enter notifyuser. For Agency, select Use no agency, as shown in Figure Figure 7-14 Basic information 2. Set the code information. For Runtime, select Python 2.7. For Handler, retain the default value. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 75

80 7 Constructing a Video Processing Service For Code Entry Mode, select Edit code inline, enter the following code, as shown in Figure 7-15: # -*- coding:utf-8 -*- import json def handler (event, context): output = 'notify user' return output Figure 7-15 Code information 3. Click Create Function. Step 3 On the notifyuser function details page, click the Configuration tab, and set the environment information. Select 128 for Memory and enter 3 for Timeout, as shown in Figure Figure 7-16 Environment information Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 76

81 7 Constructing a Video Processing Service Step 4 Click Save. ----End Function for Capturing Video Screenshots Function Parameters Create a function named screenshot for capturing video screenshots. Set the parameters listed in Table 7-5 for this function. Table 7-5 Parameters to be set for the screenshot function Parameter Function Name Description screenshot Runtime Java 8 Handler Code com.huawei.function.app.myhandle Package screenshot.zip Memory 512 Timeout 300 Environment Variables dstpath:screenshot/ Procedure Step 1 Step 2 Log in to the FunctionGraph console, choose Functions, and click Create Function. Set the function information by performing the following steps: 1. Set the basic information. For Function Name, enter screenshot. For Agency, select serverless-trust created in Creating an Agency, as shown in Figure Figure 7-17 Basic information Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 77

82 7 Constructing a Video Processing Service 2. Set the code information. For Runtime, select Java 8. For Handler, enter com.huawei.function.app.myhandle. For Code Entry Mode, select Upload file from OBS, paste the link of screenshot.zip obtained in functiongraph_05_0120.xml#functiongraph_05_0120/section , as shown in Figure Figure 7-18 Code information 3. Click Create Function. Step 3 On the screenshot function details page, click the Configuration tab, and set the environment information. Select 512 for Memory and enter 300 for Timeout. For Environment Variables, enter dstpath:screenshot/, as shown in Figure Figure 7-19 Environment information Step 4 Click Save. ----End Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 78

83 7 Constructing a Video Processing Service 7.4 Orchestrating Functions Create a workflow by using code in the workflow-videoconvert.json file. The procedure is as follows: Step 1 Step 2 Step 3 Log in to the FunctionGraph console, choose Workflows > Workflow List, and click Create Workflow. Set the workflow information. 1. For Workflow Name, enter workflow-serverless. 2. For Template, select Create from scratch. Orchestrate functions by performing the following steps: 1. Click to switch to the code pane, as shown in Figure Figure 7-20 Show Code 2. Enter the following code of the workflow-videoconvert.json file in the code box: { "states": { "gettype": { "start": true, "type": "OPERATION", "action-mode": "SEQUENTIAL", "actions": [ { "action-name": "gettypeaction", "function": "urn:fss:cnnorth-1:bb bb74682b9ca82a8803b60d8:function:default:gettype", "invocation-mode": "SYNC" } ], "next-state": "checktype" }, "checktype": { "type": "SWITCH", "choices": [ { "path": "$.f", "value": "1", "next-state": "avi2mp4" }, { "path": "$.f", "value": "2", "next-state": "flv2mp4" } ], "default": "avi2mp4" }, "avi2mp4": { "type": "OPERATION", "action-mode": "SEQUENTIAL", Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 79

84 7 Constructing a Video Processing Service "actions": [ { "action-name": "convert-avi", "function": "urn:fss:cnnorth-1:bb bb74682b9ca82a8803b60d8:function:default:convertvide o", "invocation-mode": "SYNC", "results": [ { "match": "MatchAny", "retry-interval": 1, "max-retry": 3, "next-state": "notifyuser" } ] } ], "next-state": "screenshot" }, "flv2mp4": { "type": "OPERATION", "action-mode": "SEQUENTIAL", "actions": [ { "action-name": "convert-flv", "function": "urn:fss:cnnorth-1:bb bb74682b9ca82a8803b60d8:function:default:convertvide o", "invocation-mode": "SYNC", "results": [ { "match": "MatchAny", "retry-interval": 1, "max-retry": 3, "next-state": "screenshot" } ] } ], "next-state": "screenshot" }, "notifyuser": { "type": "OPERATION", "action-mode": "SEQUENTIAL", "actions": [ { "action-name": "notification", "function": "urn:fss:cnnorth-1:bb bb74682b9ca82a8803b60d8:function:default:notify_user ", "invocation-mode": "SYNC" } ], "next-state": "end" }, "screenshot": { "type": "OPERATION", "action-mode": "SEQUENTIAL", "actions": [ { "action-name": "screenshot", "function": "urn:fss:cn- Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 80

85 7 Constructing a Video Processing Service north-1:bb bb74682b9ca82a8803b60d8:function:default:screenshot", "invocation-mode": "SYNC" } ], "next-state": "end" }, "end": { "type": "END" } } } 3. Orchestrate functions. In the gettypeaction state, add the gettype function to the function field, as shown in Figure Figure 7-21 Orchestrating functions Table 7-6 shows the mapping relationships between the states and functions. Table 7-6 Mapping relationships State gettypeaction convert-avi convert-flv notification screenshot Function gettype convertvideo convertvideo notifyuser screenshot 4. Confirm the workflow information and click Create. ----End Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 81

86 7 Constructing a Video Processing Service 7.5 Adding an Event Source Creating Workflow Triggers Create two workflow triggers to trigger the workflow when detecting uploaded AVI or FLV videos. Step 1 Step 2 Log in to the FunctionGraph console, choose Workflows > Workflow Triggers, and click Create Trigger. In the Create Trigger dialog box, set the parameters listed in Table 7-7, as shown in Figure The parameters with an asterisk (*) are mandatory. Table 7-7 Parameters for creating a trigger (AVI) Parameter *Trigger Name Description *Event Source *Bucket *Events Prefix Suffix Description Enter trigger-obs-avi. This example does not require a description. Select OBS. OBS bucket to be used as the event source. Select the serverless bucket created in Creating an OBS Bucket. Select events for triggering your workflow. Select Put and Post. When files are updated or uploaded in the serverless bucket, the workflow is triggered. Keyword for limiting notifications about objects whose names start with the matching characters. This limit can be used to filter the names of OBS objects. In this example, leave this field blank. Keyword for limiting notifications about objects whose names end with the matching characters. This limit can be used to filter the names of OBS objects. The workflow will be triggered only when AVI files are operated. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 82

87 7 Constructing a Video Processing Service Figure 7-22 Creating a trigger (for detecting AVI files) Step 3 Step 4 Click OK. Repeat steps 1 to 3 to create another trigger for detecting FLV files. Set the parameters listed in Table 7-8, as shown in Figure The parameters marked with an asterisk (*) are mandatory. Table 7-8 Parameters for creating a trigger (FLV) Parameter *Trigger Name Description *Event Source Description Enter trigger-obs-flv. This example does not require a description. Select OBS. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 83

88 7 Constructing a Video Processing Service Parameter *Bucket *Events Prefix Suffix Description OBS bucket to be used as the event source. Select the serverless bucket created in Creating an OBS Bucket. Select events for triggering your workflow. Select Put and Post. When files are updated or uploaded in the serverless bucket, the workflow is triggered. Keyword for limiting notifications about objects whose names start with the matching characters. This limit can be used to filter the names of OBS objects. In this example, leave this field blank. Keyword for limiting notifications about objects whose names end with the matching characters. This limit can be used to filter the names of OBS objects. The workflow will be triggered only when FLV files are operated. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 84

89 7 Constructing a Video Processing Service Figure 7-23 Creating a trigger (for detecting FLV files) ----End Binding the Workflow Triggers Step 1 Step 2 On the Workflow List page, click the workflow-serverless workflow. On the workflow details page that is displayed, click the Triggers tab. Step 3 Click Bind Trigger, as shown in Figure Figure 7-24 Binding workflow triggers Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 85

90 7 Constructing a Video Processing Service Step 4 In the Bind Trigger dialog box, set the parameters listed in Table 7-9. Table 7-9 Parameters for binding a trigger Parameter Trigger Type Trigger Description Click Select existing trigger. Select the trigger-obs-avi trigger created in Creating Workflow Triggers. Figure 7-25 Binding the trigger for detecting AVI files Step 5 Click OK. Step 6 Repeat steps 3 to 5 to bind the trigger for detecting FLV files, as shown in Figure Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 86

91 7 Constructing a Video Processing Service Figure 7-26 Binding the trigger for detecting FLV files Step 7 Check the two triggers bound to the workflow, as shown in Figure Figure 7-27 Bound triggers NOTE ----End After the workflow is bound with the two triggers, it will be triggered only when AVI or FLV files are uploaded or updated to the serverless bucket. Uploading or updating files in other formats (such as ZIP and TXT) will not trigger the workflow. Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 87

92 7 Constructing a Video Processing Service 7.6 Processing Videos Upload a video file to the original folder of the serverless bucket to trigger the workflow. View the execution details of each state on the workflow instance page, and check the processed video file in the relevant folder of the serverless bucket. The procedure is as follows: Step 1 Log in to the OBS console. On the object details page of the serverless bucket, upload a video file to the original folder, as shown in Figure Figure 7-28 Uploading a video file Step 2 Log in to the FunctionGraph console, and choose Workflows > Workflow List. Click the workflow-serverless workflow, click the Instances tab, and view the running status of each function, as shown in Figure Figure 7-29 Workflow instance execution details Issue 05 ( ) Copyright Huawei Technologies Co., Ltd. 88