Use Tesla to provide first GPU VM Service in China

Transcription

1 Use Tesla to provide first GPU VM Service in China Feng Zhu 专注 服务 中立

2 Outline UCloud Introduction K80 GPU VM P40 GPU VM UCloud GPU PaaS Service: UAI-Service UCloud GPU ecosystem 2

3 About UCloud Top 3 IaaS Provider in China Found in 2012 HQ in Shanghai Served 50,000+ Enterprise 3

4 Data Centers LA DC Frankfurt BJ1 BJ2 SH1 SH2 Seoul GZ HK TW ZJ Bangkok SG 14 Global Regions 4

5 5 Compute UHost SSD UHost GPU UHost UPHost UHybrid UGC Network EIP ULB VPC UDPN BWPackage UConnect Storage UDisk UDB UMem UFile UFS UDataArk CDN/Media UCDN ULive UVideo BigData UHadoop UKafka UDDP Security UADS UEWAF WAF IPS USSL UHAS UMarket Application Configurati on Backup Monitor Security Service Pre-Sales Post-Sales PM Training UCloud Product Line

6 GPU Timeline 2012 UCloud founded K80 GPU VM K80 GPU Physical Machine P40 GPU VM 2017.? P40 GPU Physical Machine 6

7 GPU Decision: Virtualization PCI Pass-through Single VM occupy GPU device Performance guaranteed Limited by PCI device number (K80 = 2 * GPU) No license Grid GPU device shared by multiple VMs No guarantee on performance no limit for PCI device License X

8 VM Advantage Flexibility for VM configuration CPU Memory Disk size GPU number are all flexible SDN network flexible Main OS all supported, Win/Linux CentOS 6.5/CentOS 7.0/Ubuntu 14.04/Ubuntu 12.04/Gentoo 2.2/Win 2008/Win 2012 Fast Deployment Based on self-defined image, can deploy 1000 VMs in 1 minute

9 VM Performance Degrade Using Pass-through Technology, almost no degradation Degradation Virtualization Bare Metal Computing 99.3% 100% Bandwidth 98% 100% % % 99.00% 98.00% Virtualization Bare Metal 97.00% Compute BW

10 UCloud GPU Virtualization DL test Caffe Performance (Ubuntu) Cases iters GPU(secs) CPU(secs) Speedup LeNet on MNIST cifar Fine-tuning for style recognition Siamese Network Training GPU CPU 10

11 UCloud GPU Virtualization DL test(2) Theano/Keras (Ubuntu) Cases iters GPU(secs) CPU(secs) Speedup imdb_lstm.py imdb_cnn.py imdb_cnn_lstm.py addition_rnn.py cifar10_cnn.py babi_rnn.py mnist_cnn.py mnist_irnn.py mnist_mlp.py

12 K80 Physical Machine Hardware Specification GPU Tesla K80 CPU Intel E5 2630*2 Memory 192G Disk 2T SSD Networking 10G NIC*4 12

13 VM Configuration - K80 VM VM GPU GPU GPU CPU 4C 8C 16C Memory 8G 16G 32G 64G Disk 100G - 1T 13

14 Flexible VM Save Cost Configuration Fixed Flexible CPU 16C 4C Memory 96G 8G Disk 1T 100G GPU 1 1 Price 4,300 CNY/month 2,200 CNY/month USD Price $615/m $315/m Flexible Fixed GPU 2GPU

15 GPU VM Features VM VPC Networking Image Self-defined images, deploy 1000 VMs in 1 min Snapshot Data backup DataArk 24 continuous data protection, call rollback to any second Hotfix Kernel patch without system shutdown Re-scale Resize CPU/Memory/Disk anytime 15

16 Storage Solution VM Disk Local SSD disk UDisk NAS, no limit on device numbers UFS NFS file system UFile Object storage UArchive Low cost cloud archive 16

17 Create GPU VM 17

18 Create GPU VM 18

19 Create GPU VM 19

20 P40 Physical Machine Hardware Specification GPU Tesla P40*4 CPU Intel E5 2650*2 Memory 256G Disk 3T SSD Networking 10G NIC*4 20

21 VM Configuration - P40 VM VM VM VM GPU GPU GPU GPU GPU GPU GPU GPU GPU CPU 4C 8C 16C 24C 32C GPU Memory 8G 16G 32G 64G 96G 128G Disk 100G - 1T 21

22 P40 Price Configuration Spec 1 Spec 2 CPU 4C 32C Memory 8G 128G Disk 100G 1T GPU 1 4 Price 4,000 CNY/month 18,200 CNY/month USD Price $570/m $2,600/m K80 P Min Max

23 UAI-Service Overview User SDK/Web TensorFlow Keras MXNet Test Env AI Products Interface Distributed Model Training Online Service AI Products Resources FPGA GPU CPU Storage 23

24 Distributed Training Layout Features User SDK/Web Deploy Eval TensorFlow Keras D & R Inference Test Env MXNet GPU Resources CPU Storage /task1/code /data /model /log 24

25 Distributed Training Process 3.Deploy 2.Test & Eval Deploy SDK/Web Test Env PS PS Worker Worker Worker Resources GPU CPU 1.Upload Storage /task1/code /data /ckpt /log Tester VM Perf 25

26 Online Inference Layout Online Inference System User SDK/Web Deploy Running Eval TensorFlow Keras Test Env MXNet Storage /task1/code /data /ckpt /log Docker Images Resources FPGA GPU CPU 26

27 Online Inference Process 3.Deploy 2.Test & Eval Deploy SDK/Web Test Env ULB Docker Docker Docker Docker Resource 1.Upload Storage /task1/code /ckpt Tester Docker Perf 27

28 Online Inference API/SDK Deploy ULB AB Test Scalable User Service Docker Service Docker Docker Resource Service Docker Perf report Model update Rollback 28

29 GPU Scenario Deep Learning Advertisement CTR HPC Gene Sequencing Weather Forecasting Face Recognition Voice Recognition Picture\Film\ACG Rendering Maya Rendering Online Rendering 3Dmax Simulator Unity

30 GPU Scenario Training Big Data Advertisement CTR Online Service User Input Neural Network Model Face Recognition Neural Network Model Neural Network Model Compute-Intensive (GPU) Voice Recognition Output Compute-Sensitive (GPU)

31 GPU Scenario: Example CTR click-through-rate estimation Precise Advertise: according to personal historical data Ad bidding & ranking:ctr click rate * Bid Price Video UGC recommendation:watching ratio

32 GPU Scenario: Example CTR click-through-rate estimation x=[weekday=wednesday, Gender=Male, City=Shanghai] x=[0,0,1,0,0,0,0 0,1 0,0,1,0 0] CTR Estimate Model Percent of Click:25%

33 Thank You 33