Advancing State-of-the-Art of Autonomous Vehicles and Robotics Research using AWS GPU Instances

Transcription

1 Advancing State-of-the-Art of Autonomous Vehicles and Robotics Research using AWS GPU Instances Adrien Gaidon - Machine Learning Lead, Toyota Research Institute Mike Garrison - Senior Systems Engineer, Toyota Research Institute Chetan Kapoor - Senior Product Manager - EC2, Amazon Web Services 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

2 AWS: Our mission Put machine learning in the hands of every developer and data scientist 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

3 AWS ML Stack Application Services Rekognition Polly Lex Comprehend Transcribe Translate Platform Services SageMaker DL AMI Spark & EMR Mechanical Turk AWS Deep Learning AMI Frameworks & Infrastructure TensorFlow Apache MXNet Cognitive Toolkit Caffe2 & Caffe PyTorch Keras Gluon GPU CPU IoT (Greengrass) Mobile 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

4 Note Update Slide AWS ML Stack Application Services Rekognition Polly Lex Comprehend Transcribe Translate Platform Services SageMaker DL AMI Spark & EMR Mechanical Turk AWS Deep Learning AMI Frameworks & Infrastructure TensorFlow Apache MXNet Cognitive Toolkit Caffe2 & Caffe PyTorch Keras Gluon GPU CPU IoT (Greengrass) Mobile 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

5 EC2 Compute Instance Types General Purpose Compute Optimized Storage and IO Optimized Memory Optimized Accelerated Computing M5 C5 I3 X1e G3 F1 T2 C4 D2 R4 P3 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

6 EC2 Accelerated Computing Instances P3 GPU Compute Instance Up to 8 NVIDIA V100 GPUs in a single instance, with NVLink for peer-to-peer GPU communication Supporting a wide variety of use cases including deep learning, HPC simulations, financial computing, and batch rendering G3: GPU Graphics Instance Up to 4 NVIDIA M60 GPUs, with GRID Virtual Workstation features and licenses Designed for workloads such as 3D rendering, 3D visualizations, graphics-intensive remote workstations, video encoding, and virtual reality applications F1: FPGA instance Up to 8 Xilinx Virtex UltraScale+ VU9P FPGAs in a single instance. Programmable via VHDL, Verilog, or OpenCL. Growing marketplace of pre-built application accelerations. Designed for hardware-accelerated applications including financial computing, genomics, accelerated search, and image processing P3 G3 F1 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

7 Amazon EC2 P3 Instances (October 2017) O n e o f t h e f a s t e s t, m o s t p o w e r f u l G P U i n s t a n c e s i n t h e c l o u d Up to eight NVIDIA Tesla V100 GPUs 1 PetaFLOPs of computational performance Up to 14x better than P2 300 GB/s GPU-to-GPU communication (NVLink) 9X better than P2 16GB GPU memory with 900 GB/sec peak GPU memory bandwidth 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

8 P3 Instances Details Instance Size GPUs GPU Peer to Peer vcpus Memory (GB) Network Bandwidth EBS Bandwidth On-Demand Price/hr* 1-yr RI Effective Hourly* 3-yr RI Effective Hourly* P3.2xlarge 1 No 8 61 Up to 10Gbps 1.7Gbps $3.06 $1.99 (35% Disc.) $1.23 (60% Disc.) P3.8xlarge 4 NVLink Gbps 7Gbps $12.24 P3.16xlarge 8 NVLink Gbps 14Gbps $24.48 $7.96 (35% Disc.) $15.91 (35% Disc.) $4.93 (60% Disc.) $9.87 (60% Disc.) Regional Availability P3 instances are generally available in AWS US East (Northern Virginia), US East (Ohio), US West (Oregon), EU (Ireland), Asia Pacific (Seoul), Asia Pacific (Tokyo), AWS GovCloud (US) and China (Beijing) Regions. Framework Support P3 instances and their V100 GPUs supported across all major frameworks (such as TensorFlow, MXNet, PyTorch, Caffe2 and CNTK) 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

9 AWS P3 vs P2 Instance G P U P e r f o r m a n c e C o m p a r i s o n P2 Instances use K80 Accelerator (Kepler Architecture) P3 Instances use V100 Accelerator (Volta Architecture) Mixed/FP16 Perf (TFLOPS) FP32 Perf (TFLOPS) 1.7X FP64 Perf (TFLOPS) 2.6X Resnet-50 8 GPU (Images/sec) X over K80 s max perf X 20 0 FP32 K80 P100 V K80 P100 V K80 P100 V K80 P100 V , Amazon Web Services, Inc. or its Affiliates. All rights reserved.

10 AWS Storage Options EFS EC2+EBS Amazon S3 Amazon Glacier Highly available, multi-az, fully managed networkattached elastic file system. For near-line, highlyavailable storage of files in a traditional NFS format (NFSv4). Create a single-az shared file system using EC2 and EBS, with third-party or open source software (e.g., ZFS, Intel Lustre, etc). For near-line storage of files optimized for high I/O performance. Secure, durable, highly-scalable object storage. Fast access, low cost. For long-term durable storage of data, in a readily accessible get/put access format. Secure, durable, long term, highly costeffective object storage. For long-term storage and archival of data that is infrequently accessed. Use for read-often, temporary working storage Use for high-iops, temporary working storage Primary durable and scalable storage for data Use for long-term, lower-cost archival of data

11 AWS Deep Learning AMI Get started quickly with easy-to-launch tutorials Hassle-free setup and configuration Pay only for what you use no additional charge for the AMI Accelerate your model training and deployment Support for popular deep learning frameworks 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

12 Agenda TRI s Mission The Data Problem The ML Bleeding Edge Infrastructure at TRI 12

13 TRI: Mission Our mission is to improve the quality of human life through advances in artificial intelligence, automated driving, and robotics. 13

14 l Source: New York Times

15 TRI s Automated Driving Vision Create a safe, robust, and versatile automated driving system that supports both series and parallel operating modes. 15

16 TRI Autonomy Software: One System, Two Modes 16

17 SAE s Levels of Automated Driving LEVEL Name Control Monitoring Vigilance Fallback Handoff Operating Conditions 0 Unassisted Human 1 Driver Assist Human and AD System Human Human Some 2 Partial Automation AD System Human Human (0 warning) Some 3 Conditional Automation AD System AD System Human (adequate warning) (assumed 15 sec) Some 4 High Automation AD System AD System AD System Some 5 Full Automation AD System AD System AD System All 17

18 SAE s Levels of Automated Driving LEVEL Name Control Monitoring Vigilance Fallback Handoff Operating Conditions 0 Unassisted Human 1 Driver Assist Human and AD System Human Human Some 2 Partial Automation AD System Human Human (0 warning) Some 3 Conditional Automation AD System AD System Human (adequate warning) (assumed 15 sec) Some 4 High Automation AD System AD System AD System Some 5 Full Automation AD System AD System AD System All 18

19 Paths to Automated Driving Emilio Frazzoli, CTO, nutonomy, MIT self-driving cars course 19

20 But There s a Different Way F-16 F-22 20

21 TRI: How We Differentiate 1. Guardian 1. Data See our YouTube channel for Cool Progress Reports! 21

22 Agenda TRI s Mission The Data Problem The ML Bleeding Edge Infrastructure at TRI 22

23 How Much Data is Enough? Goal: Machine Learning for robust 3P's (Perception, Prediction, Planning) Need: lots of labeled data ("the drug of Deep Learning") Why? YMYD: Your Model is (as good as) Your Data Both for Training (cf. Data Processing Inequality, Information Bottleneck) and of course for Validation/Testing 23

24 24

25 It takes time to find corner cases

26 "Crowdsourced steering doesn't sound quite as appealing as self driving" 26

27 The Trillion Mile Challenge: Simulation Data Gathering from Ordinary Drivers environment behavior Cloud Simulator validation Test Cars with Professional Drivers millions of miles low-res maps billions of miles high-res maps millions of miles 27

28 Agenda TRI s Mission The Data Problem The ML Bleeding Edge Infrastructure at TRI 28

29 Example Deep KoiNet 29

30 DL on AWS EC2 instances ImageNet (AI's Nürburgring) in the Cloud? - <15h on a single p3.16xlarge (cf. DAWNBench) - <1h on ~12 instances (more on that later in ) 30

31 DL on AWS EC2 instances ImageNet (AI's Nürburgring) in the Cloud? - <15h on a single p3.16xlarge (cf. DAWNBench) - <1h on ~12 instances (more on that later in ) No rusty metal: TRI Scales Up on the Bleeding Edge - p3 instances: V100! - latest CUDA, PyTorch,... 31

32 Bleeding Edge DL on AWS The Deep Stack changes fast! - 1st layer: new GPUs, OS, drivers - 2nd layer: Docker, CUDA9, AMI - 3rd layer: PyTorch, AWS Services - 4th layer: DL algorithms 32

33 Bleeding Edge DL on AWS The Deep Stack changes fast! - 1st layer: new GPUs, OS, drivers - 2nd layer: Docker, CUDA9, AMI - 3rd layer: PyTorch, AWS Services - 4th layer: DL algorithms You will be cut: med support is critical! - Continuous Integration (yes, for ML!) - Benchmarking & Dashboards - Good Docs & Customer Support - Osmosis between External and Internal Infra Ninjas 33

34 Agenda TRI s Mission The Data Problem The ML Bleeding Edge Infrastructure at TRI 34

35 Infrastructure Engineering Team Developers & Researchers Infrastructure 35

36 Vehicle Data 36

37 Data Pipeline 37

38 Fleet Operations Dashboard 38

39 TRI Toolbox 39

40 TRI Toolbox AWS 40

41 TRI Toolbox Frameworks 41

42 TRI Toolbox Open Source 42

43 Conclusion TRI s Mission + Aim to Provide: Safety, Robustness, Versatility Chauffeur & Guardian The Data Problem: How much is enough? The Trillion Mile Challenge and the Need for Sim The Infrastructure Challenge: Scaling AI Up in the Cloud We are hiring! 43