Edge-to-Cloud Compute with MxNet

Transcription

1 GTC 2017 & Edge-to-Cloud Compute with MxNet AWS & NVIDIA Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

2 Edge-To-Cloud Compute Models Overview Advantages of Hub & Spoke Case Study / Demo References & Tools

3 IIoT & AI AI/DL is primed to disrupt the Industrial Sector.

4 NVIDIA IoT

5 NVIDIA IoT Robots & Smart Manufacturing Generative Design Build Design Operate Inspection & Efficiency Predictive Maintenance Maintain & Repair

6 Amazon IoT & GreenGrass

7 Amazon IoT Software & NVIDIA Hardware Key Functions Data Ingest Compressed Inference Full Inference / Trained Model Query Model Training Deployment Models Cloud <-> Edge Cloud <-> Hub <-> Edge

8 Emerging Deployment Model Trends: Latency, Cost, Reliability, Privacy/Security Policy Cloud Elastic GPU [ Tesla ] Hub/Gateway Jetson [ Tegra ] Edge Sensors [ RasPi ]

9

10

11 Handoff

12 Use Cases Of ML at The Edge Transportation AR & VR Industrials Smart Home Robotics Body Cameras

13 In This Demo I will: -Show you why you should care about Machine Learning at the Edge -Walk through the value of running deep learning applications with MXNet on a set of edge devices. -Demonstrate how AWS services make managing deep learning at the edge easier than ever.

14 MXNet Everywhere High End: NVIDIA Jetson - 64 Bit ARM Arch64 - CUDA - Ubuntu Low End: Raspberry Pi - 32 Bit ARMv7 - ARM NEON - Raspbian

15 Easy as git clone dmlc/mxnet.git --recursive cd mxnet && make cd python && sudo python setup.py install

16 ImageNet Models - Well studied - Form the base models for most image tasks - Widely available pre-trained models Try it Yourself:

17 Running the Model # Load the network parameters from the cloud def load_model(model_url= ): urllib.urlretrieve(model_url, model.zip ) unzip(model.zip) sym, arg_params, aux_params = mx.model.load_checkpoint( Image_Model', 0) mod = mx.mod.module(symbol=sym, context= mx.cpu()) mod.bind(for_training=false,data_shapes=[('data', (1,3,224,224))]) mod.set_params(arg_params, aux_params) return mod # Predict on an Image def predict(image_path, mod): img = cv2.cvtcolor(cv2.imread(filename), cv2.color_bgr2rgb) img = cv2.resize(img, (224, 224)) img = np.swapaxes(img, 0, 2) img = np.swapaxes(img, 1, 2) img = img[np.newaxis, :] mod.forward(batch([mx.nd.array(img)])) return mod.get_outputs()[0].asnumpy()

18 Making This Production Ready With AWS Management Aggregation Logging - Rollout changes - Coordinate local devices - Handle network and hardware failure - Gather and centralize data from the Edge - Connect to existing systems in the cloud - Monitor performance - Analyze uptime - Diagnose failures

19 Enter AWS IoT and AWS Greengrass

20 On The Pi: SqueezeNet Object Classifier Iandola et. Al

21 Linking the Pi to AWS # Load the network parameters model = load_model.imagenetmodel('synset.txt', 'squeezenet_v1.1') # Run Predictions on Images Captured With the Local Camera def Callback(client, userdata, message): args = json.load(message.payload) predictions, img = model.predict_from_cam(args.model) myawsiotmqttclient.publish( sdk/test/predict, predictions,1) if(predictions[0][0] < 0.8): myawsiotmqttclient.publish( sdk/test/escalate, img, 1) mymqttclient.connect() mymqttclient.subscribe( sdk/test/control", 1, Callback)

22 On The TX1: ResNet-152 Object Detector Szegedy et. Al

23 Linking the TX1 to AWS # Load the network parameters model = load_model.imagenetmodel('synset.txt', resnet-v2') pi_model = load_model.imagenetmodel('synset.txt', 'squeezenet_v1.1') # Run Predictions on Images Captured With the Local Camera def Callback(client, userdata, message): args = json.load(message.payload) predictions = model.predict_from_img(args.img. args.model) myawsiotmqttclient.publish( sdk/test/predict, predictions, 1) # Fine tune the squeezenet model to for redeployment to Pi if(predictions[0][0] > 0.8): pi_model.update_model(args.img, predictions) mymqttclient.connect() mymqttclient.subscribe( sdk/test/escalate", 1, Callback)

24 Local Learning Loop Poorly Classified Picture Fine Tune Model With Accurate Classification Updated Model

25 Thank You! Miro Enev Aran Khanna