Presented @ GTC 2017 & Edge-to-Cloud Compute with MxNet AWS & NVIDIA Aran Khanna, Software Developer, AWS Miro Enev, Solutions Architect, NVIDIA 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Edge-To-Cloud Compute Models Overview Advantages of Hub & Spoke Case Study / Demo References & Tools
IIoT & AI AI/DL is primed to disrupt the Industrial Sector.
NVIDIA IoT
NVIDIA IoT Robots & Smart Manufacturing Generative Design Build Design Operate Inspection & Efficiency Predictive Maintenance Maintain & Repair
Amazon IoT & GreenGrass
Amazon IoT Software & NVIDIA Hardware Key Functions Data Ingest Compressed Inference Full Inference / Trained Model Query Model Training Deployment Models Cloud <-> Edge Cloud <-> Hub <-> Edge
Emerging Deployment Model Trends: Latency, Cost, Reliability, Privacy/Security Policy Cloud Elastic GPU [ Tesla ] Hub/Gateway Jetson [ Tegra ] Edge Sensors [ RasPi ]
Handoff
Use Cases Of ML at The Edge Transportation AR & VR Industrials Smart Home Robotics Body Cameras
In This Demo I will: -Show you why you should care about Machine Learning at the Edge -Walk through the value of running deep learning applications with MXNet on a set of edge devices. -Demonstrate how AWS services make managing deep learning at the edge easier than ever.
MXNet Everywhere High End: NVIDIA Jetson - 64 Bit ARM Arch64 - CUDA - Ubuntu Low End: Raspberry Pi - 32 Bit ARMv7 - ARM NEON - Raspbian
Easy as 1 2...3... 1. 2. 3. git clone https://github.com/ dmlc/mxnet.git --recursive cd mxnet && make cd python && sudo python setup.py install
ImageNet Models - Well studied - Form the base models for most image tasks - Widely available pre-trained models Try it Yourself: http://data.mxnet.io/models
Running the Model # Load the network parameters from the cloud def load_model(model_url= http://data.mxnet.io/models/imagenet/squeezenet ): urllib.urlretrieve(model_url, model.zip ) unzip(model.zip) sym, arg_params, aux_params = mx.model.load_checkpoint( Image_Model', 0) mod = mx.mod.module(symbol=sym, context= mx.cpu()) mod.bind(for_training=false,data_shapes=[('data', (1,3,224,224))]) mod.set_params(arg_params, aux_params) return mod # Predict on an Image def predict(image_path, mod): img = cv2.cvtcolor(cv2.imread(filename), cv2.color_bgr2rgb) img = cv2.resize(img, (224, 224)) img = np.swapaxes(img, 0, 2) img = np.swapaxes(img, 1, 2) img = img[np.newaxis, :] mod.forward(batch([mx.nd.array(img)])) return mod.get_outputs()[0].asnumpy()
Making This Production Ready With AWS Management Aggregation Logging - Rollout changes - Coordinate local devices - Handle network and hardware failure - Gather and centralize data from the Edge - Connect to existing systems in the cloud - Monitor performance - Analyze uptime - Diagnose failures
Enter AWS IoT and AWS Greengrass
On The Pi: SqueezeNet Object Classifier Iandola et. Al
Linking the Pi to AWS # Load the network parameters model = load_model.imagenetmodel('synset.txt', 'squeezenet_v1.1') # Run Predictions on Images Captured With the Local Camera def Callback(client, userdata, message): args = json.load(message.payload) predictions, img = model.predict_from_cam(args.model) myawsiotmqttclient.publish( sdk/test/predict, predictions,1) if(predictions[0][0] < 0.8): myawsiotmqttclient.publish( sdk/test/escalate, img, 1) mymqttclient.connect() mymqttclient.subscribe( sdk/test/control", 1, Callback)
On The TX1: ResNet-152 Object Detector Szegedy et. Al
Linking the TX1 to AWS # Load the network parameters model = load_model.imagenetmodel('synset.txt', resnet-v2') pi_model = load_model.imagenetmodel('synset.txt', 'squeezenet_v1.1') # Run Predictions on Images Captured With the Local Camera def Callback(client, userdata, message): args = json.load(message.payload) predictions = model.predict_from_img(args.img. args.model) myawsiotmqttclient.publish( sdk/test/predict, predictions, 1) # Fine tune the squeezenet model to for redeployment to Pi if(predictions[0][0] > 0.8): pi_model.update_model(args.img, predictions) mymqttclient.connect() mymqttclient.subscribe( sdk/test/escalate", 1, Callback)
Local Learning Loop Poorly Classified Picture Fine Tune Model With Accurate Classification Updated Model
Thank You! Miro Enev menev@nvidia.com Aran Khanna arankhan@amazon.com