Push straight to prod

Transcription

1 Push straight to prod API development with R and TensorFlow at T-Mobile rstudio::conf January 17 th, 2019 Heather Nolis, Machine Learning Engineer, Jacqueline Nolis, Principal Data Scientist, Nolis PAGE 1 T-MOBILE

2 What does it mean to put into production? ANALYSIS Running code once to produce a result BUILD Writing code that is continuously running Putting code into production is letting customers interact with it PAGE 2 T-MOBILE

3 The project: Goal Use machine learning to improve the customer experience Scope customer care messaging PAGE 3 T-MOBILE

4 Scenario Customer sends message: This high bill shall not pass! Goal Prep customer care agent before first response: Current bill status Method Classify the message with machine learning: [bill breakdown] Improve the prediction with customer data: Recent account activity Signal strength Bill status [overdue] PAGE 4 T-MOBILE

5 Model creation with R PAGE 5 T-MOBILE

6 Model building workflow 1. rmarkdown for exploratory analysis 2. Save the model to flat files (and log the build with rmarkdown) 3. Show model off with a shiny demo PAGE 6 T-MOBILE

7 Neural networks with R, Keras, and TensorFlow A Convolutional Neural Network (CNN) processes initial customer message and customer data Unlock my phone. Recent order: YES CNN ACCOUNT UNLOCK ORDER PAGE 7 T-MOBILE

8 PAGE 8 T-MOBILE

9 Model deployment to prod PAGE 9 T-MOBILE

10 Choosing the language If you want machine learning in production you need Python literally everyone Option Analysis Modeling Deployment Verdict PAGE 10 T-MOBILE

11 Idea! Treat R like a full programming language (because it is one) Commit is made to AI repository Jenkins builds the image Marathon orchestrates container deployment Mesos replicates and hosts the containers PAGE 11 T-MOBILE

12 Turning R into an API: plumber Create an R file for the API endpoints Start the plumber service Now make HTTP requests to R! rest_controller.r /sum function(a, b){ as.numeric(a) + as.numeric(b) } main.r library(plumber) r <- plumb("rest_contoller.r") r$run(host=' ',port=80) PAGE 12 T-MOBILE

13 Behold: containers! Example Dockerfilae # start from the rocker/r-ver:3.5.0 image FROM rocker/r-ver:3.5.0 # install the linux libraries needed for plumber RUN apt-get update -qq && apt-get install -y \ libssl-dev \ libcurl4-gnutls-dev # install plumber RUN R -e "install.packages('plumber')" Dockerfile build instructions Image the result of the Dockerfile Container Hitting run on an image # copy everything from the current directory into the container COPY / / # open port 80 to traffic EXPOSE 80 # when the container starts, start the main.r script ENTRYPOINT ["Rscript", "main.r"] PAGE 13 T-MOBILE

14 Container Struggles: Python R Keras requires Python Added Python 3.6 (Anaconda) and Python Keras installation to Dockerfile Configured Reticulate to use correct Python PAGE 14 T-MOBILE

15 Container Struggles: Security Plumber does not support HTTPS Added an Apache 2 server to reroute Text needs encryption Sodium R library HTTPS HTTPS Docker Container Apache 2 HTTP HTTP R (plumber) PAGE 15 T-MOBILE

16 Container Struggles: Size Initial size (5+ GB) Switched Pythons (Anaconda Miniconda) Removed Rstudio (rocker/tidyverse rocker/rver:3.5.0) Removed unnecessary Linux, R, Python libraries PAGE 16 T-MOBILE

17 Status: UP R-native API container TensorFlow enabled Lightweight Secure PAGE 17 T-MOBILE

18 Lessons learned PAGE 18 T-MOBILE

19 R vs Python R & Plumber close to parity with Python & Flask R advantageous for quick data exploration Language was never the project failpoint PAGE 19 T-MOBILE

20 Working Together is Critical Sr. Manager ML Engineer ML Engineer Data Scientist Product Java Engineer Data Scientist Data Scientist PAGE 20 T-MOBILE

21 Where are we today PAGE 21 T-MOBILE

22 R in prod: Apple Business Chat Can I please pay my bill Customer PAGE 22 T-MOBILE

23 And you can be here too! PAGE 23 T-MOBILE

24 Materials: PAGE 24 T-MOBILE