OPERATIONALIZING MACHINE LEARNING USING GPU ACCELERATED, IN-DATABASE ANALYTICS

Transcription

1 OPERATIONALIZING MACHINE LEARNING USING GPU ACCELERATED, IN-DATABASE ANALYTICS 1

2 Why GPUs? A Tale of Numbers 100x Performance Increase Infrastructure Cost Savings Performance 100x gains over traditional RDBMS / NoSQL / In-Mem Databases Cores Modern GPUs can consist of up to cores compared to 32 in a CPU Costs 75% reduction in infrastructure costs, licensing, staff, etc. More with Less Increase performance, throughput, capability while minimizing the costs to support the business 75% 3000 vs 32 2

3 Why a GPU Database? Leverage Innovations in CPUs and GPUs Single Hardware Platform Simplified Software Stack 3

4 What are AI, ML, and Deep Learning? AI Deep Learning ML Predict y using function on data x 4

5 AI/ML/Deep Learning Cheat Sheet No shortage of techniques and programing languages 5

6 ML Cheat Sheet Python and SQL cover almost all the algorithms in that scary spider and Kinetica supports all Python libraries! 6

7 ML/AI/Deep Learning Lifecycle 7

8 ML/AI/Deep Learning Lifecycle Create, extract, transform, and process big data: batch and streams Apply ML to data. Model pre-processing Model execution Model post-processing Within an ecosystem of general analytics Supporting a range of human and machine consumers 9

9 Typical AI Process: High Latency, Rigid, Complex Tech Stack BUSINESS USERS DATA SCIENTISTS??? SPECIALIZED AI/ DATA SCIENCE TOOLS ENTERPRISES STRUGGLE TO MAKE AI MODELS AVAILABLE TO BUSINESS EXTRACT SUBSET EXTRACTING DATA FOR AI IS EXPENSIVE AND SLOW 9

10 Kinetica: A More Ideal AI Process BUSINESS USERS Monte Carlo Risk Custom Function 2 Custom Function 3 DATA SCIENTISTS UDFs API EXPOSES CUSTOM FUNCTIONS WHICH CAN BE MADE AVAILABLE TO BUSINESS USERS 10

11 Current Inefficient Use of Python python Interpreted Single threaded Clean, transform Flow: for each member Pre-process Model execute Post-process = 11

12 Optimized SQL and Python UDF with Kinetica SQL UDF python Pre-process Binary executable code Superior optimization declarative SQL Model execute Only essential imperative model code Not relational set processing = SQL Post-process Binary executable code Superior optimization Declarative SQL 12

13 Comprehensive Solution Architecture Major U.S Retailer Massive Stream Ingestion Apache Tomcat Applications Servers Spring Endpoint oriented architecture Horizontal elastic scaling Massive Fast Analytics Fast Streaming Projects Fast Analytics Projects KINETICA: 10 Node Cluster Head Node Worker 1 Worker 9 Various ETL/ELT Full Model Pipeline 1 Various ETL/ELT Prompts Project Full Model Pipeline N Fact and dimensions tables for various Use Cases Billions of rows 13

14 Use Case Example

15 MNIST: Simple Image Processing Use Case A Parametric ModelPython Using TensorFlow Model Training Set of image files stored in Kinetica Database Python UDF in Kinetica using TensorFlow Model Serving Python UDF in Kinetica using TensorFlow Input = table TFModel table. Output = table mnist_inference_out Model Analytics SQL! 15

16 Model Training & Inference Data Model: MPP Sharding UDF train_nd_udf.py Tom 0 mnist_training Shard 0 Machine 0 Rank 0 Tom 1 Tom 2 mnist_training Shard 1 mnist_training Shard 2 Tom 3 mnist_training Shard 3 Tom 4 mnist_training Shard 4 Machine 0 Rank 0 Tom 5 Tom 6 mnist_training Shard 5 mnist_training Shard 6 Tom 7 mnist_training Shard 7 TFModel Shard 0 TFModel Shard 1 TFModel Shard 2 TFModel Shard 3 TFModel Shard 4 TFModel Shard 5 TFModel Shard 6 TFModel Shard 7 mnist_inference Shard 0 mnist_inference Shard 1 mnist_inference Shard 2 mnist_inference Shard 3 mnist_inference Shard 4 mnist_inference Shard 5 mnist_inference Shard 6 mnist_inference Shard 7 UDF UDF UDF UDF UDF UDF UDF UDF mnist_inference_out Shard 0 mnist_inference_out Shard 1 mnist_inference_out Shard 2 mnist_inference_out Shard 3 mnist_inference_out Shard 4 mnist_inference_out Shard 5 mnist_inference_out Shard 6 mnist_inference_out Shard 7

17 Thank You! Come get your copy of the O Reilly Book at Booth G.01! info@kinetica.com