Machine Learning in the Process Industry. Anders Hedlund Analytics Specialist

Transcription

1 Machine Learning in the Process Industry Anders Hedlund Analytics Specialist

2 Artificial Specific Intelligence Artificial General Intelligence Strong AI Consciousness MEDIA, NEWS, CELEBRITIES Movie Industry TERMINATOR EX-MACHINA SELF DRIVING CARS NO KNOWN PATH LEADS HERE

3 Machine Learning Introduction Learning types Unsupervised (data mining) Supervised (model training for classification or regression (rating)) Reinforcement learning (behavioural psychology) Areas Measurements, image processing, audio, sentiment analysis, predictive maintenance, economics, spam filters Techniques Keep in mind Neural networks, k-nn, clustering, SVM, Bayesian network, decision trees, deep learning (neural network with more than one hidden layer) Trends (NN->SVM->Boosting->DCNN) ML is another method than linear regression or multivariate analysis Borrows ideas from statistics (like PCA, Bayesian) Machine learning is the best technique so far for specific AI 1/ BI Nordic AB - 3

4 Unsupervised Learning Classical data-mining Visualisation is often the mean and goal No knowledge of output open loop No matching to known target Clustering (k-means), dendrogram High dimensional data means one dimension per measured variable how to visualize Can vary over time/location, but time/location can also be a variable 1/ BI Nordic AB - 4

5 Supervised Learning Features vs. expected output Features is e.g. measurements Training is key (closed loop) Result types Classification into discrete values (cat/dog) Regression analysis into continuous values (-1, 3.5, 140) Avoid overfitting Feature 1 Feature [n] Feature 2 Model Class/Rate 1/ BI Nordic AB - 5

6 Production Research and Development Samples Supervised Learning Categorize (subjective / unsupervised) Cleaning Handle missing data Features Label (expected output) Train Test Select algorithm and feature set Train Evaluate performance Collect measurements Select model (accuracy/speed) New measurements Model Classified/rated output 1/ BI Nordic AB - 6

7 Wine Quality Data Set* fixed acidity volatile acidity citric acid residual sugar chlorides free sulfur dioxide total sulfur dioxide density ph sulphates alcohol type quality red red white white red red white 8 Vinho verde (Portugal) White wine: 4898 samples Red wine: 1599 samples Histogram Red Wine 11 measured parameters Subjective quality: median of at least 3 evaluations made by wine experts. Range 0 (very bad) and 10 (very excellent) *P. Cortez, A. Cerdeira, F. Almeida, T. Matos and J. Reis. Modeling wine preferences by data mining from physicochemical properties. In Decision Support Systems, Elsevier, 47(4): , 2009.

8 Demo Python with scikit-learn k-nn classification quality 7 or above Neural network classification quality 7 or above Neural network regression Random forest regression (decision tree) Recursive feature elimination

9 Model Performance

10 k-nearest Neighbour ph Alcohol Quality Euclidian distance Neighbour rank knn = 1 knn = * Classify sample with ph = 3.4 and alcohol 10.5 * sqrt(( )^2 + ( )^2) = 0.58 Euclidian distance Majority =

11 Deep Learning Example using Tensorflow Linear Data Linearly separable data, no deep learning needed, one layer is ok hidden.py --train simdata/linear_data_train.csv --test simdata/linear_data_eval.csv --num_epochs num_hidden 1

12 Deep Learning Example using Tensorflow Moon Data Non linearly separable data hidden.py --train simdata/moon_data_train.csv --test simdata/moon_data_eval.csv --num_epochs num_hidden 1 hidden.py --train simdata/moon_data_train.csv --test simdata/moon_data_eval.csv --num_epochs num_hidden 2 hidden.py --train simdata/moon_data_train.csv --test simdata/moon_data_eval.csv --num_epochs num_hidden 2 hidden.py --train simdata/moon_data_train.csv --test simdata/moon_data_eval.csv --num_epochs num_hidden 2 hidden.py --train simdata/moon_data_train.csv --test simdata/moon_data_eval.csv --num_epochs num_hidden 3 hidden.py --train simdata/moon_data_train.csv --test simdata/moon_data_eval.csv --num_epochs num_hidden 10 Note the effect of random init No of hidden layers

13 Deep Learning Example using Tensorflow Saturn Data Non linearly separable data hidden.py --train simdata/saturn_data_train.csv --test simdata/saturn_data_eval.csv --num_epochs num_hidden 1 hidden.py --train simdata/saturn_data_train.csv --test simdata/saturn_data_eval.csv --num_epochs num_hidden 2 hidden.py --train simdata/saturn_data_train.csv --test simdata/saturn_data_eval.csv --num_epochs num_hidden 3 hidden.py --train simdata/saturn_data_train.csv --test simdata/saturn_data_eval.csv --num_epochs num_hidden 10 No of hidden layers

14 Model Tuning Underfitting Good design Overfitting

15 Approaching Machine Learning Find the cost [OPEX, CAPEX] Subjective Algorithm that you pay for Investment in time Investment in equipment Subject matter experts are rare (and expensive) Safety Environment ANNOTATION IS KEY

16 Level of interaction Reductionism Reality Artificial Specific Intelligence I M A G I N A T I O N C O M M U N I C A T I O N C U L T U R E ART C O G N I T I O N ( G E N E R A T E N E W K N O W L E D G E F R O M E X I S T I N G ) Consciousness Artificial General Intelligence Planning Reacting Lack of glue Very CPU intensive The world changes behind our back