Data Governance Simplifying Machine Learning Model Deployment

Transcription

1 Data Goverace Simplifyig Machie Learig Model Deploymet Abstract Although machie learig (ML) offers may advatages whe it comes to predictig outcomes ad idetifyig with ovel isights, deployig ML models i itself is a challegig task. Apart from assemblig the various costituet compoets of its architecture, codig teams are also required to esure the itegrity ad quality of data used for operatioalizig the models a task which is made more complex without strog data goverace, especially if a ML model is to be itegrated with third-party data sources. This paper discusses the foudatioal elemets required for settig up such a goverace framework alog with relevat details aimed at simplifyig ML model deploymet.

2 Machie Learig i Actio What if you were ruig a trasportatio busiess ad wated to reduce the cost of maitaiig egies? Machie learig (ML) techiques, i that regard, ca be a game chager. If deployed correctly, it ca help you predict just whe a bus or ay other vehicle i your fleet eeds to be recalled for proactive maiteace. I tur, this will help you miimize the impact of geeral wear ad tear, dimiish failures i the field ad the eed for a major overhaul, ad keep your fleet o the road for as log as possible so that you do't miss out o reveue opportuities. While developig such a ML model, you might soo come to realize that factors which ifluece egie use ad performace are myriad. These ca rage from the idividual egie's behavior to weather patters, the umber of passegers ferried daily, o-road traffic volumes, ad festivals ad evets o travel routes to ame just a few. Cosequetly, you will realize that although you might have complete access to some of this data, acquirig iformatio such as weather ad traffic volume forecasts meas havig to tap ito exteral data sources. Iitially, you will begi by just desigig ad developig the basic ML architecture. It ca comprise a multi-layer eural etwork (NN), a covolutioal eural etwork (CNN) or eve a recurrig eural etwork (RNN) with additioal fuctioality layers o top of it. While buildig the model, other cosideratios such as icomig data volumes, expected performace, ad the time required for traiig the model will have to be take ito accout. You will also eed to sped a cosiderable amout of time validatig, ormalizig, ad esurig that the data is labeled properly for traiig the structured algorithms. Thereafter, you will have to optimize the algorithm, reduce bias ad variace errors, ad defie target values before you fially have a model with the desired level of accuracy which is ready to be deployed i a productio eviromet. This sets the stage for the ext stage of challeges that you will have to overcome.

3 Outputs from Traiig Activities Alog with the ML model, a key set of outputs that ca be acquired post traiig activities costitutes a ivetory of data sources alog with metadata pertaiig to volume, veracity ad velocity descriptio, techical format, physical locatios, as well as the access ad cotrol descriptios of the iformatio streams. Usig them, you will be able to develop a umber of ormalizatio, trasformatio, ad classificatio rules for shapig the data ito a cosistet ad coheret format based o which you have traied your ML model. Operatioal Architecture I light of all of these ew outputs, the ML model's operatioal architecture has to be flexible eough to itegrate discrete data compoets. I case of the trasportatio busiess for example, this ca iclude daily egie metrics, real-time passeger cout as they board ad disembark from the vehicles, traffic coditios, ad daily weather forecasts o a low-volume mothly basis. Each of these sources have to be orgaized accordig to how big the associated datasets are, how ofte they are geerated, ad how they are validated. There must be policies i place for cotrollig ad maagig each distict data stream. Ad, as is the case with ay data source, you will eed the logical ad physical data models. For may compaies, this may as well be the first time they are usig exteral data sources as a key tool to streamlie their decisio-makig process. Data Quality Strategy To make sure that the ML model is fuctioig as iteded, maitaiig data quality will be a imperative. To do so, data cosistecy, relevace, ad overall accuracy will eed to be moitored while assessmets ad metrics for measurig how well data sources are performig should ideally be established. At this poit, quality gaps or bugs i the data sources such as missig or uavailable data ad the effort required for cleasig ca become a sigificat burde. I tur, this ca reder a data source early uusable ad may substatially affect the fial traied algorithms.

4 While workig with exteral data sources, you might face a high latecy problem which ca make it difficult for the ML model to derive predictive isights i ear real-time. To overcome this, you will eed to determie how much additioal computatioal power is required to ru potetially complex algorithms o demad. Implemetatio Roadmap Apart from the basics of the ML model's architecture that has bee discussed here already, data igestio, trasformatio, ormalizatio, ad loadig capabilities for multiple sources eeds to be itegrated. Despite the test results gathered from the traiig stage, the ML model's fuctioality will still eed to be assessed i terms of the quality of its output. I terms of the maiteace predictio model we have bee talkig about, whe it suggests that a specific vehicle eeds to be take off the road, the recommedatio will eed to be cross-validated for its accuracy by mechaics ad egieers. If they disagree with the predictio ad cocur that the vehicle could have efficietly operated for aother moth, the feedback ca be used to improve the ML model further. The overall goal at this stage is to establish a clear evaluatio process to collect feedback o performace ad validate them i lie with acceptable tolerace levels. Ideally, there should be strog goverace i place to keep this process operatig cotiuously. A further extesio to this is the cocept of A/B testig. As ML egieers desig ew models, their operatios teams will eed to deploy mechaisms for ruig comparative tests. There should also be a feature which eables it to pass the same data subsets through idividual models for geeratig automated comparisos. Over time, the ML model will eed to be retraied, ot oly if it produces results which deviate too far from ormal expectatios but also as part of geeral maiteace procedures. It could also be because better data sources have recetly opeed up or ew features eed to be added to the existig iformatio streams or perhaps because a bias has bee detected. As a corrective measure, ew ML models ca be released much like ay other software update. If the deploymet approach comprises a series of ML models where

5 the output of oe flows ito aother (as is the case with the maiteace predictio example), a clear framework ad a set of metrics for measurig each model's performace eed to be established. A Agile Approach There is a clear eed for cotiuous itegratio ad deploymet tools, especially because the terms of use pertaiig to differet datasets ofte vary while ew models ad data sources cotiue to emerge. If orgaizatios wish to rapidly capitalize o the beefits they offer, a agile ifrastructure will be essetial, especially whe models are ot performig as they had i the lab. This creates a true biz-devops eviromet where every stakeholder icludig data scietists, developers, ad operatios teams focus o diagosig, testig, ad deployig potetial high impact models. Strog policies for versio cotrol o both the model ad its parameters will have to be maaged ad maitaied. For example, a ew set of parameters may be a mior model release, whereas a complete ew model may be a major release. Data Goverace The implemetatio of these policies with respect to goverace is the ext step. Most orgaizatios have established data maagemet rules for iteral data sources. However, i scearios where compaies wat to use exteral sources ad third-party data brokers, clear cotrol ad quality metrics will have to be established. While usig exteral sources, ay associated procuremet policy ad purchasig rights will have to be put i place. Furthermore, specific goverace regardig associated itellectual property rights ad usage rights will have to be icorporated ito the framework, as well as adherece to the Geeral Data Protectio Regulatio (GDPR) regulatios.

6 The Way Forward Developig ad traiig ML models is challegig to say the least. Despite the umerous frameworks ad toolkits that developers have access to today, makig a ML model fully operatioal poses challeges that are ulike those associated with geeral software developmet. Apart from the skills ad capabilities required, a ML developmet project should take ote of: The impact it will have o the operatioal architecture Exteral data sources ad how they will be used o a dayo-day basis Quality of output, how it ca be measured, ad what ca be doe to improve it Operatios moitorig ad validatio As orgaizatios become icreasigly data-drive, they will come to deped o AI ad ML as the primary source for busiess itelligece. Depedig o how extesive the practice grows, it will begi to exhibit a umber of complex operatioal challeges. Imagie the trasport busiess ruig ML models for ot just evaluatig egie health but also for measurig supplier performace, facilitatig parter maagemet, ad drivig sales ad marketig campaigs. As such, at ay give time, there could be hudreds of ML models i operatios each with staggerig data requiremets ad computatioal eeds. Goverace ad chage maagemet, i such a sceario, will be the key for derivig the maximum value from operatios.

7 About The Author Ged Roberts Ged Roberts is the Global Head of Operatios ad Delivery Excellece for TCS' HiTech busiess uit. With over 30 years of experiece, he is resposible for maitaiig a sigle global service stadard across the busiess uit, ad for assurig that TCS' cliets experiece certaity ad assured service delivery. Cotact Visit the Hitech page o HiTech.Marketig@tcs.com Subscribe to TCS White Papers TCS.com RSS: Feedburer: Tata Cosultacy Services is a IT services, cosultig ad busiess solutios orgaizatio that delivers real results to global busiess, esurig a level of certaity o other firm ca match. TCS offers a cosultig-led, itegrated portfolio of IT ad IT-eabled, ifrastructure, egieerig ad assurace services. This is delivered through its uique Global Network Delivery ModelTM, recogized as the bechmark of excellece i software developmet. A part of the Tata Group, Idia s largest idustrial coglomerate, TCS has a global footprit ad is listed o the Natioal Stock Exchage ad Bombay Stock Exchage i Idia. For more iformatio, visit us at All cotet / iformatio preset here is the exclusive property of Tata Cosultacy Services Limited (TCS). The cotet / iformatio cotaied here is correct at the time of publishig. No material from here may be copied, modified, reproduced, republished, uploaded, trasmitted, posted or distributed i ay form without prior writte permissio from TCS. Uauthorized use of the cotet / iformatio appearig here may violate copyright, trademark ad other applicable laws, ad could result i crimial or civil pealties. Copyright 2018 Tata Cosultacy Services Limited TCS Desig Services I M I 03I 18 About Tata Cosultacy Services Ltd (TCS)