as a Financial Engineering Development Platform Delivering Financial / Quantitative Models to the Enterprise Eugene McGoldrick 2016 The MathWorks, Inc. 1
Development Environment for Financial Services Development Environment has increasing number of productivity tools built into the base product Test Harness modeled on JUnit, customizable and extensible Support for version control Subversion GIT API available to support other version control systems GUI development tools Guide App Designer Performance analysis tools Profiler Code Analyzer 2
in the Enterprise Goals: Enable customers to rapidly develop and deploy applications onto the desktop. Seamlessly integrate generated components into other languages, applications and enterprise production systems. 3
Integrating into production systems Requirements Development Business End Users Algorithm Developers IT professionals Software Engineers Traders Portfolio managers Business Analysts Managers Develop, Test, Implement, Install Production Component Model/Algorithm/Analytic Use everyday for business decisions Desktop Applications Production Databases Analytic Engines Web Applications Trading Systems 4
Share with non- Users: Compiler Compiler Runtime Standalone Application Excel Add-in C/C ++ Hadoop Package as a standalone executable Package as an Excel Add-in Package as a Map Reduce Application to run in Apache Hadoop 5
Compiler Workflow Application Author Toolboxes 1 2 Compiler End User Standalone Application Excel Add-in Hadoop 3 Runtime 6
in the Enterprise Goals: Enable customers to rapidly develop and deploy applications onto the desktop. Seamlessly integrate generated components into other languages, applications and enterprise production systems. 7
Share with non- Users: Compiler SDK Compiler Compiler SDK Package as components that can be integrated in C/C++, Java,.Net, or Python applications C/C ++.NET Production Server Package as deployable archives for execution on Production Server Java Python Runtime 8
Compiler SDK Workflow Application Author Toolboxes 1 Software Developer Compiler SDK 2 C/C ++.NET Production Server 3 4 Runtime Python Java 9
Production Server Directly deploy programs into production Centrally manage multiple programs & MCR versions Automatically deploy updates without server restarts Scalable & reliable Service large numbers of concurrent requests Add capacity or redundancy with additional servers Production Server(s) Use with web, database & application servers Lightweight client library isolates processing Access programs using native data types Web Server(s) HTML XML Java Script 10
Calling Functions Enterprise Application Production Server MWHttpClient object HTTP Request Broker & Program Manager Calculation Process Calculation Process Worker Pool 11
Production Deployment Workflow Development Developer Algorithm Compiler Enterprise Application Developer Server/Web Application CTF Production Server Client Library Function Call Production. Server/Web Application Client Library Production Server 12
Central Management Centrally run and manage numerical algorithms Production Server Simplifies applications Application Servers Analytics run within datacenter UI and business functionality Simplifies change management Independent update of numerical algorithms Distributed Applications 13
Benefits of the Production Server Enterprise class framework for running packaged programs Server software Manages packaged programs & worker pool Manages Runtime libraries for multiple releases Compiler Runtime (MCR) for various versions of from R2012b onward live on the server. Compiled analytics from different versions of from R2012b onward can co-exist on the server. Lightweight client library for.net and Java frameworks, C/C++, and Python are supported. Reduces the Total Cost Of Ownership for building and supporting in-house financial analytics development and deployment. 14
Flexible System to Manage Licensed on workers/worker threads not on Broker process Infinitely configurable to take advantage of existing inhouse hardware Hosted Analytics platform that is installed in house enabling rapid updating and deployment of analytics/models Accessed by any front end application by means of thin client communications library, or through JSON/Restful interface 15
Production Server Customer Configurations (1) Request Broker and 24 worker processes Can have multiple instances of the Production Server 2 request brokers and twelve worker servers 3 request brokers and eight worker servers 4 request brokers and six worker servers 6 request brokers and four worker servers Increase capacity by increasing number of servers and combining them One request broker and 48 worker processes Production Server Request Broker & Program Manager 16
Production Server Customer Configurations (2) Production Server Request Broker & Program Manager Auto Deploy CTF repository Production Server Request Broker & Program Manager 17
Easy Integration IT can efficiently integrate models/analytics in to production system Seamless integration into.net and Java development environments only a few lines of code required Time to deploy greatly reduced Only need to supply function signature from Quant to IT for implementation into Enterprise system Updates easily implemented and redeploy new model version 18
Integration Example Java Reference client library Define function signatures Define connection (server & CTF) Function function B = BlackScholes(CP,S,X,T,r,v) d2=d1-v*sqrt(t); if CP=='c' B = (S*normcdf(d1)-X*exp(-r*T)*normcdf(d2))-noise; Enterprise Application import com.mathworks.mps.client.mwclient; import com.mathworks.mps.client.mwhttpclient; import com.mathworks.mps.client.exception;; public interface BlkSchInterface { double BlackScholes(string C, double S, double X, double T, double r, double v); } MWClient client = new MWHttpClient(); BlkSchInterface blksch_1 = client.createproxy(new URL("http://192.168.240.220:9910/BlkSch1"), BlkSchInterface); double optionprice = blksch_1.blackscholes("c", BasePrice.Value, 1, 1, 1, Volatility.Value)); 19
Integration Example.NET Reference client library Define function signatures Define connection (server & CTF) Function function B = BlackScholes(CP,S,X,T,r,v) d2=d1-v*sqrt(t); if CP=='c' B = (S*normcdf(d1)-X*exp(-r*T)*normcdf(d2))-noise; Enterprise Application using Mathworks..ProductionServer.Client; public interface BlkSchInterface { double BlackScholes(string C, double S, double X, double T, double r, double v); } MWClient client = new MWHttpClient(); BlkSchInterface blksch_1 = client.createproxy<blkschinterface>(new Uri("http://192.168.240.220:9910/BlkSch1")); double optionprice = blksch_1.blackscholes("c", BasePrice.Value, 1, 1, 1, Volatility.Value)); 20
JSON/Restful API Easy API to use No client library required Function function B = BlackScholes(CP,S,X,T,r,v) d2=d1-v*sqrt(t); if CP=='c' B = (S*normcdf(d1)-X*exp(-r*T)*normcdf(d2))-noise; Enterprise Application var cp = parsefloat(document.getelementbyid('coupon_payment_value').value); var np = parsefloat(document.getelementbyid('num_payments_value').value); var ir = parsefloat(document.getelementbyid('interest_rate_value').value); var vm = parsefloat(document.getelementbyid('facevalue_value').value); // A new XMLHttpRequest object var request = new XMLHttpRequest(); //Use MPS RESTful API to specify URL var url = "http://localhost:9910/bondtools/pricecalc"; //Use MPS RESTful API to specify params using JSON var params = { "nargout":1,"rhs": [vm, cp, ir, np] }; request.open("post", url); //Use MPS RESTful API to set Content-Type request.setrequestheader("content-type", "application/json"); request.send(json.stringify(params)); 21
Where we are today with Production Server Improved throughput and overall performance 1.5 Gb/sec transfer rate for 1 through 1000 concurrent users 50 µsec latency for 1 through 1000 concurrent users R2016a R2015b 22
Development to Production Workflow 2016 The MathWorks, Inc. 23
Reduce Cost of Building and Deploying In-House Analytics Single development environment for model development and testing. Quants/Analysts/Financial Modelers do not have to rewrite code in another language. IT can efficiently integrate models/analytics in to production system Time to deploy greatly reduced Only need to supply function signature from Quant to IT for implementation into Enterprise system Updates easily implemented and redeploy new model version 24
Separation of Roles in Building and Deploying In-House Analytics Developer & Toolboxes, Access to Compiler Systems Developer Visual Studio Tools, Web Development Tools Developer: Describes function signature to the Systems developer so that they can integrate MPS models into variety of front end/server Applications Developer: Builds, tests models in, Compiles them with the compiler into a MPS ctf file and deploys it to the server Web Server Production Server Request Broker & Program Manager Models reside on the server and can be updated regularly by the development team without bringing the server down Integrates MPS functions into.net, JAVA Frameworks, C/C++ and Python so that they can be called from a variety of different client applications Web Applications Desktop Applications 25
Desktop and Server Based Excel Add-ins Desktop Excel Add-ins with compiler XLA Model Runtime Generated by Compiler Server based Excel Add-ins with Compiler SDK XLA Production Server Generated by Compiler SDK... XLA XLA Request Broker & Program Manager 26
Sharing algorithms across the organization Web Applications Compiler Web Server Production Server Portfolio Optimization Desktop Applications Pricing Application Server Risk Analytics Batch Applications Database Server 27
Web Applications - Using JAVA, C# client libraries Architecture Web Server Production Server CCRWeb Request Broker & Program Manager HTML & JavaScript front end ZeroCurveServlet genscenservlet.... computecvaservlet Database Server 28
Integration with Databases Optimize numerical processing within databases Request analytics directly from database servers Trigger requests based upon database transactions Minimize data handling using Database Toolbox Enterprise Application Database Server Stored Procedure Client Library Production Server Request Broker & Program Manager 30
Integration with Databases Native in database JSON/Restful call ORACLE, MS SQL Server, SAS support this in database Enterprise Application Database Server Stored Procedure JSON/ Restful Production Server Request Broker & Program Manager 31
Components in Production Databases Production Server can provide predictive analytics in the database Oracle (Java,.NET) Microsoft SQL Server (.NET) Microsoft Access (.NET) Netezza (JAVA) SAS (JAVA) Teradata (JAVA) Thin client with MPS Java and.net supported Database Server Central repository for models Simplifies change management Production Server Request Broker & Program Manager 32
Big Data Analytic Components - Hadoop is the analytics for big data systems such as Hadoop Hadoop Grid Embedded on-node custom algorithms can be developed in and deployed to the grid Model Runtime Compiled Hadoop Targets Compiled Hadoop Targets Compiler generated Hadoop components on each node of Hadoop Grid 33
Production Server Use Cases Big Data HADOOP Spreadsheet Applications Client Front End Applications Production Server Request Broker & Program Manager Application Servers MSMQ, Open JMS, IBM MQ Web Applications Enterprise Service/Messaging Buses Database Servers 34