Big Data Appliance in Risk Management

Transcription

1 Big Data Appliance in Risk Management Erste Group Bank Jozef Zubricky Group Credit Risk Models and Methods

2 Digital data have predictive power...

3 Web Scenarios with highest predictive power Currency Conversion Information (1.3 % Defaults) Loan Consolidation Information (4.6% Defaults)

4 Simplest method is Naïve Bayes Text classification: SPAM filter messages Term-frequency matrix Class probabilities Message 3 Msg1 Msg2 Msg3 Msg1 Msg2 Msg3 Message 2 Client 3 Message 1 Your has won 2.5 million. Client 2 Client 1 sit Mozilla Consolida Macintosh your has won million Cli1 Cli2 Cli3 sit Mozilla Consolida Macintosh SPAM 90% 5% 10% HAM 10% 95% 90% The messages with high SPAM probability classified as SPAM. Text classification: Digital scoring Lists of strings Term-frequency matrix Class probabilities Cli1 Cli2 Cli3 High risk 10% 40% 90% Low risk 90% 60% 10% Probability of high risk used as an additional variable in the scorecard.

5 Not able to implement In our traditional system, due to computational speed and ever changing underlying websites

6 Big Data appliance is build for such tasks Input Column 1: IP Address Column 2: Timestamp of click Column 3: URL of Page Visited Column 4: Webpage Text Map() Key (IP Address) Value (Timestamp) Value (URL or Web Page visited) Value (Probability of Web Page being good or bad based on Webpage text and Naïve Bayes) Reduce() Shuffle and Sort Least Risky Pages Most Risky Pages Least Risky IP Addresses Most Risky IP Addresses Key(IP Address) Value (List of probabilities of all the websites visited by IP Address per user session defined by timestamps)

7 No business case... Nobody wants to finance this just for one problem like this

8 Problem: Come up data driven model. Natural experiment data

9 Why not champion challenger? It is costly Reputation Risk

10 We found 2 natural experiments In 2007 we granted for almost one month loans without considering current instalments Until 2010 we were granting foreign exchange loans in some countries Well we though we did found natural experiments...

11 We deleted data and scripts were not working For first experiment data for 2007 were no longer available For second experiment, scripts were not working across a corridor

12 We managed to... Retrieve data from old backups. Modelling itself was quite a success. We have found our relationships

13 So we set up a project called CRANE Central Place for Model Development, Monitoring and Validation Unlimited Data History, to Utilise Past Crisis Data for Model Development Automated Data Load and Post Rollout Check to Reduce Operational Problems

14 We needed cheap storage 4 Cost of Data Warehouse Appliance 3 2 Cost of Hadoop Appliance TB 20TB 30TB 40TB 50TB

15 Big Data Technology without Big Data

16 But this is how it got in Business case based on cheap storage

17 But... How to connect it to production legacy systems? What is regulatory environment?

18 Our Environment is diverse We expanded by buying banks Different legacy IT systems Central modelling team distributed across locations Not enough storage in enterprise data warehouse

19 EBA and ESMA published Regulation draft on Big Data Type: EBA ESMA big data into Google They seek comments until People who wrote it have good insight into industry

20 Main Takeaways of Risks Transparent Security Reputation Conformist Referring to other Cybersecurity and Wrong decisions, Conformist directives aimed at data protection of difficult to control, behaviour of people client ownership of Big Data solutions. exclusion of groups when they know how the data and Risk of outsourcing of clients form their data influence transparency of use services decision making etc.

21 Thank you for your attention What is your experience with Data-mart projects? Jozef Zubricky What is your experience with Big Data usage? Jozef Zubricky