Smart Trading with Cray Systems: Making Smarter Models + Better Decisions in Algorithmic Trading

Transcription

1 Smart Trading with Cray Systems: Making Smarter Models + Better Decisions in Algorithmic Trading

2 Smart Trading with Cray Systems Agenda: Cray Overview Market Trends & Challenges Mitigating Risk with Deeper / Faster Back Testing Addressing the I/O Issue New Approaches & Technologies Sample Implementation Live Q&A with Cray Experts Getting Started COMPUTE STORE ANALYZE 2

3 Presenters Phil Filleul Director Financial Services Cray Inc. Phil is Cray s global director responsible for bringing Cray s differentiated technologies into the financial services areas that can utilize them best. He is currently focused on banks large grid workloads, quantitative fund backtesting and large-scale analytics workloads. Phil has 30 years of IT experience, mainly in banking and the capital markets, including Teradata, heading MiFID regulation at Sun Microsystems, and creating a process outsourcing business for igate Patni. Phil has a master s degree in computer science from Cambridge University. Bob Gaines Regional Dir, Fin Services Solutions Cray Inc. Bob has a passion for high-performance data analysis (HPDA) and focuses on complex risk valuation adjustment, compliance, and the quantitative workflows so common with algorithmic trading strategy development. Prior to joining Cray, Bob was at DDN Storage and Seagate, developing solutions around high performance storage with parallel file systems, addressing the massive I/O issues. Bob is certified in Six Sigma, trained in TQM, and holds a master s degree in computer science and finance from Loyola University. Dave Strenski Principal Engineer Cray Inc. Dave is a principal engineer at Cray Inc. He has degrees in surveying, civil, and mechanical engineering and has worked for Cray Inc. for 23 years. He holds a patent for a mesh algorithms for threaded fasteners and has published papers in numerical consistency, parallel programming, genomic search algorithms, and reconfigurable computing. As a hobby Dave plays with solar power and founded SolarYpsi.org. COMPUTE STORE ANALYZE 3

4 Cray: Myth vs. Reality Myths They are huge They are proprietary They are complex They are expensive Vs. Reality! They can be huge but they start less than a rack Intel, Linux, open standards, Hadoop Simpler and more productive than a grid Cost competitive, lower TCO, higher value COMPUTE STORE ANALYZE 4

5 Cray s Vision: The Fusion of Supercomputing and Big & Fast Data Supercomputing Modeling The World Cray supercomputers solving grand challenges in science, engineering and analytics Big Data Math Models Modeling and simulation augmented with data to provide the highest fidelity virtual reality results Data-Intensive Processing High throughput event processing & data capture from sensors, data feeds and instruments Data Models Integration of datasets and math models for search, analysis, predictive modeling and knowledge discovery Analytics Store Analyze Copyright 2016 Cray Inc. High Performance Data Analytics (HPDA) COMPUTE STORE ANALYZE 5

6 Trends & Market Challenges in Algorithmic Trading Bob Gaines

7 Algorithmic Trading Continues to Grow HFT trading profits are becoming smaller and smaller -- but the volume of trades are still dominating the overall market Profit Pressures # of low latency traders Copyright 2015 Cray Inc. COMPUTE STORE ANALYZE 7

8 Trends in Algorithmic Trading Boost Profitability Address Analytics Cluster Sprawl & Big Data and Grid Consolidation Performance matters Feedback cycle Pipelines not just tools Scale and I/O become bottlenecks Explore Cloud for agility and cost savings? Latency issues Data movement HPC and analytics converge Analytics proliferation Reduce cost per simulation (# of simulations /time) / (total cost/ time) Move to Open Source 8

9 Trends in Algorithmic Trading Improve Profitability (win/loss ratio) Explore New Technologies o Machine Learning o Graph Database (Neural Networks) Reduce Development Times Reduce Risk Improve Backtesting Scale Real-time Analytics Higher performance and more innovative use of memorystorage hierarchies and interconnects required here <30ms > 10MIN Performance optimizations Emergence of Latency-Sensitive Analytics Copyright 2015 Cray Inc. COMPUTE STORE ANALYZE 9

10 Mitigating Risk with Effective Backtesting Bob Gaines

11 Strategic Backtesting = Critical Improving degree of confidence Filtration Modelling Optimization Verification Data availability presents Opportunity Issues SIZE Petabytes IOPS Gigabytes Node 1. Node N. 11

12 Differentiating Capacity and Bandwidth REQUIREMENTS IOPS GB/s GB/s COMPUTE STORE ANALYZE 1 12

13 Cray Perspective on I/O in HPC Expensive compute resources sit idle during I/O Want to have highest possible bandwidth when doing I/O Disk-based PFS bandwidth is expensive Bandwidth via controllers inflates effective cost PFS is still the preferred option for scalability & permanence Many applications do I/O in bursts A cycle of Read Write Checkpoints Flash bandwidth is relatively inexpensive Effective for I/O load at beginning/end of job, during checkpointing COMPUTE STORE ANALYZE 13

14 Move up the Hierarchy for Performance Share Data for Fidelity and Productivity & local memory Shareable COMPUTE STORE ANALYZE 14 14

15 For High IOPS file data -- If it s shared Choices! Oversize the HDD FS Expensive bandwidth and overcapacity Pipe dependency Better with PFS e.g. Lustre or GPFS Replicate storage per node Provision per node Provision per node HDD Or.Shared NVRAM Not quite as fast as local But far more cost effective Easier to manage COMPUTE STORE ANALYZE 15

16 Example: NFS vs PFS (Lustre) 200G NODE 1.N sequentially 7200 models 200GB datasets to 300 nodes 92sec/ node 7.93hr aggregate NFS STORAGE 1.6PB Traditional Approach: - Risk to short cuts! > Small samples > Fewer models 300 node (7200 cores) 200G 7200 models 2.35PB Copyright 2015 Cray Inc. Parallel submission 200GB Data set to 300 nodes 300 node (7200 cores) Results Higher Degree of Confidence ID Strategies Faster to Fail Increased Scale Lost Cost/ Model ACCELERATE ACCESS TO INSIGHT 16 16

17 for Accelerating I/O Dave Strenski

18 for Holding Ticker Data Move the data closer to the processors Place the data on faster media Stripe the data for greater bandwidth Copyright 2015 Cray Inc. COMPUTE STORE ANALYZE 18

19 Reduces Resident Time of Jobs 250GB Copyright 2015 Cray Inc. 19

20 Hardware Dynamically Mountable Best price per bandwidth 4.0TB Intel P TB Intel P TB Intel P TB Intel P3600 Compliant Copyright 2015 Cray Inc. 20

21 Architecture Blade = nodes each with 2 DW DW DW DW Copyright 2015 Cray Inc. 21

22 1 node, 32 threads, 32 jobs s Write ~3GB/s Read ~5GB/s 250GB Standard POSIX File system READ GB/s WRITE GB/s Aries Network 8MB chunks DVS memory memory COMPUTE memory memory STORE memory memory ANALYZE memory memory Copyright 2015 Cray Inc. 22

23 1 node, 32 threads, 32 jobs s Write ~3GB/s Read ~5GB/s 250GB Standard POSIX File system READ GB/s WRITE GB/s Aries Network 8MB chunks DVS memory memory COMPUTE memory memory STORE memory memory ANALYZE memory memory Copyright 2015 Cray Inc. 23

24 2 node, 32 threads, 64 jobs s Write ~3GB/s Read ~5GB/s 250GB Standard POSIX File system READ GB/s WRITE GB/s Aries Network 8MB chunks DVS memory memory COMPUTE memory memory STORE memory memory ANALYZE memory memory Copyright 2015 Cray Inc. 24

25 2 node, 32 threads, 64 jobs s Write ~3GB/s Read ~5GB/s Standard POSIX File system 250GB READ GB/s WRITE GB/s Aries Network 8MB chunks DVS memory memory COMPUTE memory memory STORE memory memory ANALYZE memory memory Copyright 2015 Cray Inc. 25

26 4 node, 32 threads, 128 jobs s Write ~3GB/s Read ~5GB/s Standard POSIX File system 250GB READ GB/s WRITE GB/s Aries Network 8MB chunks DVS memory memory COMPUTE memory memory STORE memory memory ANALYZE memory memory Copyright 2015 Cray Inc. 26

27 4 node, 32 threads, 128 jobs s Write ~3GB/s Read ~5GB/s 250GB Standard POSIX File system READ GB/s WRITE GB/s Aries Network 8MB chunks DVS memory memory COMPUTE memory memory STORE memory memory ANALYZE memory memory Copyright 2015 Cray Inc. 27

28 Metrics: Latency as Data Grows 3 Single Node transmission -1 to 32 concurrent jobs Lustre Variance 213% DW Variance 20% 0 3 GB 7 GB 13 GB 26 GB 40 GB 53 GB 66 GB 79 GB 92 GB 106 GB Lustre READ Lustre WRITE READ WRITE COMPUTE STORE ANALYZE 28

29 / Scale sec 4 node transmission -- 4 to 128 concurrent jobs % lower latency READS 55 71% lower latency WRITES sec sec sec Lustre Variance 722% sec sec sec 13 GB 26 GB 53 GB 106 GB 158 GB 211 GB 264 GB 317 GB 370 GB 422 GB Lustre READ Lustre WRITE READ WRITE DW Variance 111% 29

30 8 node, 32 threads, 256 jobs s Write ~3GB/s Read ~5GB/s Standard POSIX File system READ GB/s WRITE GB/s Aries Network 8MB chunks DVS 32x 32x 32x 32x 32x 32x 32x 32x COMPUTE STORE ANALYZE memory memory memory memory memory memory memory memory 30

31 Future Directions Moving data to Attaching the to a high speed network Striping the data across multiple s What if we moved the data directly to RAM/Memory?? Aries s PGAS Network Partitioned Global Address Space memory memory COMPUTE memory memory STORE memory memory ANALYZE memory memory 31

32 Why Cray? Helping finance customer make: Better decisions beyond the box Start small and scale most complex problems Profitable Trade Ratio Optimize Sharpe Ratio Mitigate Risk Reduce Cost/ Simulation Reduce Latency Consolidate analytic workflows Copyright 2015 Cray Inc. 32

33 FOCUS: Quantitative Workflows Data-Intensive Focus Quantitative Analytics Algorithmic Trading Portfolio/ Position Risk Complex Asset Valuation Credit Default Swaps Dark Pool Assets Near Real-time Analytics/ Discovery Market Sentiment Analysis Credit Valuation Adjustment CVA/ DBA Counter Party Risk CCR CCAR Stress Test Highly Iterative Latency Driven Technologies Dense, Low TCO, Scalable Dense GPUs Supercharged I/O to Nodes Analytics Rapid Deployable Graph Engine Discover patterns in disparate datasets AGILE Open Source, HaDoop Stack Storage Lustre, Spectrum Scale Supercharged I/O 40m IOPS // 1.5TB/s++ Data Management 33

34 For More Information Read the Solution Brochure Cray-Smart-Trading.pdf Get other resources for Financial Services White papers, videos, event news and more Learn about Cray products COMPUTE STORE ANALYZE 34

35 Questions? Phil Filleul Thank You! Unbiased User Input Bob Gaines Dave Strenski _proceedings/by_sub_type.html