Safe, distributed computation for reluctant data sharers: At home in the Microsoft Azure cloud

Transcription

1 Safe, distributed computation for reluctant data sharers: At home in the Microsoft Azure cloud Phil Lavori, Professor, Biomedical Data Sciences Balasubramanian Narasimhan, Senior Research Scientist, Statistics and Biomedical Data Sciences John Mark Agosta, Principal Data Scientist, Microsoft Stanford Data Science Institute 2 November 2017

2 Outline Precision Medicine: What is the distcomp project for? DistComp What does the distcomp application do (and how does it do it)? Azure Cloud What s the promise of the cloud for this and for similar apps?

3 Challenges for precision cancer medicine Clinical trial evidence applies to only 5% of cancer patients Models based on observational (clinical) data are essential for decision support Cancer is heterogeneous Local hospital data is insufficient for modeling sharp bias/variance tradeoff We must be able to share information

4 Sharing (Informative) Data is HARD Cannot be permanently de-identified Loss of control over end use is scary Competitive forces oppose sharing data Copyright Daniel Rubin 2014

5 Sharing information instead is easy! The information is in the fitted model Model computation is distributable Copyright Daniel Rubin 2014

6 Information sharing without data sharing Courtesy Phil Lavori 2. Calculate and return local likelihoods to controller Site 1 (Stanford) 1. Propose parameter vector to sites Master/controller Site 2 (Vanderbilt) No data sharing required 3. Controller updates parameter vector to sites ITERATE Site 3 (Oxford)

7 Model Fitting with Distributed Data

8 Moving distributed applications to the Cloud

9 Borrowing the Azure cloud capabilities for implementing distributed enterprise applications. Its easier if everything runs in one place the cloud A Cloud App consists of numerous resources Each Azure tenant contains an instance of the application Since a person can view only their own tenancy--their the cloud how do apps connect?

10 Worker 2 Worker 3 Worker 1 Master A Network of distcomp cloud applications. DistComp is a peer-to-peer application, configured with the user credentials and the IDs of the apps it communicates with.

11 Resources that constitute the distcomp Azure web app

12 Repeat Shared The lunch! Formulate Model Run Distributed Model Share Model Local Install& configure distcomp app Test Model Locally Run Global Model Setting up and running experiments

13 Within each tenant, each client generates a bearer token to authenticate their request.

14 Multiple Tenant Authentication Azure Obtain Web Token ID broker Local Tenant Remote Tenant Azure Active Directory Azure Active Directory App Resource group App Resource group AD App AppServer Key vault Web node Compute node Public IP R package storage SQL server/ R server Place Request AD App AppServer Key vault Web node Compute node Public IP R package storage SQL server/ R server

15 Where do we go with this? Widespread clinical use of global models, without data privacy constraints Reduce time and effort from months to days, from model definition to putting it into use. Expand to a richer set of inference algorithms. Provide a general framework for loosely coupled distributed computation, e.g. ADMM. Full workflow around defining, running and consuming the models. And tools to support this in collaborator s current IT environments. Expedite the sharing of clinical results, at internet speed. This architecture is reusable, for secure, peer-to-peer applications and not constrained to one tenant.

16 Benefits of SDI Collaboration Stanford gains an industrial strength platform on which to build, for it s research to have largescale impact. Microsoft explores new architectural designs, based on real needs & novel computational models.