Software as a Service Multi-tenant Data Architecture. Frederick Chong Architect DPE Architecture Strategy Microsoft Corporation

Transcription

1 Software as a Service Multi-tenant Data Architecture Frederick Chong Architect DPE Architecture Strategy Microsoft Corporation

2 Agenda SIMT principles and considerations Database options for storing multi-tenant data Enabling virtual data isolation in a multitenant environment Customizing multi-tenant data model Scaling multi-tenant data

3 The SaaS Architecture Shift Single Instance Multi-tenant Scaleable Configurable Multi-tenant efficient

4 The Share vs. Isolate Continuum SLA per tenant Data Separation Economy of Scale Simpler Management Isolate Share Considerations: Business (Time to market, ROI over time) Technical (Skills, expected tenant size and load) Operation ( can I guarantee my SLA without isolating? ) Regulatory constraints ( data must be physically separate )

5 Where to store the data? Approaches: Separate database per tenant Same database, separate schemas Same database, shared schema Implications on approach for: Securing and isolating data Extending data model Scaling and partitioning data

6 Separate Database per Tenant Approach: Meta data identifies database instance for each tenant Advantages: Easy to implement data model extension Easy to restore tenant data More security isolation Tradeoff: Number of tenants per database server is low Higher management, backup cost and database server infrastructure costs When to use: When tenant has specific database isolation requirements Tenant 1Tenant 2Tenant 3

7 Same Database, Separate Schema Approach: Each tenant gets their own group of tables in the same database. Advantages: Easy to implement data model extension Moderate security isolation Better tenant scale per server Tradeoff: More difficult to restore tenant data When to use: Number of tables for the app is small (100s) Scale per server is important OK to co-locate tenant data in same database Tenant B Tenant A Tenant C

8 Same Database, Same Schema Approach: All tenants use the same set of tables in the same database. Advantages: Better tenant scale per server Cost of management and backup is lower Tradeoff: Difficult to restore tenant data Harder to implement data model extension When to use: Scale per server is important OK to co-mingle tenant data in same database Tenant B Tenant C Tenant A

9 Securing and Isolating Tenant Data

10 Virtual Data Isolation Patterns Filters Permissions Database Table SQL Command View Stored Procedure View Encryption Crypto Key

11 Current Database Access Models Impersonation + Kerberos delegation Connection is established using end user security context No database connection pooling, security vs. performance tradeoff Trusted Subsystem Use application identity to establish database connection Better scale and performance Web Server Business Logic Server SQL Server Alice App ID = Bob Impersonate/Delegate App ID = Charlie Trusted Subsystem Access Access as as Alice Charlie Security Service (e.g. Windows KDC)

12 Multi-tenant database access requires a hybrid model Database access performed using a tenant security context Facilitates locking down database objects at the tenant level Databases, tables, stored procedures, views Facilitates database auditing Enables tenant specific encryption keys Costs: Additional provisioning task for creating and deactivating tenant TS accounts Added complexity in maintaining database connections pools

13 Multi-Tenant Trusted Subsystem Model Create trusted subsystem account for every tenant Application service process starts with a generic application identity Switch to a tenant account when connecting and accessing database: Kerberos SFU, or Security credential service Application Service Request from Acme Corporation Database Service Kerberos protocol transition into Acme account Request and send Acme Kerberos credential ticket Connect as Acme security context Security Credential Service Microsoft KDC

14 Securing Access to Database Separate database connection pool for each tenant Every application server can maintain one or more connection pool Database Access Group Shared DB Dedicated DB

15 Securing Tenant Specific Tables Database access group contains application identities with database access. Create different groups of tables for each new tenant. Create a different database access identity for each new tenant. Grant table access permissions to tenant s database access account. Create database access account for each tenant Database Access Group Grant account permissions to tables Assign account to group

16 Tenant View Filter Pattern Queries and Stored Procedures Tenant- Safe Views Shared Tables TenantCustomers View SELECT * from TenantCustomers WHERE Zipcode = Customers Table CREATE VIEW TenantCustomers AS SELECT * from Customers WHERE Tenant_ID = SUSER_SID()

17 Tenant Encryption Pattern Tenant A s Public/Private Keys Database s Master Key Encrypted Private Keys Tenant B s Public/Private Keys CREATE ASYMMETRIC KEY TenantAAsymmetricKey AUTHORIZATION TenantA WITH ALGORITHM = RSA_2048 CREATE SYMMETRIC KEY TenantASymmetricKey AUTHORIZATION TenantA WITH ALGORITHM = DESX ENCRYPTION BY ASYMMETRIC KEY TenantAAsymmetricKey Encrypted Symmetric Keys OPEN SYMMETRIC KEY TenantASymmetricKey DECRYPTION BY ASYMMETRIC KEY TenantAAsymmetricKey Tenant A s Symmetric Key Encrypted Tenant Data Tenant B s Symmetric Key INSERT INTO CustomersCreditCardView VALUES( 1234, EncryptByKey(Key_GUID(TenantA SymmetricKey), ))

18 Customizing Data Models

19 Data Model Extension Tenant A Tenant B Catalog Item Product ID Description Category ID Catalog Item Product ID Description Classification Code Challenges: Defining custom fields and storing custom data for each tenant. Business logic that can handle custom fields Presentation logic that can handle custom fields Enabling queries on custom fields

20 Defining and Storing Custom Data Fields Requires meta-data/data dictionary Straight forward when storing data in: Separate Database per Tenant Same Database with Separate Schema 2 approaches when storing data in Same Database with Same Schema: Fixed set of extended fields Extensions-value pairs

21 Fixed set of extensions Approach: All tenants data in one database. Pre-defined set of custom fields Advantages: Easy to implement Maximize number of tenants per database server Tradeoff: Tendency to results in sparse table When to use: When data co-mingling is OK Easy to anticipate pre-defined custom fields Tenant ID F1 F2 C1 C2 C3 345 Ted 53 Null paid Null 777 Kay Null Null 784 Mary 45 Null Null Null 345 Ned 21 Null owe Null 438 Pat 26 Null Null yes

22 Metadata for Fixed Extensions Tenant ID C1-Label C1-Datatype C2-Label C2-Datatype C3-Label C3-Datatype 345 Null Null Payment Status ntext Null Null 777 Age tinyint Null Null Null Null 438 Null Null Null Null Paid ntext OR (Space and Time Tradeoff) Tenant ID C1-Label C1-Datatype 777 Age tinyint Tenant ID C3-Label C3-Datatype 438 Paid ntext Tenant ID C2-Label C2-Datatype 345 Payment Status ntext

23 Extension-Value Pairs Approach Offers Unlimited number/option for custom fields Extension-value pairs in separate tables Metadata table keeps track of data labels and data types for extensions Metadata Table Data Tables Tenant ID F1 F2 Record ID 764 Ted $ John $32 Null 783 Sal $ Tenant ID Extension ID Extension Label Datatype Record ID Extension ID Value Status ntext Expire datetime Affiliation ntext Gold Acme

24 Extension-value pairs Advantage Unlimited number/option for custom fields Tradeoff Increase index/query/update complexity and delay When to use OK to co-mingle tenant data Custom fields are high value features Difficult to predict custom fields

25 Scaling Multi-tenant Data

26 2 Level Data Scaling Strategy Multi-Tenant Database Multi-Tenant Database Multi-Tenant Data Scaling Strategy Single Tenant Database Single Tenant Data Scaling Strategy Databases for one tenant

27 Horizontal Tenant Partitioning Too many concurrent users or / and Too many rows Redistribute tenant user load against baseline Redistribute tenant data load against baseline

28 Single Tenant Scaling Architecture Guidance: Scaling Out SQL Server 2005 paper by Roger Wolter Replicated reference data Requires deep application introspection: Revisit data model Study reference, resource and activity data dependency, frequency of change. Identify candidates for horizontal and vertical partitioning, replication, scaleup Replicated reference data Reference data Replicated reference data Some best practices: Replicate reference data Replicated reference data replica Keep related data near replica Write Master replica Less related Use scaleup for data that cannot be partitioned (resource data) replica More related Use single master replication rather than synchronize write

29 Conclusions Single instance, multi-tenancy is hard work Without the architecture patterns, we ll be limited to the isolation option

30