Enzo Framework Developer Guide

Transcription

1 Enzo Framework Developer Guide This document provides technical information about the Enzo Framework for developers and architects. Blue Syntax Consulting specializes in the Microsoft Azure platform and provides architectural and development guidance to corporations leveraging cloud computing technologies. Created by Blue Syntax Consulting Published on 12/16/2010 Version Copyright (C) Blue Syntax Consulting Page 1

2 Contents Introduction... 3 Pre-Requisites... 3 Planning for Scalability... 3 Implementing a Linear Shard... 5 Tracking Temperatures... 5 How to Add Servers... 5 How to Create a Shard Cluster... 6 How to Create a Shard... 6 How to Add Databases... 7 Physical Configuration... 9 TEMP_DB1_ TEMP_DB2_ A Note about Referential Integrity Code Sample Implementing an Expanded Shard Overview Changing Data Configuration Code Sample Implementing a Compressed Shards Overview Security Performance Configuration Sample Code Summary About Blue Syntax Consulting Copyright (C) Blue Syntax Consulting Page 2

3 Introduction This document provides technical details on how to implement a scale out application using the Enzo Multitenant Framework. It covers high-level information explaining the various scalability models and how to configure and code against the framework to achieve the desired level of performance and scalability. Pre-Requisites The framework requires.net 4.0 and relies on SQL Azure and/or SQL Server depending on the features implemented. In addition you will need the Enzo Framework API to execute your SQL code against the underlying database model. Planning for Scalability The framework allows you to implement various scalability models, and to change the scalability model over time under most conditions. The following scalability options are supported by the Enzo framework: Scale Up Compressed Scale Out (or Compressed Shard) Linear Scale Out (or Linear Shard) Expanded Scale Out (or Expanded Shard) Although the scale up model is trivial to implement and does not require the Enzo framework per say, using the guidelines provided in this document will allow you to more easily switch to another model over time, as your application needs change. In fact, the application code does not change depending on the scalability model under most circumstances; what changes is the underlying physical configuration of the Enzo framework in the management interface. The code, if implemented correctly, should not know which scalability model is being implemented. This provides maximum application reconfiguration flexibility. Application Code Data Access Layer Application Code Data Access Layer Enzo Framework API Configuration DB DB1 DB2 DB3 DB4 Figure 1 - Enzo Framework API Model Figure 1 shows the typical difference between an application that connects to a single database using a Data Access Layer and another application that uses the Enzo Framework API. The code continues to Copyright (C) Blue Syntax Consulting Page 3

4 access the same Data Access Layer and requires very few changes in most cases. The Data Access Layer is modified to use the Enzo Framework API instead of ASP.NET or Linq to SQL and is able to make database requests across one or more databases. Once implemented, the actual physical deployment model of the databases becomes irrelevant to the Data Access Layer. Databases could be installed in SQL Azure, or moved to local SQL Server databases without any changes to application code. The Enzo Framework API depends on a configuration database; the Enzo Configuration database. When creating databases in SQL Azure or SQL Server to support the application needs, these databases must be added to the framework configuration. Once added to the Enzo Configuration database, it usually needs to be assigned one or more attributes that represents the type of data it stores. While not required, the attributes assigned to a database represent meta-data that will be used in your application code (usually in the Data Access Layer). Shards can also have attributes so they, too, can be referenced from your application code. This allows flexible data separation for certain applications. We will review this later in this document. Attributes assigned to shards and databases allow your code to define a context in which a SQL statement needs to run. Depending in the values of those attributes, a SQL statement can be executed against 0, 1 or more databases in parallel. Parallel execution is automatic and can be controlled by the management interface. An attribute is a string that is added in the management interface and assigned a value, a range of values, a date range or a regular expression. At runtime, when your code selects databases (or shards) by attribute, the value provided is checked against the select shard(s). The application databases are referred to as the physical model layer while the Enzo Framework API is referred to as the logical model layer. How you implement the physical layer impacts your scalability choices and performance characteristics. When configuring your databases, you must choose how many databases need to be deployed, and how to configure the attributes to make the databases available to your code. Copyright (C) Blue Syntax Consulting Page 4

5 Implementing a Linear Shard Before diving into the specifics of the application code, let's look at the logical implementation of your databases and let's make certain design decisions which will be used later in code samples. Tracking Temperatures Let's assume you are tracking worldwide temperatures by city and you would like to define a scalability model based on two attributes: Country and Year. Initially you are tracking two countries: the United States of America and Canada, and for the moment you are only tracking temperatures for You could have chosen the City as your attribute; that would also work; however it is up to you to define which level of scalability and performance you are looking for. As you can see, defining your attributes impacts the number of databases you will need to create. So Country and Year are defined as logical attributes and City will be kept in the database as a primary key. You will then need two databases. Let's name them arbitrarily TEMP_DB1_2010 and TEMP_DB2_2010. They belong to a shard we call ShardTemp and the shard itself belongs to a shard cluster called WorldTemp. Shard Cluster Shard Database Attribute Value WorldTemp ShardTemp TEMP_DB1_2010 Country US Year 2010 WorldTemp ShardTemp TEMP_DB2_2010 Country Canada Year 2010 This model allows you to easily scale over time and not worry too much about how many countries you will be tracking in three years from now. How to Add Servers At this point you will need to login to the Enzo Multitenant Management Studio. You can find the management portal at Before you can add clusters and shards, you will need to define at least one server. A server is a database server on which databases can be found. To add a server, select Environment Configuration -> Servers. Enter the details of the server and click on Add Server. The User ID and Password need to have sufficient rights to create databases. The server should be the fully qualified name of the database server on which databases are found. Choose Default as the Server Pool. Copyright (C) Blue Syntax Consulting Page 5

6 Note that you can add database servers from your local network. So a server name could be YOURDOMAIN\SVRNAME. Always specify the port (by default the listening port is 1433). How to Create a Shard Cluster To create a new Cluster called WorldTemp, follow these steps: Select Environment Configuration -> Clusters and Shards Type the name of the cluster, a description, and click Add Cluster A new Shard Cluster will be added to the list of clusters Figure 2 - Adding a Shard Cluster A shard cluster is an entry point for the Enzo Framework API. The API cannot query across shard clusters. So this provides a form of isolation for your application code. You could for example create a shard cluster for Development, Test and Production environments. How to Create a Shard Select the newly created cluster from the left menu (you may need to refresh the page). You will notice a screen with an area allowing you to create a new shard. Enter the name of the shard and click on Add Shard. Copyright (C) Blue Syntax Consulting Page 6

7 Figure 3 - Adding a Shard Next to the shard name you will find four links: Edit, Delete, Attributes and Default. Edit: Allows you to modify the name of the shard Delete: Deletes the shard from the configuration database; does not drop the databases Deleting a shard usually prevents the underlying databases from being accessible from your application code Attributes: View and Modify the list of attributes for this shard Default: Make this shard the Default shard Making a shard the default provides a programmatic shortcut After creating the shard, you will notice that it will appear under the WorldTemp cluster on the left menu. Click on the Default icon; this will make the shard easier to select in application code. If we had additional shards we would need to create attributes so that the correct shard could be selected by your code. How to Add Databases Click on the ShardTemp item on the left menu. You will notice a screen with the ability to add databases. Adding a database on this screen does not physically create a database; instead it adds a definition that points to a database. The database does not need to exist at the time it is being added. Specify the name of the database and a User ID and Password. A shard can be made of databases stored of different servers. Copyright (C) Blue Syntax Consulting Page 7

8 Figure 4 - Add a Database Definition After adding both databases your screen should look like the one in Figure 5. Note that none of the databases below are marked as the Default database. Marking a database the Default could be useful to simplify your code if a database would serve as the primary data store of your application. Having a default database allows your code to select a database without specifying attributes. In our case however, we will be using the attributes to select the correct database. Figure 5 - Databases Created Once the databases have been created we need to add attributes to each database. Click on the Attribute icon in front of DB1. Figure 6 shows the screen that allows you to add database attributes. Add the Country and Year attributes to both DB1 and DB2. You can switch database using the dropdown box. Once you are done, click on the 'back to ShardTemp' link. Figure 7 shows the attributes defined for DB1. Both attributes are defined as a Single Value (a string). Copyright (C) Blue Syntax Consulting Page 8

9 Figure 6 - Add a Database Attribute Figure 7 - DB1 Attributes At this point you have created a server, a shard cluster, a shard, two database references and two attributes on each database. In addition the shard ShardTemp is marked as the default shard. You are now ready to create the physical databases. Physical Configuration You now need to create the databases on the actual SQL Azure servers. Using the SQL Azure console (sql.azure.com) or SQL Server Management Studio, create two databases TEMP_DB1_2010 and TEMP_DB2_2010. Once the databases have been created, run the following statements: TEMP_DB1_2010 Open a query window in SQL Server Management Studio and paste the following T-SQL to create the tables and temperatures for 2010 for the US cities: CREATE TABLE Country (id int identity(1,1) primary key, name nvarchar(50)) CREATE TABLE City(id int identity(1,1) primary key, name nvarchar(50), countryid int not null) CREATE TABLE Temp (id int identity(1,1) primary key, value decimal, monthtaken nvarchar(3), measure char(1) not null, countryid int not null, cityid int not null) INSERT INTO country values ('US') INSERT INTO country values ('Canada') INSERT INTO City Values ('New York', 1) INSERT INTO City Values ('San Francisco', 1) INSERT INTO City Values ('Chicago', 1) INSERT INTO City Values ('Vancouver', 2) INSERT INTO City Values ('Quebec', 2) -- US Copyright (C) Blue Syntax Consulting Page 9

10 INSERT INTO Temp VALUES (29.3, 'Jan', 'F', 2, 3) INSERT INTO Temp VALUES (33.6, 'Feb', 'F', 2, 3) INSERT INTO Temp VALUES (45.1, 'Mar', 'F', 2, 3) INSERT INTO Temp VALUES (58.5, 'Apr', 'F', 2, 3) INSERT INTO Temp VALUES (70.2, 'May', 'F', 2, 3) INSERT INTO Temp VALUES (79.3, 'Jun', 'F', 2, 3) INSERT INTO Temp VALUES (83.5, 'Jul', 'F', 2, 3) INSERT INTO Temp VALUES (82.0, 'Aug', 'F', 2, 3) INSERT INTO Temp VALUES (74.7, 'Sep', 'F', 2, 3) INSERT INTO Temp VALUES (63.0, 'Oct', 'F', 2, 3) INSERT INTO Temp VALUES (47.8, 'Nov', 'F', 2, 3) INSERT INTO Temp VALUES (34.3, 'Dec', 'F', 2, 3) INSERT INTO Temp VALUES (56.3, 'Jan', 'F', 2, 2) INSERT INTO Temp VALUES (59.5, 'Feb', 'F', 2, 2) INSERT INTO Temp VALUES (60.8, 'Mar', 'F', 2, 2) INSERT INTO Temp VALUES (63.7, 'Apr', 'F', 2, 2) INSERT INTO Temp VALUES (66.2, 'May', 'F', 2, 2) INSERT INTO Temp VALUES (68.9, 'Jun', 'F', 2, 2) INSERT INTO Temp VALUES (70.0, 'Jul', 'F', 2, 2) INSERT INTO Temp VALUES (70.0, 'Aug', 'F', 2, 2) INSERT INTO Temp VALUES (71.6, 'Sep', 'F', 2, 2) INSERT INTO Temp VALUES (69.4, 'Oct', 'F', 2, 2) INSERT INTO Temp VALUES (62.4, 'Nov', 'F', 2, 2) INSERT INTO Temp VALUES (56.5, 'Dec', 'F', 2, 2) INSERT INTO Temp VALUES (35.8, 'Jan', 'F', 2, 1) INSERT INTO Temp VALUES (37.8, 'Feb', 'F', 2, 1) INSERT INTO Temp VALUES (45.9, 'Mar', 'F', 2, 1) INSERT INTO Temp VALUES (57.0, 'Apr', 'F', 2, 1) INSERT INTO Temp VALUES (67.1, 'May', 'F', 2, 1) INSERT INTO Temp VALUES (75.4, 'Jun', 'F', 2, 1) INSERT INTO Temp VALUES (80.8, 'Jul', 'F', 2, 1) INSERT INTO Temp VALUES (80.8, 'Aug', 'F', 2, 1) INSERT INTO Temp VALUES (72.9, 'Sep', 'F', 2, 1) INSERT INTO Temp VALUES (63.3, 'Oct', 'F', 2, 1) INSERT INTO Temp VALUES (52.0, 'Nov', 'F', 2, 1) INSERT INTO Temp VALUES (41.9, 'Dec', 'F', 2, 1) TEMP_DB2_2010 Similarly run the following T-SQL statement for the second database for Canada. CREATE TABLE Country (id int identity(1,1) primary key, name nvarchar(50)) CREATE TABLE City(id int identity(1,1) primary key, name nvarchar(50), countryid int not null) CREATE TABLE Temp (id int identity(1,1) primary key, value decimal, monthtaken nvarchar(3), measure char(1) not null, countryid int not null, cityid int not null) INSERT INTO country values ('US') INSERT INTO country values ('Canada') INSERT INTO City Values ('New York', 1) INSERT INTO City Values ('San Francisco', 1) INSERT INTO City Values ('Chicago', 1) INSERT INTO City Values ('Vancouver', 2) INSERT INTO City Values ('Quebec', 2) INSERT INTO Temp VALUES (18.9, 'Jan', 'F', 2, 5) INSERT INTO Temp VALUES (21.7, 'Feb', 'F', 2, 5) INSERT INTO Temp VALUES (32.2, 'Mar', 'F', 2, 5) INSERT INTO Temp VALUES (46.4, 'Apr', 'F', 2, 5) INSERT INTO Temp VALUES (62.1, 'May', 'F', 2, 5) INSERT INTO Temp VALUES (71.8, 'Jun', 'F', 2, 5) INSERT INTO Temp VALUES (77.0, 'Jul', 'F', 2, 5) INSERT INTO Temp VALUES (74.1, 'Aug', 'F', 2, 5) INSERT INTO Temp VALUES (64.6, 'Sep', 'F', 2, 5) INSERT INTO Temp VALUES (52.3, 'Oct', 'F', 2, 5) INSERT INTO Temp VALUES (37.8, 'Nov', 'F', 2, 5) INSERT INTO Temp VALUES (23.4, 'Dec', 'F', 2, 5) INSERT INTO Temp VALUES (41, 'Jan', 'F', 2, 4) INSERT INTO Temp VALUES (44.1, 'Feb', 'F', 2, 4) INSERT INTO Temp VALUES (49.8, 'Mar', 'F', 2, 4) INSERT INTO Temp VALUES (56.8, 'Apr', 'F', 2, 4) INSERT INTO Temp VALUES (63, 'May', 'F', 2, 4) INSERT INTO Temp VALUES (68.5, 'Jun', 'F', 2, 4) INSERT INTO Temp VALUES (72, 'Jul', 'F', 2, 4) Copyright (C) Blue Syntax Consulting Page 10

11 INSERT INTO Temp VALUES (71.4, 'Aug', 'F', 2, 4) INSERT INTO Temp VALUES (64.9, 'Sep', 'F', 2, 4) INSERT INTO Temp VALUES (57.2, 'Oct', 'F', 2, 4) INSERT INTO Temp VALUES (49.1, 'Nov', 'F', 2, 4) INSERT INTO Temp VALUES (42.6, 'Dec', 'F', 2, 4) A Note about Referential Integrity You may have noticed that the Country and City have the same records in both databases. This is an implementation choice; nothing prevents you from removing the Country table altogether since the database is already assigned to a country attribute of US. However in order to keep the data model flexible for potential future change, and for simpler referential integrity on the application layer, keeping the Country table makes sense in this case. Code Sample The two databases created previously are not directly visible to your application; however the Enzo Framework API knows about the physical databases since it connects to the Enzo Configuration database when the Shard Cluster object is instantiated. You code would simply provide the attribute(s) necessary to inform the Enzo Framework API which database(s) to use. string connectionstring = "Data Source=..."; ShardCluster sc = new ShardCluster(connectionString, "WorldTemp"); // Returns a DataTable with records from all databases in the default shard DataTable dtall = sc.defaultshard.executedatatable("select AVG(value), CityId FROM Temp Group By CityId"); // Returns a DataTable with records from US only Shard.DBExecutionOption dbeo1 = new Shard.DBExecutionOption(); dbeo1.attributes.add("country", "US"); DataTable dtcan = sc.defaultshard.executedatatable("select AVG(value), CityId FROM Temp Group By CityId", dbeo1); // Returns data from 2009 Shard.DBExecutionOption dbeo2 = new Shard.DBExecutionOption(); dbeo2.attributes.add("year", "2009"); DataTable dt2009 = sc.defaultshard.executedatatable("select AVG(value), CityId FROM Temp Group By CityId", dbeo2); If you were to insert records into the underlying databases you would need to specify the correct attribute. For example if you would like to insert a temperature in a US city you would specify the Country attribute of 'US' and send the INSERT statement in a manner similar to the sample code provided above. Copyright (C) Blue Syntax Consulting Page 11

12 Implementing an Expanded Shard Overview An expanded shard implementation offers a different viewpoint on your data. Instead of partitioning your data statically across database using a data domain, such as a Country code, your data becomes loosely coupled with the underlying databases. Each record could theoretically live in any database that makes up the shard. Although your code no longer knows which database holds which data, your underlying database storage can grow independently from your data domains. In other words, you could be tracking 2 countries (US and Canada) across 100 databases. Changing Data It becomes important to use the concept of a database GUID when you intend to update or delete records from an expanded shard. Indeed, the ExecuteDataTable method of the Shard object returns a database GUID for each record returned, indicating from which database a record came from. Updating or deleting the record then requires sending the database GUID back to the Enzo Framework API for processing, which is done using the BSC.EnzoAzureLib.DBLoadOption.Specific enum. Configuration Let's reuse the previous example and transform the model to use an expanded shard. Let's create a third database, TEMP_DB3_2010. Then run the following T-SQL statement which adds temperatures to New York and Quebec in Celsius: CREATE TABLE Country (id int identity(1,1) primary key, name nvarchar(50)) CREATE TABLE City(id int identity(1,1) primary key, name nvarchar(50), countryid int not null) CREATE TABLE Temp (id int identity(1,1) primary key, value decimal, monthtaken nvarchar(3), measure char(1) not null, countryid int not null, cityid int not null) INSERT INTO country values ('US') INSERT INTO country values ('Canada') INSERT INTO City Values ('New York', 1) INSERT INTO City Values ('San Francisco', 1) INSERT INTO City Values ('Chicago', 1) INSERT INTO City Values ('Vancouver', 2) INSERT INTO City Values ('Quebec', 2) INSERT INTO Temp VALUES (2.1, 'Jan', 'C', 2, 1) INSERT INTO Temp VALUES (3.2, 'Feb', 'C', 2, 1) INSERT INTO Temp VALUES (-7.4, 'Jan', 'C', 2, 1) INSERT INTO Temp VALUES (-6.1, 'Feb', 'C', 2, 1) The major difference in this model, as you can see, is that a database is no longer attached to a uniquely identifying set of attributes. This doesn't mean however that there are no attributes defined; the Year attribute could still be used to identify a shard storing 2011 data versus another shard storing 2010 data. To minimize your code changes, you need to keep the attributes that logically separate your records, so that given any attribute value the records returned remain homogeneous. In our example you would keep using the Country attribute, and have 10 databases making up the US and 8 making up Canada for Copyright (C) Blue Syntax Consulting Page 12

13 example. Although the Country/Year attribute combination no longer points to a single database, records are homogeneous given an attribute value. Although we have defined attributes for the first two databases (TEMP_DB1_2010 and TEMP_DB2_2010), we have not yet defined any attributes on TEMP_DB3_2010. At this point you need to add the Country and Year attribute to this newly created database. Let's assign the new database to the Country 'US'. So we now have three databases: Shard Cluster Shard Database Attribute Value WorldTemp ShardTemp TEMP_DB1_2010 Country US Year 2010 WorldTemp ShardTemp TEMP_DB2_2010 Country Canada Year 2010 WorldTemp ShardTemp TEMP_DB3_2010 Country US Year 2010 Code Sample As you can see from the above table there are two tables making up the US country. Selecting data in this model would look like this: string connectionstring = "Data Source=..."; ShardCluster sc = new ShardCluster(connectionString, "WorldTemp"); // Returns a DataTable with records from all databases in the default shard DataTable dtall = sc.defaultshard.executedatatable("select AVG(value), CityId FROM Temp Group By CityId"); // Returns a DataTable with records from US only Shard.DBExecutionOption dbeo1 = new Shard.DBExecutionOption(); dbeo1.attributes.add("country", "US"); DataTable dtcan = sc.defaultshard.executedatatable("select AVG(value), CityId FROM Temp Group By CityId", dbeo1); // Returns data from 2009 Shard.DBExecutionOption dbeo2 = new Shard.DBExecutionOption(); dbeo2.attributes.add("year", "2009"); DataTable dt2009 = sc.defaultshard.executedatatable("select AVG(value), CityId FROM Temp Group By CityId", dbeo2); Although the physical implementation has changed, nothing has changed at the application level. Retrieving data for the US remains the same. Behind the scenes the Enzo Framework API detects that the Country attribute with a value of US is actually assigned to two databases, and thus executes the statement against both databases transparently. Inserting data in an expanded shard should be done in a manner that attempts to evenly distribute records when they are added, so that each database given a Country has a good probability of an equal number of records. The Enzo Framework API offers a method to load records in parallel in a round-robin manner. Shard.DBExecutionOption dbeo1 = new Shard.DBExecutionOption(); dbeo1.attributes.add("country", "US"); List<string> sqls = new List<string>(); sqls.add("insert INTO Temp VALUES (32.2, 'Mar', 'F', 2, 5)"); sqls.add("insert INTO Temp VALUES (32.0, 'Feb', 'F', 2, 5)"); sqls.add("insert INTO Temp VALUES (27.1, 'Nov', 'F', 2, 5)"); sqls.add("insert INTO Temp VALUES (33.1, 'Dec', 'F', 2, 5)"); Copyright (C) Blue Syntax Consulting Page 13

14 sc.defaultshard.executeparallelroundrobinload(sqls, dbeo1); The ExecuteParallelRoundRobinLoad executes the INSERT statements in parallel against the pool of databases found under the Default Shard in the sample code above. Each database used to store US temperatures would receive two records in the example above. The expanded shard scalability model provides the most flexibility and opportunity for performance improvements since a shard could host a large number of databases independently, or in conjunction with the data domains applicable to your application. When attributes are used in the Expanded Shard model, they are not meant to refer to a single database (like in the Linear Model); they are used to identify a pool of databases that store related information. Copyright (C) Blue Syntax Consulting Page 14

15 Implementing a Compressed Shards Overview In a compressed shard scenario the design favors a smaller database footprint in an attempt to host multiple customers in the same database while keeping strong data segregation. Compressed Shards leverage database schemas heavily; each customer is assigned its own database schema although it may be co-located with other customers in the same database. In addition to leveraging schemas, this scalability model leverages the default schema property of a login. This feature of SQL Server and SQL Azure automatically set the correct schema context of users when executing SQL statements. For example, if Usr1 is assigned schema1 as it default schema, a call to "SELECT * FROM USERS" will automatically be executed under schema1, which is equivalent to "SELECT * FROM schema1.users". Because Usr1 is mapped to schema1, resolving the context to schema1 is automatic. The default schema mapping allows database code to be written without schema-specific information, letting the application resolve the schema context when the login takes place. This also means that your application should not depend on multiple schemas to drive specific features. Security From a security standpoint, you need to ensure that each Login is only allowed to logon to the correct database and the associated user is given rights to the schema it needs access to. This ensures that customer databases assigned to a specific schema are not available to other Logins and database users. Performance You should note that while this scalability model minimizes the use of databases, which itself reduces the overall cost of deployment, some databases are sharing the same underlying server resources (memory, disk, locks...). As a result your application code may be hitting certain thresholds sooner, such as the number of concurrent connections to a SQL Azure database. In fact, carefully monitoring connection pooling in this scalability model becomes absolutely critical for general database availability. Configuration Let's take our previous example and store all the databases in the same database, under a specific schema as shown below. Note the different from previous tables, where the schema is used to resolve the context of a database. Login1 (which maps to database user Usr1) is configured to use the USA schema automatically, while Login2 (which maps to database Usr2) will use the CAN schema by default. Shard Cluster Shard Database.Schema Login/User Attribute Value WorldTemp ShardTemp TEMP_DB1_2010.USA Login1/Usr1 Country US Year 2010 WorldTemp ShardTemp TEMP_DB1_2010.CAN Login2/Usr2 Country Canada Year 2010 Copyright (C) Blue Syntax Consulting Page 15

16 To implement the compressed shard you will need to take the following configuration steps. Create one database called TEMP_DB1_2010 o if you created it previously, drop it and recreate it Create two schemas: USA and CAN o Run these two statements on the newly created database CREATE SCHEMA USA CREATE SCHEMA CAN Create two logins on the master database CREATE LOGIN Login1 WITH PASSWORD = 'strongpasswordgoeshere' CREATE LOGIN Login2 WITH PASSWORD = 'strongpasswordgoeshere' Create database users mapped to these logins on the TEMP_DB1_2010 database CREATE USER Usr1 FOR LOGIN Login1 CREATE USER Usr2 FOR LOGIN Login2 Grant access to users' respective schemas o Granting access rights to users on schemas is the basis of compressed shard security GRANT SELECT, INSERT,DELETE, UPDATE ON SCHEMA ::USA TO usr1 GRANT SELECT, INSERT,DELETE, UPDATE ON SCHEMA ::CAN TO usr2 Change the default schemas of both users in the TEMP_DB1_2010 database ALTER USER usr1 DEFAULT_SCHEMA = USA ALTER USER usr2 DEFAULT_SCHEMA = CAN Run the following statements to create objects in both schemas o The beginning of each script is shown below. Note the schema used in front of objects. This is only used during the creation of those objects. Script1 CREATE TABLE USA.Country (id int identity(1,1) primary key, name nvarchar(50)) CREATE TABLE USA.City(id int identity(1,1) primary key, name nvarchar(50), countryid int not null) CREATE TABLE USA.Temp (id int identity(1,1) primary key, value decimal, monthtaken nvarchar(3), measure char(1) not null, countryid int not null, cityid int not null) INSERT INTO USA.country values ('US') INSERT INTO USA.country values ('Canada') INSERT INTO USA.City Values ('New York', 1) INSERT INTO USA.City Values ('San Francisco', 1) INSERT INTO USA.City Values ('Chicago', 1) INSERT INTO USA.City Values ('Vancouver', 2) INSERT INTO USA.City Values ('Quebec', 2) INSERT INTO USA.Temp VALUES (29.3, 'Jan', 'F', 2, 3) INSERT INTO USA.Temp VALUES (33.6, 'Feb', 'F', 2, 3) Script2 CREATE TABLE CAN.Country (id int identity(1,1) primary key, name nvarchar(50)) CREATE TABLE CAN.City(id int identity(1,1) primary key, name nvarchar(50), countryid int not null) CREATE TABLE CAN.Temp (id int identity(1,1) primary key, value decimal, monthtaken nvarchar(3), measure char(1) not null, countryid int not null, cityid int not null) INSERT INTO CAN.country values ('US') INSERT INTO CAN.country values ('Canada') INSERT INTO CAN.City Values ('New York', 1) INSERT INTO CAN.City Values ('San Francisco', 1) INSERT INTO CAN.City Values ('Chicago', 1) Copyright (C) Blue Syntax Consulting Page 16

17 INSERT INTO CAN.City Values ('Vancouver', 2) INSERT INTO CAN.City Values ('Quebec', 2) INSERT INTO CAN.Temp VALUES (18.9, 'Jan', 'F', 2, 5) INSERT INTO CAN.Temp VALUES (21.7, 'Feb', 'F', 2, 5) Change the Shard configuration in the Enzo Multitenant Framework Studio o The logical configuration does not have to change much; you do need however to change the Login/Password and the Database Name specified for each connection. Sample Code Now that the physical configuration is completed, let's take a look at the code we have previously analyzed to see what needs to change. string connectionstring = "Data Source=..."; ShardCluster sc = new ShardCluster(connectionString, "WorldTemp"); // Returns a DataTable with records from all databases in the default shard DataTable dtall = sc.defaultshard.executedatatable("select AVG(value), CityId FROM Temp Group By CityId"); // Returns a DataTable with records from US only Shard.DBExecutionOption dbeo1 = new Shard.DBExecutionOption(); dbeo1.attributes.add("country", "US"); DataTable dtcan = sc.defaultshard.executedatatable("select AVG(value), CityId FROM Temp Group By CityId", dbeo1); // Returns data from 2009 Shard.DBExecutionOption dbeo2 = new Shard.DBExecutionOption(); dbeo2.attributes.add("year", "2009"); DataTable dt2009 = sc.defaultshard.executedatatable("select AVG(value), CityId FROM Temp Group By CityId", dbeo2); Once again, the code has not changed. Selecting data from your application remains identical. Furthermore there is no need to change the SQL statements to point to a specific schema; this is done automatically based on the user assigned to a specific database. Even the first SQL statement pulling records from multiple databases will be executed with the correct schema context. Copyright (C) Blue Syntax Consulting Page 17

18 Summary This document introduced you to the various configuration modes available for scalability along with instructions on how to configure the physical and logical layers of yours shards. The following table shows you important considerations based on the scalability model chosen. You can see that inserting, deleting and updating data could be impacted depending on the scalability model chosen. For the most part however, selecting data is not affected unless the property attributes or values change themselves. Linear Shard Expanded Shard Compressed Shard Each data domain has its own dedicated database is mapped to one or more databases has its own schema in a co-located database Unique attributes map to one database one or more databases one login/user Insert records using ExecuteNonQuery ExecuteParallelRoundRobin ExecuteNonQuery Update / Delete with attribute values database GUID attribute values Choosing the correct scalability is a combination of both business and technical objectives that requires careful evaluation. However, if business of technical objectives change over time, the level of abstraction provided by the Enzo API Framework can help minimize code changes if the scalability model changes over time. Copyright (C) Blue Syntax Consulting Page 18

19 About Blue Syntax Consulting Blue Syntax Consulting provides high value-added services to customers by offering high level executive guidance on how to best leverage Cloud Computing technologies with the Microsoft Azure platform and low-level implementation services, including architecture guidance, development and testing. Blue Syntax Consulting develops leading edge technologies allowing corporations to adopt cloud computing sooner, including the Enzo Multitenant Framework and other solutions. For more information, visit or contact Copyright (C) Blue Syntax Consulting Page 19