Windows Azure Storage. Hakan Soncu

Size: px

Start display at page:

Download "Windows Azure Storage. Hakan Soncu"

MargaretMargaret Poole
5 years ago
Views:

1 Windows Azure Storage Hakan Soncu

4 West US East US South Central US

Block Id 2 Block Id 0 Block Id 3 Block Id 4

Block Id 3 Block Id 4 Block Id 2 Block Id 4

Operations PutBlock BlockId1 PutBlock BlockId3

BlockId4 PutBlockList(u=BlockId2, c=blockid0,

19 Block Id 2 Block Id 0 Block Id 3 Block Id 4 Block Id 0 Block Id 3 Block Id 5 Block Id 1 Block Id 3 Block Id 4 Block Id 2 Block Id 4 Example Blob Committed Uncommitted Sequence of Operations PutBlock BlockId1 PutBlock BlockId3 PutBlock BlockId4 PutBlock BlockId2 PutBlock BlockId4 PutBlockList(u=BlockId2, c=blockid0, u=blockid3, u=blockid4) Committed and readable version of blob

31 CloudStorageAccount account = CloudStorageAccount.FromConfigurationSetting("CloudStorageAccount"); //Initialize the local cache for drives mounted by this role instance CloudDrive.InitializeCache(localCacheDir, cachesizeinmb); //Create a cloud drive (PageBlob) CloudDrive drive = account.createclouddrive(pagebloburi); drive.create(1000 /* sizeinmb */); //Mount the network attached drive on the local file system string pathonlocalfs = drive.mount(cachesizeinmb, DriveMountOptions.None); //Use NTFS APIs to Read/Write files to drive //Snapshot drive while mounted to create backups Uri snapshoturi = drive.snapshot(); //Unmount the drive drive.unmount();

35 Table Details

41 Table

42 Scalability Query Efficiency & Speed Entity group transactions 42

43 Table for listing all movies 43

44 Why do I need multiple PartitionKeys? Does this scale? PartitionKey (Account name) RowKey (Title) Category moviesonline 12 Rounds Action moviesonline A Bug s Life Animation 100,000,000 more rows moviesonline Office Space Comedy moviesonline Platoon War 50,000,000 more rows moviesonline WALL-E Animation 44

Animation 100,000,000 more rows moviesonline Office Space Comedy

45 Client PartitionKey (Account name) RowKey (Title) Category moviesonline 12 Rounds Action moviesonline A Bug s Life Animation 100,000,000 more rows moviesonline Office Space Comedy moviesonline Platoon War 50,000,000 more rows moviesonline WALL-E Animation 45

46 Client PartitionKey RowKey (Title) (Category) (Title) Action Fast & Furious more Action movies Action The Bourne Ultimatum more Action & Animation movies Animation Animation Open Season Open Season more Animation movies more Animation movies Animation Animation The Ant Bully The Ant Bully Comedy Comedy Office Space Office Space more Comedy & SciFi movies more Comedy & SciFi movies SciFi SciFi Star Trek Star Trek more SciFi & War movies more SciFi War movies more War movies more War movies War Defiance 46

47 Log every transaction into a table for diagnostics 47

48 Timestamp as Partition Key PartitionKey (Timestamp) :00: :00: more rows :40: :40:01 Properties Applications Client more rows :30: :30: :30: :30: :30:03 48

PartitionKey (ID_Timestamp) 01_2009-10-12 05:10:00 01_2009-11-17 12:30:01

20000000 more rows Applications Client 10_2009-10-05 05:10:10 5000000 more

49 PartitionKey (ID_Timestamp) 01_ :10:00 01_ :30:01 Properties more rows 09_ :31:00 09_ :30: more rows Applications Client 10_ :10: more rows 15_ :30: more rows 19_ :20:02 19_ :30:10 49

50 50

Where PartitionKey= SciFi and RowKey = Star Trek Client PartitionKey (Category) Action Action RowKey (Title) Fast & Furious 10000 more Action movies The Bourne Ultimatum Animation Open Season 2

51 Where PartitionKey= SciFi and RowKey = Star Trek Client PartitionKey (Category) Action Action RowKey (Title) Fast & Furious more Action movies The Bourne Ultimatum Animation Open Season 2 Animation more Action & Animation movies more Animation movies The Ant Bully Comedy SciFi War Office Space more Comedy & SciFi movies Star Trek more SciFi & War movies more War movies Defiance 51

52 PartitionKey (Category) RowKey (Title) Rating Client Action Fast & Furious more movies rated > 4 Returns 1000 movies Action and Anim. movies here with rating < 4 Partition range boundary hit Animation A Bug s life more movies < 4 here Animation The Ant Bully 3 Comedy Are we there yet? 2 Return continuation More movies here Comedy Office Space more movies here Drama A Beautiful Mind more movies here War Defiance 4 52

Earth 4 More movies here Drama Seven Pounds 4 Client Horror Saw 5 3 More

53 PartitionKey (Category) RowKey (Title) Rating Action Fast & Furious 5 More movies here Comedy Office Space 5 More movies here Documentary Planet Earth 4 More movies here Drama Seven Pounds 4 Client Horror Saw 5 3 More movies here Music 8 Mile 2 More movies here SciFi Star Trek 5 More movies here 53

54 1. Fast 2. Medium 3. Slow 1 54

55 Faster Single PartitionKey and RowKey with equality Create and maintain own views that help queries Or Predicates Execute individual query in parallel instead of using OR User Interactive Cache the result to reduce scan frequency 55

56 56

57 57

61 An account can create many queues A Queue contains messages Messages 61

64 Input Queue (Work Items) Queue Guarantee delivery/processing of messages (two-step consumption)

65 read at least once pattern (not only once!) No stringent FiFo support Use Dequeue count to remove poison messages Make use of message count Max TTL = 7 days; Max VisibilityTimeout = 2 hours Use blob store for large messages

67 Thank You Questions?

Azure-persistence MARTIN MUDRA

Azure-persistence MARTIN MUDRA Storage service access Blobs Queues Tables Storage service Horizontally scalable Zone Redundancy Accounts Based on Uri Pricing Calculator Azure table storage Storage Account