Enterprise Scaling with AZURE STORAGE and AZURE SERVICE BUS Mark Cassidy, Sitecore MVP May 18th, 2017 #sugcon df
In this session. A short introduction Some real life problems Some architecture theory Some code and config Live demo with fingers crossed 2
A Short Introduction These are the technologies and concepts we will be talking about.. Azure Storage Specifically Azure Message Queue Azure Service Bus And briefly Enterprise Service Bus (ESB) Technology OMS, DMS and xdb
Raise of Hands How many of you have Worked with or know of standard Message Queues (MSMQ)? Worked with or know of ESB technology? Migrated Sitecore solutions across OMS, DMS and xdb?
The enterprise challenge. Multiple loosely coupled systems Especially with ecommerce Websites need to respond quickly Scalability In real life, systems sometimes break Data needs to be offloaded, Unconditionally and securely 5
But also. Multidisciplinary teams Not everyone knows/wants to know Sitecore Governance of systems split People working on Website rarely in the same group as ERP people Separate release schedules 6
Real life problem: Scalability. Off peak. Visitors Web Server Log Data SQL Server Storage
Real life problem: Scalability. Lunch hour. Visitors Web Server Log Data SQL Server Storage
Real life problem: Scalability. Lunch hour. Visitors Web Server Log Data SQL Server Storage
Real life problem: Scalability. One solution. Visitors Web Server Log Data SQL Server Storage
Enter => The Message Queue. MQ.
Message Queue Characteristics General properties of (almost) any MQ Guaranteed Message Delivery Efficient Routing Security Priority Based Messaging Use cases: High Performance Required Async and Sync scenarios Disconnected Systems Systems Mission-critical systems E.g. Electronic Commerce Embedded Communication to and from embedded devices that drop in and out of connectivity Off-site/outside sales E.g. travelling sales representatives Loose Coupling Connecting individual systems via MQ makes upgrading each system independently, simpler 12
Real life problem: Scalability. Lunch hour. The Message Queue Solution Web Server Log Data Message Queue Queue Handler SQL Server Storage
BONUS: Loose Coupling. The Message Queue Solution Queue Handler SQL Server Web Server Log Data Message Queue Queue Handler MongoDB DocumentDB Queue Handler
BONUS: Loose Coupling. Abstract Messages. The Message Queue Solution Web Server Log Data Message Type: Page Visit Session ID: 47111147 Visitor ID: 64787864 Url: /home/services IP: 127.0.0.1 Agent: Mozilla/5.0 Message Type: Goal Trigger Session ID: 47111147 Visitor ID: 64787864 Goal: Newsletter Subscribe Email: mark@cassidy.dk IP: 127.0.0.1 Agent: Mozilla/5.0 Message Queue Message Type: Page Visit Session ID: 47111147 Visitor ID: 64787864 Url: /home/search IP: 127.0.0.1 Agent: Mozilla/5.0 Message Type: Page Event Session ID: 47111147 Visitor ID: 64787864 Fail: Form Validation Failed Field: First Name Value: Message: Empty Field Not Allowed Message Type: Basket Event Session ID: 47111147 Visitor ID: 64787864 SKU: CBM64_64GB_MEM QTY: 1 IP: 127.0.0.1 Agent: Mozilla/5.0
Enough Theory, yea? Azure Storage - Overview Azure Blob Storage Binaries; e.g. images and video and other media Azure Queue Storage Queued Messages (duh!) Azure Table Storage Simple Table based storage; e.g. WFFM data Azure File Storage E.g. File Share. Azure Disk Storage E.g. Virtual Disks 16
Code demonstration Setting up an Azure Storage Queue Connecting to ASQ Sending and Receiving Messages Switching to MSMQ for non-cloud operations Summary 17
Plan B: Screenshots.
Plan B: Screenshots.
Plan B: Screenshots.
Plan B: Screenshots.
So what are the limitations? MQ not suited for multi point delivery Queue has to be polled (unless you consider executing a COM+ object at message time relevant :P) Messages are point-to-point 22
Enter => The Service Bus. ESB.
Key differences between MQ and SB Storage Queues Supports >80GB storage up to 7 days Server Side Logs of all transactions And a few of my own Less of a learning overhead Readily available Cheaper Service Bus Queues Push notifications Guaranteed FIFO No code change for cloud/local cloud See Service Bus for Windows Server Duplicate Detection Transactional and Atomic Role based access model AMQP 1.0 support Message Routing and Distribution More, more, MOAR More info: https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-azure-and-service-bus-queues-compared-contrasted 24
Azure Service Bus: Message Distribution. Topics. Topics and Subscribers Messages are tagged with a Topic E.g. OMS, PIM, Finance, PPM Applications publish messages Receivers subscribe to relevant Topics I was expecting more bullet points, but it IS actually this simple ;-) More info: https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dotnet-how-to-use-topics-subscriptions 25
Let s take a look Setting up Azure Service Bus is actually very similar to what we just did with Azure Storage 26
Plan B. Screenshots. 27
Plan B: Screenshots.
Plan B. Screenshots. 29
That s it! Questions? http://sitecore.stackexchange.com The number #1 community driven Q&A resource 30
Thank you! E-mail: mark@cassidy.dk Twitter & Slack: @cassidydotdk Web: https://www.cassidy.dk