Solution Brief. A Key Value of the Future: Trillion Operations Technology. 89 Fifth Avenue, 7th Floor. New York, NY

Transcription

1 89 Fifth Avenue, 7th Floor New York, NY Solution Brief A Key Value of the Future: Trillion Operations Technology

2 Printed in the United States of America Copyright 2015 Edison Group, Inc. New York. Edison Group offers no warranty either expressed or implied on the information contained herein and shall be held harmless for errors resulting from its use. The information contained in this document is based on IBM provided materials and was aggregated for IBM by the Edison Group Analyst team. All products are trademarks of their respective owners. First Publication: October 2015 Produced by: Neal Ekker, Author; Leonard Schramm, Editor; Barry Cohen, Editor-in-Chief

3 Table of Contents Emerging Storage Workloads... 1 Shifting Towards Systems of Engagement and Analytics... 1 The Advent of NoSQL... 1 Introducing IBM Trillion Operations Technology... 3 IBM FlashSystem Powers Trillion Operation Technology... 4 The Benefits of Trillion Operations Technology... 5

4 Emerging Storage Workloads Shifting Towards Systems of Engagement and Analytics Today, enterprises of all types are involved in a profound business transformation from what are known as traditional systems of record toward what have been termed systems of engagement. Basically, systems of engagement refer to the transition from current enterprise systems designed around discrete pieces of information ("records") to systems which are more decentralized, incorporate technologies which encourage peer interactions, and often leverage cloud technologies to provide the capabilities to enable those interactions. This transformation not only involves changes in business processes, and in fact in the very ways we imagine business, but also, necessarily in the underlying information technology (IT) that supports and enables business. Multiple industries, including financial services, digital marketing, healthcare, and even the Internet of Things now operate under mandatory requirements for data persistency and retention management. These new systems of engagement bring with them new challenges, such as: Reduced efficiency from real-time applications that perform small size (10s of bytes to 10s of kilobytes) operations Greater need for Resiliency at Scale to provide uninterruptable high performance Higher costs due to significant utilization of uneconomical and energy inefficient DRAM The Advent of NoSQL A new breed of data management tools is being deployed to address the requirements of many systems of engagement. Chief among these are NoSQL databases/stores that provide mechanisms for data storage and retrieval modeled in means other than the tabular relations used in traditional relational databases. The data structures used by NoSQL databases (e.g. key-value store, graph store, columnar store, or document store) differ from those used in relational databases, often making systems of engagement operations such as Big Data analytics and real-time web applications faster in NoSQL. Driven by these new application workloads, various types of NoSQL technologies have Edison: A Key Value of the Future: Trillion Operations Technology Page 1

5 increased in popularity and in use compared to traditional relational databases over the past ten years. Unfortunately, NoSQL stores suffer from significant challenges. The key-value concept lies at the heart of many NoSQL databases and enables them to offer higher performance while scaling out more easily than conventional relational databases. But today s NoSQL deployments are normally constructed by replicating multiple commodity servers filled with costly DRAM to achieve the performance necessary to power the demands of Web 2.0 applications. This approach has a number of limitations, including cost, resource utilization, and energy consumption. Edison: A Key Value of the Future: Trillion Operations Technology Page 2

6 Introducing IBM Trillion Operations Technology IBM researchers and engineers at the IBM Almaden Research Center in California and the IBM FlashSystem Development headquarters in Texas have developed an innovative new software technology that when integrated with IBM FlashSystem storage enables persistent data access at variable I/O sizes and extraordinary performance rates in key value store environments. Called Trillion Operations technology, a recent demonstration revved a NoSQL database well beyond one trillion transactions a day. Trillion Operations technology avoids the typical NoSQL model that relies on replication of costly, low density, volatile DRAM and locally attached solid state drives for key value store data repositories. Instead, Trillion Operations technology uses IBM FlashSystem 900 as a persistent, low latency, highly available, all-flash shared storage pool that can be shared across multiple key value application servers. Applications take advantage of the Trillions Operations technology library to provide clustering between application servers. In this way, the key value data can be shared and yet remain persistent. Trillion Operations technology incorporates a number of innovations. It rethinks database and storage software and optimizes end-to-end performance by reducing the number of I/O operations. Trillion Operations technology also employs a writeoptimized architecture that significantly reduces I/O activities per key value operation. These software advancements leverage the extraordinary peak read and write performance of IBM FlashSystem. A key enhancement provided by IBM Trillion Operations technology involves enabling NoSQL key value stores to provide the same levels of data protection and reliability that have been required of relational databases since the 1970s. These properties, called ACID (Atomicity, Consistency, Isolation, Durability), guarantee that key value transactions are processed reliably. Thanks to this new feature provided by Trillion Operations technology, key value stores can be used in many mission-critical data processing environments where once they could not because of their lack of reliability at scale. In addition to high speed and high availability, Trillion Operations technology enables a suite of data management capabilities, including backup/restore to prevent data loss, Edison: A Key Value of the Future: Trillion Operations Technology Page 3

7 data exports for analytics that employ Map/Reduce technologies, and snapshots to facilitate software development and testing. IBM FlashSystem Powers Trillion Operation Technology IBM s Trillion Operations technology leverages highly available and reliable IBM FlashSystem 900 storage to deliver infrastructure simplification and functional advantages when compared to the data protection schemes most often used by the commodity server clusters found in conventional NoSQL deployments. Powered by IBM FlashCore technology, IBM FlashSystem 900 delivers the extreme performance, enterprise reliability, and operational efficiencies required to provide enterprise-grade targeted application acceleration. For storage environments already equipped with robust storage management capabilities such as those provided by IBM Spectrum Virtualize or Spectrum Scale, the ultra-low latency, marketleading performance, and ease of implementation offered by IBM FlashSystem 900 make it an ideal choice to transform a business-critical application into an engine of business growth and competitive advantage. Edison: A Key Value of the Future: Trillion Operations Technology Page 4

8 The Benefits of Trillion Operations Technology Figure 1: IBM Trillion Operations Technology Offers Many Benefits at the IT Infrastructure Level Trillion Operations technology demonstrations conducted by IBM resulted in groundbreaking I/O performance. The new technology supported by IBM FlashSystem storage transferred data at a sustained rate of 5.3 million persistent key value operations per second using a single server node, and over 320 million persistent key value operations per second using a cluster of 64 application servers. Scaled up to 24 hours, that s far beyond one trillion transactions in a day. This means that one small system about the size of two pizza boxes that uses far less than one rack of equipment might easily handle all the traffic on Twitter from around the world every day. Edison: A Key Value of the Future: Trillion Operations Technology Page 5

9 Performance improvements of this magnitude can have profound implications for business, enabling an enormous amount of work to complete in a given timeframe and achieving extraordinary efficiency for time-sensitive applications like analytics, complex event processing, the Internet of Things, reservations systems, and financial program trading systems while driving increased performance from real-time applications in cloud infrastructures. Enterprises as diverse as large research institutions, financial services companies, the healthcare industry, energy utilities, and online dating sites have been turning to NoSQL technology to handle the rapid fire, high volume, variable application I/O from their systems of engagement. But IT infrastructure supporting these applications has until now been expensive, unreliable, and inefficient. IBM s Trillion Operations technology changes the equation for NoSQL systems based on key value stores and related solutions. Plus, Trillion Operations technology provides a way for additional enterprises to effectively integrate all the benefits of IBM FlashSystem storage into their business models, lowering costs while enabling ever higher levels of innovation. Edison: A Key Value of the Future: Trillion Operations Technology Page 6