Real Time for Big Data: The Next Age of Data Management. Talksum, Inc. Talksum, Inc. 582 Market Street, Suite 1902, San Francisco, CA 94104

Size: px
Start display at page:

Download "Real Time for Big Data: The Next Age of Data Management. Talksum, Inc. Talksum, Inc. 582 Market Street, Suite 1902, San Francisco, CA 94104"

Transcription

1 Real Time for Big Data: The Next Age of Data Management Talksum, Inc. Talksum, Inc. 582 Market Street, Suite 1902, San Francisco, CA 94104

2 Real Time for Big Data The Next Age of Data Management Introduction Much as been written about Big Data. Many have noted the expansion of both the volume, the variety and the velocity of data. The rise of unstructured data, mobile device data, social streams, and complex scientific modeling, to name just a few, are unquestionably altering the landscape of computing. For some, this has meant a rush to augment current data management practices and incrementally improve data stores to handle larger volume. There is certainly merit in this approach, as there s a need to do whatever is possible to handle the challenges at hand. But the nature of the changes that face us require us to not only improve existing models, but also to vigorously explore entirely new approaches in parallel. It helps to consider the pace of data expansion and where we are in the curve. The current estimate is that the volume of total data in the digital universe is 2.7 ZB, equivalent to more than 18 million Libraries of Congress. This is expected to grow by an average of 50% a year to an astounding 8 ZB of data by What s more, an estimated 80% of the data in the digital universe today was created in the past year. Not only is the volume of data, both structured and unstructured, exploding, but we re still at the very early stages of this growth. We are only on the very beginning of the exponential curve. Data Management and the Next Step The primary drivers of data growth are the expansion of Internet technologies and consumer adoption across the world, the proliferation of mobile devices and the spread of digital signal components in just about every product space imaginable. As noted previously, volume increasing, but the variety and velocity of the data is also greatly increasing. Along with the associated data quality challenges these increases are introducing, this trend is rapidly breaking current models of data operations and computing. The evolution of data management is taking its next major turn. 2 Real Time for Big Data Talksum Data Stream Appliances

3 For several decades, enterprise practices for data storage and analysis have revolved around the structured database. The paradigm of writing data to a structured store and then querying that store for both analysis and application uses has dominated computing practices. The SQL database lead to data warehousing practices, where massive amounts of data is extracted from its original form, transformed and loaded into structured formats tailored to perform specific analyses. As data has become more voluminous and complex, this model has been expanded and improved through the addition of more software to the enterprise data stack. And as the dawn of the Big Data era rose, innovation focused on changing the nature of the data store, moving toward unstructured NoSQL technologies and batch processing platforms like Hadoop. Again, useful tools to deal with large data stores, but still in keeping with the core approach of handling data by writing it to a store and then processing it by scaling software tools horizontally while incrementally enhancing processes with yet more layers of software. While this approach helps begin to solve some of today s short- term big data analytic challenges, it isn t the fundamental change to our overall data infrastructure that is needed. Data is only getting bigger and more varied, quality is a rising concern, clusters continue to grow and it s just the beginning. We have to take the next step, and consider three core concepts: - Data oriented architectures require innovation at the front end of the data workflow, the top of the data funnel, so to speak. Agility in the face of change is critical, and today s intensive ETL processes, reloading of data sets for new analyses and custom data integration practices don t accommodate flexibility, leading to rigid and un- scalable platforms. Choosing a warehouse, analytics platform and/or BI tools and then adopting custom data management practices and tools to manipulate the data is a backward approach. You have to start with the management of the top of your data funnel how you take in data, parse and transform it, route it and store it for the rest of your applications, BI tools and monitoring. The front end of the data process Real Time for Big Data Talksum Data Stream Appliances 3

4 must be flexible, high performance and massively efficient. It has to handle all kinds of data and support just about any kind of architecture, and it has to be agile. - Real time is the future. It is easy to think of real time data management as a luxury. However, if you consider the velocity, volume and variety of Big Data for a moment, it becomes clear that real time is the only rational approach for many applications. You can t store all the data in the world, or at least, the cost becomes prohibitive as volume increases. Furthermore, as velocity increases you can t rely on being able to store and then process fast enough to keep up with the data itself. You have to be able to do something meaningful at the time the data is generated, including handling new types of data and triaging data quality, or else you necessarily fall behind. While data warehousing and write, process, query approaches will always have uses, it is critical that real time management, analysis and monitoring be part of any future- proof Big Data strategy. - Hardware is important. While better utilization of commodity hardware is a laudable goal, leveraging the leaps that are occurring in computing technology is critical, especially the innovations in the storage space that promise to increase I/O significantly. Tailoring a platform to use every last bit of the resources available to it and continually integrating the latest hardware and network technologies has to be a core part of any reasonable solution to tomorrow s Big Data challenges. Of course, the question these concepts lead to is clear how do you reasonably ingest, transform, monitor, analyze and route data in real time at speeds that are fast enough to accommodate Big Data sized challenges? And how can you apply enough filtering and processing logic to real time data streams without compromising efficiency to the point that the solution is infeasible? How can you begin to create a holistic data management and analytics infrastructure that isn t made up of dozens of individual services with integration challenges on every level? Talksum Data Stream Appliances address exactly those challenges. 4 Real Time for Big Data Talksum Data Stream Appliances

5 Talksum Data Stream Appliances Talksum Data Stream Appliances are a data management and analytics platform built from the ground up to handle Big Data. The platform allows companies to act on data in real time through automated response, business process or human action. It allows users to move data where it s needed and in the format it s needed quickly and efficiently. It helps both streamline service delivery and boost overall data performance all with less infrastructure. Simply put, Talksum Data Stream is a powerful multi- tool for managing and monitoring your data in real time. Talksum Data Stream manages data acquisition, ingest and transformation, converting your data into flexibly managed event streams. It filters and routes data based on your specific business needs and data processing requirements, while also adding real time monitoring and analytics. Talksum Data Stream outputs to any external data store, application, file system or process in whatever format you need. All of this is done at very high speeds, accommodating Big Data initiatives and optimizing your entire data infrastructure. The result is a simplified data management process that gives you the ability to monitor and analyze your data in real time, while at the same time reducing the pain of acquisition, ETL and integration. TalkOS OS Level Data Processing The heart of the Talksum Data Stream appliance is TalkOS. TalkOS is a Linux based operating system developed by Talksum. TalkOS is tailored to leverage the appliance hardware in the best manner, optimizing worker thread utilization, efficiently managing disk IO and maximizing in- memory processing. TalkOS is extended by set of modules for each step in the data workflow (see Figure 1). The modular architecture is service oriented and highly extensible individual modules are instantiated for each specific function for a particular data stream. Module creation and settings are managed by the TalkOS configuration management layer, giving a great deal of flexibility in how the modules can be pieced together for various data workflows, analytic scenarios and business use cases. Real Time for Big Data Talksum Data Stream Appliances 5

6 Figure 1. TalkOS provides a modular architecture to allow flexible and highly efficient data management and processing. Let s examine the primary TalkOS module families and follow the path of data through the system. TalkOS Input Modules TalkOS Input Modules manage the acquisition, ingest and transformation of data. Dynamic Acquisition Modules proactively manage the acquisition of data that needs to be gathered from external systems by TalkOS. These configurable service modules automate data acquisition. Protocol Transformation Modules transforms incoming network protocols to an appropriate protocol for downstream process to ensure that data from any protocol (e.g. UDP, TCP, ZMQ, Unix Socket, Kernel Logging) can be appropriately handled. 6 Real Time for Big Data Talksum Data Stream Appliances

7 Event Transformation Modules parses and transforms incoming data to a CEE compliant JSON object format with the desired field structure based on configurable event transformation templates. This replaces your current work- intensive ETL processes with a frictionless NoETL approach, ensuring your data is in a format you can use where and when you need it. TalkOS Core Router The TalkOS Core Router serves as the center of communications between all other service modules, forming the core of the Talksum Data Stream appliance. Object Normalization ensures normalization of all incoming event objects before they are placed onto the core router data bus. Core Router the core router is a high performance data bus that moves data to and from all internal and external services. It allows for real time filtering on data as it moves through the bus, and routes to the desired internal service (such as an analytic or output module) based on the filter parameters and routing rules associated with it. Filters filters are simply expressed JSON rules that monitor data on the Core Router and perform actions when the filter parameter is met. The output of a filter can be sent to any other service, such as a notification module, analytics modules, a data reduction service, etc. An example of filter code is shown below. Real Time for Big Data Talksum Data Stream Appliances 7

8 Figure 2. This creates a filter that only allows data with a srcaddr equal to and a dstaddr that starts with Notifications the notification service allows for triggers to alert other service modules or external systems and stored procedures when a threshold or filter parameter has been met. For example, if the memory utilization on a server reaches a certain point, an notification can be sent to a distribution list or an external stored procedure can be triggered to provision another server. TalkOS Analytic Modules TalkOS Analytic Modules provide real time analytic capabilities. Analytic Modules, like all other TalkOS service modules, can be configured to work together for a particular analytic need, with the output being stored in the internal appliance Stream Storage for consumption by the TalkOS GraphUI or through the TalkOS API. Mathematics this module maintains a count of all actively filtered data characteristics, accessible in real time and usable by other modules. Additionally, this module enables users to enter simple mathematic equations that are expressed by filters on the core router, the results of which are accessible in real time. Aggregation all counts, filter outputs and mathematic results are aggregated, correlated and distributed by this module. This allows data to be organized and stored in more meaningful combinations for analytics and monitoring. 8 Real Time for Big Data Talksum Data Stream Appliances

9 Data Reduce allows for the real- time reduction of data to either the Stream Storage or external data store as desired. This can include, but is not limited to, real time reduction to a key- value store. Stream Storage data from the analytic modules is stored in an internal appliance store, making them usable by other internal service modules, by the TalkOS GraphUI and the TalkOS API. TalkOS GraphUI & API TalkOS has it s own native GUI layer and web socket for custom UX extensions. The native GUI includes a dashboard application for visualizing and monitoring data in real time and GUI based tools to configure key TalkOS service modules (complementing the various command line tools). TalkOS GraphUI also supports an extensible web- based UX framework that interacts with TalkOS services through a web socket connection. Importantly, the TalkOS GraphUI is based on an event- driven UX framework. This means that all actions and functions within the GUI are powered by the same common event description protocol as the core service modules. This approach both delivers an extremely efficient GUI layer from a performance standpoint, as well as allowing for very simple extensions of core GUI functionality based on the underlying event data for a particular vertical solution or business use case. TalkOS also supports an API for direct programmatic access. TalkOS Output Modules TalkOS Output Modules allow data to be transformed and sent to any external store or process as desired. All Output Modules are configurable to send data in the desired protocol, format and structure. Real Time for Big Data Talksum Data Stream Appliances 9

10 Storage Outputs outputs data to storage databases, including both SQL databases (e.g. MySQL, Postgres, Oracle, SQLite) and NoSQL databases (e.g. MongoDB, Reddis, Berkely, Couchbase). File System Outputs outputs data to file systems, writing data out as files in the appropriate format (e.g. text, JSON, XML,.csv). Network Modules routes data to other networks and network services, such as replicating to another data center or routing to a specific network device (e.g. HDFS, RFC3164, REST, AWS, Third- party APIs). Configuration Modules TalkOS uses a robust configuration management framework to manage both internal service module settings on the appliances as well as federated management of multi- appliance architectures. Importantly, these configuration modules can optionally be used to help manage devices and services external to the TalkOS platform, combining data service delivery tools and automation with real time monitoring and routing. Host Configuration allows you to manage all the servers in your datacenter as though it were an integrated heterogeneous network. Network Device Configuration manages the configuration and settings of network devices such as routers and switches. Cloud Service Configuration manages cloud service deployment and configuration to complement existing data infrastructure assets. Virtualization Management manage virtualized resources to complement existing data infrastructure assets. 10 Real Time for Big Data Talksum Data Stream Appliances

11 Hardware Specifications Talksum Data Stream Appliances offer flexible architectures based on two core sizes of appliances. Recommended deployment models range from smaller instances for internal analytics and test/dev purposes to larger, fully redundant architectures for enterprise production environments. For larger deployments, a larger appliance size option is available. All appliances feature redundant power supply, lights- out management functionality, integrated hardware RAID controller with flash- backed cache and onsite hardware support services as well as a full installation of TalkOS and all core modules. The following are basic hardware specifications for our two core appliance sizes. Description Stream Transformer ( Small ) 1 RU server best used for ingest & transformation in a multi- server environment. Can also be used as a stand alone system for test & dev. Chassis Size 1 RU 2 RU Stream Router ( Medium ) 2 RU server best used as the core router & analytics box in a multi- server environment. Can also be used as a stand alone multi- purpose system for internal analytics, test & dev. CPU 2 Intel i7 CPUs, E5600 series 2 Intel i7 CPUs, X5600 series RAM 48 GB 96 GB OS Disk 2 SAS hard disks for OS 2 SAS hard disks for OS Stream Storage 4 SAS or SATA hard disks 300GB GB 4 SAS or SATA hard disks 1.5 TB 2.0 TB PLUS 2 FusionIO NAND Storage 1.2 TB Note: where ranges are shown, configuration options and enhancements are available final specifications will be developed based on customer needs. Real Time for Big Data Talksum Data Stream Appliances 11

Top 4 considerations for choosing a converged infrastructure for private clouds

Top 4 considerations for choosing a converged infrastructure for private clouds Top 4 considerations for choosing a converged infrastructure for private clouds Organizations are increasingly turning to private clouds to improve efficiencies, lower costs, enhance agility and address

More information

When, Where & Why to Use NoSQL?

When, Where & Why to Use NoSQL? When, Where & Why to Use NoSQL? 1 Big data is becoming a big challenge for enterprises. Many organizations have built environments for transactional data with Relational Database Management Systems (RDBMS),

More information

WHITEPAPER. MemSQL Enterprise Feature List

WHITEPAPER. MemSQL Enterprise Feature List WHITEPAPER MemSQL Enterprise Feature List 2017 MemSQL Enterprise Feature List DEPLOYMENT Provision and deploy MemSQL anywhere according to your desired cluster configuration. On-Premises: Maximize infrastructure

More information

How to integrate data into Tableau

How to integrate data into Tableau 1 How to integrate data into Tableau a comparison of 3 approaches: ETL, Tableau self-service and WHITE PAPER WHITE PAPER 2 data How to integrate data into Tableau a comparison of 3 es: ETL, Tableau self-service

More information

IBM dashdb Local. Using a software-defined environment in a private cloud to enable hybrid data warehousing. Evolving the data warehouse

IBM dashdb Local. Using a software-defined environment in a private cloud to enable hybrid data warehousing. Evolving the data warehouse IBM dashdb Local Using a software-defined environment in a private cloud to enable hybrid data warehousing Evolving the data warehouse Managing a large-scale, on-premises data warehouse environments to

More information

VEXATA FOR ORACLE. Digital Business Demands Performance and Scale. Solution Brief

VEXATA FOR ORACLE. Digital Business Demands Performance and Scale. Solution Brief Digital Business Demands Performance and Scale As enterprises shift to online and softwaredriven business models, Oracle infrastructure is being pushed to run at exponentially higher scale and performance.

More information

<Insert Picture Here> MySQL Web Reference Architectures Building Massively Scalable Web Infrastructure

<Insert Picture Here> MySQL Web Reference Architectures Building Massively Scalable Web Infrastructure MySQL Web Reference Architectures Building Massively Scalable Web Infrastructure Mario Beck (mario.beck@oracle.com) Principal Sales Consultant MySQL Session Agenda Requirements for

More information

TECHNICAL OVERVIEW OF NEW AND IMPROVED FEATURES OF EMC ISILON ONEFS 7.1.1

TECHNICAL OVERVIEW OF NEW AND IMPROVED FEATURES OF EMC ISILON ONEFS 7.1.1 TECHNICAL OVERVIEW OF NEW AND IMPROVED FEATURES OF EMC ISILON ONEFS 7.1.1 ABSTRACT This introductory white paper provides a technical overview of the new and improved enterprise grade features introduced

More information

STATE OF MODERN APPLICATIONS IN THE CLOUD

STATE OF MODERN APPLICATIONS IN THE CLOUD STATE OF MODERN APPLICATIONS IN THE CLOUD 2017 Introduction The Rise of Modern Applications What is the Modern Application? Today s leading enterprises are striving to deliver high performance, highly

More information

Cisco and Cloudera Deliver WorldClass Solutions for Powering the Enterprise Data Hub alerts, etc. Organizations need the right technology and infrastr

Cisco and Cloudera Deliver WorldClass Solutions for Powering the Enterprise Data Hub alerts, etc. Organizations need the right technology and infrastr Solution Overview Cisco UCS Integrated Infrastructure for Big Data and Analytics with Cloudera Enterprise Bring faster performance and scalability for big data analytics. Highlights Proven platform for

More information

ATA DRIVEN GLOBAL VISION CLOUD PLATFORM STRATEG N POWERFUL RELEVANT PERFORMANCE SOLUTION CLO IRTUAL BIG DATA SOLUTION ROI FLEXIBLE DATA DRIVEN V

ATA DRIVEN GLOBAL VISION CLOUD PLATFORM STRATEG N POWERFUL RELEVANT PERFORMANCE SOLUTION CLO IRTUAL BIG DATA SOLUTION ROI FLEXIBLE DATA DRIVEN V ATA DRIVEN GLOBAL VISION CLOUD PLATFORM STRATEG N POWERFUL RELEVANT PERFORMANCE SOLUTION CLO IRTUAL BIG DATA SOLUTION ROI FLEXIBLE DATA DRIVEN V WHITE PAPER Create the Data Center of the Future Accelerate

More information

Big Data is Better on Bare Metal

Big Data is Better on Bare Metal WHITE PAPER Big Data is Better on Bare Metal Make big data performance a priority EXECUTIVE SUMMARY Today, businesses create and capture unprecedented amounts of data from multiple sources in structured

More information

JAVASCRIPT CHARTING. Scaling for the Enterprise with Metric Insights Copyright Metric insights, Inc.

JAVASCRIPT CHARTING. Scaling for the Enterprise with Metric Insights Copyright Metric insights, Inc. JAVASCRIPT CHARTING Scaling for the Enterprise with Metric Insights 2013 Copyright Metric insights, Inc. A REVOLUTION IS HAPPENING... 3! Challenges... 3! Borrowing From The Enterprise BI Stack... 4! Visualization

More information

Hitachi Unified Compute Platform Pro for VMware vsphere

Hitachi Unified Compute Platform Pro for VMware vsphere SOLUTION PROFILE Hitachi Unified Compute Platform Pro for VMware vsphere Accelerate Your Business-Critical Workloads to the Next-Generation Converged Infrastructure Relentless trends of increasing data

More information

Fast Innovation requires Fast IT

Fast Innovation requires Fast IT Fast Innovation requires Fast IT Cisco Data Virtualization Puneet Kumar Bhugra Business Solutions Manager 1 Challenge In Data, Big Data & Analytics Siloed, Multiple Sources Business Outcomes Business Opportunity:

More information

IBM Spectrum NAS, IBM Spectrum Scale and IBM Cloud Object Storage

IBM Spectrum NAS, IBM Spectrum Scale and IBM Cloud Object Storage IBM Spectrum NAS, IBM Spectrum Scale and IBM Cloud Object Storage Silverton Consulting, Inc. StorInt Briefing 2017 SILVERTON CONSULTING, INC. ALL RIGHTS RESERVED Page 2 Introduction Unstructured data has

More information

Oracle GoldenGate for Big Data

Oracle GoldenGate for Big Data Oracle GoldenGate for Big Data The Oracle GoldenGate for Big Data 12c product streams transactional data into big data systems in real time, without impacting the performance of source systems. It streamlines

More information

Evolving To The Big Data Warehouse

Evolving To The Big Data Warehouse Evolving To The Big Data Warehouse Kevin Lancaster 1 Copyright Director, 2012, Oracle and/or its Engineered affiliates. All rights Insert Systems, Information Protection Policy Oracle Classification from

More information

Upgrade Your MuleESB with Solace s Messaging Infrastructure

Upgrade Your MuleESB with Solace s Messaging Infrastructure The era of ubiquitous connectivity is upon us. The amount of data most modern enterprises must collect, process and distribute is exploding as a result of real-time process flows, big data, ubiquitous

More information

The PowerEdge M830 blade server

The PowerEdge M830 blade server The PowerEdge M830 blade server No-compromise compute and memory scalability for data centers and remote or branch offices Now you can boost application performance, consolidation and time-to-value in

More information

Security and Performance advances with Oracle Big Data SQL

Security and Performance advances with Oracle Big Data SQL Security and Performance advances with Oracle Big Data SQL Jean-Pierre Dijcks Oracle Redwood Shores, CA, USA Key Words SQL, Oracle, Database, Analytics, Object Store, Files, Big Data, Big Data SQL, Hadoop,

More information

Bringing OpenStack to the Enterprise. An enterprise-class solution ensures you get the required performance, reliability, and security

Bringing OpenStack to the Enterprise. An enterprise-class solution ensures you get the required performance, reliability, and security Bringing OpenStack to the Enterprise An enterprise-class solution ensures you get the required performance, reliability, and security INTRODUCTION Organizations today frequently need to quickly get systems

More information

High performance and functionality

High performance and functionality IBM Storwize V7000F High-performance, highly functional, cost-effective all-flash storage Highlights Deploys all-flash performance with market-leading functionality Helps lower storage costs with data

More information

FLORIDA DEPARTMENT OF TRANSPORTATION PRODUCTION BIG DATA PLATFORM

FLORIDA DEPARTMENT OF TRANSPORTATION PRODUCTION BIG DATA PLATFORM FLORIDA DEPARTMENT OF TRANSPORTATION PRODUCTION BIG DATA PLATFORM RECOMMENDATION AND JUSTIFACTION Executive Summary: VHB has been tasked by the Florida Department of Transportation District Five to design

More information

Embedded Technosolutions

Embedded Technosolutions Hadoop Big Data An Important technology in IT Sector Hadoop - Big Data Oerie 90% of the worlds data was generated in the last few years. Due to the advent of new technologies, devices, and communication

More information

Gain Insights From Unstructured Data Using Pivotal HD. Copyright 2013 EMC Corporation. All rights reserved.

Gain Insights From Unstructured Data Using Pivotal HD. Copyright 2013 EMC Corporation. All rights reserved. Gain Insights From Unstructured Data Using Pivotal HD 1 Traditional Enterprise Analytics Process 2 The Fundamental Paradigm Shift Internet age and exploding data growth Enterprises leverage new data sources

More information

Big Data Technology Ecosystem. Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara

Big Data Technology Ecosystem. Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara Big Data Technology Ecosystem Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara Agenda End-to-End Data Delivery Platform Ecosystem of Data Technologies Mapping an End-to-End Solution Case

More information

ELASTIC DATA PLATFORM

ELASTIC DATA PLATFORM SERVICE OVERVIEW ELASTIC DATA PLATFORM A scalable and efficient approach to provisioning analytics sandboxes with a data lake ESSENTIALS Powerful: provide read-only data to anyone in the enterprise while

More information

Taming Structured And Unstructured Data With SAP HANA Running On VCE Vblock Systems

Taming Structured And Unstructured Data With SAP HANA Running On VCE Vblock Systems 1 Taming Structured And Unstructured Data With SAP HANA Running On VCE Vblock Systems The Defacto Choice For Convergence 2 ABSTRACT & SPEAKER BIO Dealing with enormous data growth is a key challenge for

More information

VOLTDB + HP VERTICA. page

VOLTDB + HP VERTICA. page VOLTDB + HP VERTICA ARCHITECTURE FOR FAST AND BIG DATA ARCHITECTURE FOR FAST + BIG DATA FAST DATA Fast Serve Analytics BIG DATA BI Reporting Fast Operational Database Streaming Analytics Columnar Analytics

More information

REGULATORY REPORTING FOR FINANCIAL SERVICES

REGULATORY REPORTING FOR FINANCIAL SERVICES REGULATORY REPORTING FOR FINANCIAL SERVICES Gordon Hughes, Global Sales Director, Intel Corporation Sinan Baskan, Solutions Director, Financial Services, MarkLogic Corporation Many regulators and regulations

More information

Was ist dran an einer spezialisierten Data Warehousing platform?

Was ist dran an einer spezialisierten Data Warehousing platform? Was ist dran an einer spezialisierten Data Warehousing platform? Hermann Bär Oracle USA Redwood Shores, CA Schlüsselworte Data warehousing, Exadata, specialized hardware proprietary hardware Introduction

More information

Composite Software Data Virtualization The Five Most Popular Uses of Data Virtualization

Composite Software Data Virtualization The Five Most Popular Uses of Data Virtualization Composite Software Data Virtualization The Five Most Popular Uses of Data Virtualization Composite Software, Inc. June 2011 TABLE OF CONTENTS INTRODUCTION... 3 DATA FEDERATION... 4 PROBLEM DATA CONSOLIDATION

More information

Big Data Integration BIG DATA 9/15/2017. Business Performance

Big Data Integration BIG DATA 9/15/2017. Business Performance BIG DATA Business Performance Big Data Integration Big data is often about doing things that weren t widely possible because the technology was not advanced enough or the cost of doing so was prohibitive.

More information

Data 101 Which DB, When. Joe Yong Azure SQL Data Warehouse, Program Management Microsoft Corp.

Data 101 Which DB, When. Joe Yong Azure SQL Data Warehouse, Program Management Microsoft Corp. Data 101 Which DB, When Joe Yong (joeyong@microsoft.com) Azure SQL Data Warehouse, Program Management Microsoft Corp. The world is changing AI increased by 300% in 2017 Data will grow to 44 ZB in 2020

More information

Introduction to Big-Data

Introduction to Big-Data Introduction to Big-Data Ms.N.D.Sonwane 1, Mr.S.P.Taley 2 1 Assistant Professor, Computer Science & Engineering, DBACER, Maharashtra, India 2 Assistant Professor, Information Technology, DBACER, Maharashtra,

More information

ACCELERATE YOUR ANALYTICS GAME WITH ORACLE SOLUTIONS ON PURE STORAGE

ACCELERATE YOUR ANALYTICS GAME WITH ORACLE SOLUTIONS ON PURE STORAGE ACCELERATE YOUR ANALYTICS GAME WITH ORACLE SOLUTIONS ON PURE STORAGE An innovative storage solution from Pure Storage can help you get the most business value from all of your data THE SINGLE MOST IMPORTANT

More information

Building a Data-Friendly Platform for a Data- Driven Future

Building a Data-Friendly Platform for a Data- Driven Future Building a Data-Friendly Platform for a Data- Driven Future Benjamin Hindman - @benh 2016 Mesosphere, Inc. All Rights Reserved. INTRO $ whoami BENJAMIN HINDMAN Co-founder and Chief Architect of Mesosphere,

More information

Solution Brief: Archiving with Harmonic Media Application Server and ProXplore

Solution Brief: Archiving with Harmonic Media Application Server and ProXplore Solution Brief: Archiving with Harmonic Media Application Server and ProXplore Summary Harmonic Media Application Server (MAS) provides management of content across the Harmonic server and storage infrastructure.

More information

Renovating your storage infrastructure for Cloud era

Renovating your storage infrastructure for Cloud era Renovating your storage infrastructure for Cloud era Nguyen Phuc Cuong Software Defined Storage Country Sales Leader Copyright IBM Corporation 2016 2 Business SLAs Challenging Traditional Storage Approaches

More information

Modernizing Business Intelligence and Analytics

Modernizing Business Intelligence and Analytics Modernizing Business Intelligence and Analytics Justin Erickson Senior Director, Product Management 1 Agenda What benefits can I achieve from modernizing my analytic DB? When and how do I migrate from

More information

Modern Data Warehouse The New Approach to Azure BI

Modern Data Warehouse The New Approach to Azure BI Modern Data Warehouse The New Approach to Azure BI History On-Premise SQL Server Big Data Solutions Technical Barriers Modern Analytics Platform On-Premise SQL Server Big Data Solutions Modern Analytics

More information

e BOOK Do you feel trapped by your database vendor? What you can do to take back control of your database (and its associated costs!

e BOOK Do you feel trapped by your database vendor? What you can do to take back control of your database (and its associated costs! e BOOK Do you feel trapped by your database vendor? What you can do to take back control of your database (and its associated costs!) With private and hybrid cloud infrastructures now reaching critical

More information

ELASTIC SERVICES PLATFORM

ELASTIC SERVICES PLATFORM ELASTIC SERVICES PLATFORM SIMPLIFYING SERVICE DELIVERY The telecom industry is at the dawn of a bright new era - one in which new services and technologies, such as 5G, IoT, and smart cities, hold promise

More information

DriveScale-DellEMC Reference Architecture

DriveScale-DellEMC Reference Architecture DriveScale-DellEMC Reference Architecture DellEMC/DRIVESCALE Introduction DriveScale has pioneered the concept of Software Composable Infrastructure that is designed to radically change the way data center

More information

Striped Data Server for Scalable Parallel Data Analysis

Striped Data Server for Scalable Parallel Data Analysis Journal of Physics: Conference Series PAPER OPEN ACCESS Striped Data Server for Scalable Parallel Data Analysis To cite this article: Jin Chang et al 2018 J. Phys.: Conf. Ser. 1085 042035 View the article

More information

Building a Data Strategy for a Digital World

Building a Data Strategy for a Digital World Building a Data Strategy for a Digital World Jason Hunter, CTO, APAC Data Challenge: Pushing the Limits of What's Possible The Art of the Possible Multiple Government Agencies Data Hub 100 s of Service

More information

Hadoop, Yarn and Beyond

Hadoop, Yarn and Beyond Hadoop, Yarn and Beyond 1 B. R A M A M U R T H Y Overview We learned about Hadoop1.x or the core. Just like Java evolved, Java core, Java 1.X, Java 2.. So on, software and systems evolve, naturally.. Lets

More information

Cloud Computing & Visualization

Cloud Computing & Visualization Cloud Computing & Visualization Workflows Distributed Computation with Spark Data Warehousing with Redshift Visualization with Tableau #FIUSCIS School of Computing & Information Sciences, Florida International

More information

New Approach to Unstructured Data

New Approach to Unstructured Data Innovations in All-Flash Storage Deliver a New Approach to Unstructured Data Table of Contents Developing a new approach to unstructured data...2 Designing a new storage architecture...2 Understanding

More information

Software-defined Media Processing

Software-defined Media Processing Software-defined Media Processing Matthew Goldman SVP Technology, TV & Media agenda Media Processing Challenges Value of Virtualization Software Defined Media Processing Moving to the Cloud Ericsson 2017

More information

ADVANCED IN-MEMORY COMPUTING USING SUPERMICRO MEMX SOLUTION

ADVANCED IN-MEMORY COMPUTING USING SUPERMICRO MEMX SOLUTION TABLE OF CONTENTS 2 WHAT IS IN-MEMORY COMPUTING (IMC) Benefits of IMC Concerns with In-Memory Processing Advanced In-Memory Computing using Supermicro MemX 1 3 MEMX ARCHITECTURE MemX Functionality and

More information

Cisco Unified Computing System Delivering on Cisco's Unified Computing Vision

Cisco Unified Computing System Delivering on Cisco's Unified Computing Vision Cisco Unified Computing System Delivering on Cisco's Unified Computing Vision At-A-Glance Unified Computing Realized Today, IT organizations assemble their data center environments from individual components.

More information

At-Scale Data Centers & Demand for New Architectures

At-Scale Data Centers & Demand for New Architectures Allen Samuels At-Scale Data Centers & Demand for New Architectures Software Architect, Software and Systems Solutions June 4, 2015 1 Forward-Looking Statements During our meeting today we may make forward-looking

More information

5 Fundamental Strategies for Building a Data-centered Data Center

5 Fundamental Strategies for Building a Data-centered Data Center 5 Fundamental Strategies for Building a Data-centered Data Center June 3, 2014 Ken Krupa, Chief Field Architect Gary Vidal, Solutions Specialist Last generation Reference Data Unstructured OLTP Warehouse

More information

Transform to Your Cloud

Transform to Your Cloud Transform to Your Cloud Presented by VMware 2012 VMware Inc. All rights reserved Agenda Corporate Overview Cloud Infrastructure & Management Cloud Application Platform End User Computing The Journey to

More information

IBM Emulex 16Gb Fibre Channel HBA Evaluation

IBM Emulex 16Gb Fibre Channel HBA Evaluation IBM Emulex 16Gb Fibre Channel HBA Evaluation Evaluation report prepared under contract with Emulex Executive Summary The computing industry is experiencing an increasing demand for storage performance

More information

From Single Purpose to Multi Purpose Data Lakes. Thomas Niewel Technical Sales Director DACH Denodo Technologies March, 2019

From Single Purpose to Multi Purpose Data Lakes. Thomas Niewel Technical Sales Director DACH Denodo Technologies March, 2019 From Single Purpose to Multi Purpose Data Lakes Thomas Niewel Technical Sales Director DACH Denodo Technologies March, 2019 Agenda Data Lakes Multiple Purpose Data Lakes Customer Example Demo Takeaways

More information

VxRail: Level Up with New Capabilities and Powers GLOBAL SPONSORS

VxRail: Level Up with New Capabilities and Powers GLOBAL SPONSORS VxRail: Level Up with New Capabilities and Powers GLOBAL SPONSORS VMware customers trust their infrastructure to vsan #1 Leading SDS Vendor >10,000 >100 83% vsan Customers Countries Deployed Critical Apps

More information

OpenStack and Beyond Built on ProphetStor Federator

OpenStack and Beyond Built on ProphetStor Federator Whitepaper: OpenStack and Beyond Built on ProphetStor Federator OpenStack and Beyond Built on ProphetStor Federator AUGUST 2016 For more info, visit prophetstor.com OpenStack and Beyond Built on ProphetStor

More information

Data Protection for Cisco HyperFlex with Veeam Availability Suite. Solution Overview Cisco Public

Data Protection for Cisco HyperFlex with Veeam Availability Suite. Solution Overview Cisco Public Data Protection for Cisco HyperFlex with Veeam Availability Suite 1 2017 2017 Cisco Cisco and/or and/or its affiliates. its affiliates. All rights All rights reserved. reserved. Highlights Is Cisco compatible

More information

Four Steps to Unleashing The Full Potential of Your Database

Four Steps to Unleashing The Full Potential of Your Database Four Steps to Unleashing The Full Potential of Your Database This insightful technical guide offers recommendations on selecting a platform that helps unleash the performance of your database. What s the

More information

MODERNISE WITH ALL-FLASH. Intel Inside. Powerful Data Centre Outside.

MODERNISE WITH ALL-FLASH. Intel Inside. Powerful Data Centre Outside. MODERNISE WITH ALL-FLASH Intel Inside. Powerful Data Centre Outside. MODERNISE WITHOUT COMPROMISE In today s lightning-fast digital world, it s critical for businesses to make their move to the Modern

More information

IBM System Storage DCS3700

IBM System Storage DCS3700 IBM System Storage DCS3700 Maximize performance, scalability and storage density at an affordable price Highlights Gain fast, highly dense storage capabilities at an affordable price Deliver simplified

More information

2014 年 3 月 13 日星期四. From Big Data to Big Value Infrastructure Needs and Huawei Best Practice

2014 年 3 月 13 日星期四. From Big Data to Big Value Infrastructure Needs and Huawei Best Practice 2014 年 3 月 13 日星期四 From Big Data to Big Value Infrastructure Needs and Huawei Best Practice Data-driven insight Making better, more informed decisions, faster Raw Data Capture Store Process Insight 1 Data

More information

SAP Sybase SQL Anywhere Manage enterprise data in remote and mobile locations. Speaker s Name/Department (delete if not needed) Month 00, 2012

SAP Sybase SQL Anywhere Manage enterprise data in remote and mobile locations. Speaker s Name/Department (delete if not needed) Month 00, 2012 SAP Sybase SQL Anywhere Manage enterprise data in remote and mobile locations Speaker s Name/Department (delete if not needed) Month 00, 2012 The New Real-Time Business Real-Time Businesses are data-driven

More information

Paper. Delivering Strong Security in a Hyperconverged Data Center Environment

Paper. Delivering Strong Security in a Hyperconverged Data Center Environment Paper Delivering Strong Security in a Hyperconverged Data Center Environment Introduction A new trend is emerging in data center technology that could dramatically change the way enterprises manage and

More information

VMware Cloud Application Platform

VMware Cloud Application Platform VMware Cloud Application Platform Jerry Chen Vice President of Cloud and Application Services Director, Cloud and Application Services VMware s Three Strategic Focus Areas Re-think End-User Computing Modernize

More information

Build your own Cloud on Christof Westhues

Build your own Cloud on Christof Westhues Build your own Cloud on Christof Westhues chwe@de.ibm.com IBM Big Data & Elastic Storage Tour Software Defined Infrastructure Roadshow December 2 4, 2014 New applications and IT are being built for Cloud

More information

Cloud Analytics and Business Intelligence on AWS

Cloud Analytics and Business Intelligence on AWS Cloud Analytics and Business Intelligence on AWS Enterprise Applications Virtual Desktops Sharing & Collaboration Platform Services Analytics Hadoop Real-time Streaming Data Machine Learning Data Warehouse

More information

"Software-defined storage Crossing the right bridge"

Software-defined storage Crossing the right bridge Navigating the software-defined storage world shaping tomorrow with you René Huebel "Software-defined storage Crossing the right bridge" SDS the model and the promises Control Abstraction The promises

More information

REFERENCE ARCHITECTURE Quantum StorNext and Cloudian HyperStore

REFERENCE ARCHITECTURE Quantum StorNext and Cloudian HyperStore REFERENCE ARCHITECTURE Quantum StorNext and Cloudian HyperStore CLOUDIAN + QUANTUM REFERENCE ARCHITECTURE 1 Table of Contents Introduction to Quantum StorNext 3 Introduction to Cloudian HyperStore 3 Audience

More information

Oracle Big Data Connectors

Oracle Big Data Connectors Oracle Big Data Connectors Oracle Big Data Connectors is a software suite that integrates processing in Apache Hadoop distributions with operations in Oracle Database. It enables the use of Hadoop to process

More information

Abstract. The Challenges. ESG Lab Review InterSystems IRIS Data Platform: A Unified, Efficient Data Platform for Fast Business Insight

Abstract. The Challenges. ESG Lab Review InterSystems IRIS Data Platform: A Unified, Efficient Data Platform for Fast Business Insight ESG Lab Review InterSystems Data Platform: A Unified, Efficient Data Platform for Fast Business Insight Date: April 218 Author: Kerry Dolan, Senior IT Validation Analyst Abstract Enterprise Strategy Group

More information

FOCUS ON THE FACTS: SOFTWARE-DEFINED STORAGE

FOCUS ON THE FACTS: SOFTWARE-DEFINED STORAGE FOCUS ON THE FACTS: SOFTWARE-DEFINED STORAGE Table of Contents CHAPTER 1: UNRAVELING THE SDS HYPE CHAPTER 2: CRITICAL ATTRIBUTES OF SDS CHAPTER 3: THE FUTURE IS NOW CHAPTER 4: CUTTING THE HARDWARE CORD

More information

Preserving the World s Most Important Data. Yours. SYSTEMS AT-A-GLANCE: KEY FEATURES AND BENEFITS

Preserving the World s Most Important Data. Yours. SYSTEMS AT-A-GLANCE: KEY FEATURES AND BENEFITS Preserving the World s Most Important Data. Yours. SYSTEMS AT-A-GLANCE: KEY FEATURES AND BENEFITS We are the only company to integrate disk, tape, and replication in a single solution set for better near-term

More information

EMC Storage Resource Management Suite

EMC Storage Resource Management Suite EMC Storage Resource Management Suite Optimizing storage today and in the software defined datacenter of tomorrow EMC Advanced Software Division 1 New Applications/ Technologies New Consumption Models

More information

21ST century enterprise. HCL Technologies Presents. Roadmap for Data Center Transformation

21ST century enterprise. HCL Technologies Presents. Roadmap for Data Center Transformation 21ST century enterprise HCL Technologies Presents Roadmap for Data Center Transformation june 2016 21st Century Impact on Data Centers The rising wave of digitalization has changed the way IT impacts business.

More information

DataON and Intel Select Hyper-Converged Infrastructure (HCI) Maximizes IOPS Performance for Windows Server Software-Defined Storage

DataON and Intel Select Hyper-Converged Infrastructure (HCI) Maximizes IOPS Performance for Windows Server Software-Defined Storage Solution Brief DataON and Intel Select Hyper-Converged Infrastructure (HCI) Maximizes IOPS Performance for Windows Server Software-Defined Storage DataON Next-Generation All NVMe SSD Flash-Based Hyper-Converged

More information

Nowcasting. D B M G Data Base and Data Mining Group of Politecnico di Torino. Big Data: Hype or Hallelujah? Big data hype?

Nowcasting. D B M G Data Base and Data Mining Group of Politecnico di Torino. Big Data: Hype or Hallelujah? Big data hype? Big data hype? Big Data: Hype or Hallelujah? Data Base and Data Mining Group of 2 Google Flu trends On the Internet February 2010 detected flu outbreak two weeks ahead of CDC data Nowcasting http://www.internetlivestats.com/

More information

Solace JMS Broker Delivers Highest Throughput for Persistent and Non-Persistent Delivery

Solace JMS Broker Delivers Highest Throughput for Persistent and Non-Persistent Delivery Solace JMS Broker Delivers Highest Throughput for Persistent and Non-Persistent Delivery Java Message Service (JMS) is a standardized messaging interface that has become a pervasive part of the IT landscape

More information

FIVE REASONS YOU SHOULD RUN CONTAINERS ON BARE METAL, NOT VMS

FIVE REASONS YOU SHOULD RUN CONTAINERS ON BARE METAL, NOT VMS WHITE PAPER FIVE REASONS YOU SHOULD RUN CONTAINERS ON BARE METAL, NOT VMS Over the past 15 years, server virtualization has become the preferred method of application deployment in the enterprise datacenter.

More information

Qlik Sense Enterprise architecture and scalability

Qlik Sense Enterprise architecture and scalability White Paper Qlik Sense Enterprise architecture and scalability June, 2017 qlik.com Platform Qlik Sense is an analytics platform powered by an associative, in-memory analytics engine. Based on users selections,

More information

Built for Speed: Comparing Panoply and Amazon Redshift Rendering Performance Utilizing Tableau Visualizations

Built for Speed: Comparing Panoply and Amazon Redshift Rendering Performance Utilizing Tableau Visualizations Built for Speed: Comparing Panoply and Amazon Redshift Rendering Performance Utilizing Tableau Visualizations Table of contents Faster Visualizations from Data Warehouses 3 The Plan 4 The Criteria 4 Learning

More information

MidoNet Scalability Report

MidoNet Scalability Report MidoNet Scalability Report MidoNet Scalability Report: Virtual Performance Equivalent to Bare Metal 1 MidoNet Scalability Report MidoNet: For virtual performance equivalent to bare metal Abstract: This

More information

TITLE. the IT Landscape

TITLE. the IT Landscape The Impact of Hyperconverged Infrastructure on the IT Landscape 1 TITLE Drivers for adoption Lower TCO Speed and Agility Scale Easily Operational Simplicity Hyper-converged Integrated storage & compute

More information

Modernizing Healthcare IT for the Data-driven Cognitive Era Storage and Software-Defined Infrastructure

Modernizing Healthcare IT for the Data-driven Cognitive Era Storage and Software-Defined Infrastructure Modernizing Healthcare IT for the Data-driven Cognitive Era Storage and Software-Defined Infrastructure An IDC InfoBrief, Sponsored by IBM April 2018 Executive Summary Today s healthcare organizations

More information

Azure Integration Services

Azure Integration Services Azure Integration Services 2018 Microsoft Corporation. All rights reserved. This document is provided "as-is." Information and views expressed in this document, including URL and other Internet Web site

More information

How Microsoft IT Reduced Operating Expenses Using Virtualization

How Microsoft IT Reduced Operating Expenses Using Virtualization How Microsoft IT Reduced Operating Expenses Using Virtualization Published: May 2010 The following content may no longer reflect Microsoft s current position or infrastructure. This content should be viewed

More information

Top Trends in DBMS & DW

Top Trends in DBMS & DW Oracle Top Trends in DBMS & DW Noel Yuhanna Principal Analyst Forrester Research Trend #1: Proliferation of data Data doubles every 18-24 months for critical Apps, for some its every 6 months Terabyte

More information

EsgynDB Enterprise 2.0 Platform Reference Architecture

EsgynDB Enterprise 2.0 Platform Reference Architecture EsgynDB Enterprise 2.0 Platform Reference Architecture This document outlines a Platform Reference Architecture for EsgynDB Enterprise, built on Apache Trafodion (Incubating) implementation with licensed

More information

SOLUTION BRIEF NETWORK OPERATIONS AND ANALYTICS. How Can I Predict Network Behavior to Provide for an Exceptional Customer Experience?

SOLUTION BRIEF NETWORK OPERATIONS AND ANALYTICS. How Can I Predict Network Behavior to Provide for an Exceptional Customer Experience? SOLUTION BRIEF NETWORK OPERATIONS AND ANALYTICS How Can I Predict Network Behavior to Provide for an Exceptional Customer Experience? SOLUTION BRIEF CA DATABASE MANAGEMENT FOR DB2 FOR z/os DRAFT When used

More information

DEFINING SECURITY FOR TODAY S CLOUD ENVIRONMENTS. Security Without Compromise

DEFINING SECURITY FOR TODAY S CLOUD ENVIRONMENTS. Security Without Compromise DEFINING SECURITY FOR TODAY S CLOUD ENVIRONMENTS Security Without Compromise CONTENTS INTRODUCTION 1 SECTION 1: STRETCHING BEYOND STATIC SECURITY 2 SECTION 2: NEW DEFENSES FOR CLOUD ENVIRONMENTS 5 SECTION

More information

Oracle Event Processing Extreme Performance on Sparc T5

Oracle Event Processing Extreme Performance on Sparc T5 Oracle Event Processing Extreme Performance on Sparc T5 An Oracle Event Processing (OEP) Whitepaper ORACLE WHITE PAPER AUGUST 2014 Table of Contents Introduction 2 OEP Architecture 2 Server Architecture

More information

Five Common Myths About Scaling MySQL

Five Common Myths About Scaling MySQL WHITE PAPER Five Common Myths About Scaling MySQL Five Common Myths About Scaling MySQL In this age of data driven applications, the ability to rapidly store, retrieve and process data is incredibly important.

More information

CONSOLIDATING RISK MANAGEMENT AND REGULATORY COMPLIANCE APPLICATIONS USING A UNIFIED DATA PLATFORM

CONSOLIDATING RISK MANAGEMENT AND REGULATORY COMPLIANCE APPLICATIONS USING A UNIFIED DATA PLATFORM CONSOLIDATING RISK MANAGEMENT AND REGULATORY COMPLIANCE APPLICATIONS USING A UNIFIED PLATFORM Executive Summary Financial institutions have implemented and continue to implement many disparate applications

More information

IBM Storage Software Strategy

IBM Storage Software Strategy IBM Storage Software Strategy Hakan Turgut 1 Just how fast is the data growing? 128 GB/ person The World s total data per person No Problem I I think I can do this We have a problem 24 GB/ person 0.8 GB/

More information

BI ENVIRONMENT PLANNING GUIDE

BI ENVIRONMENT PLANNING GUIDE BI ENVIRONMENT PLANNING GUIDE Business Intelligence can involve a number of technologies and foster many opportunities for improving your business. This document serves as a guideline for planning strategies

More information

All-Flash Storage Solution for SAP HANA:

All-Flash Storage Solution for SAP HANA: All-Flash Storage Solution for SAP HANA: Storage Considerations using SanDisk Solid State Devices WHITE PAPER Western Digital Technologies, Inc. 951 SanDisk Drive, Milpitas, CA 95035 www.sandisk.com Table

More information

Increase Value from Big Data with Real-Time Data Integration and Streaming Analytics

Increase Value from Big Data with Real-Time Data Integration and Streaming Analytics Increase Value from Big Data with Real-Time Data Integration and Streaming Analytics Cy Erbay Senior Director Striim Executive Summary Striim is Uniquely Qualified to Solve the Challenges of Real-Time

More information