Licensed for distribution Summary TIBCO Spotfire and Qlik Sense are two of the most prominent offerings in a new generation of business intelligence platforms. They both major on data visualization, exploration and discovery, with excellent support for mobile devices and diverse data sources, and can be profitably used by people with widely varying skill sets. However, a look under the surface similarities soon reveals that they are quite different products. Spotfire is actually much more than a pure data visualization platform, and its long history is reflected in a mature set of capabilities. These include the ability to handle tens or hundreds of thousands of data points within a visualization, a set of easy-to-use advanced analytics tools (clustering for customer segmentation for example), text analytics, predictive analytics, statistical analysis and a highly configurable architecture. Qlik Sense on the other hand is essentially a pure-play data visualization and exploration platform, with excellent data discovery delivered through its associative data engine. Both products support good extensibility through scripting languages, and the ability to create new chart types if needed. Both products have been engineered for ease-of-use, and can be used in a single-user stand-alone context, or as an enterprise BI solution with hundreds or thousands of users. Somewhat ironically, Spotfire may be more suitable for novice users because of its recommendation engine. Ultimately the decision to select one or the other of these products will depend on need, and particularly the scope of analysis that must be accommodated. Spotfire can be used purely as a data visualization platform, and is used as such by many users. However as skills and requirements grow, Spotfire allows users to move into new types of analysis in a seamless and incremental fashion. This is more important than many users appreciate, since it means there is no need to embrace additional tools to grow the analytics capability of the business. It also needs to be said that visual analysis is often the first step in the analytics process, and not the destination. Sooner or later there is a need to act on analysis, and this is where Spotfire excels, since it is part of a much broader suite of products from TIBCO, which directly supports the move from insight to action. This might mean putting a predictive model into production (credit assessments, for example), or applying rules to realtime streaming data. For organizations that need a pure-play data visualization platform, Qlik Sense has advantages over many competing products. These come in the form of its associative data engine, where relationships between data sources are often automatically displayed, and the ability to extend the platform for bespoke requirements. Various APIs mean visualizations can be embedded into production applications, that unusual data sources can be accommodated, and that modified visualizations can be created. Qlik Sense does not however support any other forms of analytics. Spotfire and Qlik Sense come as stand-alone products and enterprise platforms, and are available as cloud offerings. The decision to choose Spotfire will primarily be driven by the need to scale - data volumes, analysis complexity and types of analysis. Qlik Sense will be chosen for its excellent data discovery capabilities. Finally the licensing models of these two products are quite different, although it has to be said that the Qlik Sense model is particularly complex. Copyright butleranalytics.com 1
Spotfire As one of the most complete visual analytics platforms available, Spotfire anticipates many of the emerging trends in business intelligence and data visualization. It successfully combines ease-of-use with advanced capabilities, makes pervasive visual analytics available to users of every skill level, provides powerful visual data preparation capabilities for complex data, and has advanced analytics built in. More recently it has embraced additional big data resources, including SparkSQL, SparkR, SAP Hana and enhanced support for OLAP. For enterprise scale deployment, Spotfire now supports cloud type scaling and management of the run-time architecture. This means compute resources can be programmed to automatically adapt to differing workloads. There are many productivity enhancers embedded into Spotfire: Visualizations are recommended based on the data being analyzed a capability especially valuable for novice users to kick-start their analytics experience with minimal training; Grouping - data can be grouped so that several values are treated as a single value (different spellings of a city for example). A large number of functions ease the job of data preparation, and the user interface is enhanced through support for multiple monitors, with all visualizations interlinked. Collaboration is a strong point in Spotfire where users can create a dialogue about specific issues or insights found in their data, save views and discoveries on each message within the conversation. Similarly, annotations allow users to add commentary to their analysis and be included in a dialogue. Spotfire comes in three flavors: Spotfire Desktop is a stand-alone Windows program with connectivity to a large number of data sources, Spotfire Platform is a server based implementation for departments and businesses, and Spotfire Cloud provides the same business author and consumer clients as Spotfire Platform (with a couple of analytical tools absent), but hosted in the cloud. It needs to be emphasized that Spotfire s architecture is capable of handling visualizations with hundreds of thousands of data points - without breaking a sweat. Here is a brief summary of the functionality provided by each: Spotfire Desktop runs on a Windows desktop computer and can access both local and remote data sources. These sources are too numerous to list here, but whatever data source a user might need to access, it s likely Spotfire can accommodate it. The user interface and ease-of-use factor was significantly improved with version 7, providing recommended visualizations, easier data management and more attractive visualizations. It is difficult to see how the user experience could be more accommodating, even for complex visualization tasks. Some advanced chart types are supported including heat maps, 3-D scatter plot, mapcharts, treemaps, box-plot etc. The analytical tools really distinguish Spotfire Desktop, and typically do not require any coding at all. Predictive analytics, location analytics, hierarchical clustering, forecasting and a large number of data functions come as standard. Dashboards and charts can be exported to PDF, Powerpoint, Html, or data file. Spotfire Platform is server based and provides an enterprise data exploration and visualization environment with several powerful analytical tools. A web based interface supports Copyright butleranalytics.com 2
business authoring of visualizations, and a consumer client allows viewing through the web browser. As with the desktop version, a large number of data sources can be accessed, and network graphs (an add-on for network analysis) are added to the chart types. Platform comes with the same analytical tools as Desktop with the addition of the R Runtime Server - an optimized environment for running R analytics tasks. Several services are also available including: Advanced Data Services - for integrating data from diverse sources. Automation Services - for scheduling analysis and reports, with email distribution. Location Analytics - for analysis that involves the location of objects, people and events. Statistical Services - with support for R, S+, SAS and MATLAB native functions. Various mechanisms exist to share reports and dashboards, and a Guided Analytics facility is available in all three versions to tell a visual data story. A conversation panel allows users to track comments, and a web panel supports integration with any external web tools (TIBCO offers tibbr - one such tool). Spotfire Cloud is targeted at workgroups and comes with the business authoring and consumer web based clients. Users can build their visualizations completely within the webbased client, and consumers can view them in the browser as well. A secure area is provided to share visualizations and collaborate, or if desired they can be broadcast publicly to as many consumers as necessary. An Enterprise version of Spotfire Cloud is available as a dedicated instance. Spotfire Architecture The speed of Spotfire is greatly enhanced by its clever memory and data management. For more modest data sets the in-memory processing ensures very high performance. Larger data sets can be processed in-database, and a hybrid approach called On-Demand optimizes the data held in local memory and that held in the database. This is unique to Spotfire. For more advanced analytics there is the TIBCO Enterprise Runtime for R (TERR), providing a high performance production environment to run R scripts, and TIBCO Spotfire Statistics Services (TSSS). This integrates R, S+, SAS, MATLAB and in-database analytics into the Spotfire ecosystem. The accommodation of R is worthy of note since the execution of R scripts has been parallelized and the TERR engine is compatible with the RStudio IDE (much loved by many R analysts). The Spotfire Server manages a library of analysis files, takes responsibility for authentication and authorization, persists links to external data sources and runs on Linux, Solaris and Windows. A core component in this is the Web Player. This cleverly normalizes data access by routing users to Web Player instances already accessing relevant data sources. New instances can be created as needed, giving virtually unlimited horizontal scalability. Copyright butleranalytics.com 3
Spotfire Summary The central feature of Spotfire that distinguishes it from nearly all the competition is the ability to start with easy to use desktop data visualization tools, and as needs and skills develop, to migrate to a full enterprise analytics environment. It satisfies the needs of novice users, business managers, data analysts, and data scientists in one homogeneous environment. Novice users will find a supportive, guided experience, while advanced users can develop and deploy very advanced analytics on big data using the Spark architecture. In this sense Spotfire is unique, and in the context of TIBCO s broader product architecture businesses can move all the way from insight to action in one seamless environment. Spotfire is essentially analytics without boundaries. The most important differentiators of Spotfire are: 1 A single environment for novice and expert users alike. 2 Ahead of the visual analytics curve with advanced analytics and excellent data preparation for complex data. 3 Highly scalable and programmable architecture for demanding requirements. 4 High visual analytics productivity through recommended visualizations and guided interface. 5 Part of a broader analytics capability with a direct route from insight to action. Copyright butleranalytics.com 4
Qlik Sense Qlik Sense is a drag-and-drop data visualization and discovery platform capable of addressing most business intelligence needs apart from heavy duty production reporting. It delivers an easy-to-use interface suited to all levels of skill, and has a substantial amount of in-built intelligence to help users along. Charts, tables and dashboards are its main currency, of almost any level of complexity or simplicity. Users can share their visualizations via various mechanisms and the platform was built ground up for mobile access no matter the device (visualizations are rendered in HTML5). Perhaps the most significant Qlik Sense differentiator is its associative data engine. This understands the links between various data sources and can suggest previously unsuspected relationships. It allows users to freely explore visualizations and objects, refining context through selections and keyword searches. Many suppliers use the term data discovery, but this facility adds new meaning to the term. It comes in two versions Qlik Sense Desktop, which is free to download and is not throttled in any way. It runs on a Windows desktop and is capable of accessing many data sources. Qlik Sense Enterprise runs on a server(s) and provides users with a browser based interface. Both editions have similar functionality, but the server platform offers collaboration, mobile, and governance capabilities, and can scale to serve global distributed enterprises through its excellent scalability and distributed architecture. Qlik has always offered excellent governance of its environment, and IT has the tools to ensure data is secure, unambiguous and that the right people get to the right data. Very importantly Qlik Sense is extensible. Not a very sexy attribute perhaps, but one that distinguishes the adults from the children in the world of enterprise business intelligence. In fact Qlik Sense is one large extension, built on itself! A large number of APIs are available for embedding visualizations into production applications, creating custom data connectors and building new visualization types. Developers will have absolutely no problem extending Qlik Sense, if that is what is needed. Finally Qlik Sense is fast its in-memory columnar associative engine guarantees that. Qlik Sense is the perfect fit for organizations who need easy-to-use data visualization and discovery tools, but may also want the head room to accommodate more advanced levels of sophistication. It is an enterprise solution, with its governance and developer support capabilities. Businesses looking for an enterprise production reporting platform (invoices etc) should look elsewhere, as should users who needs a few simple charts, since they would find Qlik Sense overkill. Qlik Sense does not come with any advanced statistical or predictive analytics capability, but that is not its domain. Qlik Sense Desktop Qlik Sense Desktop is the entry level product, and it can be freely downloaded the full version. Out of the box it easily connects to desktop data sources such as Excel and CSV Copyright butleranalytics.com 5
files. From version 2.0 onward, the job of connecting to other data sources has been made very much easier. Smart Data Load allows many sources to be specified, and profiles the data sources to suggest the best links between them. ODBC connectors are available for a number of sources, and Qlik DataMarket provides third-party data as a service, directly from within Qlik Sense. Once data can be accessed it is loaded into the in-memory associative data engine, where it is compressed for speed. Performance is very good provided the data fits into memory, and since it is compressed, an awful lot of data can be accommodated. Virtually every kind of chart is supported. The central construct in Qlik Sense is the sheet this can contain charts, tables, images and filters. A sheet can consist of many visualizations. The desktop version supports the distribution of visualizations via file. To add to the appeal of the desktop version, and to whet the appetite, Qlik Sense Cloud allows users to upload visualizations and share them with up to 5 users again for free. The data exploration capability is helped considerably by the associative engine. This color codes data sources to indicate the relationship they have with data sources already being used. Green indicates a data source is already being used, white means there is an association (which you may not already know about), and gray indicates there is no relationship. Qlik Sense Enterprise The server based Qlik Sense Enterprise provides similar capability to the desktop version, but with added collaboration features, mobile access, scalability, data management, governance and platform management, and a distributed architecture. Users get full functionality through a browser based interface even on mobile devices. Visualizations can be created, viewed, shared and managed through this single interface. Clever resizing of charts and objects means visualizations fit on almost any device display a natural outcome of the fact Qlik Sense was built ground up for mobile and touch devices. The Qlik Sense architecture is sophisticated perhaps one of the most sophisticated in this genre of technology. There are around a dozen components in this architecture, which can mostly be deployed on different servers for scalability. Here is a quick run-down: The Hub delivers the Qlik Sense environment through a browser interface Qlik Sense Proxy manages sessions, handles load balancing, handles identity (e.g. Active Directory), and is essentially the entry point for users coming in through the Hub, or via the Management Console. Qlik Sense Engine is in many ways the heart of the system, with its in-memory columnar database. It is RAM based and delivers visualizations, search results and run-time calculation to users. Qlik Sense Scheduler coordinates the loading of data, and looks after time based reloads and data dependencies. Qlik Sense Applications comprise data, a data model and the presentation layer. These are loaded by Qlik Sense Engine. The Management Console is for admin to manage data connections, security, auditing and other matters. A deployment console gives access to the deployment environment (Qlik Sense can run in virtual machine environments VMware vsphere, and Amazon AWS). Copyright butleranalytics.com 6
These are the main components and they provide great flexibility. It means Qlik Sense can scale in almost any manner needed, and multinational businesses can split components over different regions, if needed. Development and Administration Developers get the same drag-and-drop environment as users. Several APIs make life easier and the product extendible. The main ones are: 1 Qlik Sense API to build new visualization types. 2 Qlik Sense Mashup API/.NET SDK to embed visualizations and other functionality. 3 Qlik Sense QVX SDK to create custom data connectors 4 Qlik Sense Engine API for direct communication with the associative engine. Management of the environment is aided by a shared object library for pre-built data, expressions and visualizations. Access to resources can be tuned to user skill level, and access to data sources controlled by security rules. Streams allow fine control over who gets access to a visualization, and in which modes (read, update, modify, delete). The management and governance environment provided by Qlik Sense provides everything a large (and not so large) organization might need. OEMs can use the Qlik Analytics Platform, a new offering allowing developers to embed Qlik visualizations into applications through web mashups, or by embedding into custom applications. Qlik Sense Summary Qlik Sense can be summarized by five key concepts: 1 It is an easy to use data visualization and exploration platform. 2 It supports a level of data discovery that is probably unequalled by any other platform. 3 Extensibility is built into the product architecture. 4 It inherits the mature governed environment of QlikView. 5 It is built ground up for full use on mobile devices. These are the primary differentiators, but scalability, story-telling, and other features expected of a platform of this nature are in general above average in terms of capability. Copyright butleranalytics.com 7
Conclusion It should be clear that Spotfire and Qlik Sense are wholly different offerings, even though they are often put under the single umbrella of visual analytics. Spotfire has anticipated many of the emerging trends in visual analytics and business intelligence through its seamless support for a variety of analytical methods. As the use of analytical tools matures so users demand greater capability, and as stated earlier Spotfire enables analytics without any significant boundaries. It compares well in all aspects with competing products, and particularly its excellent performance, but also excels in the advanced functionality that is needed as experience and skills mature. Qlik Sense is essentially a pure-play visual analytics platform with no advanced analytics capability. Where requirements are defined by the need to create charts, graphs, maps, tables and dashboards, Qlik Sense is one of the most sophisticated offerings available with good extensibility and a very smart data engine. Advanced analytics (predictive analytics, statistics etc) will need additional tools. Ultimately it all depends on need and the scope of the analytical task. Spotfire without question provides the broadest scope and will appeal to users who need to execute diverse analytical tasks. Qlik Sense on the other hand provides a tight focus on the visual representation of data. Copyright butleranalytics.com 8
Ratings It is important to understand that these ratings score Qlik Sense and Spotfire for what they do, and not what they might do. So for example, Qlik Sense has no advanced analytics capability and so no scores are given. The other thing to consider is that 50% is average. Anything above that is above average - and clearly desirable. The ratings decide the tier a product is placed in. 0-25% tier 4, 25-50% tier 3, 50-75% tier 2 and 75-100% tier 1 Qlik Sense Rating - Tier 1 Data Visualization Data Discovery Extensibility Scalability Ease-of-Use Governance Developer Tools Mobile Support Data Connectivity Collaboration Performance 0 20 40 60 80 100 Rating Average for Qlik Sense 75.5% Copyright butleranalytics.com 9
Spotfire Rating - Tier 1 Data Visualization Data Discovery Extensibility Scalability Ease-of-Use Governance Developer Tools Mobile Support Data Connectivity Collaboration Performance Predictive Analytics Text Analytics Advanced Statistics 0 20 40 60 80 100 Rating Average for Spotfire 78.6% About Butler Analytics Butler Analytics is a boutique IT analyst firm specializing in business analytics technologies and methods. It was founded by Martin Butler, best known as founder of Butler Group which, prior to its acquisition, was Europe s largest indigenous IT analyst firm. Business Intelligence, predictive analytics, big data, fast data, Enterprise Decision Management and all other technologies which aid business decision making are covered. www.butleranalytics.com info@butleranalytics.com Copyright butleranalytics.com 10