The Art of Business Discovery

Size: px

Start display at page:

Download "The Art of Business Discovery"

Ariel Small
5 years ago
Views:

1 MAY 2010 A Explainer: The Art of Business Discovery Sponsored by QlikView

2 Contents Introduction 1 Consolidate, Search, Visualize 2 Letting Go of Old Paradigms 4 There Is No Database 4 There Is No SQL Or Other Queries 4 There Is No Retrieval 5 There Is No Delay 6 How QlikView Is Used: Step by Step 7 Common Questions 9 QlikView at Work 10 Fraud Analysis 10 Business Discovery Day 10 Hands-On Sales and Marketing 10

3 1 Introduction Computers are useless, the artist Pablo Picasso once said. They can only give you answers. What we need, he implied, isn t answers, but better questions. QlikView allows users to discover both questions and answers, thanks to a unique structure that allows users to unearth new questions while digging for answers. Filed under the heading of Business Intelligence (BI), QlikView is more than that. It offers users something genuinely new: not just intelligence, but business discovery. To understand the difference, it s important to first clear your mind of existing paradigms. QlikView s functionality will remind you of what s possible in BI and in spreadsheets, but if you cling too tightly to those ideas, you won t understand how QlikView works and why it may be important to your business. This paper provides a bottom-up explanation of QlikView s software, the implications of business discovery, and why those who are used to the limitations of traditional BI love QlikView. A CITO Explainer is a source of news, analysis, research, and knowledge for CIOs, CTOs, and other IT and business professionals. engages in a dialogue with its audience to capture technology trends that are harvested, analyzed and communicated in a sophisticated way to help practitioners solve difficult business problems. This document is a Explainer, a form of content intended to explain a topic that is of potential importance to CIOs and CTOs. This explainer was sponsored by QlikView in order to create a clear explanation of its products. To find out more about how explainers are created go to

4 2 Consolidate, Search, Visualize The value of QlikView and its differentiation from past approaches is best expressed in a three-step process: Consolidate, Search, and Visualize. Consolidate refers to the fact that QlikView makes the process of assembling, associating, and preparing data for analysis simple and straightforward. When data is loaded into QlikView, you explain how it is related to the other data you are looking at. This associative model along with the fact that the data is stored in-memory sets the stage for rapid-fire analysis and ease-of-use. Search represents how QlikView allows users to interact with data in the way they think associatively, with one thought, one idea, and one sequence of questions and answers leading to another. Associative search means finding not only the answers you re looking for, but also answers to questions you ve never even thought to ask. QlikView uses a construct called list boxes (explained in detail below) that allow you to highlight records matching certain criteria. You don t create queries as you would with SQL; you click checkboxes and see what matches. Just click and view (which explains the name). What sets QlikView s concept of business discovery apart from most BI applications is that you also see the records that don t match as well as the ones that do. This has significant implications in the third step. Visualize has a double meaning that refers to the visual display of summarized forms of information and the ability to see those displays change as the selected data is changed. QlikView offers powerful tools for representing data as graphs, charts, tables, and all of the visual elements you expect from a full-featured BI tool. QlikView allows you to build measures (that is, metrics, statistical calculations, and equations) that summarize data quickly, like you would in a spreadsheet. The in-memory architecture means that measures can be recalculated quickly. Every time you click a new set of search criteria in list boxes, you see those measures in the context of your selection. Watching the metrics change as the selection changes brings the data to life.

5 3 Table 1: QlikView s Associative In-Memory Architecture Versus Query-Based Systems How QlikView How Query-Based Task Works Software Works Key Differences Representing relationships between data QlikView makes associations between data sets and bring data together in a consolidated form so that all data is associated and resident in memory. Associations can be made explicitly by users or automatically by QlikView Joins in SQL statements allow users to explicitly define relationships between data SQL: Where clauses must be specified for each query; QlikView data is loaded in a way that is always associated Defining and displaying measures Users define measures that summarize and aggregate data using statistical functions and equations. Aggregates update instantly as selections change Aggregates can be defined in a query. Queries must be executed over and over as selection criteria in the Where clause changes In SQL, calculating aggregates is slower and may require many steps that must be repeated or a restructuring of the data Searching data Values are selected using list boxes. Results are immediately updated. Data that does not match is displayed. New questions arise based on seeing data that does not match Only those values that match the query are displayed. Results are displayed more slowly. More than one person may need to be involved Multiple, spontaneous questions can be asked and answered immediately in QlikView Graphics Graphics are integrated into the QlikView environment and instantly updated as selection criteria change Graphics programs need to be applied to query results. New queries must be run if selection criteria change Graphics can be adjusted and viewed immediately in QlikView

6 4 Letting Go of Old Paradigms In order to appreciate what QlikView can do for you, it s important to know what QlikView is not. Here are some concepts that you can let go of. There Is No Database One of the commonly asked questions is: What database does QlikView use? The answer is, It doesn t. QlikView s patented technology stores everything in memory. QlikView loads various data sources into an in-memory data structure where associations are automatically created. You set up the map of how each data set is related to other data sets just once at the beginning not each time you ask a question. This creates the equivalent of a massive permanent reusable SQL outer join that can be explored quickly and easily. Periodically, as needed, the in memory version of the data can be re-loaded from the original sources. There Is No SQL Or Other Queries Database queries the bedrock of traditional BI are powerful, but they ve always faced limitations. Queries in SQL and other specialized methods are often incredibly complex and difficult to construct, requiring subqueries of data subsets that can boggle the mind. (Entire books have been written on how to build better queries.) Often, the person who is asking the question doesn t have the expertise to write the query. The delay between asking the question and getting the answer can be quite long minutes, hours, or days depending on the bottleneck. It can take less than a second in QlikView. Instead of queries, QlikView presents users with list boxes in which fields in the underlying data are represented. Selecting a set of data involves just a series of clicks. Green items indicate the items you ve selected; white indicates others that are associated with the selection, and gray shows those that have been excluded from the selection. The underlying associations are made implicit by QlikView, rather than explicit by queries, so searches aren t limited by your ability to tell the database what you re looking for. The result is an unprecedented degree of interactivity for a BI application. The ability to free-associate while searching to drill down through data simply by clicking criteria is the essence of business discovery.

unknowns, the questions you haven t thought to ask. The former is the province of traditional BI; the latter is where insights are found.

7 5 List boxes show all the unique values for a field loaded in memory Figure 1: What Is a List Box? There Is No Retrieval Queries have an ontological problem as well: they can tell you known unknowns common, dependable metrics such as sales and customers by region but they can t tell you unknown unknowns, the questions you haven t thought to ask. The former is the province of traditional BI; the latter is where insights are found. A query returning a list of every widget sold to every customer in North America might be useful, but a list of who didn t buy and what didn t sell is equally if not more useful. Because QlikView holds data in memory instead of retrieving it from databases and OLAP cubes, it cannot only display what s included in your search, but also what s excluded as well. This is a real departure from traditional BI; for the first time, unknowns become known.

6 When values are selected in list boxes, the matching values in other columns are highlighted in white, values that don t match appear in gray Green highlights hl h the selected values White

8 6 When values are selected in list boxes, the matching values in other columns are highlighted in white, values that don t match appear in gray Green highlights hl h the selected values White highlights the associated data Gray highlights the non-associated data Figure 2: Matching and Non-Matching Data Are Displayed There Is No Delay Storing data in-memory means there s no database located somewhere else, there are no queries and no retrieval, which means there s no delay in returning results. QlikView displays them as clickable charts updated instantly and continuously by real-time data underneath. Charts can be measures that summarize data created on the fly by users. With a few clicks, for example, you can calculate the average price paid for every widget in North America, or the standard deviation, or any combination of more than two hundred built-in functions. Using QlikView, you can easily build business-relevant metrics. They re not defined ahead of time and stored in a cube. Traditional business intelligence historically made the best of a bad situation: coaxing insights from tremendous volumes of data beyond the handling capability of existing hardware. By pre-processing a set number of queries ahead of time, it could return results quickly. The drawback, of course, is that your deliverables or dashboards were only as good as the questions you asked ahead of time.

9 7 QlikView does nothing ahead of time and is highly optimized to take advantage of speed-ups such as multiple cores and other aspects of the latest generation of microprocessors. Because the data is associative and because it s stored in memory, you re able to ask it nearly anything, in any number of combinations. QlikView breaks the BI bottleneck in the IT department, where most requests to create new dashboards or cubes for specific projects and functions pile up for months while being subject to a strict cost-benefit analysis. QlikView empowers users to ask questions and discover the answers for themselves. How QlikView Is Used: Step by Step QlikView is one of the rare technologies that is popular with the entire range of the end-user population in a company, from the most advanced business analysts to hands-on users who want the answers right away in just a few clicks. The one area where the advanced users may be more active than beginners is in the consolidate step. The first step is understanding the meaning and structure of the data and the relationships between multiple data sources. No technology solution makes this step easier. In the load processes, QlikView captures a permanent record of the associations between data sources, whether simple or complex. With simple spreadsheet style data sets, users can usually perform this step themselves. QlikView accommodates the division of labor sometimes required through the notion of QlikView applications, which are preloaded sets of data with sets of list boxes, measures, and graphics built to support a specific type of analysis. In this way, complex data sets can be delivered and made useable by the widest population of users. But for the most part, QlikView users follow the cycle of business discovery shown in Figure 3.

8 Step 1: Consolidate Data Using g Load Scripts Using QlikView s

Each data set is related to others already in memory so that every

in various repositories Step 2: Search and Define Aggregates

What matches queries and what doesn t match is shown, leading to

Aggregate measures such as counts, sums, averages, medians, other

They are recomputed instantaneously as the selected data changes

boxes show what is selected (green) what matches (white) and what

10 8 Step 1: Consolidate Data Using g Load Scripts Using QlikView s load scripts, related data is loaded into memory. Each data set is related to others already in memory so that every field and every record is associated QlikView Load Scripts Data sets in various repositories Step 2: Search and Define Aggregates QlikView s list boxes allow data to be selected and inspected. What matches queries and what doesn t match is shown, leading to more and better questions. Aggregate measures such as counts, sums, averages, medians, other statistical functions, and the results of equations can be defined. They are recomputed instantaneously as the selected data changes Data in memory with every field associated with all others List boxes show what is selected (green) what matches (white) and what doesn t (gray) Step 3: Visualize Graphics that help visualize data update automatically as the selected data changes Figure 3: A Step-by-Step Tour Through QlikView

11 9 Common Questions Can everything really be stored in memory? Yes, because of fundamental changes in hardware since BI was first designed. The cost per gigabyte of solid-state memory is constantly falling. The number of 64-bit cores on an Intel processor is doubling every 18 months. Traditional BI applications were designed in a computing universe of scarcity. But 25 years and repeated cycles of Moore s Law have made memory cheap and hardware exponentially more powerful. Multiple cores on one chip and 64 bit processing make data processing even faster. Wal-Mart may not be able to store its terabytes of transactions in memory yet, but it will, and many businesses already can. In fact, many traditional BI vendors are switching to an in-memory approach. Does all of this need to be interactive? Can t we just send a report? Traditionally, BI applications produce one report and send it to everyone who needs it. Putting QlikView in the hands of dozens, hundreds, or even thousands of users does have the potential to place a strain on the underlying hardware. This is a serious issue, but the benefits of democratizing the tools of BI outweigh the negatives in terms of hardware costs and performance. The Stack Argument: Why shouldn t I buy BI from the biggest vendor that offers the most features? Choosing a BI application is a bit like choosing a vehicle: the most important question is Will it get you where you want to go? Sometimes the decision is bogged down in discussing features rather than finding the best solution. QlikView is like a bicycleeasy-to-use, inexpensive, maneuverable-something anyone can hop on and take for a ride. BI is a kit for building your own car that requires experts for proper assembly. You have to do a lot of work to get to the first answer. And if you want another answer, you have to do more work to change what you did to answer the new question. The kit to build a car-in software terms the complicated stack of features-may have lots of options, and for some heavy-duty, complicated jobs, that is the right approach. But most business people don t want options; they answers and the ability to ask better questions, which is where QlikView shines.

12 10 QlikView at Work Fraud Analysis Retailers using QlikView have discovered instances of fraud that had eluded their best detective efforts. One company uncovered previously invisible patterns in ordering, shipping, and returns that pointed to millions of dollars in fraudulent transactions that were accumulated a few hundred dollars at a time. It discovered the losses after using QlikView to combine and analyze formerly separate databases-its inability to do so had created the loophole in the first place. The fraud was an inside job, perpetrated by employees familiar with the limitations of the systems. Business Discovery Day One QlikView customer has instituted Business Discovery Fridays when developers are encouraged to put aside regular duties and follow the data where it leads them. This has a two-fold purpose. First, it trains developers to think associatively and to improve their pattern recognition skills. Second, by taking time for exploration, they discover helpful new projects to implement in QlikView. Hands-On Sales and Marketing Sales and marketing teams use QlikView to bring data on the road. Right before a meeting, sales staff quickly review and analyze order history to see how a customer stacks up against other customers. Is the customer buying too little of a popular product? What are other customers buying? What are the trends in orders from the customer? Armed with such information, sales staff can enter a meeting well informed and ready to ask key questions that lead to sales. Because sales staff can do this on their own, the data provided is even more powerful. Marketing staff benefit from the same hands-on approach and can perform deep exploration into who is buying what and why and which promotions are working, all without the aid of the IT department.

What s New in Spotfire DXP 1.1. Spotfire Product Management January 2007

What s New in Spotfire DXP 1.1 Spotfire Product Management January 2007 Spotfire DXP Version 1.1 This document highlights the new capabilities planned for release in version 1.1 of Spotfire DXP. In this