Data Warehouses, OLAP, and You Leslie Koorhan

Transcription

1 Seite 1 von 6 Issue Date: FoxTalk September 2000 Data Warehouses, OLAP, and You Leslie Koorhan lkoorhan@earthlink.net In this final installment of a series, Leslie Koorhan explains the twin pillars of Microsoft SQL Server OLAP Services' tools for client access, MDX and ADOMD. MDX is the SQL-like language for querying OLAP cubes, and ADOMD is the object model exposed for accessing those cubes. Together, they provide a relatively simple way to incorporate cube access from any program that can access objects. In the first article of this series, I explained what a data warehouse is, as well as the various terms used in this branch of data storage and access. Then I built a data mart from the sample TasTrade Visual FoxPro database, transferred that database into a Microsoft SQL Server 7.0 database using Data Transformation Services (DTS), and built a cube using the Microsoft SQL Server OLAP Services administrative tool, OLAP Manager. In the most recent article, I demonstrated the server-side object model of OLAP Services, Decision Support Objects (DSO). Finally, this month, I get to explain what I started out to do in the first place: getting the data out of OLAP cubes for the users. Don't get me wrong; I'm not explaining a user interface. Rather, I'm demonstrating the methods used for the programs that will get the data and then present it in whatever format is desired. This is much easier than most presentations of data, because cube data is read-only. So it can be very easy to choose a display tool. One method that I'll show is an add-in called the Microsoft Flex Grid. There will be one difference in this article that will separate it from the rest. In the sample code, I'll be using the sample OLAP cube that ships with OLAP Services, rather than the sample cube that was built several months ago in these pages. That way, anyone can use the sample code with only slight modifications, if OLAP Services is available. There are two tools that OLAP Services supplies for client-side access, MDX (Multi-Dimensional Expressions) and ADOMD (ActiveX Data Objects Multidimensional). MDX appears at first glance to be standard SQL, but it's actually a totally separate language. The second tool is simply ADO, with just a few extra objects for handling cubes. MDX As I mentioned, MDX at first glance seems to be just like a SQL SELECT command because it has SELECT, FROM, and WHERE clauses. But with a closer look, it's obvious that there's something much different going on. Between SELECT and FROM are very strange things, not at all the expressions that are expected in a normal SQL query. Instead, a typical statement looks like this: SELECT [Store].[USA].Children ON COLUMNS, Product.[ProductFamily].Members ON ROWS FROM Sales And there are other items, just as strange, in the WHERE clause, resulting in an MDX statement that looks like this: SELECT {[Education Level].[Bachelors Degree]} ON COLUMNS, {[Marital Status].[M], [Marital Status].[S]} ON ROWS, {[Product].[Food].[Frozen Foods].Children} ON PAGES FROM Sales WHERE ([Gender].[F], [Promotion Media].[Radio], Measures.[Store Sales]) In order to understand how MDX works, I'll break down the clauses into their defined roles. The SELECT clause is used to define the axes of the result set. The FROM clause lists the name of the cube that's being queried. And the WHERE clause contains what's called the slicer dimensions and/or alternate measures. In the preceding example, the WHERE clause contains both slicer dimensions and an alternate measure. Axes Since cubes are multidimensional, the results from a cube can also be multidimensional. The axes define the number of

2 Seite 2 von 6 dimensions as well as the members that will appear on those axes. Each axis represents one dimension that the writer visualizes when creating the MDX query. Each axis has an index number identifying its position within the cellset. Axes are numbered starting from zero, just like the default in arrays in Visual Basic. The first three axes can be thought of, respectively, as the x-axis, y-axis, and z-axis from geometry and 3-D graphs. And the first five axes (0-4) have aliases that can be used: COLUMNS, ROWS, PAGES, SECTIONS, and CHAPTERS. In a MDX statement, axes should always be listed in ascending numeric order, for ease of reading and maintenance. The syntax of an axis specification is the set desired on that axis followed by ON and then the axis itself, using either Axis(n) or an alias for that axis. Very often, MDX queries only specify two axes, COLUMNS and ROWS, but put multiple dimensions on one or both axes. This is known as a flattened data set. No axis needs to be specified here at all. In that case, the slicer dimensions (see the next section) will be used to filter the result and one item will appear in the results, unless there's no data at all. If there's to be more than one member on an axis, braces ('{}') must be used to enclose all of the members of that set. If there are multiple words in a dimension or level name, or numbers or reserved words, then that member name should be enclosed in brackets ('[ ]'). Sometimes it just makes good sense to use brackets all of the time just to avoid any complications. If a dimension or level name is unique within the cube, then it does need to be qualified; otherwise, the complete hierarchy must be specified with dots in between each level name. In the first of the previous examples, there are two axes. The COLUMNS axis contains all of the store states from the USA, and the ROWS axis contains the top-most product categories. In the second example, there are three axes, with COLUMNS having one position, for Bachelor's Degrees, ROWS having two positions, for Married and Single, and PAGES (the z-axis) having a position for each type of frozen food. Slicer dimensions/alternate measures The WHERE clause contains filters for the resulting set of data, as well as the option to display alternate measures. This clause acts very much the same as the WHERE clause in a normal SQL statement. The difference is that data is filtered along dimensions, causing the result data to be sliced. Only the data from the slicer dimensions will end up in the result. In the second code example previously shown, there are two slicer dimensions, female gender and radio promotions. Only the values from those two dimensions will be included in the result set. The WHERE clause can also specify an alternate measure. Without specifying such, the result set will have the default measure. The default measure is the first measure defined in a cube. To display a different measure in the result set, that measure must be specified in the WHERE clause. It's presented as if it were a dimension, where the name of the dimension is Measures. (Basically, Measures is a dimension to OLAP Services. When creating dimensions for a cube, you're allowed a maximum of 63 dimensions, because Measures is the 64th.) If there are also slicer dimensions in the WHERE clause, simply specify them along with the alternate measure. The syntax here is simpler than the axis specification. If there's more than one, then enclose all slices and alternate measures in parentheses, comma delimited. Each dimension is specified in the same way as the axis dimensions. In the second preceding code example, the measure is store sales. In the first example, the measure is unit sales because that's the default measure. Defining sets As mentioned earlier, the syntax of an axis specification is the desired set followed by ON and the axis itself. The set is the list of members that appear on an axis. There are various ways of defining that set. The first way is simply to list a comma delimited list of members, such as {[Customer].[Canada].[Toronto], [Customer].[USA]. [New York], [Customer].[France]. [Paris]}. This set would have three positions on an axis with the appropriate resulting values. If there's an inherent order to a set's naming, then an inclusive range could be specified by using a colon, as in {[2000]. [Q1:Q4]}. This set would have four positions, one for each quarter in the year Another very common method is to use functions. Although there are more than 100 built-in functions in OLAP Services, I'll only illustrate several of these. The first and most used is MEMBERS. Unlike functions in Visual FoxPro and many other languages, MEMBERS is applied after the dimension specification, as in {[Store].Members} or {[Year].Members}. The difference between these two examples is important. In the former, the axis would contain positions for all stores, broken by country, city, and name, because [Store] is the name of the dimension. In the latter, there would only be positions for each year in the cube because [Year] is the name of a level within the Time dimension. CHILDREN is another function that works in the same way as MEMBERS. With this function, only the set of values one level below the level specified would be returned. Examples are {[Store].Children} and {[1999].Children}. In the former, only positions for the countries would be used because the immediate level below the Store dimension is the top level, which is the Country level. In the latter example, if years were broken down into quarters, then there would be four positions specified. In the second example, the set could have been specified as {[1999].[Q1],[1999].[Q2],[1999].[Q3], [1999].[Q4]} or {[1999]. [Q1:Q4]}. Another function is DESCENDANTS, which is used in the normal way as the function name followed by parentheses with

3 Seite 3 von 6 arguments. The first argument is the member name; the second argument is the level name from which sets will be returned. An optional third argument allows specification of other levels instead of the one specified, or in addition to, that level. The next example illustrates the use of this function: DESCENDANTS([Food],[Brand Name]). This would result in a position for each brand name within the Food level of the Product dimension. If the third argument of BEFORE was used, then it would include a position for each member of each level between Food and Brand Name. In this case, that would be the Product Department, Category, and Subcategory levels. As mentioned earlier, multiple dimensions can be placed on one axis. One way of doing this is by using tuples. A tuple is a multiple dimension point expressed as an intersection between the desired dimensions. An example of a tuple is ([Promotion Media].[Radio], [Promotions].[Best Savings]). Notice the comma delimited list of dimensions within parentheses. One tuple represents one position on an axis, but with multiple members. Given that, the following code is an example of a set of four tuples that could be used on an axis: {([Gender].[M],[Marital Status].[S]), ([Gender].[M],[Marital Status].[M]), ([Gender].[F],[Marital Status].[S]), ([Gender].[F],[Marital Status].[M])} There's an easier way to define tuples by using the CROSSJOIN function. This function creates tuples from two other sets by creating positions for every possible combination of the two sets. A better way to express the previous example would be: {CROSSJOIN([Gender].Children,[Marital Status].Children)} PivotTable Service Microsoft SQL Server OLAP Services includes server-side components and client-side components. The server side is referred to as OLAP Server and exposes its functionality through Decision Support Objects (DSO), which was illustrated in the previous article of this series. On the client side, OLAP Services has something called PivotTable Service. This is the component that makes requests of the OLAP Server to get the data. This service also has an OLE DB provider so that OLE DB applications can request data from the OLAP cubes. This provider exposes its functionality through a special ActiveX Data Object interface, subclassed as Multidimensional (ADOMD). ADOMD has many objects for getting to the metadata as well as the data. One of the features of the PivotTable Service is that it provides cache management on the client for cube data. This means that once data has been transferred to a client, that data remains in memory, allowing for faster response times when requerying. This can also aid in drilling up through data. If a user requested data at a low level of a dimension and then requests the same data but at a higher level, then the PivotTable Service doesn't need to send the request to the OLAP Server. Instead, it will calculate the data by rolling up the cached values. It can also filter the data if the user requests a subset of the data that's already in cache. Finally, if the user wants additional data as well as what was already requested, then the PivotTable Service will only request data from OLAP Server that isn't in the local cache and combine with the values that are there. Another feature is that the PivotTable Service can create local cubes that can be stored on the client computer for later analysis, and then act as the server for that data. This means that it's easy to create a local cube (through the MDX commands CREATE CUBE and INSERT INTO) on a laptop that's attached to the network, and then later query that cube when the computer is no longer online. If possible, this would be a nice feature to explore in a future article. The ADOMD object model ADOMD's object model (see Figure 1) consists of a Catalog object that provides for the establishing of a connection to a multidimensional data store, a CubeDefs collection with a reference to every cube in the database, and a Cellset object, which is the result of a multidimensional query, the MDX statement. Within the Cellset object there's the cellset metadata as well as the Cells collection itself that has the data. Figure 1: The ADOMD object model. The Catalog object has two properties: ActiveConnection and Name. In order to connect to an OLAP server, just set the ActiveConnection property to a connection string. The connection string consists of three parts: the name of the OLE DB provider, the name of the server, and the name of the database to use. The three parts are labeled "Provider," "Data Source," and "Initial Catalog." The provider for OLAP Services is always "MSOLAP." If you're working on the server itself, you can use "LocalHost" as the Data Source name. The whole string looks like this: "Provider=MSOLAP;DataSource=LocalHost;Initial Catalog=Foodmart." Of course, you can use a different server name and database name. In the sample form mdxgrid.scx (available in the accompanying Download file), the connection string is specified in a text box. Once the ActiveConnection property is set, it can then be used with any of the other objects, such as the Cellset object in order to make a connection. But rather than do it that way, you can use the standard ADO Connection object. This is better because if you're using the ActiveConnection property of the Catalog object, then the connection is temporary and has to be

4 Seite 4 von 6 reestablished with every MDX query. By using a Connection object, the connection to the server can remain open as long as needed so that the MDX queries are handled faster. In the sample form, a Connection is used rather than the Catalog object. The CubeDef object has a Properties collection that consists of Property objects, each of which has a Name property and a Value property. The Property objects attached to a CubeDef include CatalogName, CubeName, CubeType, CubeGUID, and Description. Within the CubeDef object are the objects that define the structure of the cube. There's a Dimensions collection directly within the CubeDef object, and each Dimension object has Name, UniqueName, and Description properties. The Dimension object has a Hierarchies collection, and each Hierarchy object has Name, UniqueName, and Description properties. And this goes on and on. Also worth mentioning is the UniqueName property, which is an unambiguous name for that object. Within Hierarchy objects is a Levels collection. Each Level object has the usual properties, but also a Depth property indicating how deep the level is within that dimension or hierarchy. (If a dimension has only one hierarchy, then for all practical purposes the Hierarchy object is the same as the Dimension object.) A Level object also has a Caption property. Within a Level object, there's a collection of Members, each of which has many properties indicating the actual dimension values. Each Member object has a Name property that's the actual dimension value ("USA" would be the name of a member in the country level). Member objects can also be used to reference other level members. For example, the Children collection of a country member would be the members of the state level. All of the collections previously mentioned are zero- based, so if the Count property indicates two objects in the Levels collection, they can then be referenced as Levels(0) and Levels(1). By using the CubeDef object and its contained objects, you can discover all of the information about the cube structure. By using these objects, you can query a cube and then offer choices to the users choices based on what's there, especially if you design general-purpose classes or programs to work with any OLAP cube. Submitting MDX queries with ADOMD The Cellset object is the place where the MDX query goes and where the results end up. There are three properties and two methods that are commonly used with this object. The ActiveConnection property is where the connection string goes, or where a reference to a Connection object is linked. The State property indicates the state of the cellset. If it's open, there's data in the cellset. The Source property is the MDX query statement to be executed. The Open method sends the query to the PivotTable Service or execution, and the Close method releases the resources of the Cellset object. Within the Cellset object there's more metadata about the cellset itself, as well as the data values. Every Cellset has an Axes collection, with each axis from the query presented as one object in this collection. Each Axis object has a Positions collection. Each Position object represents one spot along the axis where a value will appear in the cellset. The position is a point along the axis. This object contains a Members collection, where each Member object represents the dimension members that make up the position. This way, whether a position on the axis is a one-dimensional item or a combination of items, the members contain that information. Figure 2 is an illustration of a dataset with each object labeled. In this diagram, the axis object highlighted would be Axes(1), the position object shown would be Positions(5), and the member objects would be Members(1) and Members(3). On the columns axis, there are eight positions, where the first, second, fifth, and sixth positions have four members each, the third and seventh positions have three members, and the fourth and eighth have only two members. Figure 2: A visual representation of a dataset returned from an MDX query with the objects marked. Finally, the data itself is in the cells of the Cellset object. To get at a Cell object, you use the Item method of the Cellset. There are three ways to reference a cell within the cellset: List the names of the members for each position. Use axis coordinates (zero-based on each axis). Use the ordinal position within the cellset, where 0 represents the first position of the array. Examples if these three ways, using Figure 2 and the cell with a value of 54, are as follows: ocellset("white","usa","usa_north","boston","1991,qtr 3") ocellset(5,4) ocellset(37) Notice that when using the first method with multiple members per position, all of the member names must be specified to identify a position. Members are listed in ascending order according to dimension nesting with the outermost dimension first, and axes are listed in ascending order. In the second method, axes are specified in increasing order. Ordinal positions can also be discovered through the Ordinal property of a Cell object. The two ways to see the contents of a cell are through the Value and FormattedValue properties. The latter is the actual data

5 Seite 5 von 6 presented in its internal data type. The FormattedValue is character data. Given all of this information, how might this be used? In the Download file, I've included a Visual FoxPro form, mdxgrid.scx, and I've taken a very simple approach. This example deals with a hard-coded MDX query that can be modified for drilling up or down within the sample Foodmart database's Sales cube. The first query that's executed is the first example shown earlier in this article. But the column axis value and the row axis value are stored as properties of the form. The cellset itself is also stored as a property of the form. So is the Connection object used, and even the FROM and WHERE clause values are stored as properties. This makes this form very easy to modify. The form has only three controls: a text box for the connection string, a command button to make the connection and retrieve the initial cellset, and a Microsoft FlexGrid control to display the results. The advantage of the FlexGrid control is that it displays row headers as well as column headers. The command button's Click event creates an ADO Connection object and opens the connection with the string found in the text box. This way, even though the string is hard-coded, you could modify it before using the command button. The Click event also populates the column, row, and from properties of the form, and then calls the DisplayData method of the form. This method doesn't actually display any data; all it does is build the MDX query, create a cellset object and then run the cellset Open method, passing the MDX query and a reference to the Connection object. Then it calls the DisplayCellset method, passing in the cellset object. This method clears the grid and sets the number of columns and rows according to the count of positions of each axis. The TextMatrix is an array representing all positions within the FlexGrid, including the row and column headers. In a FlexGrid, the zero column is where row headers go, and the zero row is where column headers go. The big problem to adjust to is that the TextMatrix lists the rows number first, then the columns number, whereas the cellset is the reverse. The next step is to set the column headers: FOR lncolumn = 1 TO THISFORM.grdResults.Cols - 1 THISFORM.grdResults.TextMatrix(0, lncolumn) = ; toadomdcellset.axes(0).positions(lncolumn - 1). ; Members(0).Caption When the row headers are displayed, it would be nice to make it pretty by indenting the headers in the grid to indicate sublevels. (This doesn't work for column headers because they're side by side.) So the next step is to find out the "base" level of the row axis. This is the top level in the original MDX query. What it means is that after fixing that level, this sample program is unable to drill up above that level. For example, if [Store States] is the level defined as the axis, then this program would be unable to show [Store Country] data, but it would be able to drill down to [Store City] data, and below. After finding out the "base" level, that number is used to put spaces in front of levels below that. * get the depth of the first level on rows lnbaseleveldepth = toadomdcellset.axes(1). ; Positions(0).Members(0).LevelDepth * display row labels FOR lnrow = 1 TO THISFORM.grdResults.Rows - 1 lcindent = ; SPACE((toAdomdCellset.Axes(1).; Positions(lnRow - 1).Members(0).LevelDepth ; - lnbaseleveldepth) * 3) THISFORM.grdResults.TextMatrix(lnRow, 0) ; = lcindent + toadomdcellset.axes(1). ; Positions(lnRow - 1).Members(0).Caption Finally, the cell data is displayed in the FlexGrid using the FormattedValue property. Notice the discrepancy between the row and column numbers of the grid and the axes numbers of the cellset. This is because the FlexGrid used the zero positions for the headers. * display the data FOR lncolumn = 1 TO THISFORM.grdResults.Cols - 1 FOR lnrow = 1 TO THISFORM.grdResults.Rows - 1 THISFORM.grdResults.TextMatrix( lnrow, lncolumn) = ; toadomdcellset.item( lncolumn - 1, ; lnrow - 1).FormattedValue The last thing to do is to save the cellset as a property of the form.

6 Seite 6 von 6 The only other action on this form is to allow the user to drill down into finer details of the cube, and then back up as far as the starting MDX query. This is done through the double click event of the FlexGrid. The example will only work for double clicks in row or column zero of the grid. It works by using the TOGGLEDRILLSTATE function of MDX. This function encompasses two other functions, DRILLDOWNMEMBER and DRILLUPMEMBER. The code takes the UniqueName property of the member that's being clicked and combines that with the previous values from the row or column axis to create a new line for the MDX query. Then it simply runs the DisplayData method. DO CASE CASE THIS.MouseCol = 0 lomember = ; THISFORM.adomdCellset.Axes(1).Positions( ; THIS.MouseRow - 1).Members(0) THISFORM.cMDXRows = "ToggleDrillState(" + ; THISFORM.cMDXRows + ", {" + ; lomember.uniquename + "})" THISFORM.DisplayData CASE THIS.MouseRow = 0 lomember = ; THISFORM.adomdCellset.Axes(0).Positions( ; THIS.MouseCol - 1).Members(0) THISFORM.cMDXColumns ; = "ToggleDrillState(" + ; THISFORM.cMDXColumns + ", {" + ; lomember.uniquename + "})" THISFORM.DisplayData ENDCASE Conclusion With MDX and ADOMD, you can query a cube and do anything with the resulting data that you'd like: display it for the user to see, print it in a report, or even do further data manipulation with the figures. There's much more than just what was presented in this article. I'd suggest that if you're interested in using these tools, you avail yourself of the excellent Help in the MSDN library for ADOMD. The Help that comes with Microsoft SQL Server OLAP Services covers the MDX syntax only. By the time you read this article, there's a strong possibility that SQL Server 2000 will have been released. Everything I've mentioned here and much, much more will be in the new release. The only change is that OLAP Services will be called Analytical Services. I hope that through this series, you've learned more than enough about data warehouses and OLAP, and have seen the many uses for it within any organization.