Enterprise Informatization LECTURE

Transcription

1 Enterprise Informatization LECTURE Piotr Zabawa, PhD. Eng. IBM/Rational Certified Consultant www:

2 Lecture 5 Analytical tools in business

3 Discussion of needs It is usually not sufficient to customize or develop an enterprise application that just supports business processes. Independently of the approach we used to make application flexible: Classical (possibly with design patterns) With business model engines we are faced with the problem of decision support by the application. Usually such decisions are made on the basis of huge amount of historical data. This problem of decision support by the application is dual to the problem of our support for the application by feeding it with business rules.

4 Discussion of needs Kinds of decision made on the basis of historical data: Selling strategy for particular products Construction of loyality programs Strategies of suppliers acquisition Strategies of foreign markets entering Strategies of products assortment management

5 Discussion of needs So far, we know how to make the single and on-line decisions on the basis of small amount of historical data. These decisions were made on the basis of simple statistical analysis or on the basis of CEP with or without time window. An example of the last approach was checking if a withdrow or bank trnasfer disposal is suspicious. However strategic decisions are usually made on very large amount of data. Such decision may be even based on all transaction of the enterprise. Moreover very complex and deep ststistical analysis is required in such situations.

6 Discussion of needs There are, generally speaking two approaches to data analysis performed for the purpose of decisons making: Manual analysis of reports Automatic decision making Manual analysis is not very interesting for us in contrast to the methods of such reports generating. Much more interesting and of growing importance is the second approach that delagates such decision to enterprise applications.

7 DSS Decision Support Systems Such systems constitute the enterprise applications that support or even perform decisions on each level of the enterprise. Sych systems were evolving gradually. Now they are called Business Intelligence. For now we will focus on complex data analysis wich is required for both approaches the maual and automatic ones.

8 Kinds of analysis Getting results of huge amount of data analysis in order to find templates or trends which cannot be classified as simple analysis is called data mining. Very complex mathematical algorithms for data segmentation and calculating probabilities of future events (prediction) are used in data maining. Tools for data mining: Associated to data base servers like Oracle [ External like Statistica [

9 Data mining techniques based on Data Mining Techniques Data Mining Crucial Concepts in Data Mining Data Warehousing On-Line Analytic Processing (OLAP) Exploratory Data Analysis (EDA) and Data Mining Techniques EDA vs. Hypothesis Testing Computational EDA Techniques Graphical (data visualization) EDA techniques Verification of results of EDA Neural Networks

10 Manual analysis of reports based on

11 Manual analysis of reports based on

12 OLAP From many ways listed above the most interesting for us is quite simple analysis, so in consequence the most popular one OLAP (Online Analytical Processing). OLAP is sometimes tied to the group of algorithms called Interactive data mining. However it is limited to the relatively simple analysis of multidimensional data tables of relational databases. OLAP tools are dedicated to fast mining of data aggregated in tables with the aid of special queries. Such queries are executed by OLAP servers associated to particular database servers.

13 OLAP vs. Data Mining Sometimes however OLAP is put into oposition with Data Mining techniques. OLAP supports the following kinds of analyses: data summarization cost allocation time series analysis what-if analysis time-series forecast, ale bez inductive inference capabilities In contrast to OLAP data mining is characterized by the ability to inference from examples (inductive inference or computational learning).

14 OLAP vs. Data Mining Another characteristic feature of OLAP that differentiates it from data mining is making accesible multidimensional views on data including hierarchical one. In practice both data mining and OLAP are used together in many ways. The most frequent approach is data preparation by data mining algorithms with suceeding OLAP analysis.

15 Kinds of OLAP systems: OLAP Multidimensional MOLAP (Multidimensional-) traditional, data stored in OLAP cubes, multidimensional views Relational ROLAP transformed data aggregated in tables of relational database servers hybrid HOLAP The notion of data warehouse is connected to thematical data grouping (like customers data, transaction data, ). The data are stored in relational databases and come from many different data sources.

16 OLAP OLAP cube:

17 OLAP case study The case study is taken from: It illustrates a growing shoe shop that transforms into shop net. The analysis needs are also growing.

18 Case study - table First we need to analyse quantity and value of sell for a particular shoe style per each month. It ca be done by addition of two tables (reports) : Quantity Total Value Tax Exclusive In this case months are in rows and styles are in columns.

19 Case study - table Enterprise grows and new shops are created. Also the number of data grows significantly. Now we can create 6 reports: Two previous ones for each or all shops Two previous ones for each or all styles months in rows, shops in columns Two previous ones for each month or for the year shops in rows, styles in columns Now we would like to take into account sex, colour, size, The dimension of the problem grows.

20 After insertion of tables for dimensions mentioned above into the database its scheme looks like the one shown to the left. This is the star with arms representing problem dimensions.

21 Case study - cube The dimensions mentioned above are called in OLAP just dimensions. There is however one more dimension the time. It does not have its representation in the above db structure.

22 Case study - cube Inspite of the notion of dimentions another notion of measuer or variables is important. These are the values that are analysed multidimensionaly in our case number of shoes and sell value. These measures (variables) are placed in elements of the multidimensional cube. This problem could be seen as the analysis of distribution of for example number of shoes in dimensions of moths, styles and shops.

23 Case study - hierarchy Now it is important to move with high flexibility along time axis (not only per months). So, we can take into accont: Month Week Year Day Season any time range But we will take into account only red time parameters.

24 Case study - hierarchy If we would build hierarchical structure with a year decomposed into months, months decomposed into weeks, weeks into days, then we will be able to map any date into all elements of this hierarchy. For example 18 th of May 2010 is mapped to 19 th week of year 2010 and this week to the year This hierarchical decomposition coould be reflected in our database in the way shown on the next diagram.

25 Case study - hierarchy

26 Case study - hierarchy In our software system data is collected from each shop everyday and this data is consolidated in the simple way in database the structure of which was presented on the last diagram. The time hierarchy is also stored in the database.

27 Case study calculation on the fly Usually there is no need for storing in database such data that can be easili calculated from stored data. Such calculations are known as formulas. An example one is VAT addition to the stored netto prices.

28 Case study - attributes Usually it is worth sorting references just for convenience and unambiguity. For example we should store UUID of the style in place of the product name. Such an approach is important during consolitation of data comming from different shops. Moreover a reference can contain many attributes. The examples connected to the shoe style are: colour, material, fason, What should be represented as attribute and what as dimension? If each shoe size has different reference, then the size should be an attribute. Otherwise the dimension.

29 Case study - attributes Thanks to the addition of attributes we can get answers from the system to the following questions: What colour of shoes was sold best in August 2009? How many women shoes made of sky was sold by a particular shop in year 2008?

30 Case study time calculation OLAP servers offer handling of more complex questions regarding time, like for example taking into account periodical events. Examples of answers obtained with the aid of such time related complex queries: How the profit from the sale changed in comparison to the analogical month from the last year? How the profit from the sale from the last month changed in comparison to the avarage profit from the sale from last 3 months? What the trend will look like for the next 12 months?

31 Case study data sparsity Let us assume that our shops net prospers very well. We have references, 500 shops which introduce new data each day. It gives 12000*500*365 cells in the cube each year. That is about 2 mld cells. Fortunately in practice there is no shop that offers full assortment. If we assume that each shop offers 600 references and works about 250 days per year we obtain only 600*500*250 cells filled by data. It gives us about 75 mln cells with data in the cube. In order to save resources we can mark dimensions as dense or thin in OLAP servers.

32 Case study pivot tables The simplified OLAP analysis limited to problems discussed above is offered by Microsoft Excel. The Pivot Tables should be used in this case. Total sale without tax "Au bon pied" Outlet (All) Sum Total VTE Reference Month Week Day Total June 2000 Week Week Total

33 The end