- Application Developers: Responsible to implement the solutions. How Oracle can meet Data Warehouse End-User Expectations

Transcription

1 ABSTRACT This paper presents the key steps and the responsibilities of a Data Warehouse or Data Mart Architect in developing a Data Warehouse or Data Mart project. The emphasis is placed on the process of gathering end-user requirements and methods to meet their expectations. Hence, the steps involved in setting the standards, providing the vision to a Data Warehouse or a Data Mart project, and documenting the specifications are stressed. These steps are essential whether the Architect is trying to implement a Data Warehouse or a Data Mart. This paper is published in RMOUG Proceedings, January 28-29, The corresponding Power Point is presented during RMOUG Training Days in Denver, Colorado. AUDIENCE Though this paper is aimed for Data Warehouse or Data Mart Architects, following roles in a Data Warehouse or a Data Mart project can be benefited from this paper: - Data Warehouse or Data Mart Architects (Program Managers): Responsible for various components in the Warehouse, proposes technical solutions to IS Managers, and validates the end-user requirements of the Data Warehouse or the Data Mart project. - IS Managers: Responsible to provide integrated enterprise-wide Information System solutions and environment. - Product Manager(s): Leader(s) from end-user community and makes sure all the requirements are specified for each release. The Product Manager also negotiates these requirements with the Product Managers, the Data Warehouse Architect, and IS Manager(s). - Application Developers: Responsible to implement the solutions. 1 / 6

2 - Business Analysts: Responsible to document the end-user requirements. INTRODUCTION How do you scope your Data Warehouse or Data Mart project? Can you define a Vision statement for what you want to do in your project? Are you aware of the components that should be defined? What do you need to communicate with your IS Manager? This paper attempts to answer these questions so that you can have successful implementation and smooth roll-out of your Data Warehouse or Data Mart. INITIATING THE WAREHOUSE For most of the Warehouse architects, the ten thousand dollar question is How do I convince my IS Manager that what I would be spending on the Warehouse project will actually be paying off by itself? Most of the times, an Architect can convince the higher authorities using Return-on-Investment (ROI) estimate. Now, the question is how do you calculate the ROI. Though there is no magical formula in which you can plug in some numbers to get the ROI, you can use the list of benefits described in Business Objectives sub-section of Envisioning: Vision/Scope section of this paper. When these benefits are quantified, you will be able to arrive at an estimate on returns of your Data Warehouse or Data Mart project. In addition, any evidence that supports the following objectives of your IS Manager would be helpful: - to build systems that support end-users - to improve quality of existing applications - to increase productivity of existing resources - to develop applications faster, and - to exploit new technologies for competitive advantage of the organization. On the other hand, if you are having tough time selling the concept of Data Warehouse to your management, either you are looking at wrong solution or you are playing the wrong role in your organization. At this time, look back and see what you are trying to do. Prepare the laundry list of requirements from end-users and present it to the IS Manager(s). If Data Warehouse is the right solution, then this list should talk itself into the Data Warehouse or Data Mart. Once the management agreed with the Data Warehouse initiative, your responsibility is to 2 / 6

3 support these requirements and discuss how you would provide solutions with a suitable solution framework and the right set of tools. KNOW THE PURPOSE What kind of data do you have to load into your Data Warehouse from internal/external sources? However academic it may sound, the right answer to this question is the data that helps you build the Warehouse that supports Strategic or Tactical decisions. The internal data in your organization will help the Warehouse to provide operational decisions while the external data helps you to disclose the relative position of your organization in the market place. In addition, to provide the Strategic or Tactical decisions, the data should also be summarized because the data to support the operational decisions is mostly at detailed level. Hence, the first and foremost expectation from your end-user should be (if not, set it!) that their Data Warehouse should answer Tactical Business Questions because, profitability and ROI are closely tied to the ability to make decisions that correctly reflect changing business conditions. What does a strategic/tactical decision means anyway and how does one get it from the Warehouse? Usually, the data from your customer provides you some Information which when analyzed becomes Knowledge. You use this Knowledge to figure your new marketing plans and strategies, which in turn provides you more data about your customer. In this whole life cycle of the use of the Warehouse data, you have seen both the ROI as well as some strategic/tactical decisions. The reports your end-user needs/asks for are two types. The first type contains the strategic/historical summaries over long-range of periods. This type of reports can be obtained from the Warehouse. The second type of reports, though no text book explains you that this might be a possibility, are based on current volatility and correlation of the data. Even if you have built your Warehouse with loading frequencies as high as 12 hours, you would never be able to get the second type of reports from the Warehouse. How do you manage these type of expectations? The Operation Data Store (ODS), that provides dynamic data, gives you some insight into such scenarios. The ODS provides your end-users a consistent overview of consolidated operational information maintained in disparate operational systems. Though many architects may believe that building an ODS is a necessary step in building a Warehouse, ODS should not be built solely to pump the data into Warehouse. Build it because you have a need for reporting on the operational data. Note that the main difference between an ODS and a Warehouse is that the data in ODS is updateable while, Warehouse contains static data. 3 / 6

4 KNOW YOUR END-USER The major concern for your end-user would be ease of use. However, the more you concentrate on it, the worst will be the performance of your Warehouse. Keep an eye on those wishes. Your responsibility to your user will be to make the right technological choices and build the right architecture. In addition, your responsibility to IS Manager is to provide supporting evidence for making such choices. Set some ground rules to both your IS Manager as well as to your user community regarding how to resolve conflicts, who owns the data, and how to communicate and maintain relationships with the users. These rules will help you concentrate on the data issues than the political issues about data ownership. Make your end-user understand the complete development process. This process is explained in the following sections with a four-phased solution framework. Also, let your end-users know that you will give them a post-project evaluation period in which they can feel free to state all their unanswered questions or unrealized wishes. However, it is your responsibility and to your advantage to conduct a post-project evaluation. Though may not applicable to all cases, billing your customer based on the usage (charge-back methods) could be a major issue. Even if is not a must for financial reasons within your organization, it is definitely an issue to determine the usage of the Warehouse. Security Do NOT push the security features into the next release because what your users see in the first release is not only what data you provide them but also how securely you are providing it. If you fail to attract the users to use the Warehouse in the first iteration, you may never have the second phase of the Warehouse! KNOW YOUR TEAM Do you have right people on your team to help meet the end-user expectations? Technical issues, political issues, human performance issues such as compatibility play major roles in the development process of a Warehouse. If necessary, you need to change the processes and 4 / 6

5 reorganize the structure. Also get the people who are more eager to learn and easy to change and train them if you need to. TAKE ONE BITE AT A TIME: ITERATIVE APPROACH Even if your enterprise Data Warehouse project is expected to take a long time such as three years, identify the short-term deliverables for the next three to six months period. These short-term deliverables should always address the solutions to immediate problems. Moreover, these solutions should fit within the corporate architecture you have planned. This is the best way to put a time-box around end-user expectations so that you can manage and have control on your product releases. Also, the iterative phased approach will be able to overcome the dilemma of architects need to know what users wants out of the Warehouse before the implementation of it while user will only discover what they want AFTER the architect delivers the Warehouse. For managing the deliverables and for smooth roll-out, the following sections describe various phases of a Data Warehouse or Data Mart project. These phases are based on Microsoft s Solutions Development Discipline TM (SDD) but expanded by Raymond James Consulting based on experiences in meeting its clients needs. The SDD framework introduces a milestone-driven process model to manage the risks inherent in a dynamic information solutions environment so that, project teams can concentrate on delivering value to the organization rather than getting trapped in political issues. This framework balances the schedule, resources/ budget, and the features/requirements in a project. As depicted in the Figure 1, each release of a Warehouse or a Data Mart contains four phases, viz., Envisioning, Planning, Developing, and Stabilizing. Each of these phases has well defined milestones. Following sections describe the purpose of each phase and the milestones to be achieved. 5 / 6

6 PHASE The the resources essential and more Data should exist where the Envisioning with Warehouse Executive Current Problem Business Overview than the also features 1: the required benefits one ENVISIONING: a consist, Excel Vision/Scope Process page. Statement: project. Objectives: phase Summary: that such of for spreadsheets for will Data the as contains example, Overview: This be pulling next Warehouse VISION deliverable: List part Summary phase This phase the the of Excel the from section next List / also primary SCOPE problems (Planning of Legacy from spreadsheets the release(s). helps the should processes various discovery features that to such systems, phase). determine consist exist The perspectives from that process inconsistencies Executive The etc. the are have Legacy the following following Current that schedule, been the should Summary systems. determines scope Process. substituting sub-sections. components and be budget, of timeliness, listed: should current This the the and features are section This not release proposed which be is in envisioning Specifications This that Customer-Oriented Internal/Operational Financial Continuous Vision Statement: Benefits Improvement Make Benefits Benefits applications, configurations, you section Scope: Overview Inclusions Exclusions User/Customer explicitly about should List document list the of consist features: all Data Profile: the in Warehouse the features the This Planning The one following section set paragraph that or phase will features the sub-sections. identifies be Data of excluded what the listed Mart various Data exactly project. here While in Warehouse the types will you, writing current drive as Warehouse an this the or release. Architect, the section, Functional Data users, Mart make are vendor the profiles Solution tools for will the each Concept: etc. be memory used type in such requirements This Warehouse section as their describes knowledge of development, their Personal what in using and would Computers what Graphical make sign-off the (PCs), User project forms Interface necessary successful, will be used software project. lists sure Specifically, Project Products Operational Deliverables Acceptance Assumptions: each any Success and milestone. assumptions Criteria Services This Factors section in made the lists Solution about all assumptions responsibilities considered of Users in or building Customers. this document. what and plans Manager), dependencies Constraints Risk Project the when leader person Assessments: the Structure: dependencies from who and in end-user for would Dependencies: the This This be success section described community responsible of should the This in (Product the to Warehouse section list manage list above the the Manager), actual describes section risks, the development. resources names consequences, are all not the of and met. anticipated features and mitigation constraints Exclusions identifies features due development quality logistics Future the assurance sub-section Features: features team team that team The of are the list important Scope: of features List to your this Features end-users section section. might in future overlap However, releases with this the as section well features (Program This Data means, PHASE The documenting defining Vision/Scope Each business Though should promising. that to Planning list Warehouse Preferred Market Glossary: requirement/feature the also and should both 2: that services demand this PLANNING: Hence, be what Demanding might each document. phase section based aid Future List project. role potentially of a your your does functional Features should IS a part FUNCTIONAL the Data define Future For Managers should from market. end-users it of source-data-driven play example, become each Warehouse various all Features specifications match in the feature straight as your may terms part define well SPECIFICATIONS perspectives. back project. agree of ties from that a to current what the document Data back a so will the business end-users that exactly some Mart that be Scope: release The you used feature should consists list of an objective. can the in (through of List Operational understanding features the actually requirements, include of of Data Features expanding Hence, change objective. deliver all that Warehouse Data you providing section management) these data what Store and will terms as services decisions you be project. (ODS) of the a in in list are athe As data In job clause so To this business triggered addition, that the summarize, becomes feature you services Pictorial Description Business User addendum report, your your performance) from if does) development going tougher. you and Depiction SQL). this Services each (two are promised the feature) these (where feature Also, provide using summarization You to (if (what three team specifications, you the need preformatted can should the feature sentences are may can a to user functional you define current need know be requirements is trigger automating specified a reports also the Window release. simple define specifications classes contents deliver this out feature and the or a terms, of great the a or the parameter of how report) objects base the and mainly report document. detail Warehouse does report, what data-model that (to with for this query other get placeholder end-users they feature or you windows following features need the that the enhance Data supports to sake, GROUP for be can sections: Mart, each built. of be the the what your BY field report of On team, be deliverables Extraction Though they particularly after consolidated code help Reporting To would data, addition, lot According product Providing If Operational architect-ed operation symmetric Oracle detection Warehouse is PHASE As developing straight some provide More - End-User solve meet the actual can you good transforming using Data which and be implementation 3: these be connect a construction, providing idea Services DEVELOPING: Tools: if Oracle great is extraction extremely always reporting a problems. to necessary creating help. don t the set Discover stored Oracle provide (what Warehouse some data, requirements valuable Features have side, providing schedules In procedures data Stored ORACLE'S this may standards appropriate knowledgeable you 3.0will data for situation, same such not necessary Procedures, getting to of ODS meet from logistics and quality accessed to your ROLE for Warehouse business pivoting, different packages. all data strategic Warehouse developers your input delivery of team development out your Packages, disparate meet drag view. such systems modified architect, and transformation In process disparate and the as addition, Most available. tactical Data test predefined systems. need drop and team of will scripts a and Mart s decision Gateways provide that to result might However, time be or non-relational Oracle s responsible your to cleansing criteria. end-user, matched, of need tools quality great making the this iterative most product(s). gateways are feature). You feeding Who When customers. others political end-user. tell you and submitting alive. your of a Logistics help time face Corporation, business from them by comes 4: Watch-outs! your of and database desk Training replication easy STABILZIING: high end-users Do schedule propose can Program Data learning your the through team. resolution. helping customers? not test support volume corporate never access: American rules. Store your functional let scripts. users For a Manager tools, technology each using query go customers, Moreover, (ODS), of and data quality that Good Oracle down team. end-users QA, Institute Though one online training specifications different and integrity is leading role, LOGISTICS, assurance news because bother not On Logistics might Replication reporting provides with support, for these a you responsibilities trying locations, used of Warehouse reporting multiple Research, you. be your other will test team, necessary solution document, tool as and Use the be IS ODS can report hand, a scripts sites for responsible the manager replication can update reference tools. them ROLL-OUT redirect architect s end-users. databases overall plan off having would should this Discoverer practically so resolution of smooth replicate that problem. your but becomes ease to their job. end-user they However, can manage updateable responsible (or Warehouse, same roll-out, built of you 3.0 the activities. end-user methods IS With optimal by department) would has data. preserved database, useful the of traffic Quality you training this been Discoverer Quality Especially, can need Data are solution. technology help to insight a your Assurance be solving assurance the by develop your requirements, preferred merged, Discoverer for to process. Mart, shoulder. not of them systems, customized short-term conflict put server by primary managing spoon 3.0 ODS Oracle s available into when each aside from times, It can and the In might Scope Collect requirements Vendors During market aim your don t know-how-to-use tool problems. Creep pitches. trap. end-user have Make list selection, Use gathering. expense Your sure not You one that right aim be this the you are reports. tool, This solve clear should communicate time tool. there can you will Not provide features. not establish just problems be goals cannot solutions that to and Make incorporate confidence a you of solution promise buy your the sure want only other. when IS you to with a all Manager. feature. your start You needed relationship organization. features with this tools are Your On list not the your the from responsibility supposed tools provided become other IS between Not Manager before hand, first to responsible by to fall a you day control tool. you into provides. and hear of identify have You that team for 3.0 lift Sign-Off Create instructions Manager Meaningless Maintain reasons from through for Overall Expectations resolving should and other Sponsorship user Guidelines why data Process recognize users. be IS sign-off the IS you Features integrity Manager. disintegrated determine cannot If user they that packages between what this with need provide which proper data. a that Get look different big resolve what tool will top for documentation occasion. they provides initiate management before any end-user issues sign-off signing. asking all communities with support. meetings. features for Make the the and cause other sure Let how you of At and end-users your users, the have violates the end-user Warehouse. same they determined possible drive the time, should requirements the alternatives give Give with IS be requirements. going them yourto If Architecture Integration Metadata Data Cleansing Promise Limit Get more scope sure less information the that to and the tools the deliver first codes you than phase. buy more. that end-users can Build exist Most be it in integrated Architects incrementally know one system and into get store are your (only excited understandable in first care metadata. phase initiating about architect scalabil Though by a It Wa is SUMMARY In the Also, end-user deliverables summary, Size Aggregates Tools Roll key a framework Outs factors satisfaction for this that smoother paper for an is building Architect discussed presented. roll-out Do Give Make Warehouses not need and them how your This load short-term to to highest easy roll-out framework understand every initiate tools. Data bit cycle level Data deliverables. of You contains Marts about data of (time Warehouse summaries. should into that their between four Warehouse. provide not project major be For the Data highly a milestone-based every versions) Mart If their involved they degree other projects, end-users. need no level longer of in details, training and of summary than they end-user six shou mon the oth BIOGRAPHICAL Consulting, received years and Nagesh CO Tel: ) (303) development Warehouses of nagesh@nagesh.com Anupindi, experience his , a Ph.D. Denver-based using of INFORMATION Raymond Oracle Information Fax: Computer Senior Oracle (303) / consulting Data James. Consultant tools. Engineering Technology Warehousing Consulting, He firm can (4563 specializing design from Solutions, be Group reached 4582 Longs University and S. Practice Data Court, development Ulster at: component-based of Warehouses, Broomfield, Rhode St. Leader Parkway, of Island Client/Server Raymond CO and Suite and analysis, 80023; Data has 100, James Marts. three systems cell: design Denver, He 6 / 6