e-warehousing with SPD

Transcription

1 e-warehousing with SPD Server Leigh Bates SAS UK

2 Introduction!e -Warehousing! Web-enabled SAS Technology! Portals! Infrastructure! Storage

3 End to End Data Warehouse

4 Web Enabled Data Warehouse

5 Web Enabled Data Warehouse WWW WWW

6 Open End-to-End e -Warehousing Benefits! Allow IT to focus on business delivery, not tool integration/maintenance! Reduce resource training requirements! Leverage investments in existing technology

7 e -Warehousing Portal WWW

8 Portals

9 Portals! Ability to search huge volumes of data! Membership! Personalisation! Activity tracking *! Channel management! Security *

10 Activity Tracking! What is the pattern of usage?! Keep it running! What information do my business users use?! Keep good information close! What information do they not use?! Keep it clean and streamlined

11 Activity Tracking! What information should they know about?! Keep users up to date with company issues! What information might be useful?! Keep people informed

12 .com vs Bricks & Mortar 60.com Bricks & Mortar Clicks & Mortar 20 15! Pure.coms will have market saturation in 2001! Pure B&M channels in rapid decline! C&M will become the norm Balance and e-speed will be the top priorities to evolve enterprise architecture Meta Group 2000

13 e -nterprise Data Warehouse WWW Portal WWW

14 e -Warehouse Loading! Access Products! V8 Transparent Access! Access to Teradata! Mainframe! SAS/Warehouse Administrator! ODS! Data Step

15 Storage Data Repositories...! SAS Tables/Data Sets! SAS Multidimensional Database! Hybrid OLAP! Access to Other Vendor s Data Structures! SAS Scalable Performance Data Server

16 HOLAP architecture Client/Server Client/Server DATA PROVIDER Cache Model Viewer WWW

17 SPD Server! Security! High Performance! High Scalability! Low Maintenance! Web data volumes are unpredictable! Very LARGE,efficient files & other Warehouse features, reduce the level of programming needed

18 Information Delivery (vs. OLTP)! Large complex read-only queries! Sequential scans common! Complex selections and calculations! Aggregation! Bulk loading! Sustained I/O throughput vital for response times! Surge pattern of growth Volume DW OLTP Time

19 Case Study - Large Telco Then...! Oracle as standard RDBMS! Typical Oracle load times:! 26 hrs load! 48 hrs index rebuild! Implemented SPD Server for 20% of DW storage Now...! Typical Load Times! 4.5 hrs load! 0 hrs index rebuild! Implemented SPD Server for 80% of DW storage! Oracle as standard RDBMS! SPD Server as standard Analytical Storage Engine 6 Terabytes

20 Other Customers Smile.co.uk (Co-op bank) Abbey National GE Capital Eli Lilly Prudential Bank Xelector.com

21 Data Analysis Mart Warehouse administrator Analysis and Modeling Enterprise Miner and related functions Data Warehouse Server Warehouse Administration Warehouse Administrator Oracle Data Access Data Management Business Intelligence ACC/Oracle (Web logs) ACC/Other Metadata SPD Server SAS/MDDB SAS/EIS and related functions SAS/IntrNet Development Enterprise Reporter AppDev Studio SAS/IntrNet Web-Based Reporting - Published - Dynamic - OLAP

22 Automated Data Partitioning SAS Data sets SAS SPD Server Data Storage Data *.SD2 Index *.SI2 Metadata Index Data B-Tree Bitmap *.MDF *.1.DPF *.2.DPF *.3.DPF *.4.DPF *.IDX *.AUX *.HBX

23 Technology Infrastructure SMP Architecture RAM HDD I/O controller HDD CPU CPU CPU CPU HDD I/O Operating system controller HDD Application Application Application Shared Everything

24 Serial Architecture!"#$%&&#" BACKPLANE '() *#+,"#--%" '() *#+,"#--%" '() *#+,"#--%"

25 Serial Architecture!"#$%&&#" BACKPLANE '() *#+,"#--%" '() *#+,"#--%" '() *#+,"#--%"

26 Parallel Architecture.!/0.%"1%" 23"%45 23"%45 23"%45 23"%45 23"%45 23"%45 23"%45 23"%45 BACKPLANE '() *#+,"#--%" '() *#+,"#--%" '() *#+,"#--%"

27 Parallel Architecture.!/0.%"1%" 23"%45 23"%45 23"%45 23"%45 23"%45 23"%45 23"%45 23"%45 BACKPLANE '() *#+,"#--%" '() *#+,"#--%" '() *#+,"#--%" Single Logical Table

28 SPD Server 3.0 Features! V8 compliant! Admin GUI! Automatic aging! SMP enablement! New enhanced bitmap indexing! VERY large tables! SQL pass-through! ASYNC parallelism! 64-bit edition!! Formats(SAS and User)! Incremental backup and restore! Extensive security - Access Control Lists (ACLs)! Row level integrity! ODBC/JDBC/htmSQL! SQL C-runtime DLL/shared library API! Dynamic refresh of Server parameters.! Parallel aggregation engine

29 e -Storage with SPDS! Very high performance store for detail data which you can get from Web logs! Use SPDS parallel group by to create summarized tables! Support for Version 8! Proven track record in the field!!!!! Scalable architecture, size and speed!! High ROI

30 Summary! e-warehousing is part of The SAS System! e is for e nterprise! SAS has the tools to build and manage e-warehouses! SAS has the storage to scale and grow e-warehouses

31 More Information?! Intelligent Warehousing Stream! Technology Centre : Theatre 9! e-intelligence Stream! Business Solutions : Theatre 9! Related Papers! SPD Server architectural design! New features in SPD Server V3! e data sources! Webhousing and reporting on e-data