e-warehousing with SPD Server Leigh Bates SAS UK
Introduction!e -Warehousing! Web-enabled SAS Technology! Portals! Infrastructure! Storage
End to End Data Warehouse
Web Enabled Data Warehouse
Web Enabled Data Warehouse WWW WWW
Open End-to-End e -Warehousing Benefits! Allow IT to focus on business delivery, not tool integration/maintenance! Reduce resource training requirements! Leverage investments in existing technology
e -Warehousing Portal WWW
Portals
Portals! Ability to search huge volumes of data! Membership! Personalisation! Activity tracking *! Channel management! Security *
Activity Tracking! What is the pattern of usage?! Keep it running! What information do my business users use?! Keep good information close! What information do they not use?! Keep it clean and streamlined
Activity Tracking! What information should they know about?! Keep users up to date with company issues! What information might be useful?! Keep people informed
.com vs Bricks & Mortar 60.com Bricks & Mortar Clicks & Mortar 20 15! Pure.coms will have market saturation in 2001! Pure B&M channels in rapid decline! C&M will become the norm Balance and e-speed will be the top priorities to evolve enterprise architecture Meta Group 2000
e -nterprise Data Warehouse WWW Portal WWW
e -Warehouse Loading! Access Products! V8 Transparent Access! Access to Teradata! Mainframe! SAS/Warehouse Administrator! ODS! Data Step
Storage Data Repositories...! SAS Tables/Data Sets! SAS Multidimensional Database! Hybrid OLAP! Access to Other Vendor s Data Structures! SAS Scalable Performance Data Server
HOLAP architecture Client/Server Client/Server DATA PROVIDER Cache Model Viewer WWW
SPD Server! Security! High Performance! High Scalability! Low Maintenance! Web data volumes are unpredictable! Very LARGE,efficient files & other Warehouse features, reduce the level of programming needed
Information Delivery (vs. OLTP)! Large complex read-only queries! Sequential scans common! Complex selections and calculations! Aggregation! Bulk loading! Sustained I/O throughput vital for response times! Surge pattern of growth Volume DW OLTP Time
Case Study - Large Telco Then...! Oracle as standard RDBMS! Typical Oracle load times:! 26 hrs load! 48 hrs index rebuild! Implemented SPD Server for 20% of DW storage Now...! Typical Load Times! 4.5 hrs load! 0 hrs index rebuild! Implemented SPD Server for 80% of DW storage! Oracle as standard RDBMS! SPD Server as standard Analytical Storage Engine 6 Terabytes
Other Customers Smile.co.uk (Co-op bank) Abbey National GE Capital Eli Lilly Prudential Bank Xelector.com
Data Analysis Mart Warehouse administrator Analysis and Modeling Enterprise Miner and related functions Data Warehouse Server Warehouse Administration Warehouse Administrator Oracle Data Access Data Management Business Intelligence ACC/Oracle (Web logs) ACC/Other Metadata SPD Server SAS/MDDB SAS/EIS and related functions SAS/IntrNet Development Enterprise Reporter AppDev Studio SAS/IntrNet Web-Based Reporting - Published - Dynamic - OLAP
Automated Data Partitioning SAS Data sets SAS SPD Server Data Storage Data *.SD2 Index *.SI2 Metadata Index Data B-Tree Bitmap *.MDF *.1.DPF *.2.DPF *.3.DPF *.4.DPF *.IDX *.AUX *.HBX
Technology Infrastructure SMP Architecture RAM HDD I/O controller HDD CPU CPU CPU CPU HDD I/O Operating system controller HDD Application Application Application Shared Everything
Serial Architecture!"#$%&&#" BACKPLANE '() *#+,"#--%" '() *#+,"#--%" '() *#+,"#--%"
Serial Architecture!"#$%&&#" BACKPLANE '() *#+,"#--%" '() *#+,"#--%" '() *#+,"#--%"
Parallel Architecture.!/0.%"1%" 23"%45 23"%45 23"%45 23"%45 23"%45 23"%45 23"%45 23"%45 BACKPLANE '() *#+,"#--%" '() *#+,"#--%" '() *#+,"#--%"
Parallel Architecture.!/0.%"1%" 23"%45 23"%45 23"%45 23"%45 23"%45 23"%45 23"%45 23"%45 BACKPLANE '() *#+,"#--%" '() *#+,"#--%" '() *#+,"#--%" Single Logical Table
SPD Server 3.0 Features! V8 compliant! Admin GUI! Automatic aging! SMP enablement! New enhanced bitmap indexing! VERY large tables! SQL pass-through! ASYNC parallelism! 64-bit edition!! Formats(SAS and User)! Incremental backup and restore! Extensive security - Access Control Lists (ACLs)! Row level integrity! ODBC/JDBC/htmSQL! SQL C-runtime DLL/shared library API! Dynamic refresh of Server parameters.! Parallel aggregation engine
e -Storage with SPDS! Very high performance store for detail data which you can get from Web logs! Use SPDS parallel group by to create summarized tables! Support for Version 8! Proven track record in the field!!!!! Scalable architecture, size and speed!! High ROI
Summary! e-warehousing is part of The SAS System! e is for e nterprise! SAS has the tools to build and manage e-warehouses! SAS has the storage to scale and grow e-warehouses
More Information?! Intelligent Warehousing Stream! Technology Centre : Theatre 9! e-intelligence Stream! Business Solutions : Theatre 9! Related Papers! SPD Server architectural design! New features in SPD Server V3! e data sources! Webhousing and reporting on e-data