Terabyte-class data analysis for CRM in service provider

Transcription

1 Terabyte-class data analysis for CRM in service provider NTT COMWARE CORPORATION Ryo Mukae ( mukae.ryo ryo@nttcom.co..co.jp )

2 NTT COMWARE CORPORATION Achievement in building the fundamental operating system and network of NTT ERP Business Application system EAI Web business support service Communication control system System Integration Middleware Database Fee collection management system CRM Network construction and maintenance IP Network Data Mining Center Web server construction Copyright (C) NTT COMWARE CORPORATION

3 About Data Mining Center The comprehensive analysis center equipped with Large-scale DWH and the OLAP& data mining tool Large-scale DWH - NCR Teradata ( 24CPU + 8.6TB Disk ) - Oracle9i ( 4CPU + 2.6TB Disk ) + OLAP & Data Mining - SAS Products (mentioned later) - SPSS Clementine - BrioQuery Explorer - Microsoft Excel Analysis Flow Business inquiry Grasp actual condition of customer business in past - present - Subject presentation -Goal set Data extraction Customer history etc. accumulated data NTT Comware Data Mining center Know-how Special staff Relief Large scale Strict Security Business Guide Line Offer business guide line to achieve the goal * The names of companies and products used here are the registered trademark or trademark of each company. Copyright (C) NTT COMWARE CORPORATION

4 The Analyzed Provider - A major ISP which has access points nationwide - The number of account = approximately 1 million - The access log of about 1.2 billion observations (=600GB) per month Copyright (C) NTT COMWARE CORPORATION

5 Background of the Analysis - Due to the enormous amount of access log, log analysis had not been carried out until the present - In order to analyze, large-scale analysis environment and advanced analysis know-how were required Analysis trial service, which is one of the services currently offered by the Data Mining Center of our company, was utilized in the form of outsourcing, and analysis consulting was carried out for several themes. Copyright (C) NTT COMWARE CORPORATION

6 Analysis Environment NCR / WorldMark5200 Teradata Sun / Enterprise4500 SAS 24CPU 8.6TB Disk 4CPU 2.6TB Disk SAS/CONNECT SAS/STAT SAS/ACCESS to Oracle SAS/ACCESS to Teradata Teradata Enterprise Miner SAS Integration Technologies SAS/GRAPH WebHound Base SAS 100Base-TX SAS ECO(STAT Plus) 10User & Enterprise Guide ClientPC * The names of companies and products used here are the registered trademark or trademark of each company. Copyright (C) NTT COMWARE CORPORATION

7 Analyzed Data Access Log - Account ID - Access Date - Access Time - Requested URL -Etc Join Key Customer attributes - Account ID -Sex - Age (6 Categories) - Charge Plan - Discount service(y/n) -Etc + The Access Log from August 1, 2001 to August 31, Customer attributes as of August 31, 2001 Copyright (C) NTT COMWARE CORPORATION

8 Analyzed Main Themes - Comprehensive analysis Monthly / Weekly / Daily traffic status The access status classified by web site genre - Customer Behavior Clustering Contents, browsing frequency, stay time by genre, etc. are made from raw log data, and the users are clustered using the k-means method, SOM KOHONEN network, etc. Furthermore, the correlation between contents and the users needs are grasped by studying each cluster. - PR site analysis It grasps whether a PR page within a provider is linked to profits, from the number of repeated visit by user, or stream information. Copyright (C) NTT COMWARE CORPORATION

9 Analyzed Main Themes - Churn analysis Service subscription status and customer attribute information, such as the access status before the cancellation, usage of discount service, and the charge course, are combined, to identify the cancellation pattern. - Click stream analysis By using link analysis and MDS, the click stream of the user in a site is understood, and the relevance of contents are grasped. - Other + Classification of customer behavior pattern by measuring the number of access bytes per unit time. + Study changes in the access status observed during the subscription period. + And so on. Copyright (C) NTT COMWARE CORPORATION

10 Processing of Access Log The following processes are carried out in order to efficiently analyze the large volume of log made up of 1.2 billion observations - Grant session information Session information is given to a log by access interval. - Grant Web Site code A site master is created and URL, which consists of dozens of o bytes of character sequence, is managed as 5 bytes code. - Create summary according to the theme The target access log and customer attribute information are combined according to the theme, and the summary only for the specified theme is created. Copyright (C) NTT COMWARE CORPORATION

11 PR Site analysis About PR page The page is intended by the provider to increase the overall access while introducing new-arrival sites and tie-up sites. It is updated on every Monday. --- The following analysis was carried out to this page. (1) Analyze the effect of publication, by looking at the changes in the number of access to the site before and after the site was published as a part of PR, during the period subjected to analysis. (2) Analyze the effect of publication, by analyzing the number of stream from PR page to the site published on the page, during the period subjected to analysis. (3) Analyze the effect of PR page included in the provider s s site contents, by looking at the number of access to the PR page. Copyright (C) NTT COMWARE CORPORATION

12 PR Site analysis --- (1) The effect of publication is analyzed by looking at the change before and after the publication in the number of accesses to the site introduced in the PR page during the period of analysis. Site A introduced on the PR site on 20 August Introduced on the PR site Copyright (C) NTT COMWARE CORPORATION

13 PR Site analysis --- (2) The effect of publication is analyzed by analyzing the number of stream from the PR page to the site introduced on the page during the period of analysis. The number of stream from the PR site Copyright (C) NTT COMWARE CORPORATION

14 PR Site analysis --- (3) By looking at the number of access to the PR page, the effect of PR page as the provider s s site contents is analyzed. The number of access user Monday PR Site Update Copyright (C) NTT COMWARE CORPORATION

15 Analysis result - Analysis is possible even if the volume of log is as large as 1,200 million observations - From Customer Relationship Management perspective, it becomes possible to discover personal hobby and interest areas from customers behavior history data, including the things that can be done through customer clustering. It can be used to understand the strength and weakness of the contents provided, and can be applied for improvement. Copyright (C) NTT COMWARE CORPORATION

16 Issues Deployment of measures to link the analysis results to profit, when a full-scale analysis is to be implemented - What contents? Prize? Advertisement? - With what kind of means? Within the in-house website? DM? Telephone? Mail? Copyright (C) NTT COMWARE CORPORATION

17 To the future Based on the result from this initiative, the following future initiatives i are planned: - Analysis with more depth, by adding the other information such as questionnaire results, etc. - Visualization of stream information, using Enterprise Miner software - Proposal of Balanced Score Card implementation, based on the access cess log analysis NTT COMWARE will continue to regard SAS as business intelligence tool, and intend to actively provide advanced solution services to the customers in a wide-variety of industry. It is certain that more value will be delivered to the customers through the integration of technology and knowledge between SAS and NTT COMWARE, in the form of joint projects between the two companies. Copyright (C) NTT COMWARE CORPORATION

18 Best Relationship Copyright (C) NTT COMWARE CORPORATION