AT&T Community Forum Unstructured Data Analysis to Detect Latent Change

Transcription

1 AT&T Community Forum Unstructured Data Analysis to Detect Latent Change 1

2 The AT&T Community Forum The Forum The AT&T Community forum (forums.att.com) is a website that allows users to post questions or comments about any issue related to the AT&T businesses including: Wireless U-verse DIRECTV Internet Digital Life The Community Forum was built as a cost play to divert call volume from customer service centers. The Business Question How can AT&T detect and intervene in customer service issues earlier in the customer dispute cycle than they currently do? 2

3 The Data Data was scraped using Python s Scrapy library from AT&T into a SQL Server database. The data encompasses posts from 2011 to the present and includes: 55,681 unique users 43,937 posts 140,234 replies USERS Username UserURL UserClass UserState MemberSince TotalPosts TotalSolutions AverageRating POSTS PostID PostUserName PostURL PostUserViewCount PostReplyCount PostTitle PostDate PostTime PostText Solved SolvedText Forum REPLIES PostID PostTitle PostUserName PostURL ReplyUserName ReplyNumber ReplyUserViewCount ReplyDate ReplyTime ReplyText 3

4 Initial Analysis Python s Scikit-Learn library was then use to perform unsupervised text clustering to explore the data using both Latent Semantic Indexing and K-means clustering. Sample LSI Output [(0, '0.364*"att" *"phone" *"servic" *"call" *"custom" *"time" *"bill" *"account" *"one" *"back"'), (1, '-0.688*"phone" *"tv" *"connect" *"channel" *"servic" *"internet" *"number" *"work" *"time" *"plan"'), (2, '-0.437*"servic" *"phone" *"call" *"connect" *"bill" *"att" *"custom" *"work" *"channel" *"tv"'), (3, '-0.726*"att" *"call" *"channel" *"servic" *"phone" *"back" *"account" *"plan" *"day" *"tech"'), (4, '0.525*"account" *"number" *" " *"servic" *"phone" *"address" *"bill" *"att" *"name" *"inform"'), (5, '0.419*"channel" *"bill" *"..." *"att" *"connect" *"month" *"plan" *"servic" *"hd" *"internet"'), Sample K-Means Output Cluster 16 words:['alarm', 'disabl', 'door', 'button', 'set', 'set alarm'] Cluster 17 words:['att', 'custom', 'bill', 'time', 'call', 'issu'] Cluster 18 words:['cancel', 'want', 'want cancel', 'order', 'att', 'phone'] Cluster 19 words:['messag', 'myatt', 'login', 'includ', 'featur', 'click'] Cluster 20 words:['dish', 'room', 'tv', 'solut', 'track technician', 'time'] 4

5 SAS Topic Clustering Enterprise Miner was used to explore the text data and select topics based upon an iterative process of manually reviewing the clusters generated. 5

6 SAS Topic Clustering Low Resolution Clustering Medium Resolution Clustering High Resolution Clustering 6

7 SAS Topic Clustering Enterprise Miner was used to explore the text data and select topics based upon an iterative process of manually reviewing the clusters generated. User Selected Topics Topics Terms Documents Accounts Billing Customer Service Data + Unlimited Data Directv Issues Account GoPhone + Refills Insurance + Warranty Passwords Sim Cards Software Updates Text Messaging Troubleshooting Unlock Codes Uverse Wifi Connection Wireless Contracts + Upgrades

8 Sentiment Generation using SAS Enterprise Miner was used to assign sentiment scores to each post based upon the AFINN Sentiment Score matrix. 1. Data was exported from the SQL database into a CSV File 2. The posts were imported into SAS EM and validated for completeness 3. The posts were parsed into tokens and filtered based upon the AFINN Start List 4. Each post sentiment was then calculated based upon the occurrence of AFINN Start Words and their associated positive or negative scores 5. Sentiment scores were saved into the dataset 8

9 Topics and Sentiment Average sentiment is generally positive across the data, with negative sentiment only around Billing 9

10 Latent Change Detection - Topic There are several instances of sentiment change across time periods that supports using sentiment to detect customer service issues in topic areas. June 2015 July

11 Latent Change Detection There are several instances of sentiment change across time periods that supports using sentiment to detect customer service issues across geographies. June 2015 July

12 Issues to LDA LD Issue Issue Description Solution Data Bias Periodicity of Posts Data selected from only one source may over or under represent sentiment in a customer population Time periods with small number of posts may make latent detection difficult due to a lack of signal strength Gather data from multiple social medial sources to reduce bias Reduce detection sensitivity or increase number of samples from other social media sources 12

13 Conclusion This sample of data does appear to indicate that social media data may contain latent change information about customer service issues and sentiment for the AT&T customer base. Next steps would include sampling from multiple social media sources like twitter and google+ to eliminate selection bias and increase sample size. Overall, AT&T should use text analytics as another detection mechanism to identify issues with customer satisfaction. While the forum does provide a business case for reduced contact with customer care centers, proactive monitoring and intervention would further reduce costs to AT&T 13