Parameter Optimization &

Size: px

Start display at page:

Download "Parameter Optimization &"

Eric Watkins
5 years ago
Views:

2015 2 Parameter Optimization & Word Cloud by

1 Parameter Optimization & Word Cloud by hoseong@dm.snu.ac.kr, zoon@snu.ac.kr,

1 Word2vec / Doc2vec Parameters Word2vec parameter, word dimension, window size, training epoch parameter - Dimension - Windows - Training epoch - Methods (PV-DM/PV-DBOW) -

2 1 Word2vec / Doc2vec Parameters Word2vec parameter, word dimension, window size, training epoch parameter - Dimension - Windows - Training epoch - Methods (PV-DM/PV-DBOW) - Concatenate/average - Negative Sampling, Subsampling of Frequent words w11 Dimension Average/concatenate distributed memory (PV-DM) distributed bag of words (PV- DBOW) Brand ID w1 w2 w10 2

3 2 Brand Vector embedding category., Brand, word,. 1 2 Electronics Clothing Samsung review 1 LOREAL review 1 D&G review 1 Electronics Clothing Beauty LOREAL review 500 D&G review 500 Samsung review 500 Beauty 3

4 3 category review 3000., 50 review. Dataset Electronics, Clothing, Beauty category review Electronics : 2,902,574 - Beauty : 1,207,767 - Clothing : 796,366 Category review, 50 sampling 150 dataset 1 Doc2vec Review dataset parameter setting 2 Brand2vec Brand Vector parameter setting 4

5 3 1 1 Doc2vec review. document vector training/test set training set logistic regression. window size, dimension, epoch test accuracy Dimension, Window size accuracy 0.88 Epoch accuracy Dim : 500, windows

6 3 2 2 Brand2Vec Brand Vector. Brand Vector training/test set training set logistic regression. window size, dimension, epoch test accuracy Dimension, Window size accuracy Epoch accuracy Dim : 500, windows

7 3 Brand 1 & 2 1, 2 dimension 500 category,, Document window size, vector., Dimension 500, window 8, epoch 10 Parameter setting 1) Brand vector, Word vector dimension : 500 2) Window : 8 3) Epoch : 10 4) Model : Distributed Memory model 5) 300 6) Average / concatenation average Brand ID w average w1 w2 w10 7

2015 2 Brand2vec : Distributed representation of Brands and Applications

8 Brand2vec : Distributed representation of Brands and Applications Word Cloud by hoseong@dm.snu.ac.kr, zoon@snu.ac.kr,

9 UGC(User generated content), - Counting-based, (, x, x, reproducible) - Counting-based Word2vec - UGC - Word representation 3. : Brand2Vec - Brand2Vec Brand2vec - Parameter search (, ) 4. application - Dataset description (Amazon dataset ) - Parameter search - Application ( 1. Positioning, 2. Keyword extraction ) 5. Discussion (,, ) - Summary - Limitation & Future work 9

10 1 -Counting-based -UGC(User generated content), -Counting-based, ( : / :,, reproducible ) Word2vec Brand2vec -, - - Review / Social Media - bias Noise - *File = File-based or Unstructured data Source : Enterprise Strategy Group,

11 2 Text - 1) UGC mining,, - Text mining Business. - Social Media [2], sentiment [4], LDA(Latent Dirichlet Allocation) [7] - SKT Social Media Buzz 1) - LG Social Media. 2) -,, LG CNS Social Media Text. 1) 2) www. etnews.com/

12 2 Distributed 2) Word Representation Text Text. Discrete., Neural network Discrete representation Distributed Representation ) One-hot vector Word-word / Word-Document Matrix ) Word2vec, Glove dog cat pig dog cat pig / - similarity - - Neural Network continuous vector - Bengio[1] Mikolov[5] (Word2vec) - similarity, similarity 12

13 2 Gradient 2) Word representation Word2Vec C window, p w O w I Maximize W V N W N V Neural Network. Descent W V N, W N V representation. - Vocabulary size V, Hidden layer size N (=Word Vector Size) - Input node One-hot encoding x 1 =[1,0, 0], x 2 =[0,1, 0], x V =[0,0, 1] - W V N W N V random x k, x T k W = W k = v wi window v wi h. Training Input p w O w I = exp(v T w O v wi ) V j=1 exp(v w T j v wi ) [6] : Reference max p w O w I Gradient Descent (+Negative Sampling) 13

14 2 2) Word representation Word2Vec ( class). Word Representation Doc2vec [3] Paragraph, Class2vec[8] / class. w3 Word2Vec, window Average/concatenate Doc2Vec ( ) ID ( ) Class_ID Paragraph ID w1 w2 Class2Vec 4 Review Positive 3 Negative class 14

15 3 Proposed Method 1) Brand2vec Distributed Memory Model[3] & Class2vec[8] Social media product review data vector? ) Samsung Review Brand_ID = Class_brands_Samsung ID Samsung Review Vector w4 Samsung Coca cola Average/concatenate Apple Nike Hyundai Brand ID w1 w2 w3 15

16 3 Proposed Method 2) Brand2vec Parameter Optimization Word2vec parameter, word dimension, window size, training epoch parameter - Dimension - Windows - Training epoch - Methods (PV-DM/PV-DBOW) - Concatenate/average - Negative Sampling, Subsampling of Frequent words w11 Dimension Average/concatenate distributed memory (PV-DM) distributed bag of words (PV-DBOW) Brand ID w1 w2 w10 16

17 3 Proposed Method 2) Brand2vec Parameter Optimization Brand Vector embedding category., Brand, word,., 2. 1 Brand Vector 2 Document Vector Electronics Clothing Samsung review 1 LOREAL review 1 D&G review 1 Electronics Clothing Beauty LOREAL review 500 D&G review 500 Samsung review 500 Beauty 17

18 4 Logitech, 1) Dataset Description Amazon Review Data[4] Electronics review, 3 token. 9, review, Sony, Generic review. Number of Reviews by Brands Electronics review 5,566,912 reviews Review, brand 2,959,904 reviews Review 10 Brand 2,944,904 reviews Total tokens 218,288,099 token Number of unique words 886,768 words Year Num of Reviews , ,405, ,708 Brands 9,557 Review 10 Brands 5,

19 4 2) Parameter search 1 Doc2vec review. document vector training/test set training set logistic regression. window size, dimension, epoch test accuracy Dimension, Window size accuracy 0.88 Epoch accuracy Dim : 500, windows

20 4 2) Parameter search 2 Brand2Vec Brand Vector. Brand Vector training/test set training set logistic regression. window size, dimension, epoch test accuracy Dimension, Window size accuracy Epoch accuracy Dim : 500, windows

21 4 Brand - 2) Parameter search 1, 2 dimension 500 category., Document window size, vector., Dimension 500, window 8, epoch 10 Parameter setting 1) Brand vector, Word vector dimension : 500 2) Window : 8 3) Epoch : 10 4) Model : Distributed Memory model 5) 300 6) Average / concatenation average Preprocessing 1) #,?,@ 2) 3) Brand ID w8 300 average w1 w2 w7 21

22 4 Brand 3) Application #1 Product positioning Modeling Samsung, Apple Brands vector Brand Vector, Vector cosine similarity. Vector, Brand. Samsung Brand Vector vs 다른 Brand Vector Rank Brand Vector Similarity 1 brand_acer brand_toshiba brand_proscan brand_lenovo brand_sharp brand_hp brand_tcl brand_dell brand_vizio brand_kocaso brand_seiki brand_hannspree brand_le Pan brand_pipo brand_asus brand_ocz brand_tabletexpress brand_marquis brand_double Power brand_kaser Samsung Brand Vector vs Word Vectors Rank Word Vectors Similarity 1 samsung samsung's smart google inch ativ lg series plasma smarttv viera dlp microsdxc multitasking dual-core snappy quad-core random bd android Canon Brand Vector vs 다른 Brand Vector Rank Brand Vector Similarity 1 brand_nikon brand_focus Camera brand_fujifilm brand_sigma brand_pentax brand_tamron brand_tokina brand_rokinon brand_sse brand_vivitar brand_olympus brand_bower brand_leica brand_zeikos brand_bower Camera brand_holga brand_ricoh brand_raynox brand_elite Brands Inc brand_herofiber Canon Brand Vector vs Word Vectors Rank Word Vectors Similarity 1 canon zoom dslr telephoto nikon slr bodies megapixel macro point-and-shoot minolta full-frame low-light cannon mm wide-angle panoramic lens body nex

4 3) Application #1 Product positioning Brand2Vec

141820 2 StarTech 0.122915 3 SIB 0.103958 4 HP 0.

23 4 3) Application #1 Product positioning Brand2Vec representation Vector Brand Property. Computer, Earphone, Camera Vector Brand Vector. (computer + desktop)/2 Rank Brand Cosine Sim 1 Dell StarTech SIB HP Cooler Master (earphone+ headphone)/2 Rank Brand Cosine Sim 1 Sennheiser Monster JVC Monoprice Bose (camera + cameras)/2 Rank Brand Cosine Sim 1 Canon Nikon Neewer Case Logic Panasonic

Sony Panasonic Motorola LG RCA Pyle Pioneer Philips JVC

Toshiba HP Dell Corsair Cooler Master Crucial Western Digital

Apple Neewer Canon Nikon OtterBox GriffinTechnology Case Logic

Cheetah Monoprice Mediabridge BlueRigger 4 3) Application #1

24 Sony Panasonic Motorola LG RCA Pyle Pioneer Philips JVC Monster Sennheiser Bose Logitech Microsoft SIB Samsung Asus Toshiba HP Dell Corsair Cooler Master Crucial Western Digital Seagate Netgear TP-LINK Garmin Foscam Roku Google SanDisk Apple Neewer Canon Nikon OtterBox GriffinTechnology Case Logic SANOXY Targus Generic eforcity BELKIN StarTech VideoSecu Cheetah Monoprice Mediabridge BlueRigger 4 3) Application #1 Product positioning Review 50 Agglomerative hierarchical clustering. vector cosine similarity pair-distance matrix, Ward s minimum variance method. Canon, Nikon. PC PC PC Camera Phone acc PC acc Audio Media 24

25 4 Dendrogram 3) Application #1 Product positioning 50 Brands Brands Vector T-SNE Brand2Vec Brand. Phone Acc Roku Google Media players PC components Cooler Master Corsair Foscam Netgear OtterBox TP-LINK Panasonic Sony Griffin Technology JVC Pyle Sennheiser Bose Samsung SanDisk Pioneer Philips PC Asus HP Dell LG Monster Toshiba RCA Apple SIB Monoprice Motorola BELKIN Microsoft StarTech Garmin Generic Logitech SANOXY Mediabridge eforcity Targus BlueRigger Crucial Western Digital Seagate Case Logic Canon Neewer Camera Nikon TV mounts VideoSecu Cheetah 25

26 4 3) Application #2 Keyword extraction. Samsung review,. 'the' 'i' 'and' 'to' 'it' 'a' 'is' 'this' 'for' 'of' 'my' 'with' 'that' 'in' 'on' 'you' 'have' 'was' 'not' 'the' 'i' 'it' 'and' 'to' 'a' 'is' 'my' 'this' 'for' 'of' 'with' 'that' 'you' 'in' 'on' 'have' 'was' 'apple' 'the' 'i' 'it' 'and' 'a' 'to' 'is' 'this' 'for' 'of' 'my' 'that' 'on' 'with' 'mouse' 'keyboard' 'you' 'in' 'but'

27 4 P 3) Application #2 Keyword extraction Brands word.,. Review A j [1] ) A {good, nice, new } Brand Conditional Probability Similarity ) Brand P HP new = exp HP new exp HP new +exp Dell new + Brand Vectors B i, P(B i A j ) = exp(b i A j ) brands exp(b brand A j ) ) i {Apple, Samsung } [1] 3 Review 3 Random Sampling, NLTK Part of Speech tagging,? 27

28 4 3) Application #2 Keyword extraction, Desktop 9 P(B i A j ). Samsung, Apple, Microsoft. Review PMI [1] [1] window size = 3, 10 word 28

29 4 3) Application #2 Keyword extraction : Samsung Samsung Brand, Samsung 3d Active Glasses Samsung.. glasses, 3d Samsung 3d Active Glasses much cheaper than any other active 3d glasses Dish Network Network, TV Photographer, reviews I should mention that my signal provider is dish network 29

30 4 3) Application #2 Keyword extraction : Apple Extra,. Protective, Apple issue. Trackpad, mouse.. Extra ( ) I recommend buying the or gb and spending the extra money gb Case, plastic, cover Protective case, cover( ) cash, spend recommend a protective carry case 30

31 4 XBOX 3) Application #2 Keyword extraction : Microsoft Microsoft keyboard,. Sculpt, Keyboard Ergonomic Products Elite, positioning Xbox Controller Elite Hot keys 31

32 5 Discussion 1) Text mining Frequency.,,., Brand Vector,. 32

33 5 Discussion 2) Brand Vector,,, Social Media vector representation. clustering, keyword. Brand2vec Keyword Brand2Vec [0.28, 0.56, 0.13, 0.91] [0.32, 0.65, 0.18, 0.25] [0.32, 0.65, 0.18, 0.25] 혜리 ( 예시 ) word distance 단발 0.xx 애교 0.yy 이쁜 0.zz 귀여운 0.ww 김연아 ( 예시 ) word distance 홍보 0.aa 피겨 0.bb 아름다운 0.cc 응원 0.dd 33

34 6 3) Summary & Future work Word2Vec, Brand Vector., conditional probability Brand keyword. parameter search, Brand. Summary - Review Vector representation - Brand Vector - Hierarchical Clustering, Positioning Map - Brand Product Property - Conditional Probability Brand - Brand2Vec,, Future work - Electronics, Brand - review data - Model Sentiment Sentiment analysis 34

35 7 Reference [1] Bengio, Yoshua, et al. "A neural probabilistic language model." The Journal of Machine Learning Research 3 (2003): [2] He, Wu, Shenghua Zha, and Ling Li. "Social media competitive analysis and text mining: A case study in the pizza industry." International Journal of Information Management 33.3 (2013): [3] Le, Quoc V., and Tomas Mikolov. "Distributed representations of sentences and documents." arxiv preprint arxiv: (2014). [4] McAuley, Julian, et al. "Image-based recommendations on styles and substitutes." Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, [5] Mikolov, Tomas, et al. "Efficient estimation of word representations in vector space." arxiv preprint arxiv: (2013). [6] Mostafa, Mohamed M. "More than words: Social networks text mining for consumer brand sentiments." Expert Systems with Applications (2013): [7] Rong, Xin. "word2vec Parameter Learning Explained." arxiv preprint arxiv: (2014). [8] Sachan, Devendra Singh, and Shailesh Kumar. "Class Vectors: Embedding representation of Document Classes." arxiv preprint arxiv: (2015). [9] Tirunillai, Seshadri, and Gerard J. Tellis. "Mining marketing meaning from online chatter: Strategic brand analysis of big data using latent Dirichlet allocation." Journal of Marketing Research 51.4 (2014):

INDEX. PRODUCT OVERVIEW OUR PARTNERS THE PROCESS USPs THE RANGE MARKETING & EVENTS

INDEX. PRODUCT OVERVIEW OUR PARTNERS THE PROCESS USPs THE RANGE MARKETING & EVENTS TECHSCHEME BROCHURE INDEX PRODUCT OVERVIEW OUR PARTNERS THE PROCESS USPs THE RANGE MARKETING & EVENTS PRODUCT OVERVIEW The biggest and best range in a tech benefit. Techscheme is a technology benefit that