Hierarchical Classification of Amazon Products

Transcription

1 Hierarchical Classificatin f Amazn Prducts Bin Wang Stanfrd University, bwang4@stanfrd.edu Shaming Feng Stanfrd University, superfsm@ stanfrd.edu Abstract - This prjects prpsed a hierarchical classificatin strategy fr Amazn prducts. Classificatin strategy and the inference efficiency were the tw fcuses. A strategy t apply general classificatin algrithms t hierarchical classificatin prblem was prpsed. Parallel prgraming and hashing techniques were applied t imprve inference efficiency. I. INTRODUCTION Autmatic hierarchical classificatin is drawing mre attentins nwadays. With the explsin f infrmatin n the Internet, managing infrmatin by classifying them int hierarchical categries is becming mre and mre imprtant. Internet encyclpedias like Wikipedia, Medias like newspapers and nline retailers like Amazn are the majr demand surces. This prject aims t slve the hierarchical classificatin prblem fr Amazn prducts. In this prblem, each prduct will be classified int a hierarchical categry where bth the destinatin categry and the parental categries are clearly specified. There are tw majr challenges in this prblem efficiency and accuracy. Due the huge size f Amazn prduct catalg, there are enrmus pssible assignments fr a prduct. Hence finding the mst pssible assignment will be a cmputatinal cnsuming as well as time cnsuming prcess. An efficient classificatin strategy must be develped s that each prduct can be classified fast enugh t keep retailers and custmers n Amazn frm waiting fr t lng. Besides efficiency, accuracy is als an imprtant cnsideratin. If the prducts cannt be classified accurately, there will be trubles fr bth custmers wh want t search fr a prduct and fr retailers wh want t maintain their catalg. These tw challenges in hierarchical classificatin are als the fcuses f this prject. After the prblem frmulizatin in sectin II, ur methds will be presented in sectin III and be evaluated in sectin IV. Cnclusins and future wrks are presented in sectin V. A. Set II. PROBLEM FORMULIZATION The dataset is a cllectin f Amazn prducts with highly detailed prduct infrmatin and their preassigned labels (categries). The labels are rganized hierarchically as shwn in Figure 1. It is different frm a simple tree structure since a nde may have multiple parents. E.g. Nde in Figure 1 has nde 1.2 and nde 2 as its parents. Such structure is called Directed Acyclic Graph (DAG) [1]. As what the name implies, the relatins between ndes are directed and there is n cycle in the graph Figure 1. Categry Hierarchy There are19103 labels in ttal in the directed acyclic graph. Each label is a nde in the graph and its sublabels (sub-categries) are its children ndes. The graph has a single rt. Mst f the label ndes has a depth (distance frm rt nde, with each edge a length f 1) f 4, where the maximum depth is 9. Als, each prduct may multiple labels which lcate at different places in the graph. The label infrmatin f a prduct is stred in the attribute BrwseNdes. They are described as a path frm rt nde t its label. Since the prduct may have different labels, there may be multiple paths in this attribute.

2 Each prduct has 19 attributes where each attribute cntains multiple prperties and sub-prperties with rich infrmatin fr a certain aspect. In this prject Pruned Editrial Reviews was the particular attribute that was used t build ur base classifier. They are the cncise prduct descriptin prvided by the vendr r sme trusted parties. B. Accuracy Imprvement Imprving the accuracy fr hierarchical classificatin in general is t brad a tpic fr a term prject. Hence we narrwed dwn the scpe t finding a gd strategy fr applying a given base classificatin algrithm. Given a classificatin algrithm as ur base classificatin algrithm, such a Naïve Bayes r lgistic regressin, there are many ways fr them t be applied t a hierarchical classificatin prblem. And fr different prblems, the best base algrithm may als vary. Hence, it is mre imprtant t find a gd strategy fr applying the base algrithm, which was set as the majr gal fr accuracy imprvement in this prject. Detailed strategies will be shwn in next sectin. C. Efficiency Imprvement There are arund 20,000 categries and mre than 330,000 prducts in the catalg. Efficiency will then be an imprtant cncern. T imprve the inference efficiency, i.e. t minimize the time used fr predictin, bth prgramming techniques and algrithm imprvement were explred. D. Evaluatin Metrics Bth accuracy and efficiency were evaluated in this prject. Accuracy per layer (depth) was the majr criteria fr different strategies. This is because this metric des nt nly capture hw well a strategy is ding, but als reflects the advantage and disadvantage f a strategy in detail. Inference time and training time were the majr criteria fr efficiency evaluatin. Inference time refers t the time that an algrithm needs t make a predictin while training time stands fr the time needed n training. Inference time is a mre crucial requirement fr applicatins since it affects user experience and system setup directly. A shrter training time will make develpment less expensive. III. METHODOLOGY A. Efficiency Imprvement i. Parallel Cmputing The training and predictin f every ndes were applied in a breadth first search (BFS) rder. It means fr ndes at the same layer (depth), the data selectin, training and predictin culd be dne in parallel. Due t the existence f glbal interrupt lck (GIL) in pythn, nly a single cre can actually be utilized in pythn s multi-thread mdule. Thus multiprcess mdule must be applied t speed up the perfrmance. Hwever, writing speed t glbal data then became a prblem fr different prcesses, s the balance between cmputing and writing speed needs t be carefully handled. Till nw, 8x speedup was achieved n a 4 cre CPU with hyper-threading. ii. Classifier and Hashing After training a classifier, it will be saved t disk fr later use. Since a label may appear multiple times during predictin, hashing the classifiers will bst up the speed. In the riginal prgram, all data were saved as a list which was prcessed ne by ne in rder t filter ut thse that belng t a certain label s they can be trained. Hwever, during the predictin prcess, if the grandchild f a nde needs t be predicted, there is n need t search fr the data again since we already have all the needed infrmatin when predicting that nde s child. Thus a data hash table is created t hld the data fr later use in next layer in the BFS rder. B. Accuracy Imprvement i. Base Classifier Multinmial Naïve Bayes classifier was chsen as the base classifier in this prject. Pruned editrial review f each prduct was used as the input feature. The reviews were first encded int ASCII characters and then stemmed. After stemming, sme stpwrds were remved t reduce the feature dimensin. At last, term frequency inverse dcument frequency (tf-idf) was applied t weight each wrd prperly. ii. Classificatin Strategy Flat Strategy Flat strategy is brutal style strategy used as a base line. In this strategy, the hierarchical structure is

3 ignred. In ther wrds, all ndes in the graph are cnsidered equally. Fr example, bk and art bk desn t have any relatinship anymre. S all the labels will be trained tgether in this strategy. Tp-Dwn Strategy: In a tp-dwn strategy, a path t a label frm the rt will be predicted fr a given. Cnsidering the directed graph f the labels, several algrithms can be applied n generating a label path. Greedy Search: Greedy search is a special case f K- search which will be cvered in the sectin belw. In greedy search, when predicting frm any label nde, the child with mst prbability will be chsen. K-Beam Search: K- search is mre cmplicated and has lts f variatins. The general idea f K- search is that instead f chsing the child with mst prbability directly, it will chse K children with highest prbability, then at the very end (leaves f the graph) the mst pssible ne will be picked. The variatin f this algrithm lcates where when cunting the prbability, multiple ways f calculatin can be used: nde prbability, path prbability and discunted prbability. Figure 2 shws an example fr future explanatin. Given a prduct, assuming it has 70% prbability t be a Bk and 30% prbability t be Kitchen fr classifier Rt. And fr classifier Bk, it has a 40% chance t be Art bk, and 60% chance t be Fictin bk, similarly fr the classifier Kitchen. Rt 70% 30% Bk Figure 2 Classificatin Example Nde Prbability: Kitchen Art Fictin Dish Chp. 40% 60% 70% 30% Nde prbability desn t have any histrical infrmatin when cunting prbabilities. In the abve example with K=2, under classifier rt, bth bk and kitchen will be kept in the result because they are the 2 nde with highest pssibilities. Hwever, when predicting the next level f labels, the result wuld becme Dish (70% prbability) and Fictin (60% prbability), which are higher than Art (40% prbability) and Chpsticks (30% prbability). In ther wrds, infrmatin abut their parent will be ignred. Path Prbability: Fr path prbability, the multiplicatin f the prbabilities f every nde n the path wuld be used t chse the highest K prbabilities. In K=2 case, bk and kitchen wuld be kept fr the first step withut any change cmparing t nde prbability. But fr the next level f predictin, Art and Fictin will be kept as their prbabilities are 0.7 * 0.4 = 0.28 and 0.7 * 0.6 = 0.42, which are higher than the prbabilities f Dish (0.3 * 0.7 = 0.21) and Chpsticks (0.3 * 0.3 = 0.09). Discunted Prbability: Discunted prbability is a variatin f path prbability with weight n the prbability f different levels. In path prbability in the example, Art is calculated as 0.7 * 0.4 = Here a cefficient is added, s the prbability becmes 0.7α * 0.4β, where the value f α and β needs t be adjusted at run time. Cmbined Strategy Since different strategies may achieve the best perfrmance at different layer, it is better t cmbine strategies t maximize the verall perfrmance. First f all, different variatin and parameters f K- search will be tested, t find the parameters (α, β and K) with highest accuracy fr each layer. Then fr a given prduct, if all the variatins give a label in same layer, the result frm the methds that has the highest test accuracy in that layer will be chsen. If different variatins predict a label in different depth, nly thse results that are at the best perfrmance f each methd are cnsidered. Amng all the cnsidered results, the ne with highest testing accuracy with deeper layer will be chsen. Let s take the fllwing situatin where 3- search perfrms best at depth 3, 2- search perfrms best at depth 4 and greedy search perfrms best n reset f the depths as an example. Fr a given prduct, if all methds predict the label t be in

4 depth 3, the result f 3- search wuld be chsen since it perfrms best at depth 3. If 3- search predicts the result t be a label at depth 2, 2- search predicts a label in depth 4 and greedy search predicts the label t be in depth 8. Only the result f 2- search and greedy search wuld be cnsidered since nly greedy search and 2- search gave predicts at the layer where their testing accuracy is better than ther variatins. And fr this case, the result f greedy search wuld be chsen since it perfrms better at the deeper layer. This methdlgy ensures that n degrading will happen in the balance f accuracy at different layers. IV. RESULT AND EVALUATION A. Feature Engineering Table 1 shws the feature selectin prcess fr the base classifier in this prject. Thugh imprvement f base classifier is nt ur majr cncern. Sme effrt was spent t make sure it has a mderate perfrmance. Table 1 Feature selectin fr base classifier Accuracy in % Train Test Train Test Train Test Reviews in UTF-8 cding Reviews in ASCII cding Reviews with stemming Reviews with stemming and stp wrd B. Flat Strategy Table 2 Flat Classificatin Perfrmance Accuracy in % Train Test Train Test Train Test Mn-layer Predictin As we can see frm the result f flat strategy, the accuracy f bth training and testing data are extremely lw, which means flattening the labels and train them in a traditinal way desn t fit the needs f hierarchical classificatin prblem at all. C. Tp-Dwn strategy Table 3 shws the predictin accuracy fr different tp-dwn strategy with varius parameter settings. After experimenting with different discunting parameters, n discunt perfrms best in this case. Table 3 Tp-Dwn Strategy Perfrmances Greed y K(2)- K(3)- K(4) - bea m K(5)- Cmbined Strategy L L L L L L L L L As shwn in the table, the greedy search (K(1)- ) prvides a fairly gd accuracy cmparing t K- search. Hwever, thrugh Layer 2 t Layer 4, K(2)- perfrms better than greedy search. Hence the greedy search will be majr parameter set. When cmbined strategy was applied, it achieve the best verall accuracy. D. Efficiency Imprvement Table 4 Efficiency Imprvement Single Prduct Predictin Ttal Predictin Training Original 20~200 ms s ~1 day Imprved 1.12~3.12 ms s ~2 hurs Here is a cmparisn between initial versin f ur K(3)- prttype and the final ptimized versin with hashes and parallel cmputing.

5 Ttal predictin time was imprved by 18X and training time was imprved by 12X, which is reasnable with 8 prcesses and hashing. Als, due t the limitatin f multi-threading mdule in pythn interpreter as explained in abve sectin, it is believed that a great leap in predictin time will be achieved with C r ther language where multi-cres and gd memry sharing efficiency are available. V. CONCLUSION AND FUTURE WORK This prject explred the strategies f applying base classifier n a hierarchical classificatin prblem as well as methds t bst bth inference and training speed. Cmbining greedy search and K- search with different parameters after experiments is the best strategy that was fund. Parallel prgraming and hashing techniques were prpsed as the slutins t imprve efficiency. But it is interesting that the accuracy f K- search des nt increase with the increase f K. This implies the fact that the discunted path prbability cannt truly reflect hw gd a path is. Hence lking fr a better way f scring different search strategies fr each path will be the mst imprtant wrk fr the future. Acknwledgement: We want t say thank yu t Prf. Ashutsh Saxena and Aditya Jami fr prviding the prject materials and the weekly advice. Als we want t say thank yu t Prf. Andrew Ng and CS229 TA s fr their help during the quarter. REFERENCE [1] Sun, Aixin, and Ee-Peng Lim. "Hierarchical text classificatin and evaluatin." Mining, ICDM 2001, Prceedings IEEE Internatinal Cnference n. IEEE, [2] Babbar, Rhit, et al. "On Flat versus Hierarchical Classificatin in Large-Scale Taxnmies." Advances in Neural Infrmatin Prcessing Systems