DOMAIN ADAPTATION BY ITERATIVE IMPROVEMENT OF SOFT-LABELING AND MAXIMIZATION OF NON-PARAMETRIC MUTUAL INFORMATION. M.N.A. Khan, Douglas R.

Transcription

1 DOMAIN ADAPTATION BY ITERATIVE IMPROVEMENT OF SOFT-LABELING AND MAXIMIZATION OF NON-PARAMETRIC MUTUAL INFORMATION M.N.A. Khan, Douglas R. Heisterkamp Department of Computer Siene Oklahoma State University, Stillwater, OK {mohk, ABSTRACT Domain adaptation (DA) algorithms address the problem of distribution shift between training and testing data. Reent approahes transform data into a shared subspae by minimizing the shift between their marginal distributions. We propose a method to learn a ommon subspae that will leverage the lass onditional distributions of training samples along with reduing the marginal distribution shift. To learn the subspae, we employ a supervised tehnique based on nonparametri mutual information by induing soft label assignment for the unlabeled test data. The approah presents an iterative linear transformation for subspae learning by repeatedly updating test data preditions via soft-labeling and onsequently improving the subspae with maximization of mutual information. A set of omprehensive experiments on benhmark datasets is onduted to prove the effiay of our novel framework over state-of-the-art approahes. Index Terms Mutual information, soft-labeling, subspae, transfer learning. 1. INTRODUCTION To build a model for objet lass detetion or regression problem, it is generally assumed that training and testing data are sampled from the same distribution. This assumption is often hallenged in real life senario, i.e. the dataset on whih a model is trained (referred to as soure domain) may vary signifiantly from the test data distribution (target domain) (Figure 1). This may degrade test auray or performane of the trained model and entails the neessity of adapting that model suh that it an overome the distribution differene among training and testing datasets, widely known as dataset bias, domain shift or domain adaptation [1, 2, 3]. In this paper, we will refer the terms domain and data interhangeably. The ultimate goal is to ompensate the distribution divergene between soure and target domains suh that a lassifier trained using soure data an also perform well on diversely distributed but related target data. Formally, soure and target data are represented with same feature enodings, but their marginal probability distributions Domain Adaptation Fig. 1: Example of distribution differenes for the same biyle lass. DA methods will try to resolve this domain shift so that a lassifiation model an perform effetively aross domains. will be different. Here, both domains have the same set of lass labels. One pratial example is, learning an image lassifiation model with images generated from high-resolution amera whereas deploying that appliation into a devie with low-resolution amera. To deal with the domain shift problem, two different settings are usually onsidered: i) unsupervised domain adaptation [4, 5, 6, 7], where no labeled data available in target domain and ii) semi-supervised domain adaptation [8, 9], where only a few labeled data are available in target domain along with abundant labeled data of soure domain. In this paper, we will fous on the more hallenging unsupervised ase. One popular way to deal with the unsupervised ase is to find a ommon feature subspae that expresses shared strutures between soure and target data. For example, Fernando et al.[5] proposed a linear projetion funtion to align soure and target distributions. Some other approahes fous in instane re-weighting of soure data to math with target data in order to minimize their distribution differenes [6, 10, 11, 2]. Most of these works deal with orreting marginal distribution shift [5, 4, 6], ignoring the lass onditional distribution of soure data. Therefore, the learned subspae may not be optimal in terms of lass separation. This motivates us to utilize lass disriminative information of soure data to learn a ommon feature subspae with goals of resolving the distribution divergene and reating a disriminative subspae for unlabeled target data. A supervised tehnique based on maximization of nonparametri mutual information (MI) between data and orre-

2 sponding lass labels has been proved effetive in learning a disriminative subspae [12, 13]. Following prior work in domain adaptation setting with labeled soure and unlabeled target data [7], an iterative method is proposed. At eah iteration, a subspae is learned with MI maximization and then target data lass preditions are reated with soft-labeling (probability that a point belongs to a lass) by utilizing neighboring soure data in the learned subspae. The soft assignment of lass labels for target data is integrated into the objetive funtion [12] of MI maximization and influenes the next iteration subspae learning. These two steps ontinue till onverging to a final subspae. In summary, the ontributions of this work are i) utilizing lass label distributions of soure data along with all domain data distributions to develop a domain adaptation framework, ii) extending supervised method of MI to support unlabeled target data by induing soft-labeling and iii) proposing an iterative approah of ommon subspae learning based on maximization of non-parametri MI indued with soft-labeling. 2. SUBSPACE LEARNING BY MAXIMIZING SOFT-LABELING INDUCED QUADRATIC MUTUAL INFORMATION (QMI-S) Aording to information theoreti literature, Mutual Information (MI) is defined as a measure of dependene between random variables. Assume that X is a random variable representing d-dimensional data x R d and C is a disrete random variable representing lass labels {1, 2,..., N }, where N is the total number of lasses. Let p(x) be the density funtion of x and P () be the lass prior probability. Using Renyi s entropy, quadrati mutual information (QMI) is a non-parametri estimation of MI and is defined as [13], I(X, C) = (p(x, ) P ()p(x)) 2 dx x = p(x, ) 2 dx + P () 2 p(x) 2 dx x x 2 p(x, )P ()p(x)dx x = V in + V all 2V btw (1) Bouzas et al. proposed a subspae learning algorithm using trae-norm formulation of QMI [12]. The data distribution p(x) is estimated by a Parzen window method using a Gaussian kernel. A multivariate Gaussian N (x; µ, Σ) with mean vetor µ and ovariane matrix Σ is, N (x; µ, Σ) = 1 (2π)d Σ e( 1 2 (x µ)t Σ 1 (x µ)) A Parzen window density estimation of p(x) with IID samples x i is n 1 p(x) = n N ( x; x i, σ 2 I ) where n is the ardinality of dataset. We extend [12] by introduing a soft lass labeling into the QMI formulation. The lass prior probability P () an be expressed as n n P () = P ( x i )P (x i ) = P ( x i ) 1 n = S where P ( x i ) is the probability that x i belongs to lass (soft-labeling). For notational onveniene, we will further refer P () as S. The joint distribution p(x, ) an be expressed as p(x, ) = P ( x)p(x) = 1 n P ( x i )N ( x; x i, σ 2 I ) n Following [12] s approah, a trae-norm derivation using the above expressions for p(x), P () and p(x, ) is presented in Table 1 and allows Eq.(1) to be rewritten as where M = 1 n 2 ( I(X, C) = tr { Φ T MΦ } (2) z z T ) ( ) S 2 + n 2 11 T ( ) S 2 1 n 2 z T (3) where z = [P ( x 1 ), P ( x 2 ),..., P ( x n )] T R n 1 is the soft-labeling introdued into QMI and Φ R n m represents the mapped data points from the original feature spae to a kernel Hilbert spae using a mapping funtion ψ : X H, i.e., K = ΦΦ T. This formulation is referred to as QMI-S. A k-dimensional subspae is learned by finding a linear transformation W = Φ T A, A R n k, that maximizes QMI-S. That is, maximize I(X, C) = tr { W T Φ T MΦW } whih with unit ovariane onstraints beomes (see [12]) A tr { A T KM KA } = argmax A T KA=I tr{a T (4) KKA} with M = (M + M T )/2 (5) where M is a symmetri form of M. Centralized Gaussian kernel K R n n is defined as K = K g E n K g K g E n +E n K g E n, where K g (i, j) = N ( x i x j ; 0, 2σ 2 I ) and E n R n n onsists of elements eah equal to 1 n. The trae ratio problem of Eq.(4) an be approximated as a ratio trae optimization [14] and A is found using the generalized eigen value deomposition method [15]. After finding A, the data is projeted by X p = ΦW = KA.

3 Table 1: Derivation of V in, V all and V btw. Here K = ΦΦ T, z = [P ( x 1 ), P ( x 2 ),..., P ( x n )] T R n 1 and 1 = [1, 1,..., 1] T R n 1 V in = p(x, ) 2 dx V all = P () 2 p(x) 2 dx V btw = p(x, )P ()p(x)dx x x x n n n 2 P ( x i )P ( x j )N ( x i x j ; 0, 2σ 2 I ) = n n S 2 1 n 2 N ( x; x i, σ 2 I ) N ( x; x j, σ 2 I ) = n n ( ) 1 S n 2 P ( x i )N ( x i x j ; 0, 2σ 2 I ) j=1 j=1 i j ( ) n 2 z T Kz n 2 tr { T Kz z } n 2 S 2 1 T K1 n 2 S z T K1 ( ) { ( ) } n 2 S 2 tr { K11 T } n 2 tr { ( )} S K 1 z T { ( n 2 tr ( ) Φ T z z T Φ n 2 S 2 tr { Φ T (11 T )Φ } = 1 n 2 tr Φ T 1 ) } S z T Φ Construt kernel matrix K from input data X Step-I Compute M to find a low-dimensional subspae by QMI-S maximization Step-II Update label preditions for projeted target data onvergene Projeted soure and target data Fig. 2: Proposed method for iterative subspae learning based on QMI-S maximization. 3. ITERATIVE IMPROVEMENT OF SOFT-LABELING AND QMI-S SUBSPACE The data available are X R n d onsisting of soure domain data X s R ns d, target domain data X t R nt d and ground truth labels of soure data [y 1, y 2,..., y ns ] T, where n s and n t represent soure and target data size respetively and n = n s + n t. P ( x i ) is defined as follows. For x s i X s, labels are known so hard labeling an be used (i.e, P ( x s i ) = 1 if = y i else 0). On the other hand, target data are unlabeled, so a full distribution is used. For target data, P ( x t i ) is initialized with uniform label distribution i.e., for x t i X t, P ( x t i ) = 1 N for eah {1, 2,..., N }. If soure labels are from a lassifier instead of ground truth then the lassifier s P ( x s i ) an be used instead of hard labels. The proposed iterative QMI-S (Figure 2) onsists of, Step-I: M is omputed using Eq. (3) and (5). To find A for learning QMI-S subspae, Eq.(4) an be rewritten as KM KU = KKUΛ, where Λ is a diagonal matrix of eigen values and U is a matrix of orresponding eigen vetors. For omputational effiieny, we substitute KU with a new variable V i.e. KU = V, multiply both sides by K 1 and obtain M V = V Λ. This is a standard eigen problem where V and Λ represent matrix of eigen vetors and eigen values respetively. As M has rank N 1, the N 1 vetors with largest eigen values are seleted from V. Hene, A will be A = K 1 2 V and the projeted data X p R n k will be omputed as X p = KA = K 1 2 V. Step-II: Target data preditions P ( x t i ) are updated by applying a lassifier trained with projeted soure data. This will eventually update M of Step-I for the next iteration. P ( x s i ) will be remained same through out the iterations. The overall proess is summarized in Algorithm 1. Convergene riterion The proposed algorithm will Algorithm 1 Subspae learning based on iterative QMI-S. 1: Input: Data matrix X=[X s ;X t ] R n d where soure data X s R ns d and target data X t R nt d, soure data labels [y 1, y 2,..., y ns ] T. 2: Output: X p R n k, k-dimensional projeted data. 3: Initialization: For x t i X t, P ( x t i ) = 1 N for eah {1, 2,... N }. For x s i X s, P ( x s i ) = 1 if = y i and P ( x s i ) = 0 otherwise, for eah {1, 2,... N}. 4: Compute a entralized Gaussian kernel matrix, K R n n. 5: repeat Step-I: 6: Compute M matrix using Eq.(3) and (5). 7: Solve standard eigen problem, M V = V Λ. 8: Compute projeted data X p = K 1 2 V. Step-II: 9: Train a lassifier f using projeted soure data Xp s and apply it to update P ( x t i ) with soft-labeling. 10: until onvergene (defined in Setion 3). reah onvergene when subspae hange in two suessive iterations will be negligible. The subspae is defined by the basis vetors V. The differene beween two k-dimensional subspaes an be approximated as a subspae distane on a Grassmannian [16]. One suh distane metri measures the prinipal angle θ between V i and V i+1 of iterations i and i + 1 respetively [17, 16]. A onvergene threshold ɛ is set and the algorithm terminates when θ ɛ. At this state, lass preditions for target data are stabled. 4. DATASET AND EXPERIMENTS The proposed method is implemented and tested against popular benhmark datasets. Offie is a widely used image database for domain adaptation [18]. It ontains three different domains (Amazon, DSLR, Webam) of images aptured with varied settings and image onditions. Images of Amazon are downloaded from amazon site, DSLR ontains images aptured with high-resolution DSLR amera and Webam ontains images aptured with low-resolution web amera. Additionally, a popular dataset for objet reognition Calteh-256 [19] is used. The experiments will be onduted using these 4 domains with 10 ommon ategories seleted from eah of them (Bike, BakPak, Calulator, Headphone, Keyboard, Laptop, Monitor, Mouse, Mug, Projetor). From these 4 domains, a total of 12 DA sub-problems an be

4 Table 2: Comparative results in terms of lassifiation auray(%) of target data for 12 different sub-problems. Eah sub-problem onsists of soure target, where soure or target represents any of the four domains: C(Calteh-256), A(Amazon), W(Webam) and D(DSLR). Methods C A C W C D A C A W A D W C W A W D D C D A D W Avg Origfeat PCA GFK SA TCA TFL TJM QMI-H QMI-S reated, eah of whih ontains one soure and one target domain. Experimental setup The image representations published by Gong et al.[4] are used and the experimental protool of [6, 7] is followed. Input data are whitened with PCA preserving 95% of the data variane. Gaussian kernel σ is set to median of the pair-wise distanes of data in original feature spae. A K-nn lassifier with K set to log(n s ) + 1 heuristi [20] is used in line 9 of Algorithm 1. Convergene threshold ɛ = is used. The proposed QMI-S will be ompared with 7 other methods (see Table 2). They an be ategorized as follows, Without adaptation: Origfeat and PCA indiate the lassifiation auray of target data in original feature spae and PCA subspae respetively. Adaptation based on subspae alignment: It inludes geodesi flow kernel (GFK) [4], subspae alignment (SA) [5], transfer omponent analysis (TCA) [21] and transfer feature learning (TFL) [7]. Adaptation based on subspae alignment+instane reweighting: inludes transfer joint mathing (TJM) [6]. Analysis For eah of the 12 sub-problems with souretarget ombination (C A, C W et.), we reported the lassifiation auray(%) of target domain data. For all methods, auray is determined by a one nearest neighbor lassifier in the projeted spae or quoted from [6]. Our method shows improved performane ompared to others and outperforms them in 7 out of 12 sub-problems. In terms of average auray over all 12 ases, our method is 3.75% ahead of the losest average auray (TJM). We also onduted experiments with hard-labeling (denoted as QMI-H in Table 2) for target preditions. Iterative QMI-S approah learly shows its superiority as it onservatively updates target data labels, whereas QMI-H is aggressive and one a target point is falsely labeled, it is prone to stik with this label in suessive iterations. Figure 3 shows the affinity struture in the learned feature spae for three different methods inluding ours for the sub-problem C A. Projeted data are obtained using Origfeat, TJM and iterative QMI-S method. For illustration, 1051 projeted data from 5 different lasses are hosen Sr Trg Sr Trg (a) (b) () Fig. 3: Assesing the quality of feature subspae by similarity matrix for sub-problem C A using (a) original feature spae, (b) TJM and () iterative QMI-S. with the first set is 584 data from soure domain sorted by lass and the seond set is 467 data from target domain sorted by their predited labels. A similarity matrix with 25-nearest neighbors is onstruted (see Figure 3). The top-left and bottom-right sub-matries of eah sub-figure represent similarity inside a domain (soure or target). Iterative QMI-S exhibits more ompat blok diagonal struture (Figure 3()) ompared to TJM (Figure 3(b)). The top-right and bottomleft sub-matries represent similarity aross domains with better ompat blok diagonal struture generated by our method (whih represents better within-lass similarity). In terms of omputational ost, the average over all 12 sub-problems of the number of iterations of QMI-S until onvergene is 25. The dominate ost in eah iteration is solving an eigen value deomposition of a n n for a set of largest eigenvalues and eigenvetors. 5. CONCLUSION This paper proposes a domain adaptation algorithm based on soft-labeling indued quadrati mutual information. Unlike other subspae alignment methods, the goal is to utilize lass onditional distribution of soure domain to learn a ommon subspae with improved lass separation suh that a lassifier trained with projeted soure data an be applied to projeted target data effetively. In future work, we are planning to inorporate instane weighting into this framework in order to failitate subspae learning only with losely related data samples aross domains along with minimizing the impat of unrelated soure samples.

5 6. REFERENCES [1] Antonio Torralba, Alexei Efros, et al., Unbiased look at dataset bias, in IEEE Conferene on Computer Vision and Pattern Reognition, 2011, pp [2] Jiayuan Huang, Arthur Gretton, Karsten M Borgwardt, Bernhard Shölkopf, and Alex J Smola, Correting sample seletion bias by unlabeled data, in Pro. Advanes in Neural Information Proessing Systems, 2006, pp [3] Sinno Jialin Pan and Qiang Yang, A survey on transfer learning, IEEE Transations on Knowledge and Data Engineering, vol. 22, no. 10, pp , [4] Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman, Geodesi flow kernel for unsupervised domain adaptation, in IEEE Conferene on Computer Vision and Pattern Reognition, 2012, 2012, pp [5] Basura Fernando, Amaury Habrard, Mar Sebban, and Tinne Tuytelaars, Unsupervised visual domain adaptation using subspae alignment, in IEEE International Conferene on Computer Vision, 2013, 2013, pp [6] Mingsheng Long, Jianmin Wang, Guiguang Ding, Jiaguang Sun, and Philip S Yu, Transfer joint mathing for unsupervised domain adaptation, in IEEE Conferene on Computer Vision and Pattern Reognition, 2014, pp [7] Mingsheng Long, Jianmin Wang, Guiguang Ding, Jiaguang Sun, and Philip S Yu, Transfer feature learning with joint distribution adaptation, in IEEE International Conferene on Computer Vision, IEEE, 2013, pp [8] Jeff Donahue, Judy Hoffman, Erid Rodner, Kate Saenko, and Trevor Darrell, Semi-supervised domain adaptation with instane onstraints, in IEEE Conferene on Computer Vision and Pattern Reognition, 2013, 2013, pp [9] Hal Daumé III, Abhishek Kumar, and Avishek Saha, Frustratingly easy semi-supervised domain adaptation, in Pro. of the 2010 Workshop on Domain Adaptation for Natural Language Proessing, 2010, pp [11] Wen-Sheng Chu, Fernando De la Torre, and Jeffrey F Cohn, Seletive transfer mahine for personalized faial ation unit detetion, in IEEE Conferene on Computer Vision and Pattern Reognition, 2013, 2013, pp [12] Dimitrios Bouzas, Nikolaos Arvanitopoulos, and Anastasios Tefas, Graph embedded nonparametri mutual information for supervised dimensionality redution, IEEE Trans. on Neural Networks and Learning Systems, vol. 26, [13] Kari Torkkola, Feature extration by non parametri mutual information maximization, The Journal of Mahine Learning Researh, vol. 3, pp , [14] Yangqing Jia, Feiping Nie, and Changshui Zhang, Trae ratio problem revisited, IEEE Trans. on Neural Networks, vol. 20, no. 4, pp , [15] Keinosuke Fukunaga, Introdution to statistial pattern reognition, Aademi press, [16] Jayaraman J Thiagarajan and Karthikeyan Natesan Ramamurthy, Subspae learning using onsensus on the grassmannian manifold, in IEEE International Conferene on Aoustis, Speeh and Signal Proessing (ICASSP). IEEE, 2015, pp [17] Jihun Hamm and Daniel D Lee, Grassmann disriminant analysis: a unifying view on subspae-based learning, in Pro. 25th International Conferene on Mahine learning. ACM, 2008, pp [18] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell, Adapting visual ategory models to new domains, in Pro. ECCV 2010, pp Springer, [19] G. Griffin, A. Holub, and P. Perona, Calteh-256 objet ategory dataset, Teh. Rep. 7694, California Institute of Tehnology, [20] Ulrike Von Luxburg, A tutorial on spetral lustering, Statistis and Computing, vol. 17, no. 4, pp , [21] Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang, Domain adaptation via transfer omponent analysis, IEEE Trans. on Neural Networks, vol. 22, no. 2, pp , [10] Minmin Chen, Kilian Q Weinberger, and John Blitzer, Co-training for domain adaptation, in Pro. Advanes in Neural Information Proessing Systems, 2011, pp