Geodesic Flow Kernel for Unsupervised Domain Adaptation

Transcription

1 Geodesic Flow Kernel for Unsupervised Domain Adaptation Boqing Gong University of Southern California Joint work with Yuan Shi, Fei Sha, and Kristen Grauman 1

2 Motivation TRAIN TEST Mismatch between different domains/datasets Object recognition Ex. [Torralba & Efros 11, Perronnin et al. 10] Video analysis Ex. [Duan et al. 09, 10] Pedestrian detection Ex. [Dollár et al. 09] Other vision tasks Performance degrades significantly! Images from [Saenko et al. 10]. 2

3 Unsupervised domain adaptation Source domain (labeled) D Target domain (unlabeled) Objective = {( x, y ), i = 1,2,!, N}~ P ( X, Y) S i i S D = {( x,?), i = 1,2,!, M}~ P ( X, Y) T i T The two distributions are not the same! Train classification model to work well on the target 3

4 Challenges How to optimally, w.r.t. target domain, define discriminative loss function select model, tune parameters How to solve this ill-posed problem? impose additional structure 4

5 Examples of existing approaches Correcting sample bias Ex. [Shimodaira 00, Huang et al. 06, Bickel et al. 07] Assumption: marginal distributions are the only difference. Learning transductively Ex. [Bergamo & Torresani 10, Bruzzone & Marconcini 10] Assumption: classifiers have high-confidence predictions across domains. Learning a shared representation Ex. [Daumé III 07, Pan et al. 09, Gopalan et al. 11] Assumption: a latent feature space exists in which classification hypotheses fit both domains. 5

6 Our approach: learning a shared representation Key insight: bridging the gap Fantasize infinite number of domains Integrate out analytically idiosyncrasies in domains Learn invariant features by constructing kernel Source F() t áz z, z i j Target T æf(0) ö ç ç! T = ç F() t x ç ç! ç T (1) è F ø ñ 6

7 Main idea: geodesic flow kernel F() t Source 2 Target 1 z T æf(0) ö ç ç! T = ç F() t x ç ç! ç T (1) è F ø 3 áz, z ñ i j 4 1. Model data with linear subspaces 2. Model domain shift with geodesic flow 3. Derive domain-invariant features with kernel 4. Classify target data with the new features 7

8 Modeling data with linear subspaces Assume low-dimensional structure Target Source Ex. PCA, Partial Least Squares (source only) 8

9 Characterizing domains geometrically Source subspace Target subspace Grassmann manifold G( dd, ) Collection of d-dimensional subspaces of a vector D space R ( d < D) Each point corresponds to a subspace 9

10 Modeling domain shift with geodesic flow F(0) Source F(1) F (),0 t t 1 Target Geodesic flow on the manifold starting at source & arriving at target in unit time flow parameterized with one parameter closed-form, easy to compute with SVD 10

11 Modeling domain shift with geodesic flow Subspaces: F(0) Source F(1) F (),0 t t 1 Target Domains: Source Target 11

12 Modeling domain shift with geodesic flow Subspaces: F(0) Source F(1) F (),0 t t 1 Target Domains: Along this flow, points (subspaces) represent intermediate domains. 12

13 Domain-invariant features z = [ F(0) T x,!, F( t) T x,!, F(1) T x] Source F(0) F(1) F (),0 t t 1 Target More similar to source. 13

14 Domain-invariant features z = [ F(0) T x,!, F( t) T x,!, F(1) T x] Source F(0) F(1) F (),0 t t 1 Target More similar to target. 14

15 Domain-invariant features z = [ F(0) T x,!, F( t) T x,!, F(1) T x] Source F(0) F(1) F (),0 t t 1 Target Blend the two. 15

16 Measuring feature similarities with inner products z z = T T T [ F(0) x,!, F( t) x,!, F(1) x i i i i = T T T [ F(0) x,!, F( t) x,!, F(1) x ] j j j j ] More similar to source. More similar to target. áz, z ñ: Invariant to either source or target. i j 16

17 Learning domain-invariant features with kernels We define the geodesic flow kernel (GFK): Advantages 1 T T T T i, z j ( () ) ( () ) 0 i j dt xi Gx j áz ñ = ò F t x F t x = Analytically computable Robust to variants towards either source or target Broadly applicable: can kernelize many classifiers 17

18 Contrast to discretely sampling GFK (ours) [Gopalan et al. ICCV 2011] F(0) F (),0 t t 1 F(1) áz, z ñ = ò 1 0 i j ( F() t x ) ( F ( t) x ) dt = x Gx T T T T i j i j No free parameters Dimensionality reduction GFK is conceptually cleaner and computationally more tractable. Number of subspaces, dimensionality of subspace, dimensionality after reduction 18

19 Recap of key steps 1 Source subspace 2 Target subspace ( " = )(+) - )(/) - x )(0) - 3! " ázi, z j ñ = $ % &$ ' 4 19

20 Experimental setup Four domains Features Bag-of-SURF Classifier: 1NN Average over 20 random trials Caltech-256 DSLR Amazon Webcam 20

21 Classification accuracy on target 40 No adaptation [Gopalan et al.'11] GFK (ours) Accuracy (%) W-->C W-->A C-->D C-->A A-->W A-->C D-->A SourceàTarget 21

22 Classification accuracy on target 40 No adaptation [Gopalan et al.'11] GFK (ours) Accuracy (%) W-->C W-->A C-->D C-->A A-->W A-->C D-->A SourceàTarget 22

23 Classification accuracy on target 40 No adaptation [Gopalan et al.'11] GFK (ours) Accuracy (%) W-->C W-->A C-->D C-->A A-->W A-->C D-->A SourceàTarget 23

24 Which domain should be used as the source? DSLR Caltech-256 Webcam Amazon 24

25 Automatically selecting the best We introduce the Rank of Domains measure: Intuition Geometrically, how subspaces disagree Statistically, how distributions disagree 25

26 Automatically selecting the best Possible sources Our ROD measure Caltech Amazon 0 DSLR 0.26 Webcam 0.05 Accuracy (%) No adaptation [Gopalan et al.'11] GFK (ours) W-->A C256-->A D-->A SourceàTarget Caltech-256 adapts the best to Amazon. 26

27 Semi-supervised domain adaptation Label three instances per category in the target 60 No adaptation [Gopalan et al.'11] [Saenko et al.'10] GFK (ours) Accuracy (%) W-->C W-->A C-->D C-->A A-->W A-->C D-->A SourceàTarget 27

28 Analyzing datasets in light of domain adaptation Cross-dataset generalization [Torralba & Efros 11] Accuracy (%) Self Cross (no adaptation) Cross (with adaptation) PASCAL ImageNet Caltech

29 Analyzing datasets in light of domain adaptation Cross-dataset generalization [Torralba & Efros 11] Accuracy (%) Self Cross (no adaptation) Cross (with adaptation) 70 Performance 60 drop! PASCAL ImageNet Caltech-101 Caltech-101 generalizes the worst. Performance drop of ImageNet is big. 29

30 Analyzing datasets in light of domain adaptation Cross-dataset generalization [Torralba & Efros 11] Accuracy (%) Self Cross (no adaptation) Cross (with adaptation) Performance drop becomes smaller! PASCAL ImageNet Caltech-101 Caltech-101 generalizes the worst (w/ or w/o adaptation). There is nearly no performance drop of ImageNet. 30

31 Summary Unsupervised domain adaptation Important in visual recognition Challenge: no labeled data from the target Geodesic flow kernel (GFK) Conceptually clean formulation: no free parameter Computationally tractable: closed-form solution Empirically successful: state-of-the-art results New insight on vision datasets Cross-dataset generalization with domain adaptation Leveraging existing datasets despite their idiosyncrasies 31

32 Future work Beyond subspaces Other techniques to model domain shift From GFK to statistical flow kernel Add more statistical properties to the flow Applications of GFK Ex., face recognition, video analysis 32

33 Summary Unsupervised domain adaptation Important in visual recognition Challenge: no labeled data from the target Geodesic flow kernel (GFK) Conceptually clean formulation Computationally tractable Empirically successful New insight on vision datasets Cross-dataset generalization with domain adaptation Leveraging existing datasets despite their idiosyncrasies 33