Package hsiccca. February 20, 2015

Transcription

1 Package hsiccca February 20, 2015 Type Package Title Canonical Correlation Analysis based on Kernel Independence Measures Version 1.0 Date Author Billy Chang Maintainer Billy Chang Canonical correlation analysis that extracts nonlinear correlation through the use of Hilbert Schmidt Independence Criterion and Centered Kernel Target Alignment. License GPL-2 NeedsCompilation no Repository CRAN Date/Publication :21:57 R topics documented: hsiccca-package hsiccca hsicccafunc ktacca ktaccafunc sumwtdiff Index 10 1

2 2 hsiccca hsiccca-package Canonical Correlation Analysis based on Kernel Independence Measures Details Canonical correlation analysis that extracts nonlinear correlation through the use of Hilbert Schmidt Independence Criterion and Centered Kernel Target Alignment. Package: hsiccca Type: Package Version: 1.0 Date: License: GPL-2 Billy Chang: References Chang et. al. (2013) Canonical Correlation Analysis based on Hilbert-Schmidt Independence Criterion and Centered Kernel Target Alignment. ICML Gretton et. al. (2005) Measuring statistical dependence with Hilbert-Schmidt Norm. In Algorithmic Learning Theory Cortes et. al. (2012) Algorithms for learning kernels based on centered alignments. JMLR 13: hsiccca Canonical Correlation Analysis based on the Hilbert-Schmidt Independence Criterion. Usage Given two multi-dimensional data sets, find pairs of canonical projection pairs that maximize the HSIC criterion. hsiccca(x, y, M, sigmax = NULL, sigmay = NULL, numrepeat = 5, numiter = 100, reltolstop = 1e-04)

3 hsiccca 3 Arguments x y M sigmax sigmay numrepeat numiter reltolstop The x-variable data matrix. One row per observation. The y-variable data matrix. One row per observation. Number of canonical projection pairs to extract. The bandwidth parameter for the Gaussian kernel on the x-variable set. A positive value. The smaller the smoother. If NULL, set to median(dist(x)), and will be updated automatically for extracting different pairs of canonical projection. The bandwidth parameter for the Gaussian kernel on the y-variable set. A positive value. The smaller the smoother. If NULL, set to median(dist(y)), and will be updated automatically for extracting different pairs of canonical projection. Number of random restarts. Maximum number of iterations for extracting each pair of canonical projections. Convergence threshold. Algorithm stops when relative change in cost from consecutive iterations is less than the threshold and will then move on to find the next pair of canonical vectors. Details Value Optimization is done by gradient descent, where Nelder-Mead is used for step-size selection. Nelder Mead may fail to increase the cost at times (when stuck at local minima). User may consider restarting the algorithm when this happens. A list containing: The M canoncial projection vectors for the x-variable set. Each column corresponds to a projection vector. The M canoncial projection vectors for the y-variable set. Each column corresponds to a projection vector. Note Current implementation is slow and requires high storage for large sample data. Sample size > 2000 not recommended. Billy Chang References Chang et. al. (2013) Canonical Correlation Analysis based on Hilbert-Schmidt Independence Criterion and Centered Kernel Target Alignment. ICML Gretton et. al. (2005) Measuring statistical dependence with Hilbert-Schmidt Norm. In Algorithmic Learning Theory 2005.

4 4 hsicccafunc See Also ktacca, hsicccafunc Examples set.seed(1) numdata <- 100 numdim <- 3 x <- matrix(rnorm(numdata*numdim),numdata,numdim) y <- matrix(rnorm(numdata*numdim),numdata,numdim) z <- runif(numdata,-pi,pi) y[,1] <- cos(z)+rnorm(numdata,sd=0.1); x[,1] <- sin(z)+rnorm(numdata,sd=0.1) y[,2] <- x[,2]+rnorm(numdata,sd=0.5) x <- scale(x) y <- scale(y) fit <- hsiccca(x,y,2,numrepeat=2,numiter=10) par(mfrow=c(1,2)) for (K in 1:2) plot(x%*%fit$[,k],y%*%fit$[,k]) hsicccafunc Canonical Correlation Analysis based on the Hilbert-Schmidt Independence Criterion. Usage Given two multi-dimensional data sets, find a pair of canonical projection pairs that maximizes the HSIC criterion. Called by hsiccca, and intended for internal use, but users may play with it for potential finer controls. hsicccafunc(x, y, = NULL, = NULL, sigmax, sigmay, numiter = 20, reltolstop = 1e-04) Arguments x y sigmax sigmay numiter reltolstop The x-variable data set. One row per observation. The y-variable data set. One row per observation. Initial projection vector for the x data set. Randomly set if NULL. Initial projection vector for the y data set. Randomly set if NULL. The bandwidth parameter for the Gaussian kernel on the x-variable set. A positive value. The smaller the smoother. The bandwidth parameter for the Gaussian kernel on the y-variable set. A positive value. The smaller the smoother. Maximum number of iterations. Convergence threshold. Algorithm stops when relative changes in cost from consecutive iterations is less than the threshold.

5 hsicccafunc 5 Details Optimization is done by gradient descent, where Nelder-Mead is used for step-size selection. Nelder Mead may fail to increase the cost at times (when stuck at local minima). User may consider restarting the algorithm when this happens. Value A list containing: cost The canoncial projection vector for the x-variable set. The canoncial projection vector for the y-variable set. A vector of (negative) cost values at each iteration. Note Current implementation is slow and requires high storage for large sample data. Sample size > 2000 not recommended. Billy Chang References Chang et. al. (2013) Canonical Correlation Analysis based on Hilbert-Schmidt Independence Criterion and Centered Kernel Target Alignment. ICML Gretton et. al. (2005) Measuring statistical dependence with Hilbert-Schmidt Norm. In Algorithmic Learning Theory See Also hsiccca Examples set.seed(1) numdata <- 100 numdim <- 2 x <- matrix(rnorm(numdata*numdim),numdata,numdim) y <- matrix(rnorm(numdata*numdim),numdata,numdim) z <- runif(numdata,-pi,pi) y[,1] <- cos(z)+rnorm(numdata,sd=0.1); x[,1] <- sin(z)+rnorm(numdata,sd=0.1) x <- scale(x) y <- scale(y) fit <- hsicccafunc(x,y,sigmax=1,sigmay=1) plot(x%*%fit$,y%*%fit$)

6 6 ktacca ktacca Canonical Correlation Analysis based on the Centered Kernel Target Alignment. Given two multi-dimensional data sets, find pairs of canonical projection pairs that maximize the Centered Kernel Target Alignment Algorithm. Usage ktacca(x, y, M, sigmax = NULL, sigmay = NULL, numrepeat = 5, numiter = 100, reltolstop = 1e-04) Arguments x y M sigmax sigmay numrepeat numiter reltolstop The x-variable data matrix. One row per observation. The y-variable data matrix. One row per observation. Number of canonical projection pairs to extract. The bandwidth parameter for the Gaussian kernel on the x-variable set. A positive value. The smaller the smoother. If NULL, set to median(dist(x)), and will be updated automatically for extracting different pairs of canonical projection. The bandwidth parameter for the Gaussian kernel on the y-variable set. A positive value. The smaller the smoother. If NULL, set to median(dist(y)), and will be updated automatically for extracting different pairs of canonical projection. Number of random restarts. Maximum number of iterations for extracting each pair of canonical projections. Convergence threshold. Algorithm stops when relative change in cost from consecutive iterations is less than the threshold and will then move on to find the next pair of canonical vectors. Details Optimization is done by gradient descent, where Nelder-Mead is used for step-size selection. Nelder Mead may fail to increase the cost at times (when stuck at local minima). User may consider restarting the algorithm when this happens. Value A list containing: The M canoncial projection vectors for the x-variable set. Each column corresponds to a projection vector. The M canoncial projection vectors for the y-variable set. Each column corresponds to a projection vector.

7 ktaccafunc 7 Note Current implementation is slow and requires high storage for large sample data. Sample size > 2000 not recommended. Billy Chang References Chang et. al. (2013) Canonical Correlation Analysis based on Hilbert-Schmidt Independence Criterion and Centered Kernel Target Alignment. ICML Cortes et. al. (2012) Algorithms for learning kernels based on centered alignments. JMLR 13: See Also hsiccca, ktaccafunc Examples set.seed(1) numdata <- 100 numdim <- 3 x <- matrix(rnorm(numdata*numdim),numdata,numdim) y <- matrix(rnorm(numdata*numdim),numdata,numdim) z <- runif(numdata,-pi,pi) y[,1] <- cos(z)+rnorm(numdata,sd=0.1); x[,1] <- sin(z)+rnorm(numdata,sd=0.1) y[,2] <- x[,2]+rnorm(numdata,sd=0.5) x <- scale(x) y <- scale(y) fit <- ktacca(x,y,2,numrepeat=2,numiter=10) par(mfrow=c(1,2)) for (K in 1:2) plot(x%*%fit$[,k],y%*%fit$[,k]) ktaccafunc Canonical Correlation Analysis based on the centered kernel target alignment. Usage Given two multi-dimensional data sets, find a pair of canonical projection pairs that maximizes the kernel alignment criterion. Called by ktacca, and intended for internal use, but users may play with it for potential finer controls. ktaccafunc(x, y, = NULL, = NULL, sigmax, sigmay, numiter = 20, reltolstop = 1e-04)

8 8 ktaccafunc Arguments x y sigmax sigmay numiter reltolstop The x-variable data matrix. One row per observation. The y-variable data matrix. One row per observation. Initial projection vector for the x data set. Randomly set if NULL. Initial projection vector for the y data set. Randomly set if NULL. The bandwidth parameter for the Gaussian kernel on the x-variable set. A positive value. The smaller the smoother. The bandwidth parameter for the Gaussian kernel on the y-variable set. A positive value. The smaller the smoother. Maximum number of iterations. Convergence threshold. Algorithm stops when relative changes in cost from consecutive iterations is less than the threshold. Details Value Optimization is done by gradient descent, where Nelder-Mead is used for step-size selection. Nelder Mead may fail to increase the cost at times (when stuck at local minima). User may consider restarting the algorithm when this happens. A list containing: cost The canoncial projection vector for the x-variable set. The canoncial projection vector for the y-variable set. A vector of (negative) cost values at each iteration. Note Current implementation is slow and requires high storage for large sample data. Sample size > 2000 not recommended. Billy Chang References Chang et. al. (2013) Canonical Correlation Analysis based on Hilbert-Schmidt Independence Criterion and Centered Kernel Target Alignment. ICML Cortes et. al. (2012) Algorithms for learning kernels based on centered alignments. JMLR 13: See Also ktacca

9 sumwtdiff 9 Examples set.seed(10) numdata <- 100 numdim <- 2 x <- matrix(rnorm(numdata*numdim),numdata,numdim) y <- matrix(rnorm(numdata*numdim),numdata,numdim) z <- runif(numdata,-pi,pi) y[,1] <- cos(z)+rnorm(numdata,sd=0.1); x[,1] <- sin(z)+rnorm(numdata,sd=0.1) x <- scale(x) y <- scale(y) fit <- ktaccafunc(x,y,sigmax=1,sigmay=1) plot(x%*%fit$,y%*%fit$) sumwtdiff Sum of Weighted Pairwise Outer Differences. Usage Given weights matrix Wt, find sum of weighted pairwise outer product of differences, i.e. sum_i,j Wt_ij(x_i-x_j)(x_i-x_j)^T. Internal use only. sumwtdiff(wt, x) Arguments Wt x Weight matrix, nrow(x)-by-nrow(x) data matrix, one observation per row. Value the weighted sum of outer product of pairwise differences. Billy Chang

10 Index Topic package hsiccca-package, 2 hsiccca, 2, 5, 7 hsiccca-package, 2 hsicccafunc, 4, 4 ktacca, 4, 6, 8 ktaccafunc, 7, 7 sumwtdiff, 9 10