The R.huge Package. September 1, 2007

Transcription

1 The R.huge Package September 1, 2007 Version Date Title Methods for accessing huge amounts of data Author Henrik Bengtsson Maintainer Henrik Bengtsson Depends R (>= 2.0.0), R.oo, R.utils The package currently provides a class representing a matrix where the actual data is stored in a binary format on the local file system. This way the size limit of the data is set by the file system and not the memory. Please consider this package to in an alpha/early-beta version. License LGPL version 2.1 or newer (the releases) URL ContribURL DevelURL LazyLoad TRUE R topics documented: [.FileMatrix [.FileVector [< -.FileMatrix [< -.FileVector AbstractFileArray FileMatrix FileVector Non-documented objects R.huge-package as.character.abstractfilearray as.character.filematrix

2 2 R topics documented: as.matrix.filematrix as.vector.abstractfilearray clone.abstractfilearray close.abstractfilearray colnames.filematrix delete.abstractfilearray dim.abstractfilearray dimnames.abstractfilearray finalize.abstractfilearray flush.abstractfilearray getbasename.abstractfilearray getbyrow.filematrix getbytespercell.abstractfilearray getclonenumber.abstractfilearray getcomments.abstractfilearray getdataoffset.abstractfilearray getdimensionorder.abstractfilearray getextension.abstractfilearray getfilesize.abstractfilearray getname.abstractfilearray getpath.abstractfilearray getpathname.abstractfilearray getsizeofcomments.abstractfilearray getsizeofdata.abstractfilearray getstoragemode.abstractfilearray isopen.abstractfilearray length.abstractfilearray names.filevector ncol.filematrix nrow.filematrix open.abstractfilearray readalls.abstractfilearray readcontiguouss.abstractfilearray readheader.abstractfilearray reads.abstractfilearray rowmeans.filematrix rowsums.filematrix rownames.filematrix setcomments.abstractfilearray writealls.abstractfilearray writeemptydata.abstractfilearray writeheader.abstractfilearray writes.abstractfilearray Index 47

3 [.FileMatrix 3 [.FileMatrix Gets a subset of elements of a file matrix Gets a subset of elements of a file matrix. ## S3 method for class 'FileMatrix': this[i, j, drop=false] i j drop A numeric vector or row indices. A numeric vector or column indices. If TRUE, if only a single row or column is retrived, it is returned as a vector. Returns a matrix (or possibly a vector). *[<-(). For more information see FileMatrix. [.FileVector Gets a subset of elements of a file vector Gets a subset of elements of a file vector. ## S3 method for class 'FileVector': this[i=null] i A numeric vector or element indices.

4 4 [< -.FileMatrix Returns a vector. *[<-(). For more information see FileVector. [< -.FileMatrix Assigns values to a subset of elements of a file matrix Assigns values to a subset of elements of a file matrix. ## S3 replacement method for class 'FileMatrix': this[i, j] <- value i j value A numeric vector or row indices. A numeric vector or column indices. s to be assigned to the selected elements. Returns itself. *[(). For more information see FileMatrix.

5 [< -.FileVector 5 [< -.FileVector Assigns values to a subset of elements of a file vector Assigns values to a subset of elements of a file vector. ## S3 replacement method for class 'FileVector': this[i=null] <- value i value A numeric vector or element indices. s to be assigned to the selected elements. Returns itself. *[(). For more information see FileVector. AbstractFileArray Class representing a persistent array stored in a file Package: R.huge Class AbstractFileArray Object ~~ ~~+--AbstractFileArray Directly known subclasses: FileByteMatrix, FileByteVector, FileDoubleMatrix, FileDoubleVector, FileFloatMatrix, FileFloatVector, FileIntegerMatrix, FileIntegerVector, FileMatrix, FileShortMatrix, FileShortVector, FileVector

6 6 AbstractFileArray public static class AbstractFileArray extends Object Note that this is an abstract class, i.e. it is not possible to create an object of this class but only from one of its subclasses. For a vector data type, see FileVector. For a matrix data type, see FileMatrix. AbstractFileArray(filename=NULL, path=null, storagemode=c("integer", "double"), byt filename path storagemode The name of the file storing the data. An optional path where data should be stored. The storage mode() of the data elements. bytespercell The number of bytes each element (cell) takes up on the file system. If NULL, it is inferred from the storagemode argument. dim Details dimnames dimorder A numeric vector specifying the dimensions of the array. An optional list of dimension names. The order of the dimensions. comments An optional character string of arbitrary length. nbroffreebytes The number of "spare" bytes after the comments before the data section begins. The purpose of this class is to be able to work with large arrays in R without being limited by the amount of memory available. Data is kept on the file system and elements are read and written whenever queried. Fields and Methods Methods: as.character as.vector clone close delete dim dimnames finalize flush getbasename getbytespercell Returns a short string describing the file array. Returns the elements of a file array as an R vector. Clones a file array. Closes a connection to the data file of the file array. Deletes the file array from the file system. Gets the dimension of the file array. Gets the dimension names of a file array. Internal: Clean up when file array is deallocated from memory. Internal: Flushes the write buffer. Gets the basename (filename) of the data file. Gets the number of bytes per element in a file array.

7 AbstractFileArray 7 getclonenumber Gets the clone number of the file array. getcomments Gets the comments for this file array. getdataoffset Gets file position of the data section in a file array. getdimensionorder Gets the order of dimension. getextension Gets the filename extension of the file array. getfilesize Gets the size of the file array. getname Gets the name of the file array. getpath Gets the path (directory) where the data file lives. getpathname Gets the full pathname to the data file. getsizeofcomments Gets the number of bytes the comments occupies. getsizeofdata Gets the size of the data section in bytes. getstoragemode Gets the storage mode of the file array. isopen Checks whether the data file of the file array is open or not. length Gets the number of elements in a file array. open Opens a connection to the data file of the file array. readalls Reads all values in the file array. readcontiguouss Reads sets of contiguous values in the file array. readheader Read the header of a file array data file. reads Reads individual values in the file array. setcomments Sets the comments for this file array. writealls Writes all values to a file array. writeemptydata Writes an empty data section to the data file of a file array. writeheader Writes the header of a file array to file. writeheadercomments - writes Writes values to a file array. Methods inherited from Object:,<-, [[, [[<-, as.character, attach, attachlocally, clearcache, clone, detach, equals, extend, finalize, gc, getenvironment, getfields, getinstanciationtime, getstaticinstance, hasfield, hashcode, ll, load, objectsize, print, save Maximum number of elements It is only the header that is kept in memory, not the data, and therefore the maximum length of a array that can be allocate, is limited by the amount of available space on the file system. Since element names (optional) are stored in the header, these may also be a limiting factor. Element names The element names are stored in the header and are currently read and written to file one by one. This may slow down the performance substantially if the dimensions are large. For optimal opening performance, avoid names. For now, do not change names after file has been allocated. File format The file format consist of a header section and a data section. The header contains information about the file format, the length and element names of the array, as well as data type (storage mode()), the

8 8 FileMatrix size of each element. The data section, which follows immediately after the header section, consists of all data elements with non-assigned elements being pre-allocated with zeros. For more details, see the source code. Limitations The size of the array in bytes is limited by the maximum file size of the file system. For instance, the maximum file size on a Windows FAT32 system is 4GB (2GB?). On Windows NTFS the limit is in practice 16TB. References [1] New Technology File System (NTFS), Wikipedia, wiki/ntfs. FileMatrix Class representing a persistent matrix stored in a file Package: R.huge Class FileMatrix Object ~~ ~~+--AbstractFileArray ~~~~~~~ ~~~~~~~+--FileMatrix Directly known subclasses: FileByteMatrix, FileDoubleMatrix, FileFloatMatrix, FileIntegerMatrix, FileShortMatrix public static class FileMatrix extends AbstractFileArray FileMatrix(..., nrow=null, ncol=null, rownames=null, colnames=null, byrow=false)

9 FileMatrix 9 Details... passed to AbstractFileArray. nrow, ncol The number of rows and columns of the matrix. rownames, colnames Optional row and column names. byrow If TRUE, data are stored row by row, otherwise column by column. The purpose of this class is to be able to work with large matrices in R without being limited by the amount of memory available. Matrices are kept on the file system and elements are read and written whenever queried. The purpose of the class is not to provide methods for full matrix operations, but instead to be able to work with subsets of the matrix at each time. For more details, AbstractFileArray. Fields and Methods Methods: [ Gets a subset of elements of a file matrix. [<- Assigns values to a subset of elements of a file matrix. as.character Returns a short string describing the file matrix. as.matrix Returns the elements of a file matrix as an R matrix. colnames Gets the column names of a file matrix. getbyrow Checks if elements are stored row by row or not. getcolumnoffset - getmatrixindicies - getoffset - getrowoffset - ncol Gets the number of columns of the matrix. nrow Gets the number of rows of the matrix. readfullmatrix - reads - rowmeans Calculates the means for each row. rownames Gets the row names of a file matrix. rowsums Calculates the sum for each row. writes - Methods inherited from AbstractFileArray: as.character, as.vector, clone, close, delete, dim, dimnames, finalize, flush, getbasename, getbytes- PerCell, getclonenumber, getcomments, getdataoffset, getdimensionorder, getextension, get- FileSize, getname, getpath, getpathname, getsizeofcomments, getsizeofdata, getstoragemode, isopen, length, open, readalls, readcontiguouss, readheader, reads, setcomments, writealls, writeemptydata, writeheader, writeheadercomments, writes Methods inherited from Object:,<-, [[, [[<-, as.character, attach, attachlocally, clearcache, clone, detach, equals, extend, final-

10 10 FileMatrix ize, gc, getenvironment, getfields, getinstanciationtime, getstaticinstance, hasfield, hashcode, ll, load, objectsize, print, save Column by column or row by row? If the matrix elements are to be accessed more often along rows, store data row by row, otherwise column by column. Supported data types The following subclasses implement support for various data types: FileByteMatrix (1 byte per element), FileShortMatrix (2 bytes per element), FileIntegerMatrix (4 bytes per element), FileFloatMatrix (4 bytes per element), and FileDoubleMatrix (8 bytes per element). Examples verbose <- $getverbose(true); pathname <- "example.rmatrix"; if (isfile(pathname)) { file.remove(pathname); if (isfile(pathname)) { stop("file not deleted: ", pathname); } } # # Create a new file matrix # verbose && enter(verbose, "Creating new matrix"); # The dimensions of the matrix nrow <- 20; ncol <- 5; X <- FileByteMatrix(pathname, nrow=nrow, ncol=ncol, byrow=true); verbose && exit(verbose); verbose && enter(verbose, "Filling it with data"); rows <- c(1:4,7:10); cols <- c(1); x <- 1:length(rows); writes(x, rows=rows, cols=cols, values=x); verbose && exit(verbose);

11 FileMatrix 11 verbose && enter(verbose, "Getting data again"); y <- reads(x, rows=rows, cols=cols); verbose && exit(verbose); stopifnot(all.equal(x,y)); verbose && enter(verbose, "Setting data using [i,j]"); i <- c(20:18, 13:15); j <- c(3:2, 4:5); n <- length(i) * length(j); values <- 1:n; X[i,j] <- values; verbose && enter(verbose, "Validating"); print(x); print(x[]) print(x[i,j]); stopifnot(all.equal(as.vector(x[i,j]), values)) verbose && exit(verbose); verbose && exit(verbose); # # Open an already existing file matrix # verbose && enter(verbose, "Getting existing matrix"); Y <- FileByteMatrix(pathname); verbose && exit(verbose); print(y[]) Y[5,1] <- 55; print(y[]) print(x[]) # Note, X and Y refers to the same instance # # Clone a matrix # Z <- clone(x) Z[5,1] <- 66; print(z[]) print(y[]) # Remove clone again delete(z); # # Close all matrices # close(x); close(y); # Remove original matrix too delete(x); gc();

12 12 FileVector FileVector Class representing a persistent vector stored on file Package: R.huge Class FileVector Object ~~ ~~+--AbstractFileArray ~~~~~~~ ~~~~~~~+--FileVector Directly known subclasses: FileByteVector, FileDoubleVector, FileFloatVector, FileIntegerVector, FileShortVector public static class FileVector extends AbstractFileArray FileVector(..., length=null, names=null) Details... passed to AbstractFileArray. length names The number of elements in the vector. Optional element names. The purpose of this class is to be able to work with large vectors in R without being limited by the amount of memory available. Data is kept on the file system and elements are read and written whenever queried. For more details, AbstractFileArray. Fields and Methods Methods: [ Gets a subset of elements of a file vector. [<- Assigns values to a subset of elements of a file vector. names Gets the element names of a file vector.

13 FileVector 13 Methods inherited from AbstractFileArray: as.character, as.vector, clone, close, delete, dim, dimnames, finalize, flush, getbasename, getbytes- PerCell, getclonenumber, getcomments, getdataoffset, getdimensionorder, getextension, get- FileSize, getname, getpath, getpathname, getsizeofcomments, getsizeofdata, getstoragemode, isopen, length, open, readalls, readcontiguouss, readheader, reads, setcomments, writealls, writeemptydata, writeheader, writeheadercomments, writes Methods inherited from Object:,<-, [[, [[<-, as.character, attach, attachlocally, clearcache, clone, detach, equals, extend, finalize, gc, getenvironment, getfields, getinstanciationtime, getstaticinstance, hasfield, hashcode, ll, load, objectsize, print, save Supported data types The following subclasses implement support for various data types: FileByteVector (1 byte per element), FileShortVector (2 bytes per element), FileIntegerVector (4 bytes per element), FileFloatVector (4 bytes per element), and FileDoubleVector (8 bytes per element). Examples verbose <- $getverbose(true); pathname <- "example.rvector"; if (isfile(pathname)) { file.remove(pathname); if (isfile(pathname)) { stop("file not deleted: ", pathname); } } # # Create a new file vector # verbose && enter(verbose, "Creating new vector"); # The length of the vector length <- 1e6; X <- FileDoubleVector(pathname, length=length); verbose && exit(verbose); print(x); verbose && enter(verbose, "Filling it with data"); idxs <- c(1:4,7:10); x <- 1:length(idxs);

14 14 R.huge-package writes(x, indices=idxs, values=x); verbose && exit(verbose); verbose && enter(verbose, "Getting data again"); y <- reads(x, indices=idxs); verbose && exit(verbose); stopifnot(all.equal(x,y)); verbose && enter(verbose, "Getting and setting data using [i,j]"); print(x[1:20]) i <- 13:15; X[i] <- 99:98; print(x[1:20]) verbose && exit(verbose); delete(x); rm(x); gc(); Non-documented objects Non-documented objects This page contains aliases for all "non-documented" objects that R CMD check detects in this package. Almost all of them are generic functions that have specific document for the corresponding method coupled to a specific class. Other functions are re-defined by setmethods3() to default methods. Neither of these two classes are non-documented in reality. The rest are deprecated methods. R.huge-package Package R.huge The package currently provides a class representing a matrix where the actual data is stored in a binary format on the local file system. This way the size limit of the data is set by the file system and not the memory. Please consider this package to in an alpha/early-beta version. This package should be considered to be in "alpha" version.

15 as.character.abstractfilearray 15 Requirements This package requires the R.oo package [1] and the R.utils package. Installation and updates To install this package, see To get started To get started, see: 1. FileVector. 2. FileMatrix. How to cite this package Please cite [1] below. License The releases of this package is licensed under LGPL version 2.1 or newer. The development code of the packages is under a private licence (where applicable) and patches sent to the author fall under the latter license, but will be, if incorporated, released under the "release" license above. References 1 H. Bengtsson, The R.oo package - Object-Oriented Programming with References Using Standard R Code, In Kurt Hornik, Friedrich Leisch and Achim Zeileis, editors, Proceedings of the 3rd International Workshop on Distributed Statistical Computing (DSC 2003), March 20-22, Vienna, Austria. as.character.abstractfilearray Returns a short string describing the file array Returns a short string describing the file array.

16 16 as.character.filematrix as.character(x,...) Returns a character string. For more information see AbstractFileArray. as.character.filematrix Returns a short string describing the file matrix Returns a short string describing the file matrix. ## S3 method for class 'FileMatrix': as.character(x,...) Returns a character string. For more information see FileMatrix.

17 as.matrix.filematrix 17 as.matrix.filematrix Returns the elements of a file matrix as an R matrix Returns the elements of a file matrix as an R matrix, that is, imported into memory (if possible). ## S3 method for class 'FileMatrix': as.matrix(x,...) Returns a matrix. For more information see FileMatrix. as.vector.abstractfilearray Returns the elements of a file array as an R vector Returns the elements of a file array as an R vector, that is, imported into memory (if possible). as.vector(x,...)

18 18 clone.abstractfilearray Returns a vector. *as.vector(). For more information see AbstractFileArray. clone.abstractfilearray Clones a file array Clones a file array including the file on the file system. clone(con, copydata=true,...) copydata If TRUE, also the data file is copied, otherwise not. Returns the new AbstractFileArray object. For more information see AbstractFileArray.

19 close.abstractfilearray 19 close.abstractfilearray Closes a connection to the data file of the file array Closes a connection to the data file of the file array. close(con,...) Returns TRUE if the file was closed. If the file is not opened, an exception is thrown. *isopen(). *open(). For more information see AbstractFileArray. colnames.filematrix Gets the column names of a file matrix Gets the column names of a file matrix. ## S3 method for class 'FileMatrix': colnames(this,...)

20 20 delete.abstractfilearray Returns a character vector, or NULL. *dimnames(). *rownames(). For more information see FileMatrix. delete.abstractfilearray Deletes the file array from the file system Deletes the file array from the file system. If the file array is open, it is first closed. delete(this,...) Returns (invisibly) TRUE if the file was successfully deleted (or did not exist in the first place), otherwise FALSE. For more information see AbstractFileArray.

21 dim.abstractfilearray 21 dim.abstractfilearray Gets the dimension of the file array Gets the dimension of the file array. dim(x) Returns a double vector of length two. For more information see AbstractFileArray. dimnames.abstractfilearray Gets the dimension names of a file array Gets the dimension names of a file array. dimnames(x)

22 22 finalize.abstractfilearray Returns a character vector. For more information see AbstractFileArray. finalize.abstractfilearray Internal: Clean up when file array is deallocated from memory Internal: Clean up when file array is deallocated from memory. finalize(this,...) Details The finalizer of a file array makes sure to close the file connection, if it is open. Returns nothing. For more information see AbstractFileArray.

23 flush.abstractfilearray 23 flush.abstractfilearray Internal: Flushes the write buffer Internal: Flushes the write buffer. flush(con,...) Returns nothing. For more information see AbstractFileArray. getbasename.abstractfilearray Gets the basename (filename) of the data file Gets the basename (filename) of the data file. getbasename(this,...)

24 24 getbyrow.filematrix Returns a character string. *getpathname(). *getbasename(). For more information see AbstractFileArray. getbyrow.filematrix Checks if elements are stored row by row or not Checks if elements are stored row by row or not. ## S3 method for class 'FileMatrix': getbyrow(this,...) Returns TRUE if the elements are stored row by row, otherwise FALSE. For more information see FileMatrix.

25 getbytespercell.abstractfilearray 25 getbytespercell.abstractfilearray Gets the number of bytes per element in a file array Gets the number of bytes per element in a file array. getbytespercell(this,...) Details In R, an integer Returns an integer. For more information see AbstractFileArray. getclonenumber.abstractfilearray Gets the clone number of the file array Gets the clone number of the file array. getclonenumber(this,...)

26 26 getcomments.abstractfilearray Returns an integer. For more information see AbstractFileArray. getcomments.abstractfilearray Gets the comments for this file array Gets the comments for this file array. getcomments(this,...) Returns a character vector. *setcomments(). For more information see AbstractFileArray.

27 getdataoffset.abstractfilearray 27 getdataoffset.abstractfilearray Gets file position of the data section in a file array Gets file position of the data section in a file array. getdataoffset(this,...) Returns a double (since the largest integer is only ). For more information see AbstractFileArray. getdimensionorder.abstractfilearray Gets the order of dimension Gets the order of dimension. getdimensionorder(this,...)

28 28 getextension.abstractfilearray Returns an numeric vector. For more information see AbstractFileArray. getextension.abstractfilearray Gets the filename extension of the file array Gets the filename extension of the file array. getextension(this,...) Returns a character string. *getname(). For more information see AbstractFileArray.

29 getfilesize.abstractfilearray 29 getfilesize.abstractfilearray Gets the size of the file array Gets the size of the file array (in bytes). getfilesize(this,...) Returns the number of bytes as For more information see AbstractFileArray. getname.abstractfilearray Gets the name of the file array Gets the name of the file array from the filename of the data file. getname(this,...)

30 30 getpath.abstractfilearray Returns a character string. *getbasename(). For more information see AbstractFileArray. getpath.abstractfilearray Gets the path (directory) where the data file lives Gets the path (directory) where the data file lives. getpath(this,...) Returns a character string. *getpathname(). *getbasename(). For more information see AbstractFileArray.

31 getpathname.abstractfilearray 31 getpathname.abstractfilearray Gets the full pathname to the data file Gets the full pathname to the data file. getpathname(this,...) Returns a character string. *getpath(). *getbasename(). For more information see AbstractFileArray. getsizeofcomments.abstractfilearray Gets the number of bytes the comments occupies Gets the number of bytes the comments occupies. getsizeofcomments(this, comments=this$header$comments,...) comments The comments to be measured.

32 32 getsizeofdata.abstractfilearray Returns an integer. For more information see AbstractFileArray. getsizeofdata.abstractfilearray Gets the size of the data section in bytes Gets the size of the data section in bytes. getsizeofdata(this,...) Returns a double (since the largest integer is only ). *getbytespercell(). *length(). For more information see AbstractFileArray.

33 getstoragemode.abstractfilearray 33 getstoragemode.abstractfilearray Gets the storage mode of the file array Gets the storage mode of the file array. getstoragemode(this,...) Returns a character string. For more information see AbstractFileArray. isopen.abstractfilearray Checks whether the data file of the file array is open or not Checks whether the data file of the file array is open or not. isopen(this,...)

34 34 length.abstractfilearray Returns nothing. *open(). *close(). For more information see AbstractFileArray. length.abstractfilearray Gets the number of elements in a file array Gets the number of elements in a file array. length(x) Returns a double (since the largest integer is only ). *dim(). For more information see AbstractFileArray.

35 names.filevector 35 names.filevector Gets the element names of a file vector Gets the element names of a file vector. ## S3 method for class 'FileVector': names(x,...) Returns a character vector. For more information see FileVector. ncol.filematrix Gets the number of columns of the matrix Gets the number of columns of the matrix. ## S3 method for class 'FileMatrix': ncol(this,...) Returns a double.

36 36 nrow.filematrix *dim(). For more information see FileMatrix. nrow.filematrix Gets the number of rows of the matrix Gets the number of rows of the matrix. ## S3 method for class 'FileMatrix': nrow(this,...) Returns a double. *dim(). For more information see FileMatrix.

37 open.abstractfilearray 37 open.abstractfilearray Opens a connection to the data file of the file array Opens a connection to the data file of the file array. open(con,...) Returns (invisibly) itself. *isopen(). *close(). For more information see AbstractFileArray. readalls.abstractfilearray Reads all values in the file array Reads all values in the file array. readalls(this, mode=getstoragemode(this), size=getbytespercell(this), offset=g

38 38 readcontiguouss.abstractfilearray mode The storage mode to read. size The number of bytes each values allocates on file. offset The file offset to the first value on file..checkargs If TRUE, arguments are validated, otherwise not. Returns a numeric vector. *readseqsofs() and reads(). For more information see AbstractFileArray. readcontiguouss.abstractfilearray Reads sets of contiguous values in the file array Reads sets of contiguous values in the file array. A set of contiguous values are values that are connecting without a break. It is much faster to read contiguous sequences at once than read each value separately. readcontiguouss(this, indices, lengths=1, mode=getstoragemode(this), size=getb indices length mode size offset.checkargs A numeric vector of start (first) indices in each of the contiguous sets. A numeric vector specifying the length of each of the contiguous sets. The storage mode to read. The number of bytes each values allocates on file. The file offset to the first value on file. If TRUE, arguments are validated, otherwise not.

39 readheader.abstractfilearray 39 Returns a numeric vector. *readalls() and reads(). For more information see AbstractFileArray. readheader.abstractfilearray Read the header of a file array data file Read the header of a file array data file. readheader(this,...) Returns list structure. *writeheader(). For more information see AbstractFileArray.

40 40 reads.abstractfilearray reads.abstractfilearray Reads individual values in the file array Reads individual values in the file array. reads(this, indices=null, mode=getstoragemode(this), size=getbytespercell(this indices mode size offset A numeric vector of indices. The storage mode to read. The number of bytes each values allocates on file. The file offset to the first value on file..checkargs If TRUE, arguments are validated, otherwise not. Returns a numeric vector. *readcontiguouss() and reads(). For more information see AbstractFileArray.

41 rowmeans.filematrix 41 rowmeans.filematrix Calculates the means for each row Calculates the means for each row. ## S3 method for class 'FileMatrix': rowmeans(x,..., docount=false)... passed to *rowsums(). docount If TRUE, the number of included values are counted and returned as an attribute. Returns a numeric vector. *rowsums(). For more information see FileMatrix. rowsums.filematrix Calculates the sum for each row Calculates the sum for each row. ## S3 method for class 'FileMatrix': rowsums(x, na.rm=false, docount=false, rows=null, columns=null,...)

42 42 rownames.filematrix na.rm docount rows columns Returns a numeric vector. If TRUE, NA values are excluded. If TRUE, the number of included values are counted and returned as an attribute. An integer vector of rows for which the sum should be calculated. If NULL, all rows are considered. An integer vector of columns for which the sum should be calculated. If NULL, all columns are considered. *rowmeans(). For more information see FileMatrix. rownames.filematrix Gets the row names of a file matrix Gets the row names of a file matrix. ## S3 method for class 'FileMatrix': rownames(this,...) Returns a character vector, or NULL.

43 setcomments.abstractfilearray 43 *dimnames(). *colnames(). For more information see FileMatrix. setcomments.abstractfilearray Sets the comments for this file array Sets the comments for this file array. setcomments(this, comments=null,...) comments Returns (invisibly) the old comments. A character vector of new comments. *getcomments(). For more information see AbstractFileArray. writealls.abstractfilearray Writes all values to a file array Writes all values to a file array. writealls(this, values, mode=getstoragemode(this), size=getbytespercell(this),

44 44 writeemptydata.abstractfilearray values A numeric vector of values to be assigned. mode The storage mode to be used. size A integer specifying the number of bytes per cell. offset A integer specifying the file offset of the data section. Returns nothing. *writes(). For more information see AbstractFileArray. writeemptydata.abstractfilearray Writes an empty data section to the data file of a file array Writes an empty data section to the data file of a file array. writeemptydata(this,...) Returns nothing. For more information see AbstractFileArray.

45 writeheader.abstractfilearray 45 writeheader.abstractfilearray Writes the header of a file array to file Writes the header of a file array to file. writeheader(this,...) Returns nothing. *writeheader(). For more information see AbstractFileArray. writes.abstractfilearray Writes values to a file array Writes values to a file array. writes(this, indices=null, values, mode=getstoragemode(this), size=getbytesper

46 46 writes.abstractfilearray indices values mode size offset order An integer vector of indices to be updated. If NULL, all cells are considered. A numeric vector of values to be assigned. The storage mode to be used. A integer specifying the number of bytes per cell. A integer specifying the file offset of the data section. If TRUE, the data is reordered before being written.... Additional arguments passed to *writealls(). Returns nothing. *writealls(). For more information see AbstractFileArray.

47 Index Topic IO [.FileMatrix, 2 [.FileVector, 2 [< -.FileMatrix, 3 [< -.FileVector, 4 as.character.abstractfilearray, 14 as.character.filematrix, 15 as.matrix.filematrix, 16 as.vector.abstractfilearray, 16 clone.abstractfilearray, 17 close.abstractfilearray, 18 colnames.filematrix, 18 delete.abstractfilearray, 19 dim.abstractfilearray, 20 dimnames.abstractfilearray, 20 finalize.abstractfilearray, 21 flush.abstractfilearray, 22 getbasename.abstractfilearray, 22 getbyrow.filematrix, 23 getbytespercell.abstractfilearray, 24 getclonenumber.abstractfilearray, 24 getcomments.abstractfilearray, 25 getdataoffset.abstractfilearray, 26 getdimensionorder.abstractfilearray, 26 getextension.abstractfilearray, 27 getfilesize.abstractfilearray, 28 getname.abstractfilearray, getpath.abstractfilearray, 29 getpathname.abstractfilearray, 30 getsizeofcomments.abstractfilearray, 30 getsizeofdata.abstractfilearray, 31 getstoragemode.abstractfilearray, 32 isopen.abstractfilearray, 32 length.abstractfilearray, 33 names.filevector, 34 ncol.filematrix, 34 nrow.filematrix, 35 open.abstractfilearray, 36 readalls.abstractfilearray, 36 readcontiguouss.abstractfilearray, 37 readheader.abstractfilearray, 38 reads.abstractfilearray, 39 rowmeans.filematrix, 40 rownames.filematrix, 41 rowsums.filematrix, 40 setcomments.abstractfilearray, 42 writealls.abstractfilearray, 42 writeemptydata.abstractfilearray, 43 writeheader.abstractfilearray, 44 writes.abstractfilearray, 44 Topic classes AbstractFileArray, 4 FileMatrix, 7

48 48 INDEX FileVector, 11 Topic documentation Non-documented objects, 13 Topic internal [.FileMatrix, 2 [.FileVector, 2 [< -.FileMatrix, 3 [< -.FileVector, 4 as.character.abstractfilearray, 14 as.character.filematrix, 15 as.matrix.filematrix, 16 as.vector.abstractfilearray, 16 clone.abstractfilearray, 17 close.abstractfilearray, 18 colnames.filematrix, 18 delete.abstractfilearray, 19 dim.abstractfilearray, 20 dimnames.abstractfilearray, 20 finalize.abstractfilearray, 21 flush.abstractfilearray, 22 getbasename.abstractfilearray, 22 getbyrow.filematrix, 23 getbytespercell.abstractfilearray, 24 getclonenumber.abstractfilearray, 24 getcomments.abstractfilearray, 25 getdataoffset.abstractfilearray, 26 getdimensionorder.abstractfilearray, 26 getextension.abstractfilearray, 27 getfilesize.abstractfilearray, 28 getname.abstractfilearray, 28 getpath.abstractfilearray, 29 getpathname.abstractfilearray, 30 getsizeofcomments.abstractfilearray, 30 getsizeofdata.abstractfilearray, 31 getstoragemode.abstractfilearray, 32 isopen.abstractfilearray, 32 length.abstractfilearray, 33 names.filevector, 34 ncol.filematrix, 34 Non-documented objects, 13 nrow.filematrix, 35 open.abstractfilearray, 36 readalls.abstractfilearray, 36 readcontiguouss.abstractfilearray, 37 readheader.abstractfilearray, 38 reads.abstractfilearray, 39 rowmeans.filematrix, 40 rownames.filematrix, 41 rowsums.filematrix, 40 setcomments.abstractfilearray, 42 writealls.abstractfilearray, 42 writeemptydata.abstractfilearray, 43 writeheader.abstractfilearray, 44 writes.abstractfilearray, 44 Topic methods [.FileMatrix, 2 [.FileVector, 2 [< -.FileMatrix, 3 [< -.FileVector, 4 as.character.abstractfilearray, 14 as.character.filematrix, 15 as.matrix.filematrix, 16 as.vector.abstractfilearray, 16 clone.abstractfilearray, 17 close.abstractfilearray, 18 colnames.filematrix, 18 delete.abstractfilearray, 19 dim.abstractfilearray, 20 dimnames.abstractfilearray,

49 INDEX finalize.abstractfilearray, 21 flush.abstractfilearray, 22 getbasename.abstractfilearray, 22 getbyrow.filematrix, 23 getbytespercell.abstractfilearray, 24 getclonenumber.abstractfilearray, 24 getcomments.abstractfilearray, 25 getdataoffset.abstractfilearray, 26 getdimensionorder.abstractfilearray, 26 getextension.abstractfilearray, 27 getfilesize.abstractfilearray, 28 getname.abstractfilearray, 28 getpath.abstractfilearray, 29 getpathname.abstractfilearray, 30 getsizeofcomments.abstractfilearray, 30 getsizeofdata.abstractfilearray, 31 getstoragemode.abstractfilearray, 32 isopen.abstractfilearray, 32 length.abstractfilearray, 33 names.filevector, 34 ncol.filematrix, 34 nrow.filematrix, 35 open.abstractfilearray, 36 readalls.abstractfilearray, 36 readcontiguouss.abstractfilearray, 37 readheader.abstractfilearray, 38 reads.abstractfilearray, 39 rowmeans.filematrix, 40 rownames.filematrix, 41 rowsums.filematrix, 40 setcomments.abstractfilearray, 42 writealls.abstractfilearray, 42 writeemptydata.abstractfilearray, 43 writeheader.abstractfilearray, 44 writes.abstractfilearray, 44 Topic package R.huge-package, 13 Topic programming [.FileMatrix, 2 [.FileVector, 2 [< -.FileMatrix, 3 [< -.FileVector, 4 as.character.abstractfilearray, 14 as.character.filematrix, 15 as.matrix.filematrix, 16 as.vector.abstractfilearray, 16 clone.abstractfilearray, 17 close.abstractfilearray, 18 colnames.filematrix, 18 delete.abstractfilearray, 19 dim.abstractfilearray, 20 dimnames.abstractfilearray, 20 finalize.abstractfilearray, 21 flush.abstractfilearray, 22 getbasename.abstractfilearray, 22 getbyrow.filematrix, 23 getbytespercell.abstractfilearray, 24 getclonenumber.abstractfilearray, 24 getcomments.abstractfilearray, 25 getdataoffset.abstractfilearray, 26 getdimensionorder.abstractfilearray, 26 getextension.abstractfilearray, 27

50 50 INDEX getfilesize.abstractfilearray, 28 getname.abstractfilearray, 28 getpath.abstractfilearray, 29 getpathname.abstractfilearray, 30 getsizeofcomments.abstractfilearray, 30 getsizeofdata.abstractfilearray, 31 getstoragemode.abstractfilearray, 32 isopen.abstractfilearray, 32 length.abstractfilearray, 33 names.filevector, 34 ncol.filematrix, 34 nrow.filematrix, 35 open.abstractfilearray, 36 readalls.abstractfilearray, 36 readcontiguouss.abstractfilearray, 37 readheader.abstractfilearray, 38 reads.abstractfilearray, 39 rowmeans.filematrix, 40 rownames.filematrix, 41 rowsums.filematrix, 40 setcomments.abstractfilearray, 42 writealls.abstractfilearray, 42 writeemptydata.abstractfilearray, 43 writeheader.abstractfilearray, 44 writes.abstractfilearray, 44 *[, 3, 4 *[<-, 2, 3 *as.vector, 17 *close, 33, 36 *colnames, 42 *dim, 33, 35 *dimnames, 19, 42 *getbasename, 23, 29, 30 *getbytespercell, 31 *getcomments, 42 *getname, 27 *getpath, 30 *getpathname, 23, 29 *isopen, 18, 36 *length, 31 *open, 18, 33 *readalls, 38 *readcontiguouss, 39 *readseqsofs, 37 *rowmeans, 41 *rowsums, 40 *rownames, 19 *setcomments, 25 *writealls, 45 *writeheader, 38, 44 *writes, 43 [, 8, 11 [,FileMatrix-method ([.FileMatrix), 2 [,FileVector-method ([.FileVector), 2 [.FileMatrix, 2 [.FileVector, 2 [< -.FileMatrix, 3 [< -.FileVector, 4 [<-, 8, 11 [<-,FileMatrix-method ([< -.FileMatrix), 3 [<-,FileVector-method ([< -.FileVector), 4 [<-.FileMatrix ([< -.FileMatrix), 3 [<-.FileVector ([< -.FileVector), 4 AbstractFileArray, 4, 7, 8, 11, 15, 17 33, 36 39, AbstractFileArray.as.character (as.character.abstractfilearray), 14 AbstractFileArray.as.vector (as.vector.abstractfilearray), 16 AbstractFileArray.clone (clone.abstractfilearray), 17 AbstractFileArray.close (close.abstractfilearray),

51 INDEX AbstractFileArray.delete (delete.abstractfilearray), 19 AbstractFileArray.dim (dim.abstractfilearray), 20 AbstractFileArray.dimnames (dimnames.abstractfilearray), 20 AbstractFileArray.finalize (finalize.abstractfilearray), 21 AbstractFileArray.flush (flush.abstractfilearray), 22 AbstractFileArray.getBasename (getbasename.abstractfilearray), 22 AbstractFileArray.getBytesPerCell (getbytespercell.abstractfilearray), 24 AbstractFileArray.getCloneNumber (getclonenumber.abstractfilearray), 24 AbstractFileArray.getComments (getcomments.abstractfilearray), 25 AbstractFileArray.getDataOffset (getdataoffset.abstractfilearray), 26 AbstractFileArray.getDimensionOrder (getdimensionorder.abstractfilearray), 26 AbstractFileArray.getExtension (getextension.abstractfilearray), 27 AbstractFileArray.getFileSize (getfilesize.abstractfilearray), 28 AbstractFileArray.getName (getname.abstractfilearray), 28 AbstractFileArray.getPath (getpath.abstractfilearray), 29 AbstractFileArray.getPathname (getpathname.abstractfilearray), 30 AbstractFileArray.getSizeOfComments (getsizeofcomments.abstractfilearray), 30 AbstractFileArray.getSizeOfData (getsizeofdata.abstractfilearray), 31 AbstractFileArray.getStorageMode (getstoragemode.abstractfilearray), 32 AbstractFileArray.isOpen (isopen.abstractfilearray), 32 AbstractFileArray.length (length.abstractfilearray), 33 AbstractFileArray.open (open.abstractfilearray), 36 AbstractFileArray.readAlls (readalls.abstractfilearray), 36 AbstractFileArray.readContiguouss (readcontiguouss.abstractfilearray), 37 AbstractFileArray.readHeader (readheader.abstractfilearray), 38 AbstractFileArray.reads (reads.abstractfilearray), 39 AbstractFileArray.setComments (setcomments.abstractfilearray), 42 AbstractFileArray.writeAlls (writealls.abstractfilearray), 42 AbstractFileArray.writeEmptyData (writeemptydata.abstractfilearray), 43 AbstractFileArray.writeHeader (writeheader.abstractfilearray), 44 AbstractFileArray.writes (writes.abstractfilearray), 44 as.character, 5, 8 as.character,abstractfilearray-method (as.character.abstractfilearray),

52 52 INDEX 14 as.character,filematrix-method (as.character.filematrix), 15 as.character.abstractfilearray, 14 as.character.filematrix, 15 as.matrix, 8 as.matrix (Non-documented objects), 13 as.matrix,filematrix-method (as.matrix.filematrix), 16 as.matrix.filematrix, 16 as.vector, 5 as.vector (Non-documented objects), 13 as.vector,abstractfilearray-method (as.vector.abstractfilearray), 16 as.vector.abstractfilearray, 16 character, 5, 15, 19, 21, 23, 25, 27, 29, 30, 32, 34, 41, 42 clone, 5 clone,abstractfilearray-method (clone.abstractfilearray), 17 clone.abstractfilearray, 17 close, 5 close (Non-documented objects), 13 close,abstractfilearray-method (close.abstractfilearray), 18 close.abstractfilearray, 18 colnames, 8 colnames (Non-documented objects), 13 colnames,filematrix-method (colnames.filematrix), 18 colnames.filematrix, 18 delete, 5 delete (Non-documented objects), 13 delete,abstractfilearray-method (delete.abstractfilearray), 19 delete.abstractfilearray, 19 dim, 5 dim,abstractfilearray-method (dim.abstractfilearray), 20 dim.abstractfilearray, 20 dimnames, 5 dimnames,abstractfilearray-method (dimnames.abstractfilearray), 20 dimnames.abstractfilearray, 20 double, 20, 26, 31, FALSE, 19, 23 FileByteMatrix, 4, 7 FileByteMatrix (FileMatrix), 7 FileByteVector, 4, 11 FileByteVector (FileVector), 11 FileDoubleMatrix, 4, 7 FileDoubleMatrix (FileMatrix), 7 FileDoubleVector, 4, 11 FileDoubleVector (FileVector), 11 FileFloatMatrix, 4, 7 FileFloatMatrix (FileMatrix), 7 FileFloatVector, 4, 11 FileFloatVector (FileVector), 11 FileIntegerMatrix, 4, 7 FileIntegerMatrix (FileMatrix), 7 FileIntegerVector, 4, 11 FileIntegerVector (FileVector), 11 FileMatrix, 2 5, 7, 14 16, 19, 23, 35, FileMatrix.[ ([.FileMatrix), 2 FileMatrix.[<- ([< -.FileMatrix), 3 FileMatrix.as.character (as.character.filematrix), 15 FileMatrix.as.matrix (as.matrix.filematrix), 16 FileMatrix.colnames (colnames.filematrix), 18 FileMatrix.getByRow (getbyrow.filematrix), 23 FileMatrix.ncol (ncol.filematrix), 34 FileMatrix.nrow (nrow.filematrix), 35 FileMatrix.rowMeans (rowmeans.filematrix), 40 FileMatrix.rownames (rownames.filematrix), 41

53 INDEX 53 FileMatrix.rowSums (rowsums.filematrix), 40 FileShortMatrix, 4, 7 FileShortMatrix (FileMatrix), 7 FileShortVector, 4, 11 FileShortVector (FileVector), 11 FileVector, 3 5, 11, 14, 34 FileVector.[ ([.FileVector), 2 FileVector.[<- ([< -.FileVector), 4 FileVector.names (names.filevector), 34 finalize, 5 finalize,abstractfilearray-method (finalize.abstractfilearray), 21 finalize.abstractfilearray, 21 flush, 5 flush (Non-documented objects), 13 flush,abstractfilearray-method (flush.abstractfilearray), 22 flush.abstractfilearray, 22 getbasename, 5 getbasename (Non-documented objects), 13 getbasename,abstractfilearray-method (getbasename.abstractfilearray), 22 getbasename.abstractfilearray, 22 getbyrow, 8 getbyrow (Non-documented objects), 13 getbyrow,filematrix-method (getbyrow.filematrix), 23 getbyrow.filematrix, 23 getbytespercell, 5 getbytespercell (Non-documented objects), 13 getbytespercell,abstractfilearray-method getclonenumber,abstractfilearray-method (getclonenumber.abstractfilearray), 24 getclonenumber.abstractfilearray, 24 getcolumnoffset (Non-documented objects), 13 getcomments, 6 getcomments (Non-documented objects), 13 getcomments,abstractfilearray-method (getcomments.abstractfilearray), 25 getcomments.abstractfilearray, 25 getdataoffset, 6 getdataoffset (Non-documented objects), 13 getdataoffset,abstractfilearray-method (getdataoffset.abstractfilearray), 26 getdataoffset.abstractfilearray, 26 getdimensionorder, 6 getdimensionorder (Non-documented objects), 13 getdimensionorder,abstractfilearray-method (getdimensionorder.abstractfilearray), 26 getdimensionorder.abstractfilearray, 26 getextension, 6 getextension (Non-documented objects), 13 getextension,abstractfilearray-method (getextension.abstractfilearray), 27 getextension.abstractfilearray, 27 getfilesize, 6 getfilesize (Non-documented (getbytespercell.abstractfilearray), objects), getbytespercell.abstractfilearray, 24 getclonenumber, 6 getclonenumber (Non-documented objects), 13 getfilesize,abstractfilearray-method (getfilesize.abstractfilearray), 28 getfilesize.abstractfilearray, 28 getmatrixindicies (Non-documented objects),

54 54 INDEX 13 getname, 6 getname,abstractfilearray-method (getname.abstractfilearray), 28 getname.abstractfilearray, 28 getoffset (Non-documented objects), 13 getpath, 6 getpath,abstractfilearray-method (getpath.abstractfilearray), 29 getpath.abstractfilearray, 29 getpathname, 6 getpathname (Non-documented objects), 13 getpathname,abstractfilearray-method (getpathname.abstractfilearray), 30 getpathname.abstractfilearray, 30 getrowoffset (Non-documented objects), 13 getsizeofcomments, 6 getsizeofcomments (Non-documented objects), 13 getsizeofcomments,abstractfilearray-method (ncol.filematrix), 34 ncol.filematrix, 34 (getsizeofcomments.abstractfilearray), Non-documented objects, nrow, 8 getsizeofcomments.abstractfilearray, nrow (Non-documented objects), nrow,filematrix-method getsizeofdata, 6 (nrow.filematrix), 35 getsizeofdata (Non-documented nrow.filematrix, 35 objects), 13 NULL, 19, 41, 45 getsizeofdata,abstractfilearray-method numeric, 2 5, 27, 37 41, 43, 45 (getsizeofdata.abstractfilearray), 31 getsizeofdata.abstractfilearray, 31 getstoragemode, 6 getstoragemode (Non-documented objects), 13 getstoragemode,abstractfilearray-method 36 open.abstractfilearray, 36 (getstoragemode.abstractfilearray), 32 getstoragemode.abstractfilearray, 32 integer, 24 26, 31, 33, 41, 43, 45 isopen, 6 isopen (Non-documented objects), 13 isopen,abstractfilearray-method (isopen.abstractfilearray), 32 isopen.abstractfilearray, 32 length, 6 length,abstractfilearray-method (length.abstractfilearray), 33 length.abstractfilearray, 33 list, 5, 38 matrix, 2, 16 mode, 5, 6 NA, 41 names, 11 names,filevector-method (names.filevector), 34 names.filevector, 34 ncol, 8 ncol (Non-documented objects), 13 ncol,filematrix-method Object, 4, 5, 7, 11 open, 6 open (Non-documented objects), 13 open,abstractfilearray-method (open.abstractfilearray), R.huge (R.huge-package), 13 R.huge-package, 13 readalls, 6 readalls (Non-documented objects), 13

55 INDEX 55 readalls,abstractfilearray-method (rowsums.filematrix), 40 (readalls.abstractfilearray), rowsums.filematrix, readalls.abstractfilearray, setcomments, 6 36 setcomments (Non-documented readbinfragments (Non-documented objects), 13 objects), 13 setcomments,abstractfilearray-method readcontiguouss, 6 (setcomments.abstractfilearray), readcontiguouss 42 (Non-documented objects), setcomments.abstractfilearray, readcontiguouss,abstractfilearray-method TRUE, 2, 8, 17 19, 23, 37, 39 41, 45 (readcontiguouss.abstractfilearray), 37 readcontiguouss.abstractfilearray, 37 readfullmatrix (Non-documented objects), 13 readheader, 6 readheader (Non-documented objects), 13 readheader,abstractfilearray-method (readheader.abstractfilearray), 38 readheader.abstractfilearray, 38 reads, 6, reads (Non-documented objects), 13 reads,abstractfilearray-method (reads.abstractfilearray), 39 reads.abstractfilearray, 39 rowmeans, 8 rowmeans (Non-documented objects), 13 rowmeans,filematrix-method (rowmeans.filematrix), 40 rowmeans.filematrix, 40 rownames, 8 rownames (Non-documented objects), 13 rownames,filematrix-method (rownames.filematrix), 41 rownames.filematrix, 41 rowsums, 8 rowsums (Non-documented objects), 13 rowsums,filematrix-method vector, 2 5, 17, 19 21, 25, 27, 34, 37 43, 45 writealls, 6 writealls (Non-documented objects), 13 writealls,abstractfilearray-method (writealls.abstractfilearray), 42 writealls.abstractfilearray, 42 writebinfragments (Non-documented objects), 13 writeemptydata, 6 writeemptydata (Non-documented objects), 13 writeemptydata,abstractfilearray-method (writeemptydata.abstractfilearray), 43 writeemptydata.abstractfilearray, 43 writeheader, 6 writeheader (Non-documented objects), 13 writeheader,abstractfilearray-method (writeheader.abstractfilearray), 44 writeheader.abstractfilearray, 44 writeheadercomments (Non-documented objects), 13 writes, 6 writes (Non-documented objects), 13