Introduction to idistance Data Structures.

Transcription

1 Introduction to idistance Data Structures. idistance Workshop CREEM, University of St Andrews March 18, 2016 Contents 1 Introduction 2 2 Load the Package 3 3 Data Assessment Checking the data Components to the dsdata class object Summarising the Data Plotting Plotting the data Plotting with additional aesthetics Mapping Changing the coordinate system Detection Data 15 6 Spatial Covariate Data Creating a covdata object How to create the dsdata object? make.dsdata function Conversion from dsm style data to dsdata class Conversion of MRSea data to dsdata class Datasets within the idistance package The mrsea data The weeds data The Eastern Tropical Pacific (ETP) data A toy dataset; toy

2 1 Introduction In this tutorial you will look at the two data structures, dsdata and covdata, which are integral to fitting models using idistance. 2

3 2 Load the Package require(idistance) What datasets are there in the idistance package? data(package='idistance') Table 1: Table of data sets within the idistance package Dataset Description Name in Package Covariate Name ETP Blue Whales ETP Striped Dolphin ETP Short beaked Dolphin Eastern Tropical Pacific Data. Boat Based line transect whales strdolphin sbdolphin Marine Renewable data Data from the MRSea package mrsea mrsea.depth Weeds Data from the DSpat package weeds - Toy dataset single transect with detections toy1 - Details of each of the main example data sets with this package can be found in section 8. sst 3 Data Assessment idistance stores line and point transect data in R objects of the class dsdata. A useful example to understand this data structure is the toy1 data set included in the package. We can load the data set and learn about its class via data(toy1) class(toy1) ## [1] "dsdata" "list" This data set contains one transect with some detections about the line. 3.1 Checking the data In order to obtain more information about the dsdata class as well as other data sets included in idistance simply check out the documentation by calling?dsdata. In the following we will introduce the most important functions needed to inspect and create dsdata objects. First, let us check that the data is in the correct format for idistance sanity(toy1) ## Checking if data set has all the necessary fields... [OK] ## Checking if $effort has all the necessary columns... [OK] ## Checking if transects are sorted... [OK] 3

4 ## Checking if all segments are inside the mesh boundary... [OK] ## Checking if all detections are inside the mesh boundary... [OK] ## Checking if any distances are NA or NaN... [OK] ## Checking if $effort has distance column... [OK] 3.2 Components to the dsdata class object str(toy1, max.level=1) ## List of 4 ## $ geometry : chr "euc" ## $ effort :Classes 'effort' and 'data.frame': 175 obs. of 13 variables: ## $ mesh :List of 7 ##..- attr(*, "class")= chr "inla.mesh" ## $ mesh.coords: chr [1:2] "x" "y" ## - attr(*, "class")= chr [1:2] "dsdata" "list" The survey area and the respective triangulation are described by an inla.mesh object stored in the field named mesh. Constructing a suitable mesh for a given survey is an quite extensive subject and more details will be provided later in the course. Data about the surveys transects as well as the detected species is stored in the field effort. The format that this table has to be in is very specific and is explained in the help file (?make.dsdata). The other two fields are quite straight forward to set. The geometry field defines the manifold that the survey data lives on, e.g. the Euclidean plane. Lastly, mesh.coords holds an array of characters that identify the names of the spatial coordinates used throughout your data. Once we have these data fields at hand a dsdata object can be created using the constructor make.dsdata (see section??). head(toy1$effort) ## strat trans seg det x y distance obs.x ## <NA> NA NA NA NA ## ## ## ## ## ## obs.y start.x start.y end.x end.y ## 175 NA ## ## ## ## ## head(na.omit(toy1$effort)) 4

5 ## strat trans seg det x y distance obs.x ## ## ## ## ## ## ## obs.y start.x start.y end.x end.y ## ## ## ## ## ## Summarising the Data idistance provides a summary method that serves some meta information about the object. The statistics function provides information about the total number and length of all transects and segments, the number of detections and the area of the mesh. statistics(toy1) ## Number of transects: N_tr = 1 ## Total length of transects: L = units ## Number of segments: N_s = 1 ## Total length of segments: L_s = units ## Number of detections N = 174 ## Total area of mesh A_m = square mesh units The summary function provides the output of the statistics function as well as information about the data fields. summary(toy1) ## # data fields # ## Geometry: euc ## Coordinate names: x y ## Effort columns names: ## strat trans seg det x y distance obs.x obs.y start.x start.y end.x end.y ## # statistics # ## Number of transects: N_tr = 1 ## Total length of transects: L = units ## Number of segments: N_s = 1 ## Total length of segments: L_s = units ## Number of detections N = 174 ## Total area of mesh A_m = square mesh units 5

6 4 Plotting The toy1 data set is reasonably uninteresting for plotting so we use a more interesting data set for this section. One of the datasets, mrsea, is based in a region of offshore windfarm development. Aerial surveys were conducted across the region both before and after construction, with the intention of discovering whether there was any impact of the windfarm construction on the local diving bird population. For more details about this data set, see section 8.1. data(mrsea) summary(mrsea) ## # data fields # ## Geometry: euc ## Coordinate names: x y ## Effort columns names: ## trans Transect.label season impact depth segment.label length Effort mid.x mid.y s ## # statistics # ## Number of transects: N_tr = 208 ## Total length of transects: L = units ## Number of segments: N_s = 9232 ## Total length of segments: L_s = units ## Number of detections N = 2373 ## Total area of mesh A_m = square mesh units 4.1 Plotting the data There are a number of ways to plot the various aspects of the data and all use the ggplot2 setup. First, the most useful function is plot which, when given a dsdata object, will plot all main aspects; the segments, the detections, the mesh, the inner boundary of the mesh and the outer boundary of the mesh. plot(mrsea) 6

7 We can also plot the individual components separately, or build up a plot of our own choosing: 1. Segments ggplot() + gg.seg(mrsea) 2. Detections ggplot() + gg.det(mrsea) 7

8 3. The mesh ggplot() + gg.mesh(mrsea) 4. The outer and inner boundary of the mesh 8

9 ggplot() + gg.bnd(mrsea) + gg.int(mrsea) 5. The area of effort (swath) ggplot() + gg.swath(mrsea, width=500, alpha=0.1, linetype='blank') 6. Putting it all together and adding some personalisation ggplot() + gg.int(mrsea) + gg.swath(mrsea, width=500, alpha=0.05, linetype='blank') + gg.det(mrsea, col='blue', alpha=0.3) + gg.seg(mrsea, size=0.3) 9

10 4.2 Plotting with additional aesthetics You can also use the plot function and then add some aesthetics to one or more of the components. For example, using another dataset that contains group size, you can change the point size for the detections to represent group size. data(whales) plot(whales) + gg.det(whales, aes(size=grpsize)) 4.3 Mapping If the data set has a proj.4 string attached you can plot the data onto googlemaps

11 In their own words: proj.4 is a standard Unix filter function which converts geographic longitude and latitude coordinates into Cartesian coordinates (and vice versa), and it is a C API for software developers to include coordinate transformation in their own software. Make sense? Thought not! Essentially, for idistance, you need a string that gives relevant information about the projection/coordinate system you are working in. The mrsea data structure has two additional list entries, p4s and mesh.p4s, which have been manually added to the dsdata object. These show that the data are in UTMs (Universal Transverse Mercator) and gives the UTM zone. str(mrsea, max.level=1) ## List of 6 ## $ effort :Classes 'effort' and 'data.frame': obs. of 23 variables: ## $ geometry : chr "euc" ## $ mesh.coords: chr [1:2] "x" "y" ## $ mesh :List of 7 ##..- attr(*, "class")= chr "inla.mesh" ## $ p4s : chr "+proj=utm +zone=32" ## $ mesh.p4s : chr "+proj=utm +zone=32" ## - attr(*, "class")= chr [1:2] "dsdata" "list" If you don t already have the rgdal package, install it now as is required for this mapping exercise. Note that the default conversion of the remap function is to latitude and longitude. library(ggmap) # requires the 'rgdal' library dset = remap(mrsea) # Map to lon/lat, which is the default of remap() mymap <- get_map(c(11.8, 54.55), zoom = 10, source="google", maptype="satellite", crop=false) ggmap(mymap) + gg.seg(dset, col='grey') + gg.det(dset, alpha=0.3, size=0.3) + gg.int(dset, col='black') 11

12 Using the ggplot2 faceting function, you can create a lattice of plots by any covariate that is available in the effort data. p<-ggmap(mymap) + gg.seg(dset, col='grey') + gg.det(dset, alpha=0.3, size=0.3) + gg.int(dset, col='black') p + facet_wrap(~impact) Figure 1: Map showing the location of the mrsea detections before (left) and after (right) the construction of a wind farm. 12

13 p + facet_wrap(~season) 13

14 Figure 2: Map showing the location of the mrsea detections across the four seasons of the survey. 14

15 4.4 Changing the coordinate system Take the mrsea effort table and turn all of the coordinates/distances in to kilometres rather than meters. We can use the remap() function to do this for us. mrsea.km<-remap(mrsea, p4s = "+proj=utm +zone=32 +units=km", mesh.p4s = "+proj=utm +zone=32 +uni Plot as before, but the data coming in was in kilometres rather than meters. plot(mrsea.km) 5 Detection Data In order to have a look at the detection data alone, there is a function to subset the effort data accordingly: Returning briefly to the toy dataset: head(detdata(toy1)) ## strat trans seg det x y distance obs.x ## ## ## ## ## ## ## obs.y start.x start.y end.x end.y ## ## ##

16 ## ## ## Using this function, we can plot a histogram of detection distances: hist(detdata(toy1)$distance, main='histogram of Detection Distances', xlab='distance') Alternatively, you may use the effort data directly: hist(toy1$effort$distance) Note that there is an equivalent of detdata() for the segments; segdata() segdata(toy1)[1:2,] ## strat trans seg det x y distance obs.x obs.y start.x start.y ## <NA> NA NA NA NA NA -1 1 ## NA <NA> <NA> <NA> <NA> NA NA NA NA NA NA NA ## end.x end.y ## ## NA NA NA 6 Spatial Covariate Data The dsdata object does not contain any information relating to spatial covariates. The covariate data is in an object of class covdata. This contains the value of the covariate at each node on 16

17 the mesh. data(mrsea.depth) class(mrsea.depth) ## [1] "covdata" "list" str(mrsea.depth, max.level = 1) ## List of 5 ## $ mesh :List of 7 ##..- attr(*, "class")= chr "inla.mesh" ## $ values :'data.frame': 329 obs. of 1 variable: ## $ mesh.coords: chr [1:2] "x" "y" ## $ time.coords: NULL ## $ mesh.p4s : chr "+proj=utm +zone=32" ## - attr(*, "class")= chr [1:2] "covdata" "list" The plot function works on this data class too. plot(mrsea.depth) + gg.bnd(mrsea.depth) + gg.int(mrsea.depth) All values outside of the inner mesh boundary have been interpolated across the mesh. 6.1 Creating a covdata object The data in the MRSea package comes with a data frame to make predictions. This gives the depth covariate across a grid and we shall turn this into a covdata object 17

18 require(mrsea) data(predict.data.re) preddata<-prednamechange(predict.data.re) This data frame consists of one grid repeated a number of times across some covariates. For depth just one instance of the grid is required so subset the prediction data first. s preddata<-preddata[1:length(unique(preddata$segment.id)),] Now use the covdata.import function to create the covdata class object. The column name of the covariate of interest must be specified here. depth.covdata<- covdata.import(dframe = preddata, colname = "depth", data=mrsea) ## Note: method with signature CsparseMatrix#Matrix#missing#replValue chosen for function [<-, ## target signature dgcmatrix#ngcmatrix#missing#numeric. ## "Matrix#nsparseMatrix#missing#replValue" would also be valid class(depth.covdata) ## [1] "covdata" "list" # plot(depth.covdata) 7 How to create the dsdata object? 7.1 make.dsdata function If you have the effort data available, in the format specified in the help file, you can use the function make.dsdata() to create the dsdata class object. This function automatically creates a mesh, unless specified otherwise, and does the sanity check on the data. dset<-make.dsdata(effort=mrsea$effort) ## make.dsdata(): Checking if the effort table has all the necessary columns... [OK] ## make.dsdata(): You did not provide a geometry. However, your effort table has 'x' and 'y' col ## make.dsdata(): You did not provide mesh.coords but geometry is set to 'euc'. Will assume defa ## make.dsdata(): You did not provide a mesh. A default mesh will be constructed. ## make.dsdata(): Done with constructing the data set. Running sanity checks... ## Checking if data set has all the necessary fields... [OK] ## Checking if $effort has all the necessary columns... [OK] ## Checking if transects are sorted... [OK] ## Checking if all segments are inside the mesh boundary... [OK] ## Checking if all detections are inside the mesh boundary... [OK] ## Checking if any distances are NA or NaN... [OK] ## Checking if $effort has distance column... [OK] 18

19 This is the equivalent of specifying the following: dset<-make.dsdata(effort=mrsea$effort, mesh = NULL, geometry = NULL, mesh.coords = NULL, mesh.args = NULL) If you have details about the mesh or the geometry then you can add them here. You will learn about making a customised mesh later in the course. 7.2 Conversion from dsm style data to dsdata class If you are familiar with the data format used in the dsm package then you can use the following code to convert that style of data to the dsdata class. For this example we will use the Mexican dolphin dataset from within the dsm package. require(dsm) data("mexdolphins") Have a look at the structure of the data set. It is a list object containing four data frames, a spatial polygon of predictions and a spatial polygon data frame of the survey area. str(mexdolphins, max.level=1) ## List of 6 ## $ distdata :'data.frame': 47 obs. of 10 variables: ## $ obsdata :'data.frame': 47 obs. of 5 variables: ## $ pred.polys :Formal class 'SpatialPolygons' [package "sp"] with 4 slots ## $ preddata :'data.frame': 1374 obs. of 6 variables: ## $ segdata :'data.frame': 387 obs. of 8 variables: ## $ survey.area:formal class 'SpatialPolygonsDataFrame' [package "sp"] with 5 slots mexdolphins$segdata[1:2,] ## longitude latitude x y Effort Transect.Label Sample.Label ## ## ## depth ## ## mexdolphins$obsdata[1:2,] ## object Sample.Label size distance Effort ## ## mexdolphins$distdata[1:2,] 19

20 ## object size distance Effort detected beaufort latitude longitude ## ## ## x y ## ## mexdolphins$preddata[1:2,] ## latitude longitude x y depth area ## ## dset = import.dsmdata(mexdolphins, covar.col=8) ## Loading required package: splancs ## ## Spatial Point Pattern Analysis Code in S-Plus ## ## Version 2 - Spatial and Space-Time analysis ## Warning in inla.nonconvex.hull(loc, convex = -0.01): Resolution (40,40) too small for convex/concave radius ( , ). ## Resolution >=(105,52) required for more accurate results. ## make.dsdata(): Checking if the effort table has all the necessary columns... [OK] ## make.dsdata(): You did not provide a geometry. However, your effort table has 'x' and 'y' col ## make.dsdata(): You did not provide mesh.coords but geometry is set to 'euc'. Will assume defa ## make.dsdata(): Done with constructing the data set. Running sanity checks... ## Checking if data set has all the necessary fields... [OK] ## Checking if $effort has all the necessary columns... [OK] ## Checking if transects are sorted... [OK] ## Checking if all segments are inside the mesh boundary... [OK] ## Checking if all detections are inside the mesh boundary... [OK] ## Checking if any distances are NA or NaN... [OK] ## Checking if $effort has distance column... [OK] summary(dset) ## # data fields # ## Geometry: euc ## Coordinate names: x y ## Effort columns names: ## mid.x mid.y Effort trans depth seg quadrant.r angle.r start.x start.y end.x end.y ## # statistics # ## Number of transects: N_tr = 45 ## Total length of transects: L = units ## Number of segments: N_s =

21 ## Total length of segments: L_s = units ## Number of detections N = 47 ## Total area of mesh A_m = e+12 square mesh units plot(dset) y e+00 5e+05 1e+06 x 7.3 Conversion of MRSea data to dsdata class This is a slightly more complicated affair. First, the data is converted to the dsm data format (list of data frames), and then converted to the idistance format. 21

22 require(mrsea) data("dis.data.re") data("predict.data.re") MRSea style to dsm style First change the names of the relevant columns to be in-line with the requirement for the import.dsm function. disdata<-mrseanames2dsmnames(dis.data.re) 1. Make the segment data require(dplyr) segdata<-disdata[,c("transect.label", "Transect.label", "Sample.Label", "segment.label", "length", "Effort", 'x', 'y', "season", "impact", "depth")] segdata<- distinct(segdata, Sample.Label) 2. Make the detection data # effort, object and distance. # Not taken x and y as these are segement mid points not detection locations distdata<-makedistdata(disdata) 3. Make the observation data obsdata<-makeobsdata(disdata) 4. Update coordinate names in the prediction data preddata <- predict.data.re preddata<-prednamechange(preddata) Create idistance data dsmdata = list(obsdata = obsdata, distdata = distdata, segdata = segdata, preddata = preddata) dset = import.dsmdata(dsmdata, covar.col=11) 22

23 ## Warning in inla.nonconvex.hull(loc, convex = -0.01): Resolution (40,40) too small for convex/concave radius (516.26,516.26). ## Resolution >=(105,58) required for more accurate results. ## make.dsdata(): Checking if the effort table has all the necessary columns... [OK] ## make.dsdata(): You did not provide a geometry. However, your effort table has 'x' and 'y' col ## make.dsdata(): You did not provide mesh.coords but geometry is set to 'euc'. Will assume defa ## make.dsdata(): Done with constructing the data set. Running sanity checks... ## Checking if data set has all the necessary fields... [OK] ## Checking if $effort has all the necessary columns... [OK] ## Checking if transects are sorted... [OK] ## Checking if all segments are inside the mesh boundary... [OK] ## Checking if all detections are inside the mesh boundary... [OK] ## Checking if any distances are NA or NaN... [OK] ## Checking if $effort has distance column... [OK] plot(dset) 8 Datasets within the idistance package 8.1 The mrsea data The data for this example are from the MRSea package developed for analysing data collected in the marine renewables sector. If you do not have the package you can find it on the CREEM software webpage (it is not on CRAN), The data are aerial surveys conducted over a region, off the coast of Denmark, where a windfarm was to be constructed. The data here are simulated from a model fitted to the original data with a known impact on animal numbers and distribution added. In this instance, there is a baseline distribution of animals (impact=0) and a post impact distribution (impact=1). The 23

24 animal numbers remain roughly the same but there is a re-distribution of animals away from the construction region to elsewhere. The impact region is roughly in the middle as seen in Figure 1. data(mrsea) plot(mrsea) 8.2 The weeds data data(weeds) plot(weeds) 24

25 plot(weeds) + gg.det(weeds, aes(group=as.factor(seen), colour=as.factor(seen))) + scale_color_discrete(name='seen') plot(weeds) + facet_wrap(~seen) 8.3 The Eastern Tropical Pacific (ETP) data There are data sets for three species from the ETP surveys included in the package: 1. Blue whales; whales 25

26 data("whales") plot(whales) 2. Short beaked dolphins; sbdolphin data("sbdolphin") plot(sbdolphin) 3. Striped dolphins; strdolphin 26

27 data("strdolphin") plot(strdolphin) 8.4 A toy dataset; toy1 data(toy1) plot(toy1) 27