Examples, examples: Outline Overview of todays exercises Basic scripting Importing data Working with temporal data Working with missing data Interpolation in 1D Some time series analysis Linear regression Getting started Preparing data Data analysis
Exercise Overview Today we are going to be doing lots of exercises. Today's dataset consists of 2001 hourly buoy measurements from a buoy near Bodega Bay (#46013): http://www.ndbc.noaa.gov/maps/monterey_bay.shtml Today we will: Import the data Learn how matlab deals with dates and times Deal with missing data Fill in missing data Do some autocorrelations of wind and air temperature Do a linear fit between wind and air temperature
Basic Scripting Matlab commands can be stored in a text file with a.m extension and then run using the filename. Change the working directory to your student directory (z:\home\wfcb\wfcb??\). Use the edit command to being editing a text file.» edit Put some commands in the file and save it as myfile.m: a = 1:10; plot( a, a.^2 ) Run your commands by either hitting F5 or typing from the command line:» myfile Get in the habit of saving commands in m files.
Importing data The data for this class is located in: z:\pkg\wfcb\class3\46013h2001.txt Take a look at data (use edit). There are many ways to get data into matlab. One popular way is through the Excel toolbox. Many other commands: load, importdata, textread, imread (try: help fileformats) We have used load. Drawback: Requires rigid format. Another more flexible command is importdata. More flexible tries to load data no matter what Works on lots of different formats (text, images, HDF,...) Exercise: Use importdata to load todays dataset. Next slide explains the result of loading data.
Structures The result of loading a text file with importdata is often a structure. Many functions return structures. Structures are like bags of variables. If you have loaded the data into a variable a, then the column names would be accessed using:» a.colheaders Structures are a convenient way to keep together related variables. Add an element to a structure like so:» a.buoy_num = 46013 Take a look at the elements of the result of importdata. Which element contains the numerical data?
Temporal data The first four columns of the data are year, month, day and hour Matlab has its own way of dealing with dates and times Numbers days from Year 0, January 1 at 00:00! Noon on January 1, 2000 is 730486.5 to matlab. Matlab has functions to convert to and from matlab format: datenum: converts year, month,... to matlab date numbers datevec: converts numbers to individual parts datestr: converts numbers to human readable dates datetick: converts numeric ticks on a plot to dates Exercises: Put dataset dates in matlab format (datenum) Oops dateset dates are in GMT (subtract 8 hours for PST) Plot PST date vs. air temp (ATMP) and fix x axis labels (datetick)
Initial plot of air temperature Looks pretty bad air temperatures of 1000ºC are still unusual. Why does it look this way? Try scaling y axis limits to something more reasonable (ylim)
Missing Data The problem in the last plot is missing data In this dataset, 99, 999 and 9999 all indicate missing data. Exercises: Create a new variable of data with all missing data replaced by NaN. Use logical operations (last class) to do this. == is equivalent is logical or Replot data
Plot of air temperature with NaNs Try zooming in on the second half of March. Hint: Use datetick('keeplimits') to create date ticks after zoom. Do you see any data gaps? What dates?
Interpolation in 1D We want to fill in all that missing data using linear interpolation. Exercise: Use the interp1 function to fill in the missing data for air temperature and water temperature (WTMP). Hints: You will need to create new variables with the results. Use isnan to remove bad data (remember ~ is logical not). Use interp1 to get values at all points. Where data already exists, it will return the same value; elsewhere, it will interpolate.
Replot data Exercises: Plot air temperature and water temperature on a figure with different line colors. Put a black circle around all data points that were interpolated (i.e. originally missing data). Hint: use isnan and look at line styles in help plot. Don't forget to hold on to your first plot. Zoom in on second half of March again Can you figure out how to fill in the dots? Don't worry if you can't do this.
Time Series Analysis: Autocorrelations Now we want to look first at how water and air temperatures vary in time through autocorrelations. Autocorrelations are a measure of how well data at one point in time reflects data at a future or past point in time. It is basically a lagged correlation. Exercises: Compute the autocorrelations for air and water temp (hint: use the xcov and set the scaling option to 'coeff'). Plot both autocorrelation functions on the same graph. Label x and y axes. Remember that lags are in hours. Zoom in to a sensible time period (i.e. a few days). What does the slower decline of one of the autocorrelations means? What do the oscillations in the curves mean?
Plot of autocorrelations Slower decline of water temp autocorrelation means that it is more stable. Oscillations every 24 hours mean that temps have a diurnal cycle, but air temp fluctuates more.
Linear Regression Exercise: Make a scatter plot of air temperature against water temperature using the plot function. Clearly there is a relationship. Matlab can do basic linear regression with polyfit (more robust fits need Stats Toolbox or code available online). Exercises: Use polyfit to regress air temp vs. water temp. Add a red line to your scatter plot with the linear fit using the polyval function. Can you plot the residuals of the fit?
Next Class For next class, we will continue working with this dataset. Also, think about datasets and analyses that you could bring into the class and we could work on. Must be general enough that everyone can get involved. Must not be too complex for us to work through it in a class. I will choose one or two to work on.