Probability An Example

Transcription

1 Probability An Example For example, suppose we have a data set where in six cities, we count the number of malls located in that city present: Each count of the # of malls in a city is an event # of Malls City Outcome #1 Outcome #2 Outcome #3 Outcome #4 Sample Space We might wonder if we randomly pick one of these six cities, what is the chance that it will have n malls?

2 Discrete Random Variables The random variable from our example is a discrete random variable, because it has a finite number of values (i.e. a city can have 1 or 2 malls, but it cannot have 1.5 malls) Any variable that is generated by counting a whole number of things is likely to be a discrete variable (e.g. # of coin tosses in a row with heads, questionnaire responses where one of a set of ordinal categories must be chosen, etc.) A discrete random variable can be described by a probability mass function

3 Probability Mass Functions Probability mass functions have the following rules that dictate their possible values: 1. The probability of any outcome must be greater than or equal to zero and must also be less than or equal to one, i.e. 0 P(x i ) 1 for i = {1, 2, 3,, k-1, k} 2. The sum of all probabilities in the sample space must total one, i.e. i=k Σ P(x i ) = 1 i=1

4 Continuous Random Variables Continuous random variables can assume all real number values within an interval, for example: measurements of precipitation, ph, etc. Some random variables that are technically discrete exhibit such a tremendous range of values, that is it desirable to treat them as if they were continuous variables, e.g. population Discrete random variables are described by probability mass functions, and continuous random variables are described by probability density functions

5 Probability Density Functions Probability density functions are defined using the same rules required of probability mass functions, with some additional requirements: 1. The function must have a non-negative value throughout the interval a to b, i.e. f(x) 0 for a x b 2. The area under the curve defined by f(x), within the interval a to b, must equal 1: f(x) a area=1 b x

6 Probability Density Functions Suppose we are interested in computing the probability of a continuous random variable falling within a range of values bounded by lower limit c and upper limit d, within the interval a to b How can we find the probability of a value occurring between c and d? We need to calculate the shaded area if we know the density function, we could use calculus: P(x) c d = d f(x) c f(x) dx a c x d b

7 Simple Descriptive Statistics These are ways to summarize a number set quickly and accurately The most common way of describing a variable distribution is in terms of two of its properties: Central tendency describes the central value of the distribution, around which the observations cluster Dispersion describes how the observations are distributed First we ll look at measures of central tendency

8 Measures of Central Tendency - Mean 3. Mean cont. A standard geographic application of the mean is to locate the center (a.k.a. centroid) of a spatial distribution by assigning to each member of the spatial distribution a gridded coordinate and calculating the mean value in each coordinate direction Bivariate mean or mean center For a set of (x,y) coordinates, the mean center (x,y) is computed using: x = i=n Σ x i i=1 n y = i=n Σ y i i=1 n

9 Measures of Dispersion Variance, Standard Deviation, Z-scores 2. Variance etc. cont. Standard deviation is calculated by taking the square root of variance: σ = i=n Σ (x i µ) 2 i=1 N Population standard deviation S = i=n Σi=1 (x i x) 2 n - 1 Sample standard deviation Why do we prefer standard deviation over variance as a measure of dispersion? Magnitude of values and units match means

10 Discrete Probability Distributions The first part of today s lecture will introduce three kinds of discrete probability distributions that are useful for us to examine: 1. The Uniform Distribution 2. The Binomial Distribution 3. The Poisson Distribution Each of these probability distributions is appropriately applied in certain situations to help us understand particular geographic phenomena

11 The Uniform Distribution While the idea of a uniform distribution might seem a little simplistic and perhaps useless, it actually is well applied in two situations: 1. When the probabilities are of each and every possible outcome are truly equal (e.g. the coin toss) 2. When we have no prior knowledge of how a variable is distributed (i.e. when we are dealing with complete uncertainty), we first distribution we should use is uniform, because it makes no assumptions about the distribution

12 The Binomial Distribution The binomial distribution provides information about the probability of the repetition of events when there are only two possible outcomes, e.g. heads or tails, left or right, success or failure, rain or no rain any nominal data situation where there are only two categories / outcomes possible The binomial distribution is useful for describing when the same event is repeated over and over, characterizing the probability of a proportion of the events having a certain outcome over a specified number of events

13 The Poisson Distribution Here, the counts of points per quadrat form the frequencies we use to check Poisson probabilities: Regular Low variance Mean 1 σ 2 :x is low Random Clustered Variance Low variance Mean Mean 0 σ 2 :x ~ 1 σ 2 :x is high

14 The Normal Distribution Most naturally occurring variables are distributed normally (e.g. heights, weights, shoe sizes, annual temperature variations, test scores, IQ scores, etc.) For example, if we take the land surface temperature values for Maryland Climate Division 6 on May 24,2002 from the LST GRID we are using in lab #6 and generate a histogram of these values, we can see that the values are basically normally distributed:

15 LST Distribution Mean = degrees F Standard Deviation = degrees F

16 Simple Linear Regression A means that we can use to characterize a probabilistic relationship like the one we saw in the previous slide is using simple linear regression, a linear model with the following characteristics: y (dependent) a error: ε b x (independent) y = a + bx + ε x is the independent variable y is the dependent variable b is the slope of the fitted line a is the intercept of the fitted line ε is the error term

17 Least Squares Method The least squares method operates mathematically, minimizing the error term ε for all points We can describe the line of best fit we will find using the equation ŷ = a + bx, and you ll recall that from a previous slide that the formula for our linear model was expressed using y = a + bx + ε y ŷ We use the value ŷ on the line to estimate the true value, y (y - ŷ) The difference between the two is (y - ŷ) ŷ = a + bx This difference is positive for points above the line, and negative for points below it

18 The Strength of Relationships Source: Earickson, RJ, and Harlin, JM Geographic Measurement and Quantitative Analysis. USA: Macmillan College Publishing Co., p. 209.

19 Coefficient of Determination (R 2 ) The regression sum of squares (SSR) expresses the improvement made in estimating y by using the regression line: n y ŷ y SSR = Σ (ŷ i -y) 2 i = 1 The total sum of squares (SST) expresses the overall variation between the values of y and their mean y: n SST = Σ (y i -y) 2 i = 1 The coefficient of determination (R 2 ) expresses the amount of variation in y explained by the regression line (the strength of the relationship): R 2 = SSR SST

20 Definition of Spatial Analysis A method of analysis is spatial if the results depend on the locations of the objects being analyzed i.e. if you move the objects and the results change, or the results are not invariant under relocation, spatial analysis is being applied To conduct a spatial analysis requires both attributes and locations of objects Conveniently, GIS has been designed to store both we usually assemble geographic information in a GIS so that we might analyze it

21 Spatial Relationships are at the Core of Spatial Analysis Most spatial analyses are, when reduced to their simplest components, based on a few kinds of fundamental questions: How near is Feature A to Feature B What features contain other features? What features are adjacent to other features? What features are connected to other features? From these topological building blocks, we can develop all sorts of spatial analysis approaches to answer many complex questions

22 Types of Spatial Analysis There are literally thousands of spatial analysis techniques, with new ones developed all the time We will consider six categories of spatial analyses, each having a distinct conceptual basis: 1. Queries and reasoning 2. Measurements 3. Transformations 4. Descriptive summaries 5. Optimization 6. Hypothesis testing Chapter 13 Chapter 14

23 1. Queries and Reasoning A basic function of GIS is the ability to query data layers Queries can be attribute-based (e.g. show me all the pixels in a GRID with an LST value > 80 degrees) or location-based (e.g. find all the counties in Maryland Climate Division 6 that are adjacent to Baltimore County) A GIS can respond to queries by presenting data in appropriate documents (e.g. a View and/or a Table) It is often useful to be able to display two or more documents at once If we have performed a query that selects features from a theme based on some criteria, we can see the selected features in both a View and a Table

24 2. Measurements Many tasks require measurement from maps The measurement of a distance between two points The measurement of area, e.g. the area of a parcel of land (represented as a polygon feature) Such measurements are tedious and inaccurate if made by hand On the other hand, measurement using GIS tools and digital databases is fast, reliable, and accurate

25 3. Transformations The category encompassing transformations of spatial data includes many analytical approaches that can be applied using either the vector or raster spatial data models, or combining both together Transformations create new attributes and objects, based on some simple rules: They involve geometric construction or calculation They may also create new fields, either from existing fields or from discrete objects

26 Buffering (Proximity Analysis) Buffering: The delineation of a zone around the feature of interest within a given distance. For a point feature, it is simply a circle with its radius equal to the buffer distance:

27 Feature in Feature Transformations These transformations determine whether a feature lies inside or outside another feature The most basic of these transformations is point in polygon analysis, which can be applied in various situations: The application is usually one of generalization: Assign many points to containing polygons For example, this is used to assign crimes to police precincts, voters to voting districts, accidents to reporting counties, etc.

28 Point in Polygon Analysis Overlay point layer (A) with polygon layer (B) In which B polygon are A points located?» Assign polygon attributes from B to points in A Example: Comparing soil mineral content at sample borehole locations (points) with land use (polygons)... A B

29 Line in Polygon Analysis Overlay line layer (A) with polygon layer (B) Example: Assign land use attributes (polygons) to streams (lines): In which B polygons are A lines located?» Assign polygon attributes from B to lines in A A B

30 Polygon Overlay Analysis UNION overlay polygons and keep areas from both layers INTERSECTION overlay polygons and keep only areas in the input layer that fall within the intersection layer IDENTITY overlay polygons and keep areas from input layer

31 Overlay of Fields Represented as Rasters A B The two input data sets are maps of (A) travel time from the urban area shown in black, and (B) county (red indicates County X, white indicates County Y). The output map identifies travel time to areas in County Y only, and might be used to compute average travel time to points in that county in a subsequent step

32 Spatial Interpolation Often, we have a geographic phenomenon that we wish to represent using a field (e.g. elevation), but the values of that field have been measured at sample points There is a need to estimate the complete field from the discrete samples, in order to estimate values at points where the field was not measured create a contour map by drawing isolines between the data points, or a raster digital elevation model which has a value for every cell Methods of spatial interpolation are designed to solve this problem

33 Inverse Distance Weighting (IDW) i point i known value z i location x i weight w i distance d i unknown value (to be interpolated) location x z (x) = w izi w The estimate is a i i weighted average w = i 2 1 d i Weights decline with distance

34 A semivariogram: Each cross represents a pair of points. The solid circles are obtained by averaging within the ranges or bins of the distance axis. The solid line represents the best fit to these five points, using one of a small number of standard mathematical functions.

35 Neighborhood Operations In raster overlay analysis, we compared each cell in a raster layer with another cell in the same position on another layer In neighborhood operations, we look at a neighborhood of cells around the cell of interest to arrive at a new value: Cell of Interest A 3x3 neighborhood An input layer Neighborhoods can be of any possible size; we can use a 3x3 neighborhood for any cell except on the edge of the layer

36 The Mean Operation Revisited 1/ 9 1/ 9 1/ 9 1/ 9 1/ 9 1/ 9 1/ 9 1/ 9 1/ 9 In the mean operation, each cell in the neighborhood is used in the same way: Input Layer Result Layer 3 4 5

37 Kernel Function Example The result of applying a 150km-wide kernel to points distributed over California A typical kernel function

38 4. Descriptive Summaries Descriptive summaries attempt to summarize useful properties of data sets in one or two statistics We have already dealt with this branch of analysis fairly extensively in the previous section of this course The mean or average is widely used to summarize data centers are the spatial equivalent of means, applied in the two directions there are several ways of defining centers

39 Point Pattern Analysis Standard distance is just one of many ways to summarize the pattern found in a set of points: Clustered Regular Random Point patterns can be characterized by the distance between neighboring points. If we define d i as the distance between a point and its nearest neighbor, the average distance between neighboring points can be written as: n D Σ d A = i = 1 i n David Tenenbaum GEOG 070 UNC-CH Spring 2005

40 The Nearest Neighbor Index We can calculate the expected distance (D E ) between randomly distributed points using: 1 D E 2 A n = where A is the area and n is the # of points We can determine the degree to which a set of points is randomly distributed by comparing the actual distance between the points (D A ) with the expected distance (D E ), taking the ratio between the two, known as the nearest neighbor index (NNI): NNI = Random points: D A ~ D E, ΝΝΙ 1 Clustered points: D A ~ 0, ΝΝΙ 0 Scattered points: D A larger up to max. ΝΝΙ = D D A E

41 Frag. Stats. Example To the left are three images of part of the state of Rondonia in Brazil, collected in 1975, 1986, and 1992 Note the increasing fragmentation of the natural habitat as a result of settlement (forest canopy appears deep red, locations of development appear cyan blue) Such fragmentation can adversely affect the success of wildlife populations, and its extent can be assessed using fragmentation statistics

42 5. Optimization Spatial analysis can be used to solve many problems of design, such as where is the best place to build a new x The decision as to where to build a new facility is often approached from the point of view of maximizing access, or minimizing travel time from a certain catchment or service area, e.g. if we identify a developing area where the nearest hospital is an unacceptably long drive away, we may know we want to locate a hospital in that area but where should we put it to best serve the residents in the area and minimize overall travel time for the area? To do, we can identify the point of minimum aggregate travel (MAT)

43 Uncertainty, Error, and Sensitivity Geographic databases are collections of observations or measurements of phenomena They can include position and attribute information, and relationships Almost all observations and measurements are subject to error and uncertainty, and these depend upon: The accuracy of the measuring instrument The care taken by the observer when conducting the observation The clarity of the relevant definitions that determine how the measurement is performed and many other factors too

44 Example Confusion Matrix A grand total of 304 parcels have been checked in this confusion matrix. The rows of the table correspond to the land use class of each parcel as recorded in the database (observations), and the columns to the class as recorded in the field (reference). The numbers appearing on the principal diagonal of the table (from top left to bottom right) reflect correct classifications Observations A B C D E Total A Reference B C D E Total

45 Precision and Accuracy These related concepts are often confused: Precision refers to the exactness associated with a measurement (i.e. closely clustered) Accuracy refers to the extent of systematic bias in the measurement process (i.e. centered on the middle) x x x x x x x x x x x x x x x x x x x x Precise & Accurate Precise & Inaccurate Imprecise & Accurate Imprecise & Inaccurate

46 A Positional Uncertainty Map Plot of the 350m contour for the State College, Pennsylvania, U.S.A. topographic quadrangle. The contour has been interpolated from the U.S. Geological Survey's digital elevation model for this area Uncertainty in the location of the 350m contour based on an assumed RMSE of 7m. The Gaussian distribution with a mean of 350 m and a standard deviation of 7 m gives a 95% probability that the true location of the 350 m contour lies in the colored area, and a 5% probability that it lies outside.

47 Areal Error in a Square For example, suppose we measure a square area (100m sides) by surveying the position of each of its corners The errors for each corner are subject to bivariate Gaussian distributions with standard deviations in x and y of 1 m (dashed circles) The red polygon shows one possible surveyed square (one realization of the error model) In this case, the measurement of area is subject to a standard deviation of 200 m 2 ; a result such as 10, is quite likely, though the true area is 10,000 m 2. In principle, the result of 10, should be rounded to the known accuracy and reported as as 10,000

48 D8 Analysis Sequence Assume we now have a raster DEM and we want to use it find a watershed and drainage network through D8 analysis We can follow this sequence of analysis steps, each of which involves a neighborhood analysis operation: Fill Sinks Slope Aspect Flow Direction Flow Accumulation StreamLink & StreamOrder Watershed D8 Analysis

49 Fill Sinks We need a DEM that does not have any depressions or pits in it for D8 drainage network analysis The first step is to remove all pits from our DEM using a pitfilling algorithm This illustration shows a DEM of Morgan Creek, west of Chapel Hill

50 Flow Direction and Accumulation Slope and aspect are needed to produce flow direction, which assigns each cell a direction of steepest descent Flow accumulation uses flow direction to find the number of cells that drain to each cell Taking the log of accumulation makes the pattern much easier to see

51 Stream Links, Order, and Basins By selecting a threshold value for flow accumulation, we can produce a stream network This network can divided into stream links, which can in turn be assigned stream order values using network analysis methods Threshold=1 gives the watershed