Homework 2; Down load instructions and data from Web Can load.dbf in to ArcMap and use Select by Attributes

Transcription

1 Homework 2; Down load instructions and data from Web Can load.dbf in to ArcMap and use Select by Attributes Submit via Moodle using HomeWork2 Answer Sheet Quiz

2 Homework on Website

3

4 Homework 2; Down load instructions and data from Web Can load.dbf in to ArcMap and use Select by Attributes Submit via Moodle using HomeWork2 Answer Sheet Quiz

5

6

7 Homework 2 After downloading zip file, unzip and add the.dbf to and ArcMap project. Then Open the table and use Select by Attributes to query the table answering the questions.

8

9 Note: also on the Website are extra credit options Extra Lab - LabX (practice digitizing) 15 pts Extra Homework (more data queries) 15 pts

10 Standard GIS Data Model Linked spatial and attribute data Geographic Data 1 2 Tabular Data ID Age Type Pavement Ln Bitumen 2 2 2Ln Concrete Ln Dirt etc. ID Lake Size Quality etc

11 For each layer there is typically a one-to-one relationship between geographic features (point, line, or polygon) and records in a table

12 Terminology Database an organized collection of data Table data organized in rows and columns Attribute a variable or item Record a collection of attributes Domain the range of values an attribute may take Index/key attribute(s) used to identify, organize, or order records in a database

13 Common components of a database: Attribute (or item or field) Record (or tuple) ID AREA Perim Class Code a 11z a 119f integer domain real domain alphanumeric domain (a string)

14 A DBMS provides Translation (many views on the data) Protection (e.g., against errors due to simultaneous updates)

15 Multi-tiered architecture Isolates end user (upper tier) from changes at lower tiers Allows integration of multiple divergent data sources

16 Common Database Models: Flat File Hierarchical Network Relational

17 Flat File Data in a text or other lightly formatted file. Little structure, crossreferencing or linking among entries. Often in a row/column format Advantages: Transparent, easily transportable Disadvantages: Little structure, few error safeguards

18 Relational Database Model Minimal rowcolumn structure Items/records with specified domains (possible values) Advantages: Minimum structure, easy programming, flexible Disadvantages: Relatively slow, a few restrictions on attribute content

19 Relational Databases Are Most Common Flexible Relatively easy to create and maintain Computer speeds have overcome slow response in most applications Low training costs Inertia many tools are available for RDBMS, large personnel pool

20 Hybrid Database Coordinate data as a connected structure Attribute data as a relational table

21 Eight Fundamental Operations Restrict (query) subset by rows Project subset by columns Product all possible combinations Divide inverse of product

22 Eight Fundamental Operations Union combine top to bottom Intersect row overlap Difference row non-overlap Join (relate) combine by a key column

23 Main Operations with Relational Tables Query / Restrict Conditional selection Calculation and Assignment Sort rank based on attributes Relate/Join Temporarily combine two tables by an index

24 Query / Restrict Operations with Relational Tables Set Algebra Uses operations less than (<), greater than (>), equal to (=), and not equal to (<>). Boolean Algebra uses the conditions OR, AND, and NOT to select features. Boolean expressions are evaluated by assigning an outcome, True or False, to each condition.

25 Query / Restrict Operations with Relational Tables Each record is inspected and is added to the selected set if it meets one to several conditions AND, OR and NOT may be applied alone or in combinations AND typically decreases the number of records selected OR typically increases the number of records selected NOT Is the negation operation and is interpreted as meaning select those that do not meet the condition following the NOT.

26 Query / Restrict simple, AND

27 Query / Restrict OR, NOT

28 Operation Order is Important in Query (D OR E) AND F may not be the same as D OR (E AND F) NOT (A and B) may not be the same as [ NOT (A) AND NOT (B)] Typically need to clarify order with delimiters

29 NAME

30

31

32

33

34

35

36 Structured Query Language (SQL) A standard system for query syntax Uniformly interpreted set of operations, e.g., CREATE INSERT SELECT Anybody can (and it appear everybody has) create a database engine that fits under SQL then we can switch vendors and upgrade whenever we want.in theory.

37 SQL Example (Creating a file of points where roads and streams intersect) CREATE TABLE informix.culverts ( se_row_id serial primary key, shape ST_Point); CREATE INDEX culverts_ix2 ON culverts (shape ST_Geometry_Ops) USING RTREE; INSERT INTO geometry_columns VALUES ( "hamlet", "informix", "culverts", "shape", NULL, 1, -- The code for a Point. NULL, 0 );... Source:

38 Recap Query / Restrict Set Algebra Uses operations less than (<), greater than (>), equal to (=), and not equal to (<>). Boolean Algebra uses the conditions OR, AND, and NOT to select features. Boolean expressions are evaluated by assigning an outcome, True or False, to each condition. SQL - Structured Query Language

39 1. 2. SELECT AREA > 8.0 COLOR = blue OR ID = F Practice Do not turn in 3. NOT (COLOR = blue) 4. ID < c 5. COLOR = blue AND AREA > 10 OR ID = g

40 1. SELECT AREA > COLOR = blue OR ID = F 3. NOT (COLOR = blue) 4. ID < C 5. COLOR = blue AND AREA > 10 OR ID = g

41

42

43

44

45 OUTER JOIN

46 INNER JOIN OUTER JOIN

47 Main Operations with Relational Tables Query / Selection Conditional selection Calculation and Assignment Sort rank based on attributes Relate/Join Temporarily combine two tables by an index

48 Calculation and Assignment Slope = steep Aspect = 45.2 Cost = [ M * U + cos (distance) ] / (F P/R*T)

49 Main Operations with Relational Tables Query / Selection Conditional selection Calculation and Assignment Sort rank based on attributes Relate/Join Temporarily combine two tables by an index

50 Sort ordering by attribute values Simple sort ascending AREA Name AREA class Type Emily, Lake 52, Limnetic zone Emily, Lake 58, Limnetic zone 60, Shallow lakes 64, Shallow lakes 70, Shallow lakes Long Lake 88, Limnetic zone 143, Littoral zone Sleepy Eye Lake 170, Littoral zone Mud Lake 193, Shallow lakes Goldsmith Lake 201, Littoral zone Emily, Lake 336, Littoral zone 349, Limnetic zone 384, Littoral zone Emily, Lake 420, Limnetic zone Savidge Lake 479, Littoral zone Emily, Lake 545, Limnetic zone Dog Lake 635, Littoral zone Duck Lake 1,126, Limnetic zone Wita Lake 1,354, Littoral zone 1,418, Limnetic zone Ballantyne Lake 1,428, Limnetic zone Washington, Lake 1,914, Limnetic zone 1,937, Limnetic zone 4,040, Limnetic zone Compound sort ascending Type, then descending AREA within Type Name AREA class Type 4,040, Limnetic zone 1,937, Limnetic zone Washington, Lake 1,914, Limnetic zone Ballantyne Lake 1,428, Limnetic zone 1,418, Limnetic zone Duck Lake 1,126, Limnetic zone Emily, Lake 545, Limnetic zone Emily, Lake 420, Limnetic zone 349, Limnetic zone Long Lake 88, Limnetic zone Emily, Lake 58, Limnetic zone Emily, Lake 52, Limnetic zone Dog Lake 635, Littoral zone Wita Lake 1,354, Littoral zone Savidge Lake 479, Littoral zone 384, Littoral zone Emily, Lake 336, Littoral zone Goldsmith Lake 201, Littoral zone Sleepy Eye Lake 170, Littoral zone 143, Littoral zone Mud Lake 193, Shallow lakes 70, Shallow lakes 64, Shallow lakes 60, Shallow lakes

51 Main Operations with Relational Tables Query / Selection Conditional selection Calculation and Assignment Sort rank based on attributes Relate/Join Temporarily combine two tables by an index

52 Tables in GIS Attribute tables are often huge We have to maintain our tables (change values, remove, add records or items) Different people/applications are interested in different subsets of attributes (columns) We often break our tables up into pieces (many tables), and use relational joins as needed to combine them back together

53 A Relational Join

54 Relational Tables Relational tables have many advantages, but If improperly structured, they may suffer from: Poor performance Inconsistency Redundancy Difficult maintenance This is common because most users do not understand the concepts Normal Forms in relational tables.

55 Relational Tables Relational tables have many advantages, but If improperly structured, table may suffer from: Poor performance Inconsistency Redundancy Difficult maintenance This is common because most users do not understand the concepts Normal Forms in relational tables.

56 Tables in Non-normal Form repeat columns, dependent data, empty cells by design

57 Normal Forms Are Good Because: It reduces total data storage Changing values in the database is easier It insulates information it is easier to retain important data Many operations are easier to code

58 1 st Normal Forms in Relational Tables Tables are in first normal form when there are no repeat columns Advantages: easy to code queries (can look in only one column) Disadvantages: slow searches, excess storage, cumbersome maintenance

59 2 nd Normal Forms in Relational Tables 2NF if: it is in 1NF and if every non-key attribute is functionally dependent on the primary key What is a key? An item or set of items that may be used to uniquely identify every row What is functional dependency? If you know an item (or items) for a row, then you automatically know a second set of items for the row this means the second set of items is functionally dependent on the item (or items)

60 Keys Item(s) that uniquely identify a row STATE can be a key, but not REGION, SIZE, or POPULATION

61 Keys Item(s) that uniquely identify a row Sometimes we need >1 column to form a key, e.g., Parcel-ID and Own-ID together may form a key

62 Functional Dependency Knowing the value of an item (or items) means you know the values of other items in the row e.g., if we know the person s name, then we know the address In our example, if we know the Parcel-ID, we know the Alderman, Township name, and other Township attributes: Parcel-ID - > Alderman Parcel-ID - > Thall_add Parcel-ID - > Tship-ID Parcel-ID - > Tship_name

63 Moving from First Normal Form (1NF to Second Normal Form (2NF), we need to: Identify functional dependencies Place in separate tables, one key per table

64 3 rd Normal Forms in Relational Tables Remove transitive functional dependencies A transitive functional dependency is when A -> B (if we know A, then we know B) and B -> C (if we know B, then we know C) So A -> C (if we know A, then we know C). To be in 3NF, we must identify all transitive functional dependencies, and remove them, typically by splitting the table(s) that contain them

65

66 In our example, one transitive functional dependency: Parcel-ID -> Tship-ID, Alderman Tship-ID -> Tship_name, Thall_add

67

68 Bad Things in (unormalized) Relational Tables: Repeat (or similar) variables e.g., parcel #, owner 1, owner2, owner3, owner 4 Multiple dependencies per record e.g., owner name, house#, street, city, county, zipcode, state, country Repeat records Many blank cells

69 Normal Forms Summary No repeat columns (create new records such that there are multiple records per entry) Split the tables, so that all non-key attributes depend on a primary key. Split tables further, if there are transitive functional dependencies. This results in tables with a single, primary key per table.

70 Normal Forms Are Good Because: It reduces total data storage Changing values in the database is easier It insulates information it is easier to retain important data Many operations are easier to code

71

72 In ArcGIS 10 Per capita energy use < 4,000 OR (population > 40 million) and (car theft <1) The order of operations is assumed to be this When you don t include parentheses Per capita energy use < 4,000 OR ((population > 40 million) and (car theft <1))

73

74

75

76 A Small Exercise in Normal Forms

77 Earthworm Invasions Study to identify the cause and extent of exotic earthworm infestations Many sample sites, 10 meters in diameter, over a range of conditions. The worms present were counted and identified by species. GPS location was taken along with the type of soil, amount/depth of forest litter and vegetation at each sample site is also noted. 5 different Earthworm species are found in varying amounts at each site These data were organized in an attribute data file associated with each sampling point

78 Maple without Worms

79 Maple with Worms

80 Earthworm Invasions Study to identify the cause and extent of exotic earthworm infestations Many sample sites, 10 meters in diameter, over a range of conditions. The worms present were counted and identified by species. GPS location was taken along with the type of soil, amount/depth of forest litter and vegetation at each sample site is also noted. 5 different Earthworm species are found in varying amounts at each site These data were organized in an attribute data file associated with each sampling point

81 Exotic Earthworm Study Non-normalized Form Repeat columns Site Table

82 Normal Forms Summary No repeat columns (create new records such that there are multiple records per entry) Split the tables, so that all non-key attributes depend on a primary key. Split tables further, if there are transitive functional dependencies. This results in tables with a single, primary key per table.

83 Un-normalized File Class Exercise Only (not to be turned in) Retain for Exam Study

84 You remove duplicate columns to create a 1st normal form

85 1st normal form no repeat columns

86 2nd normal form Functional Dependencies Site/Species/Count Table Break up the 1 st normal table. Retain the Site/Species/Count table and create a second table with eight items, using Site as the primary key and removing the dependent items from the 1st NF table Site Table

87 2nd normal form Functional Dependencies Site/Species/Count Table Break up the 1 st normal table. Retain the Site/Species/Count table and create a second table with eight items, using Site as the primary key and removing the dependent items from the 1st NF table Site Table

88 3nd normal form - Remove transitive functional dependencies Veg-code -> Vegetation Soil -> Description Split the 2 nd normal table these into several tables, and removing the transitively dependent columns from the site table. Site/Species/Count Table Veg Table Site/CodeTable Soil Table

89 3nd normal form - Remove transitive functional dependencies Veg-code -> Vegetation Soil -> Description So split these into their own tables, and remove the transitively dependent columns from the site table, above Vegetation Table Site/Code Table Soil Table

90 Exotic Earthworm Study 3rd Normal Form Site/Code Table Site/ Species/ Count Table Vegetation Table Soil Table

91 Source Site file Exotic Earthworm Invasion Study Site/ Species/ Count Table Site/Code Table Use with another file to analyze proximity to source How do you connect the Source Site and Site/CodeTable? Vegetation Table Soil Table

92 Source Site file Exotic Earthworm Invasion Study Site/ Species/ Count Table Site/Code Table Use with another file to analyze proximity to source How do you connect the Source Site and Site/Code Table? The UTM Coordinates Vegetation Table Soil Table