Python Best Practices

Transcription

1 Python Best Practices for GIS Analysts Thomas Laxson GIS Programmer/Analyst Washington State Department of Natural Resources

2 Why Follow Python Best Practices? Save time Facilitate collaboration Develop transferable skills

3 globalnerdy.com

4 Python GIS ArcPy Concise High-level Not pythonic Broad Inconsistent ArcGIS is the state govt standard Open Source Verbose Lower-level Pythonic Focused Fiona Shapely GeoPandas GDAL Rasterio Laspy Object-oriented

5 Coding Principles DRY WET Abstraction Principle Agency package: single module to store file paths Single Responsibility Principle SLOC doesn t exist in a vacuum Write clear and intuitive interfaces

6 Norman Doors invisionapp.com brentmanke.com

7 User Centered Design oss.adm.ntu.edu.sg

8 Style PEP 8 Python s official style guide "Sometimes style guide recommendations just aren't applicable. When in doubt, use your best judgment. Not comprehensive Examples: Indent 4 spaces Lines 79 characters Do not use compound statements Classes use CapWordsNames Constants use UNDERSCORE_SEPARATED_UPPERCASE_NAMES All other variables use underscore_separated_lowercase_names No use for camelcase or CamelCase Comments are complete sentences, adhering to Strunk and White

9 Style Hungarian Notation Systems Hungarian DON T USE IT Indicates data type of the variable >>> str_name = Clark >>> int_age = 29 Apps Hungarian Indicates the use of the variable >>> fc_roads = r C:\\my_db.gdb\roads Example prefixes: fc feature class tbl table gdb geodatabase dir system directory fn field name lyr layer

10 Style Community Consensus Always use variables Use reverse notation E.g., fc_roads_paved paved_roads_fc Concatenate using Advanced String Formatting (PEP 3101) >>> John is {}.format(age) Concatenate paths using os.path.join >>> dir_demo = os.path.join(dir_root, 'My_Subdir')

11 Forward Compatibility Python 2 Only print my_variable 4 <> 2 for k, v in my_dict.iteritems(): keys = my_dict.keys() for value in xrange(10): input( Enter an integer: ) Python 2 and 3 print(my_variable) 4!= 2 for k, v in dict.items() keys = list(my_dict.keys()) for value in range(10): int(input( Enter an integer: ))

12 Documentation PEP 257 Docstrings NumPy-style docstring: def my_function(arg1, arg2): My description of the function. Further details or clarification. Parameters arg1 : int Description of arg1 arg2 : str Description of arg2 Returns bool Description of return value PyDoc, Doxygen, DoxyPy, ReadTheDocs.org

13 Code Examples arcpy.addfield_management("my_feature_class", "LOREM", "TEXT"); print arcpy.getmessages() arcpy.addfield_management("my_feature_class", "IPSUM", "TEXT"); print arcpy.getmessages() arcpy.addfield_management("my_feature_class", "DOLOR", "TEXT"); print arcpy.getmessages() arcpy.addfield_management("my_feature_class", "SIT", "TEXT"); print arcpy.getmessages() arcpy.addfield_management("my_feature_class", "AMET", "TEXT"); print arcpy.getmessages() arcpy.addfield_management("my_feature_class", "CONSECTETUR", "TEXT"); print arcpy.getmessages() arcpy.addfield_management("my_feature_class", "ADIPISCING", "TEXT"); print arcpy.getmessages() fc_dissolved = "my_feature_class fns_text = ('LOREM', 'IPSUM', 'DOLOR', 'SIT', 'AMET', 'CONSECTETUR', 'ADIPISCING') for fn in fns_text: arcpy.addfield_management(fc_dissolved, fn, 'TEXT') print(arcpy.getmessages())

14 rows = arcpy.da.updatecursor("streams", ["STREAM_TYPE", "BUFFER_DISTANCE"]) for row in rows: if row[0] == 1: row[1] = 100 elif row[0] == 2: row[1] = 50 elif row[0] == 3: row[1] = 25 elif row[0] == 4: row[1] = 15 else: row[1] = 10 rows.updaterow(row) del row, rows

15 fc_streams = 'STREAMS' fn_stream_type = 'STREAM_TYPE' fn_buffer = 'BUFFER_DISTANCE' with arcpy.da.updatecursor(fc_streams, [fn_stream_type, fn_buffer]) as rows: for row in rows: if row[0] == 1: row[1] = 100 elif row[0] == 2: row[1] = 50 elif row[0] == 3: row[1] = 25 elif row[0] == 4: row[1] = 15 else: row[1] = 10 rows.updaterow(row)

16 d_streams = {1:100, 2:50, 3:25, 4:15} with arcpy.da.updatecursor(fc_streams, [fn_stream_type, fn_buffer]) as rows: for row in rows: row[1] = d_streams.get(row[0], 10) rows.updaterow(row)

17 def set_values_by_dict(table, key_field, value_field, dictionary, default_value): with arcpy.da.updatecursor(table, [key_field, value_field]) as cur: for row in cur: row[1] = dictionary.get(row[0], default_value) cur.updaterow(row) d_streams = {1:100, 2:50, 3:25, 4:15} set_values_by_dict(fc_streams, fn_stream_type, fn_buffer, d_streams, 10) d_streams = {1:100, 2:50, 3:25, 4:15} v_default = 10 set_values_by_dict(fc_streams, fn_stream_type, fn_buffer, d_streams, v_default) set_values_by_dict(fc_streams, fn_stream_type, fn_buffer, {1:100, 2:50, 3:25, 4:15}, 10)

18 rows = arcpy.da.updatecursor("streams", ["STREAM_TYPE", "BUFFER_DISTANCE"]) for row in rows: if row[0] == 1: row[1] = 100 elif row[0] == 2: row[1] = 50 elif row[0] == 3: row[1] = 25 elif row[0] == 4: row[1] = 15 else: row[1] = 10 rows.updaterow(row) del row, rows d_streams = {1:100, 2:50, 3:25, 4:15} set_values_by_dict(fc_streams, fn_stream_type, fn_buffer, d_streams, 10)

19 rows = arcpy.updatecursor("my_feature_class", "", "", FIELD_NAME1; NOTES; OLD_NOTES") for row in rows: if "\n" in row.field_name1 or "\r" in row.field_name1: row.field_name1 = row.field_name1.replace('\n', ' ') row.field_name1 = row.field_name1.replace('\r', ' ') if "\n" in row.notes or "\r" in row.notes: row.notes = row.notes.replace('\n', ' ') row.notes = row.notes.replace('\r', ' ') if "\n" in row.old_notes or "\r" in row.old_notes: row.notes = row.notes.replace('\n', ' ') row.notes = row.notes.replace('\r', ' ') rows.updaterow(row) del rows, row

20 fc_mine = 'my_feature_class' with arcpy.da.updatecursor(fc_mine, ['FIELD_NAME1', 'NOTES', 'OLD_NOTES']) as rows: for row in rows: for index, value in enumerate(row): for v_replace in ['\n', '\r']: value = value.replace(v_replace, ' ') rows.updaterow(row)

21 def replace_characters(table, fields, strings_to_replace, replacement_string): with arcpy.da.updatecursor(table, fields) as cur: for row in cur: for index in range(len(fields_text)): value = row[index] if value: for string in strings_to_replace: value = value.replace(string, replacement_string) row[index] = value cur.updaterow(row) return fc_mine = 'my_feature_class' replace_characters(fc_mine, ['\n', '\r'], ' ', ['FIELD_NAME1', 'NOTES', 'OLD_NOTES'])

22 def replace_characters(table, fields, strings_to_replace, replacement_string): """(str, collection, str, str) -> None Replaces occurrences of the string(s) in the fields' values with the replacement string. Returns a count of the values that were changed. Parameters: table - The path/name of the feature class within which you wish to replace character fields A collection of fields on which the string replacement will be processed strings_to_replace - A collection of strings that you will be replaced in the feature class' attribute table replacement_string - A string with which you wish to replace all occurrences of all strings_to_replace in the feature class' attribute table. """

23 Packaging packaging.python.org DemoPackage/ demopackage/ init.py cursorwrapper.py LICENSE.txt MANIFEST.in README.rst setup.py

24 init.py import cursorwrapper all = ['cursorwrapper'] version = '0.0.0' author = 'Thomas A. Laxson' license = 'MIT' author_ = 'my @example.com' maintainer_ = 'my @example.com' url = 'mywebsite.example.com'

25 Installing Packages PyPI Python Package Index pypi.python.org Use command prompt > python -m pip install packagename > C:\Python27\ArcGISx6410.4\python.exe m pip install packagename > python -m pip install C:\\path\to\package

26 Version control Package distribution Virtual environments IDEs Programming paradigms

27 Programming Paradigms Imperative Functional Object-oriented (OOP) Procedural + OOP

28 Editor / Development Environment IDLE NotePad++ PyScripter PyCharm Eclipse/PyDev/Liclipse

29 Version Control git vs svn Github vs BitBucket

30 Virtual Environments Isolated workspaces for Python installations/configurations virtualenv Create a virtual environment > virtualenv C:\path\to\my\ve Operate within the virtual environment > C:\path\to\my\ve\Scripts\activate Now you can install packages, run scripts, etc. ArcPy compatibility USGS article: Calling arcpy from an external virtual Python environment