THE SPACE BETWEEN THE DOTS
GEOG 6161 - Capstone in GIS
The goal of this project is to help tell the story of Razorback Sucker migration in the Colorado River Basin. A large and unique fish, the Razorback Sucker is found nowhere else in the world. It was listed as endangered in 1991 before state and federal agencies worked together to help restore its population. By helping tell the comeback story and their migration patterns, the Division of Wildlife Resources (DWR) will be able to secure more funding to assist in their efforts to protect Utah’s wildlife.
The detection data is inconsistent at best; this project helps to fill out “the space between the dots.” This is done through the creation of a series of Python scripts that can help fill out those time gaps with logical fish movement. The DWR wants to produce this logical fish movement on a large scale, so the process being easy to automate quickly was a high priority as well.
Methodology Background
As established in the introduction, this project focuses on Razorback Sucker encounter point locations in the Upper Colorado River Basin. The encounter dataset provided by the DWR contains over 600,000 records and attribute fields including the location of the encounter, the river in which the encounter was recorded, the individual fish with its unique ID number, the individual encounter with its unique ID number, and the date and time that the encounter occurred.
In order to create the Python scripts, ArcGIS ModelBuilder was used. ModelBuilder allows the user to connect tools and commands together to streamline a workflow as well as run that workflow after all of the errors have been debugged. The user can then export the ModelBuilder to a Python script or simply save it and run it in the future as a tool. In the model figure below, the orange bubbles represent inputs and the teal bubbles represent outputs.
Phase 1: Eliminate Single-Encounter Inidividuals
The first step in the project was to eliminate individual fish records that only have one encounter point, as fish movement cannot be modeled for fishes with a single location. This led to the creation of the first Python script, one that goes through an entire dataset or subset of data and deletes single encounter individuals. It uses the frequency tool to count the number of times that a fish’s individual ID appears in the dataset and creates a new table that associates that count with the individual ID. Then it joins that newly created table back to the original dataset or subset and selects all of the individual IDs that have a frequency count of one associated with them. Once those records are selected, the Python script deletes them from the dataset. Finally, it sorts the dataset in ascending order in the individual ID field. This whole process can be seen below in its ModelBuilder form.
Figure 1: Eliminate Single-Encounter Individuals Model
Phase 2: Movement Line Generation (Part 1: Create and Project a New Geodatabase)
The second step of the project was to generate a movement line for the fish between encounter points, and do so in a way that allows it to be automated or at least mostly automated. The key to this movement line is that it needed to snap to the river lines in the Upper Colorado River Basin rather than just travel directly between encounter points as the crow flies. After much trial and error, the Network Analyst Toolset was determined to be the best if not only way for this to occur in an automated manner. Essentially, the river lines became the network that movement would occur in and the encounter points for a single individual were loaded in as “stops” along the network. The Network Analyst could also load these “stops” in sequentially based on the date that each encounter occurred, solving the sequencing problem as well.
This step of the project occurred in two Python scripts. The first of these scripts set up the usage of the Network Analyst Toolset. In order for that toolset to work, all of the layers had to be in the same projection and in the same feature dataset. The script creates a new geodatabase and a feature dataset within it. Then, it asks the user to input a river line feature, a boundary polygon that is necessary for the Network Analyst, and the encounter points for a single individual fish. It reprojects the features to a common project, and loads them all into the newly created feature dataset within the geodatabase. This whole process can be seen below in its ModelBuilder form.
Figure 2: Create and Project Geodatabase Model
Phase 2: Movement Line Generation (Part 2: Line Generation)
The second script involved with the movement line creation creates the movement line itself using the Network Analyst Toolset. After inputting the feature dataset with all of the necessary layers created in the previous script and setting the source feature class, it creates and builds the network along the river lines. Once the network is created, it asks the user to specify the encounter points used for the “stops” in the network. It is also at this step that it filters the points based on the encounter date. Finally, it solves the network by creating the shortest route that runs between the encounter points along the river lines from the earliest encounter date to the latest. This whole process can once again be seen below in its ModelBuilder form.
Figure 3: Movement Line Generation Model
Phase 3: Animate the Line
The final step of the project is to animate this newly created movement line between fish encounter points. After exporting the movement line to its own feature and out of the route tool, the animation process could begin. The line is densified before being split at its vertices. This process splits the single movement line into many segments, sometimes hundreds or thousands. Then the script adds a field to the feature that is used for range animation using the time slider. This whole final process can be seen below in its ModelBuilder form.
Figure 4: Line Animation Model
Results and Discussion
Early on the process of developing the project, it became clear the most important thing in this project, besides finding a way to generate the movement line in a streamlined fashion, is the ability to automate the entire overall process. That is when it became clear that creating Python scripts in ModelBuilder would be the most effective way to accomplish the project goals. In ModelBuilder, once all of the tools are put together and the errors are ironed out, all the user has to do it hit the play button and it will run everything in order. The fewer the buttons there are to hit, the better it is for automation and ModelBuilder was the best route to take in that regard. Then, after the model is built, the user can then export the ModelBuilder to a Python script or simply save it and run it in the future as a geoprocessing tool in ArcGIS.
The Network Analyst Toolset also was a very effective method for generating the movement line. Early on, the goal of the movement line was to be snapped to the river lines as well as progress from the earliest encounter date to the latest. The Network Analyst Toolset is able to accomplish both of those things in one fell swoop. In the figure below, this movement line is shown. The line runs along the river and in between the “stops.” Stop number 1 is the encounter for that particular fish with the earliest date, while stop number 2 has the next encounter date, and so on. When exported and animated, the line is shown to double up between stops 2 and 3 as the fish travels to stop 2 before circling back to stop 3. There were other ways to accomplish this line creation, but none of them were nearly as efficient or allowed for automation.
Figure 5: Movement Line
Conclusion
When this project began, the goal was to fill in “the space between the dots” or the area between fish encounter points and do so in a way that would be automatable to speed up the overall process. Not only was that accomplished, but it was done so in a way that is easy to understand and will be useful on fish data that exists anywhere. These Python scripts are not specific to the Upper Colorado River Basin; they work on any river network consisting of river lines, points, and a boundary polygon. This project also showed the wide range of abilities that the Network Analyst Toolset has, and how it can be used for more things than just routes on actual roads. By providing the DWR with a versatile toolset, these scripts will be useful to them for years to come and should certainly outlast the original Razorback Sucker data for which they were created.
Skills Learned / Used
Communication Skills
Data Models and Structures
GIS Analysis
GIS Workflow
Model Building
Programming and Scripting
Project Design
Project Management
Structured Query Language