How do you document and share your GIS scripts and geoprocessing models?

Many of the python scripts I wrote were run as scheduled tasks. I created an inventory document for the scripts. It included a summary of what the script did, what server it ran on, what day(s) it ran, what time it ran, and the location of origin repository. I think a further step could have created a visual showing when the various scripts were running.

Reproducibility: By sharing your code and documenting how it was created, others can follow the same steps and achieve the same results. Quality Control: By carefully explaining the methods and assumptions, you can catch errors and inconsistencies before they become problematic. Reusability: Sharing your GIS scripts and models with others allows them to reuse and build upon your work. This can save time and effort and encourage collaboration and innovation. Transparency: Others can see precisely how your analysis was conducted and evaluate the validity of your results. Learning: By carefully explaining your methods and assumptions, you may be able to find new insights or identify areas where you need to enhance your understanding.

Getting into the practice of documenting our work, we achieve a few goals: First, and perhaps foremost, we add value to the models, scripts and representations of data by adding rationale, explanations, commentaries, references and general knowledge content into a purely technical and computational task. Secondly, the process itself help us clarify the scope and extent of our thinking process. Thirdly, getting used to, and adopting a workflow that includes systematic documentation for models and scripts, is a very good professional practice. It allows others to understand the context, content and often, limitations of our work, ask the right questions, and sometimes improve the quality and accuracy of our work.

I like to use comments & Jupyter notebooks, but have found that different methods call for different formats. We save Arcade snippets in text files and we have a reference folder where we store them for future uses (see Reusability above). Every request gets a document where we just store conversations, decisions, and methods. Works well for creating after action reports or when you have to go back to find out why something was done the way it was.

One of my favorite practices is to work with GitHub repositories to add relevant documentation for GIS data, scripts and models. These repositories become the points of reference for the processes modeled. Another one for GIS scripts and models that address research, or experimentation, is to document and publish the tools and related processes in the form of an article or white paper. There are a number of these open-data article repositories, apart from the traditional scientific journal path, such as ArXiV, Research Square, etc.

Again, Github is a good place to host your models and make them available to the wider professional community. ArcGIS online and public GIS data repositories are another set of ways to achieve this.

Coding standards and best practices are guidelines that help developers write clean, maintainable, and efficient code. One commonly used coding standard for Python is PEP 8, which defines naming conventions, code layout, and comment procedures. Following PEP 8 can help ensure that your code is readable and consistent. However, it is essential to note that different organizations may have specific coding standards tailored to their needs and requirements.

Here are some insights that can help you to make your GIS scripts and models more accessible and reusable: 1. Think about generalizability: how can you can generalize your models to go beyond the specific case study or implementation. 2. Modularity: try to code and develop in ways that other parts can be "plug-and-play" with a wider scope. 3. Interoperability: How can your models and scripts can work with other existing (or think about future) scripts and models to address even more robust problems or tackle more complex challenges. 4. Agile programming: This is self-explanatory 5. Develop your code inan object-oriented fashion. Use classes that can easily become parts of libraries. When possible use existing standards and protocols.

Relative paths for output files can help to ensure your code will still work even if the data location or resource changes. In certain situations, including timestamps in your output filenames can help avoid overwriting or confusion. This is especially important when running automated or batch scripts that generate multiple output files. When using version control, include a thorough Readme file that provides instructions with required dependencies, parameters, screenshots, etc. Unit tests also help ensure your code is compatible with different platforms and environments. This can be useful to identify any issues or incompatibilities that arise and ensure your code is portable and consistent.

FME (Feature Manipulation Engine) is a powerful data integration platform heavily used for GIS scripting and geoprocessing tasks. In addition to running externally, an interoperability extension in ArcGIS Pro provides seamless integration between FME and ArcGIS Pro, which offers the ability to quickly convert data between different formats, validate data quality, and perform spatial analysis. Safe Software also offers an Enterprise platform called FME Server that can handle large volumes of data and processing tasks. Whitebox Geospatial is a geospatial analysis platform that provides various tools and functions for working with raster and vector data. It is open source but does offer paid extensions to expand the toolsets available.

In addition to pure Python, or Python-related tools and libraries, I find it useful to work with JSON and geoJSON related strcutures. Today a lot of the cloud services, including SQL services, no-SQL databases, are interacting with geoJSON data and data models, such as DTDL digital twin models or 2D and 3D visualization programs.