ServiceNow Deployment Pipeline - Part 3: Solution Design

Introduction

After laying the groundwork in my first two articles (part 1, part 2), it's time to dive into the next level - the design of the deployment pipeline app.

Many customers assume that a solution’s scope, timeline, and costs will naturally emerge from vague, unstructured, and fragmented requirement documents, overlooking the need for thorough analysis and planning. And they think that a few configurations are all it takes to build the right solution, since the underlying technology already provides everything needed. However, with over 20 years of project experience, I can confidently say that every new project is an unpredictable journey - an exploration through the fog of uncertainty, where assumptions are constantly challenged. My deployment pipeline app was no exception. Despite being both the requester and the implementer, the path to a refined solution was anything but straightforward.

To avoid getting lost in this fog of uncertainty and lack of knowledge, you need a structured approach, and in most cases that means creating a solution design. This might be a Word document based on any proven template or a collection of individual pages in your company wiki. The medium is not as important as the content, and in the following chapters I will show you a way to achieve this using my deployment pipeline application as an example.

Solution Design

In the IT space, solution design refers to the process of creating a detailed blueprint or plan for implementing a specific solution to a problem or challenge. It involves analyzing requirements and constraints to design an effective solution that addresses business objectives, user needs, technological limitations, and available resources. Here are some key definitions and aspects of solution designs:

1. Blueprint for Implementation: The solution design serves as the first step towards deploying any software solution, acting as a blueprint that guides the entire project from planning to implementation.
2. Bridge Between Processes and IT: It translates process requirements into concrete technical solutions, ensuring that business processes are efficiently digitalized and supported by IT systems.
3. System Design: The solution design defines the features and requirements of a solution, benefiting various organizational aspects by optimizing communication, defining business goals, and proposing technologies for maintainability and scalability.
4. Holistic Approach: It encompasses functional, technical, and UI/UX aspects to create a scalable and robust solution that aligns with business needs and technical strategies.
5. Iterative Process: It is iterative, involving multiple rounds of feedback and refinement until stakeholders verify that the planned solution addresses the problem effectively.

After this high-level explanation, let's get a little more specific and do the first step with by creating a system context diagram.

System Context Diagram

A system context diagram, also known as a level 0 data flow diagram, is a high-level graphical representation that defines the boundary between the system you're analyzing (in this case, my deployment pipeline application) and its environment. It focuses on illustrating the system's interactions with external entities like users, other systems, and organizations, rather than internal processes within the system itself.

It serves as a solid foundation for diagramming and documenting a software system, enabling you to step back and gain a broader perspective. The focus isn’t on details at this stage but rather on providing a high-level overview of the system landscape.

Key components of a system context diagram:

• The System: This is the main focus of the product/solution being designed and represented as a single circle or box labeled with a name.
• External Entities (Actors/Terminators): These are entities outside the system that interact with it. They can be people (end-users, IT staff, managers, etc.), external systems (other applications like HR systems, data repositories, monitoring tools, etc.) or organizations (departments within the company, partner organizations, etc.)
• Arrows: These represent the main types of interactions with the surrounding entities. They should be labeled to help understand the purpose of the interaction.

A solution design benefits from an system context diagram in several ways:

• Defines Scope and Boundaries: It helps architects establish what’s included in the system and what’s external, preventing scope creep and ensuring focus.
• Identifies Integration Points: By mapping interactions with external entities, it highlights necessary interfaces (e.g., APIs or data flows), guiding technical requirements and reducing oversight in connectivity design.
• Enhances Stakeholder Alignment: Its simplicity makes it an effective communication bridge between technical teams and business stakeholders, ensuring everyone understands the system’s role and dependencies before diving into complex internals.
• Supports Risk Mitigation: Early visibility into external dependencies (e.g., a third-party service’s reliability) allows architects to plan for contingencies, improving the solution’s robustness.

For the Deployment Pipeline application, the context diagram might look like this:

Since you cannot expect every reader to be able to understand the entities presented, you should explain them briefly. These explanations can then later become part of the project glossary:

Deployment

Refers to the technical process of transferring all deployment artifacts in a planned and automatic approach from a source instance to a target instance and processing these artifacts according to their nature (e.g. installing a custom application).

Deployment Record

A deployment record consolidates all associated deployment artifacts, encapsulating the comprehensive technical and organizational details required to define and execute a single deployment effectively.

Deployment Artifacts

The following artifact types are in scope and can be added to a deployment record by a deployment pipeline user:

• Custom Application: A custom application (table "sys_app") is a tailored software solution built from scratch using its development tools, such as ServiceNow Studio, to address specific business needs or processes not met by existing store apps or out-of-the-box features. Custom applications can be transferred to another instance via ServiceNow's application repository, a Git repository or via update sets.
• Customized Store Application: A store application (table "sys_store_app") refers to an application originally sourced from the ServiceNow Store. Some store application allow customizing the contained application files or adding new ones to align with unique business requirements, processes, or workflows that the standard version doesn’t fully address. These customizations are bundled in individual versions with the baseline version of the underlying store app. Customizations of a store app can be moved to another instance the same way as a custom application.
• Update Set: An update set is a container (table "sys_update_set") that groups a collection of configuration changes (table "sys_update_xml") made to the ServiceNow platform. Update sets are exported on the source instance to an XML file and then imported, previewed and committed on the target instance.
• Scripts/Code: Most customers rely on operations teams to execute deployments on ServiceNow instances using precompiled runbooks, which frequently include scripts to handle instance-specific configuration adjustments or system tasks not embedded within the deployment package. While ServiceNow offers Fix Scripts as a solution, I find them less favorable - partly because they don’t execute with update set-only deployments. This makes the ability to automatically run tailored scripts on the target instance a valuable alternative for ensuring seamless post-deployment adjustments.
• Data: The transfer of pure data (not captured as application files or configuration changes) during a ServiceNow deployment is typically done to ensure that baseline data, essential lookup table records, or initial configuration values are available in the target instance. Although there are options to capture pure data records within an application or update set, it is not recommended to go that way for many reasons.

Actors / Personas

• Deployment Pipeline Admin: That application-specific administrator has been granted administrative privileges specifically scoped to the Deployment Pipeline application to configure the application without requiring full system-wide admin rights (i.e., the global "admin" role).
• Deployment Pipeline User: These users represent the primary group for which the Deployment Pipeline application was designed.
• Deployment Approver: Depending on the criticality of the target instance, it must be possible for appropriately named users to be able to approve or reject deployments to the respective environment.
• Any user with the admin role: A deployment pipeline application is a valuable tool, but it can also be a very dangerous one. Without proper security measures in place, unauthorized users could make far-reaching and destructive changes to a ServiceNow instance that they may not even have access to. Since the deployment pipeline application is typically installed on a DEV instance, which usually has many developers with "admin" privileges, all the loopholes that the "admin" role naturally provides must be closed.

External Systems

• ServiceNow Instance: In addition to the current DEV instance on which the deployment pipeline application is installed, this includes all remote instances on which a deployment is to be performed.
• Application Repository: A centralized location used primarily for managing and deploying custom applications and customized store applications across different customer instances, allowing for version control and lifecycle management of these applications.
• Git Repository: A central storage location for managing and tracking changes in files and directories, which can be integrated with ServiceNow to enhance collaboration and workflow management by synchronizing data between Git repositories like GitHub and ServiceNow instances.

In the next step, you can approach the finished solution design by brainstorming a list of use cases.

Use Cases

Summarized and simplified, a use case is a (detailed) description of how a user or system interacts with a solution to achieve a specific goal, outlining the steps, actors, and expected outcomes in a given scenario. It serves as a practical tool in solution design to define requirements, guide development, and ensure the system meets real-world needs effectively.

While there are no standardized formal requirements for system context diagrams, use cases are very well-defined in the IT industry and there are sophisticated templates and notations for documenting use cases (see Wikipedia). However, I recommend starting with just writing down the headlines for the most important activities for every persona. Later, you can describe the use cases in more detail and then derive the corresponding user stories from them.

Deployment Pipeline Administrator

• Initial setup and configuration of the application.
• Create all the necessary configurations, such as the connections to the target instances.
• Onboard application users.
• Monitor issues.

Deployment Pipeline User

• Creation and planning of deployments, including collecting all required deployment artifacts.
• Trigger deployments directly or ask for approval.
• Cancel deployments.

Deployment Approver

• Approve a deployment request

Now let's zoom into the box in the middle of the system context diagram and unlock the next level of detail. This is where a component diagram comes into play.

Component Diagram

Component diagrams provide a high-level overview of the system, showing the major components and their relationships. This helps everyone involved understand the overall architecture without getting lost in the details of more technical views or even code.

Although there are standard notations for component diagrams (e.g. in UML), I recommend not following them too closely for solution design documents. For their target audience, which is always more business than technical, ease of understanding is more important than 100% compliance with a notation standard.

Based on the previous analysis you now can start thinking about high-level building blocks and their relations to each other. In the following component diagram these relations are represented by arrows which indicate a "uses" or "communicates with" relation:

Some hints and tips regarding the creation of component diagrams - based on my experiences:

• Use coloring as much as possible, but don't overdo it. It is more important to be consistent in your color choices, so that objects of the same type are colored the same. I often see complicated diagrams where the elements are almost indistinguishable to the human eye. Especially when arrows and boxes have the same style (black, solid lines of the same thickness), the chaos becomes perfect.
• Use rounded corners and lines whenever possible. This is more comfortable and easier to read.
• Adjust the font size to the size of the chart. Most drawing programs do not do this automatically, so as the diagram size increases, the ratio of font size to expansion becomes worse and worse. The result is that the text can only be read with a magnifying glass when the chart has to be reduced to fit the given slot (e.g. screen with a restricted resolution, or here on LinkedIn the images need to have a fixed width).
• Distinguish between machine-to-maschine interactions or relationships (solid arrows) and those of human users (dashed arrows), as these are two completely different subjects - with different principles, rules, and thus implications for implementation.
• An application is a closed system that lives in a closed platform. Therefore, you should never connect arrows from a component directly to an external system, but route them through so-called ports. After all, you don't go out of your house directly through the wall, but use the door, do you?
• These ports (or technically speaking: interfaces) are shown in my component diagram as small colored boxes on the edges of a system to show that there are “pass-through” points (for data, signals, interactions, etc.). For the technical forms of communication, capabilities such as REST messages or scripted REST APIs can be utilized in ServiceNow. And human users also interact with a system through (user) interfaces, which are technologically different and can be a mobile app in ServiceNow, a service portal, or a workspace. That is why those UIs have a different port color in my diagrams.

Wrapping up

This article is not intended to be a complete guide to writing a solution design document, as other elementary chapters are missing to make such a document complete. Some examples are:

• Introduction & Overview: purpose, scope, audience, project context, history, referrences to related sources
• Current situation: what is available/existing at the moment on customer side, what are the current challenges and pain points
• Requirements Overview: references to provided requirements documents, summarized/extracted business/functional/non-functional requirements and constraints
• Non-functional Design Considerations: performance, security, scalability, reliability
• Assumptions and Risks: conditions assumed true, potential risks and their possible mitigations, constraints
• Implementation Considerations: proposed development approach (in ServiceNow stick to the well-defined Now Create methology), testing strategy, deployment plan, dependencies, etc.
• Appendices: glossary, references, etc.

For those eager for concrete implementation insights, the upcoming articles will delve deeper into the technical details, offering a closer look behind the scenes. Until then, I look forward to your feedback and questions, and wish you an exciting time on your own adventures in the ServiceNow universe.