How to build a sustainable and automated IIS monitoring system using C# and Azure DevOps

Introduction

For any web application, its availability and reliability are critically important. This is especially true for .NET applications hosted on Microsoft's Internet Information Services (IIS) web server. Disabling IIS components such as Application Pools and Web Sites can cause the Web application to become unavailable to users.

The traditional, manual approach to IIS monitoring and recovery is time-consuming and error-prone. Incidents are only responded to after the problem has already been identified and the application is no longer available. This is certainly not desirable.

This article describes an automated approach to monitoring and recovering IIS using Infrastructure as Code (IaC) principles. The main components of our approach are:

1. A .NET Core Windows service that periodically checks the status of IIS and attempts to automatically recover it in case of any issues.
2. Azure DevOps CI/CD pipelines that build, test, and deploy our monitoring service.
3. Azure Application Insights integration for real-time monitoring and alerts.

This approach ensures increased availability and reliability of our IIS applications while reducing the need for manual intervention.

Application Architecture

At the heart of our system is the .NET Core Windows service, which we call IISMonitoringService. This service periodically runs and performs the following functions:

1. Checks the status of important IIS services (such as W3SVC and WAS). If any of them are not running, it attempts to restart them.
2. Check the status of specified websites. If any are not running, it attempts to start them.
3. Checks the status of specified Application Pools. If any are not running, it attempts to start them.
4. Logs all actions to the Windows Event Log.
5. Sends telemetry (such as actions and errors) to Azure Application Insights for real-time monitoring.

The configuration of the monitoring service (such as the list of websites and Application Pools to check, the interval of checks, etc.) is stored in the app.config file, allowing us to manage the service's behavior without changing the code.

CI/CD and IaC Setup

Our goal is to fully automate the lifecycle management of the monitoring service, from code changes to deployment. For this, we use Azure DevOps Build and Release pipelines.

The Build pipeline performs the following tasks:

1. Builds the .NET Core project.
2. Runs unit tests.
3. Performs static code analysis.
4. Creates a deployment package.

The Release pipeline performs the following tasks:

1. Copies the deployment package to the IIS server(s).
2. Stops the monitoring service if it is already running.
3. Replaces the service's binary files with the new ones.
4. Replaces the app.config file with the new version if it has changed.
5. Starts the monitoring service.

All necessary artifacts, such as build scripts, deployment scripts, and configuration files, are stored in a version control system (e.g., Git). This is a core principle of Infrastructure as Code.

Monitoring and Alerting

With our system, we no longer need to wait for users to discover and report issues. Our monitoring service sends telemetry to Azure Application Insights, allowing us to monitor the status of IIS in real-time and automatically detect potential problems.

In Application Insights, we set up alerts for the following scenarios:

1. When an Exception is logged (indicating that something failed to restart).
2. When the number of restart attempts exceeds a certain threshold (indicating a recurring issue).

When alerts are triggered, a notification is sent to the appropriate parties (e.g., via email or SMS), or an automated action is initiated (e.g., rolling back the application to a previous version).

Example

1.Create a new .NET project named IISMonitoringService:

2. Add an app.config file with the following content:

3.Create an Azure DevOps Build Pipeline that builds the .NET project and creates the deployment package:

4.Create an Azure DevOps Release Pipeline that deploys the service to an IIS server:

5.In the home directory of the project, create a .gitignore file with the following content:

bin/

obj/

*.user

6.Create a Git repository and push it to Azure DevOps.

7.Run the Build and Release pipeline in Azure DevOps.

8.Error Handling and Retry Logic: Our service will directly log an error and move to the next component if any component fails to restart. We can improve this behavior by adding Error Handling and Retry Logic:

9.Configuration validation:

It is a good practice to validate the configuration when starting the service. This will prevent the service from crashing due to misconfiguration.

10. Configure alerts in Azure Application Insights:

Simply recording telemetry in Application Insights is not enough. We also need to configure alerts to let us know when something goes wrong.

- In the Azure Portal, open your Application Insights resource.

- Go to the "Alerts" tab and create a new alert rule.

- Select a signal (eg, Exceptions count), select a Condition (eg, greater than 0), and select a Period (eg, In the last 5 minutes).

- Select the Action Group to which you want to send an alert (eg, send an email or SMS).

- Save the alert rule.

11. Final remarks:

- Make sure your IIS server is running and connected to the Internet so it can send telemetry to Application Insights.

- Ensure that the monitoring service user has sufficient rights to manage IIS components.

- Remove sensitive information (such as the Application Insights instrumentation key) from configuration files before uploading them to Git. Preferably manage this through Azure Key Vault or some similar service.

- Don't forget to change CheckInterval and APPINSIGHTS_INSTRUMENTATIONKEY according to your need in app.config file.

After that, the monitoring service will be installed on your IIS server and will automatically start checking and restarting the specified IIS components if they are not running. All events will be logged to the Windows Event Log and also sent to Azure Application Insights (if you provide an instrumentation key in the app.config file).

If you later need to change the monitoring settings, simply edit the app.config file, commit the changes to Git, and Release Pipeline will automatically update the configuration on your server.

Example 2

1.Let's create a new PowerShell module called IISMonitoringDSC:

New-ModuleManifest -Path 'IISMonitoringDSC.psd1' -Description 'DSC Resource for IIS Monitoring Service' -Author 'YourName'

2.In the same directory, create the IISMonitoringDSC directory and inside the file IISMonitoring.ps1 with the following content:

This is a DSC resource that installs, configures, and runs the IIS Monitoring Service with the desired configuration.

3.Create a configuration.ps1 file with the following content:

4.Let's create a start.ps1 script that will compile the configuration and run it:

Now, when we want to install and configure the IIS monitoring service, we can simply run the start.ps1 script with administrator rights. PowerShell DSC provides service installation, configuration, and startup with specified settings.

The main advantages of this approach are:

1. Infrastructure management with code that allows us to store and version control configuration files.

2. Quick and easy deployment. We can easily start the monitoring service on different servers by simply running the start.ps1 script.

3. Declarative style. We only describe the desired configuration and let PowerShell DSC achieve it.

Summary

This modern, DevOps-oriented approach to IIS monitoring and automated recovery is particularly effective for large organizations where responsibilities are distributed across different teams. Developers focus on application functionality, DevOps engineers ensure its resilience, operations teams ensure its availability, and business owners focus on customer satisfaction.

Although implementing this system requires some effort, its long-term benefits are significant. It not only reduces downtime and increases customer satisfaction but also frees up IT resources from low-value, routine tasks, allowing them to focus on innovation and optimization.