Azure DevOps and Vorteil.io ... even I couldn't stuff it up ...

As any IT architect, engineer or support person who has previously worked with me can tell you - I like to tinker, play and solve problems. However, that does not imply that I'm good at ANY of the aforementioned. There are 3 things I love though:

1. Linux (or UNIX) operating systems and command lines. I like the elitism of it (see image below) ... or I enjoy the punishment of typing out every command I would like to run, watching it print garbage to the screen in text format and then pondering over kill -9 <pid>.
2. Complexity and over-complicating something which should've been simple for the sake of automating it
3. The thrill of rm -rf * ...

So when a customer asked us to put together a demonstration of how Vorteil.io would operate with Azure DevOps ... well let's just say I wasn't in a happy place.

So let's cut right to it, what did I think of Azure and Azure DevOps:

• The Azure interface is clean, easy to use and unbelievably user friendly. I expected this from the organisation who gave us graphical interfaces with mountains in the background which my kids use in school - but Azure was far better than what I expected!
• Azure DevOps (for someone who is NOT a CI/CD pipeline developer) was remarkably easy to use. If anything, the issues I had in the process of deploying Vorteil.io machines was the azure templates I needed to create to deploy virtual machines
• Documentation was confusing at first - because I'm impatient. I should have read "The Basics".

The end-state: a .NET Core application, on Vorteil.io, on Azure!

To make this happen, the end-state solution looks something like the image below:

In summary, the following happens:

1. A change on the Github project initiates a pipeline in my Azure DevOps.
2. The Pipeline essentially produces a .NET Core package which is a stand-alone application.
3. Once the Pipeline completes, a Release is triggered
4. The Release copies the standalone package to a Vorteil.io repository server (an Ubuntu machine hosted in Azure). The Vorteil.io repository server creates the Vorteil.io Azure image based on the Vorteil.io VCFG file stored within the standalone .NET Core package.
5. The Release then creates an Azure Virtual Machine based on the templates and parameters stored within the standalone .NET Core package.

So now on to how it was done!

Prerequisites

Apart from creating an Azure account to allow access to Azure Compute, Azure Storage and Azure DevOps, the only prerequisite was to create a virtual machine for the Vorteil.io repository server (which also acts as the Vorteil.io machine builder). I simply used an Ubuntu public image from the Azure marketplace and created a machine with the following specifications:

1. Size: Standard A2 v2 (2 vcpus, 4 GiB memory)
2. Operating system: Linux (ubuntu 18.04)

To install the Vorteil.io repository I used wget to get the binary and install it using the support website knowledge article.

To support Azure as an endpoint for the Vorteil.io repository I configured it using the Vorteil.io command line interface (see my very first paragraph on why I like text) and the command:

vorteil-user@vorteil-repo:/# vorteil provisioners new azure azure-vorteil --storage-account-name="vorteildemodiag" --storage-account-key="<removed because it's a key>" --container="vorteil-demo" --location="Australia East" --resource-group="vorteil-demo" --key-file="/home/vorteil/azure.conf"

Again, these steps are documented on the Vorteil.io support site. Now we're ready to provision Azure Vorteil.io templates and we have a mechanism for the Pipelines to build machines!

The chosen application

Because I know NOTHING about .NET Core, I decided to use the first Google match to a search ".Net Core game exmaple" ... then I clicked on "Showing results for .Net Core game example" because that's how we roll in IT.

I forked the code from the TicTacToe .NET Core repository to my personal repository (thanks to the Packt for this).

I'm not going to lie about the next couple of things I added to the personal instance of the TicTacToe repository - I figured this out after a couple of iterations of building, breaking and frustrating myself with not having read and understood the principles of Azure DevOps.

I added 4 new files to my project:

• .vorteilproject: this file contains Vorteil.io configuration information
• tictactoe.vcfg: this file contains the Vorteil.io build information, more on this here

To allow my 2 files above to be added to the .NET Core standalone package I also had to modify the .NET project file (I had to read up on .NET Core). The file which I modified was TicTacToeGame.csproj and I added the following lines:

  <ItemGroup>
     <None Include="tictactoe.vcfg" CopyToPublishDirectory="Always" /> 
     <None Include=".vorteilproject" CopyToPublishDirectory="Always" />
  </ItemGroup>

• The Azure template file
• The Azure template parameters file

Azure Pipelines

This is was easy. Link your pipeline to the Gitub instance of the repository, select the type of build (in my case .NET Core - but Azure DevOps suggested this based on the files in the repository in any case) and modify the azure-pipelines.yml file to suit your needs.

In my case, the file contents below:

# ASP.NET Core
	# Build and test ASP.NET Core projects targeting .NET Core.
	# Add steps that run tests, create a NuGet package, deploy, and more:
	# https://docs.microsoft.com/azure/devops/pipelines/languages/dotnet-core
	

	trigger:
	- master
	

	pool:
	  vmImage: 'ubuntu-latest'
	

	variables:
	  buildConfiguration: 'Release'
	

	steps:
	- task: UseDotNet@2
	  inputs:
	    packageType: 'sdk'
	    useGlobalJson: true
	

	- script: dotnet build --configuration $(buildConfiguration)
	  displayName: 'dotnet build $(buildConfiguration)'
	

	- task: DotNetCoreCLI@2
	  displayName: 'dotnet publish $(buildConfiguration)'
	  inputs:
	    command: publish
	    publishWebProjects: False
	    arguments: '--self-contained --runtime linux-x64 --configuration $(BuildConfiguration) --output $(Build.ArtifactStagingDirectory)'
	    zipAfterPublish: false
	

	- task: PublishBuildArtifacts@1
	  inputs:
	    pathtoPublish: '$(Build.ArtifactStagingDirectory)' 
	    artifactName: 'Drop'

PLEASE NOTE: a lot of this content was auto-generated - I would have had ZERO knowledge or skill to do this!

The parts of the file which I modified are:

• added the "--self-contained" and "--runtime linux-x64" flags
• added the "zipAfterPublish: false" option because I didn't want to unzip it on the Vorteil.io repository server (laziness only)

That was it ... I could run the pipelines and have a successful build! On to the Release!

Azure Releases

For Releases I created a new release to match the Pipeline built in the previous section. I added 5 tasks to the Release configuration.

Deployment group job

The deployment group job is a built-in function which copies the artefacts produced by the Pipeline into the Release. This allows a user to run tasks, configurations and jobs against these artefacts contained in the directory. Pretty standard stuff thus far.

Copy artefacts to Vorteil.io repository

This is a "Copy files over SSH" pre-defined task in Azure DevOps. The tasks simply copies the compiled stand-alone artefacts over to the Vorteil.io repository server (Ubuntu machine in Azure).

It creates a directory:

/opt/projects/$(appname)_$(Release.ReleaseName)

where $appname is a defined variable which contains the application name (in this case tictactoe) and $Release.ReleaseName refers to the Release details. This makes every image and virtual machine unique in the Release process.

Remote shell - import shared libraries from Ubuntu to Vorteil.io package

This is an "SSH" predefined task in Azure DevOps. The tasks connects to the Vorteil.io repository and runs the following commands:

vorteil projects import-shared-objects /opt/projects/$(appname)_$(Release.ReleaseName)

This command instructs the Vorteil.io repository to analyse the project we've just copied over and import all shared libraries which are dynamically linked. The next step is for illustrative purposes only, and shows how a user can import any additional Linux libraries which are not imported during the "import-shared-objects" command:

cp /usr/lib/x86_64-linux-gnu/libssl.so.1.* /usr/lib/x86_64-linux-gnu/libcrypto.so.1.* /opt/projects/$(appname)_$(Release.ReleaseName)/lib/x86_64-linux-gnu/

This is a critical step in the process as it ensures that the Vorteil.io image contains the required custom libraries an application might need.

Run shell - provision to Azure Image from Vorteil.io Repository

Time for the Vorteil.io magic to happen! This is an "SSH" predefined task in Azure DevOps. The tasks connects to the Vorteil.io repository and runs the following commands:

vorteil images provision --name=$(appname)-$(Release.ReleaseName) --force --provisioner=azure-vorteil /opt/projects/$(appname)_$(Release.ReleaseName)

This command instructs the Vorteil.io repository to build an Azure image and provision it to the Azure instance previously configured as the target. Once this command completes, an image exists (see below):

Create Azure Virtual Machine

This is an "Azure resource group deployment" predefined task in Azure DevOps. This is the tasks which creates the Virtual Machine based on the image provisioned by Vorteil.io in the screenshot above.

I'm not going to lie - this was a challenge. The 2 most important files at this point is the Azure template file and the Azure template parameters file.

However, to ensure uniqueness when the virtual machines are deployed, some of the parameters are overwritten during the Release. As an example:

• -networkInterfaceName "tictactoe-$(Release.ReleaseName)"
• -publicIpAddressName "tictactoe-$(Release.ReleaseName)"
• -virtualMachineName $(appname)-$(Release.ReleaseName)
• -templateName $(appname)-$(Release.ReleaseName)

The "templateName" matches the "--name=$(appname)-$(Release.ReleaseName)" arguments in the Vorteil.io repository image creation.

Azure Compute

What does all of this lead to? A CI/CD pipeline which:

1. deploys a fully functional .NET Core application without a Linux operating system on Azure;
2. a virtual machine with 500MB of memory and 1 CPU, still capable of running the application without ANY operating system overhead
3. a virtual machine disk - 1 GB in size! (it's actually much smaller, but I don't think Azure reports anything smaller)
4. a virtual machine which during idle times uses only 0.05% CPU

Hope you enjoyed it ...!

Feel free to reach out to me with any questions or comments:

[email protected]