DevOps: Automating .Jar release with Python

Intro

DevOps (Development and Operations) is a set of automated software practices that combine software development (Dev), testing and IT operations (Ops) to shorten the software development life cycle while delivering features, fixes, and updates frequently in alignment with the business' objectives.

DevOps teams use tools to automate and accelerate processes, which helps to increase reliability. A DevOps toolchain helps teams tackle important DevOps fundamentals including continuous integration, continuous delivery, automation, and collaboration. In a recent survey, 99% of respondents said that DevOps had a positive impact on their organization. The benefits of DevOps include faster and easier releases, team efficiency, increased security, higher quality products, and consequently happier teams and customers.

Reference: Atlassian - DevOps

Automating with Python

In this article, we will look at a portion of DevOps: automation of processes. P.S This process can as well be automated by another tool out of the box (however, this process is for learning purpose and demonstration). To achieve automation, we need to know how its done manually first, lets look at the underlying approaches of our automation program.

Problem Statement

Jack have a SpringBoot / Java .jar project build and ready to use on the server. Jack manually runs the .jar file and manages the deployment, monitoring and tailing his .jar project using available terminal commands. However this has been tedious for Jack from adding new changes to his project, build .jar, transfer the .jar to the server, stop the running old file, start a new one and tail it if need be.

Proposed Solution

To help Jack automate some of these processes, assume Jack has build his .jar executable app called `app.jar` and is using a Windows OS Server. P.S There are many options to automate the process we will look at below which include: bash script on Linux (.sh) or Windows Command Prompt scripting language (.bat). However, will use python as it will run on both environments. We will create a basic python command line project, using basic python with no third-part libraries to 1. Tail service (view its running logs) 2. Check service status (is it running / not) 3. Start service 4. Restart service 5. Stop service (if its already running) 1. Start Service

Jack got used to running a app.jar via terminal using the java command

(Option 1)

C:\Projects\example> java -jar app.jar        

But every time he exits the terminal, his program also stops. So he looked around and started to use the command where he don't need to see console output (Option 2) and outputs his project logs to app.log file to tail later.

C:\Projects\example> javaw -jar app.jar >> app.log        

2. Tail Service

If you have used Unix systems, you might have come across the tail command - its used to print the last N number of data of the given input (our log file in this case). But on Windows OS, its a bit different, On CMD (default, old command prompt) we can use the command

C:\Projects\example> type "app.log" | more          

type is a built-in command which displays the contents of a file on Windows OS and more is used to display the contents of a lengthy file one screen at a time. To achieve an almost similar approach to tail on Linux, Jack looked at using a PowerShell command below, which is used to open the "app.log" file, display its current content, and then continue to monitor the file for any changes.

PS C:\Projects\example> Get-Content -Path "app.log" -Wait        

3. Stop Service

This is a bit tricky, once he started his project, how does he stop it? If he use Option 1, he can easily close the terminal or via CRTL-C. With Option 2, its not that straight forward because the application is started in the background, the OS assigns a PID to the running app.jar service and to close it, Jack will have to manually use Task Manager, CMD or from within his application if he has that mechanism in place.

Via CMD

To shutdown a java process started in the background with javaw, Jack used 2 commands.

1. Tasklist - used to display a list of currently running processes on the computer together with their PIDs.

2. taskkill - ends one or more tasks or processes, usually by using a PID. But using Tasklist can be tedious as he has to lookup for the services running java and mistakenly, he might stop other java services running which depends on Java. To avoid that mistake, Jack resort to finding only the PIDs running java with the jps command.

jps - JVM Process Status Tool packaged in JDK is a tool that displays all the Java processes that are running on that machine.

C:\Projects\example> jps -m  
10416 sem-project.jar   
13444 Jps -m  
11708 app.jar --spring.profiles.active=live        

From the sample output above, Jack noticed his service is running with PID 11708 and he stops it using the command

C:\Projects\example> taskkill /F /PID 11708        

Automating the boring task

Now that we understood the manual tedious approach, lets initiate automation. To achieve this, we will mimic running commands on terminal using python's subprocess module.

a built-in module that allows us to create new child processes, a valuable tool for executing external functions or commands in your Python code. Think of it as a way to programmatically interact with your computer using Python instead of manually entering commands in the terminal.

Check running service

In this snippet, we fire up our command, split the result by \n and loop through to check if our app.jar is present. If we found it, we split the line by space to get the PID which is the first item and return True.

Start service

In this snippet, we first check if the service is already running. We only fire our run command if the service is not already running

Tail service

To get application logs, we tailed the .log file which the app writes its logs to.

Kill service

To shutdown the service, we will call our check_running_process function to be sure the service is running and get its PID. We then take this PID and pass it to our kill_service function.

From here, we have automated the tasks that Jack performs manually, restarting and stopping the service will reuse the functions we already defined. To conclude, our script will have an old CLI menu to abstract all these "basic" technicalities behind, with the options mapped to the functions we created.

Remarks

We have seen that we can automate some tedious repetitive tasks we face in our project deployments. Python can achieve just that using its in-built libraries. However, we can achieve the same using other advanced tools or taking the cloud approach or scripting languages like bash on linux and .bat on windows. In python, we can check the OS we are running on and adapt our functions to run the commands for the current host machine, that way we can have our script running on both platforms.