Process Mining and Software Development

Almost every engagement that I have starts with my clients wanting to create a new program, or they want to do an automation.

In almost every situation, however, my clients have an idea of the application that they want and the outcome that they want, but they DO NOT know WHY they need that application or automation.

They know that there is a problem, and they THINK that the solution is something new or automating something. What we do then is talk to them about the problem, and what the requirements for the application or automation is, and we start to build the software.

When we get done... We sometimes give the client the outcome that they want. Don't get me wrong, we build the software to spec, and use Agile process to give the client exactly what they are asking for. BUT, because the client didn't understand the WHY, our solution doesn't always solve their problems.

Taking an introspective look, I ask myself, "Self!" (cause that is how I address myself) "Self!", I asked, "How can I have more successful engagements? We build the software that the client asked for, why didn't it solve the problem?" and I answer myself "Self!" (cause that is how I reply to myself) "Self!", I answered, "Do you or your clients really understand the problem they are trying to solve? You know what they want to BUILD, but do you or they really know WHY?"

So... How do I find the WHY when my clients don't even know the why? In walks something very popular in Europe, but not so much in the US, with one of those great big collars, and a powdered wig, and you know all of that stuff that is popular in Europe... Process & Task mining.

What is it? Process mining is looking at your data to determine and map out your work processes. Task mining is looking at the actual work that is being done by individuals performing the process. What these two do together is that Process Mining gives you the overall data and analytics, over a large time period, on the process from beginning to end, and all of the mess in between. What Task Mining gives you is the HOW individuals are actually doing their work over the time of the mining recording. Contrast this to your stated process SOP to show what the theory of the process is against the actual way work is done.

What this gives the software developer is a very clear picture of where and what areas of work are causing the most pain, more often than not THIS is the WHY. If we see that the SOP is supposed to run sequentially with steps 1-10, but we see that 90% of the time the actual process is step 1 - 5, then back to 3, then jump to 6 then back to 2, then finally 7-10, any program or automation will need to address this constant rework in order to get the desired improvements.

A major issue with requirements gathering, even in Agile where requirements and adjustments are made as the software is being created, is that if the client doesn't KNOW that there is this rework occurring, it will NEVER be mentioned as a requirement. What I do see, more often than not, is that the new program will attempt to recreate the existing process and end up with the same problems as the previous iteration of the system. The WHY wasn't addressed.

Again, when taking down requirements, there may be requirements that are not discovered, because work that occurs is not seen as part of the process. The process may not state that a lookup has to occur, or that a notification needs to be done, or a subprocess needs to be completed, because it is just something that we "do" not something that is written down as SOP. Many times this is just seen as "work" and not something that is part of any process. The inclusion of these activities can greatly improve the adoption and efficiency of the team. But, if they aren't discussed, they aren't included.

This is where Task Mining comes into the picture. How is that process ACTUALLY completed. Where Process Mining will show what is going on in a given dataset, Task Mining will show the WORK or the additional datasets that need to be included in the Process Mining workload.

Task mining will show that the people doing the work are opening attachments from email, and doing manual copying of data from the attachment into their system. If that is seen as just "work" a major opportunity for efficiency and reduction of input errors is lost. HOWEVER, Task Mining uncovers what is happening and can be pointed out as a requirement for automation or requirements for a new system.

What is also a major benefit is that a comparison can be made between the pre and post application deployment. Most of the time we just ask, do you think this program is helping you? And the users say yes or no, and we can make a rough guess on the total ROI. With Process Mining, you can actually SEE if the application improved or hurt overall efficiency or performance. Using this very measurable metric, multiplying by your average cost per FTE will give you a very real ROI figure.

This direct knowledge of the return is a great way to show that automation helped and makes an appealing argument for additional investment into automation or software.

One of the reasons that the Process Mining/Task Mining aid to requirements are so popular in Europe is that once a company starts doing projects where Process/Task Mining is used, they target very specific goals and measures and see where improvement needs to occur. This means that the company is much more efficient in where they are investing in automation and software.

The WHY is known, therefore their instances of programs that do not solve their problems are greatly reduced.