Key Takeaways from My Early Process Mining Journey
It has been a year since I started my process mining discovery journey. Over the past 12 months, I’ve faced various challenges, experienced several ‘aha’ moments, and gained a great deal of valuable learning. After applying process mining techniques to millions of event logs for numerous processes, I wanted to share insights on what worked well and where I encountered difficulties.
In a previous article, I summarized the six steps for applying process mining to business operations. While I’m not here to introduce a "seventh step," I do want to dive deeper into some of the critical lessons I learned—often through failure—along the way.
Key lessons learned from my early process mining journey:
1. Progress over Perfection
Just like with any data analytics application, the most time-consuming and often tedious part of process mining is the data pre-processing step, where datasets are cleaned, modified, and transformed. However, there’s a key difference between process mining and other data analytics projects when it comes to preparing the data.
In traditional data analytics, the goal is usually to create a near-perfect dataset before beginning the analysis, to guarantee quality output or prevent double handling. Process mining, however, requires a more agile approach to data preparation. While basic data cleansing (such as removing duplicates, standardizing data, and formatting timestamps) is necessary, I think the most effective strategy is to build a preliminary process model with the initial dataset and use it to uncover data quality issues early on. This process should be repeated until both the dataset and the process model reach optimal quality.
There are two main advantages to this approach:
I believe that having multiple iterations of the process model and revisiting the data set when necessary provides a shortcut to the quality outcomes.
领英推荐
2. Understanding the Process Variants
Once the optimal process model is built, the next step is to analyze the variations from the main process flow, often referred to as process variants to the "happy path."
Process variants are deviations or alternative paths that can emerge within a single business process. These variations usually appear due to differences in how tasks are performed, the complexity of the workflows, or exceptions in the standard process.
For processes with many steps, the number of variants can quickly become overwhelming, especially with large data sets. Even slight deviations, such as skipping a step or changing the order of tasks, can create new variants. As the number of variants grows, it becomes difficult to identify which are relevant and which are just noise or rare exceptions. Some variants may be harmless deviations, while others may signal inefficiencies or bottlenecks. The challenge lies in distinguishing which variants are meaningful and worth investigating versus those that are inconsequential to the overall process performance.
Interpreting why certain variants occur is also difficult. Process mining reveals what the variants are, but it doesn’t always explain why they happen. Understanding the root causes of process variations may require further investigation into external factors (e.g. human behaviour or system limitations) and contextual factors (e.g. customer specific needs or location). I had to invest a significant amount of unplanned time and effort to better understand the process variants.
3. Analysis Paralysis
The process mining tools provide a powerful and unique capability to reveal the details of how a business runs certain operations. Discovering and mapping out every detail of a process can lead to a comprehensive understanding of the process but may leave little time to act on the insights gained.
Focusing too heavily on the discovery phase can lead to "analysis paralysis," where too much time is spent analyzing and refining the process model without making decisions on what needs to be improved. This can cause teams to overlook opportunities for quick, impactful improvements. I believe by dedicating more time to improvement early on, organizations can often resolve simple bottlenecks or inefficiencies that deliver immediate benefits, rather than getting bogged down in perfecting the process model.
These early lessons shaped my approach to process mining and helped me refine my techniques for more effective outcomes. I hope these insights can help others avoid some of the bumps I hit along the way and make their own process mining journey a little smoother.