What Process Mining Is Not¶
In addition to understanding the typical Process Mining Use Cases it is also important to understand what process mining is not. By placing clear boundaries, the concept of process mining becomes less vague and you will be better able to explain the differences compared to existing approaches to your colleagues. This, in turn, will help you to manage expectations of what process mining can do.
What you will learn:
- What makes process mining different from other tools, techniques, or methods that people might think of when they see or hear process mining for the first time.
- How process mining can be combined with these other approaches.
BI or Reporting Tool¶
Process mining is an analysis tool while BI-dashboards are for monitoring and reporting. These are different use cases. A process mining analysis can result in a new KPI that then should be monitored, but it can also lead to a process change (see Possible Outcomes).
In a dashboard or reporting environment, you focus on a limited number of characteristics that you want to see every day. That’s what the ‘key’ in Key Performance Indicator (KPI) stands for. The idea is that there are a few important, often aggregated metrics that indicate whether your process is working as expected. In contrast, with process mining as an analysis tool you explore the process from the ground up and into many different directions. The goal is to first understand the process in detail.
Process mining is often used in interactive workshop sessions, where you analyze the data together with a process expert. The process expert can point out things you might otherwise miss and, together, you can immediately answer any question that you come up with right there and then. With a dashboard tool, you can’t have such interactive analysis sessions but that’s also not their purpose. Their purpose is monitoring, not analysis.
Process mining tools and BI-dashboards can be best kept separate, because they follow different paradigms and fulfill different purposes. This means that it is not typically the BI department that should have the responsibility for the process mining initiatives in your company (unlike they have an active role in supporting process analysis projects in your company), because process mining is IT Project and it is very different from setting up dashboards and reports according to the requirements of the business units. This also means that process mining does not replace the existing dashboard tools that most companies have already in place.
Process mining tools and BI-dashboards are complementary and there are many scenarios, where they can be used together. For example, if you want to set up a new monitoring environment, then process mining can provide valuable input about which KPIs should be monitored in the future and where the measure points should be placed in the data. Or, if you already have a BI tool in place, you might find yourself in a situation where some KPIs are out of bounds. To identify the root cause of this problem, you then often have to go to the process level. A process mining analysis can help you to investigate the process in more detail and to find out what is going wrong.
Process Modeling Tool¶
With process mining, the process maps are automatically created from the transaction data in the information systems. In contrast, in a process modeling tool the process map is manually drawn.
Business Process Management (BPM) professionals often refer to the manual process mapping that is performed through classical workshops and interviews as process discovery. This can be confusing for people learning about process mining, where the automatic construction of the process based on the IT data is also referred to as discovery.  For a while, some people tried to use the term automated process discovery to distinguish process mining from the traditional way of manual process mapping, but the term has not caught on and process mining is now used in most situations.
Process mining and process modeling are used for different purposes. With a process modeling tool, the perceived ‘As is’ process is described and the process is documented. In this documentation, often various dimensions are captured. For example, the responsibilities and boundaries of organizational units are illustrated via swim lanes. The decision logic at choice points in the process can be captured via decision rules, etc. Often, the documented processes are then published to the rest of the company through a portal. The purpose is to create a shared understanding of the processes in the organization.
While manually modeled processes create a reference point for the typical process, for example, to help explain new employees how the process should work, process mining discovers what is actually happening based on the history logs from the IT systems. This ‘As is’ reality as discovered by process mining is typically much more complex than if that same process would be captured in a traditional process modeling exercise. With process mining, you see a lot more detail and every little exception that occurred in the process. You will realize that many processes allow for much greater degrees of freedom and flexibility than you might have thought.
This is not necessarily a bad thing: For example, a lot of the service processes are knowledge-intense processes and enabling people to get the job done may be better than forcing them into a single route that won’t fit their case half of the time (or to drive them to work outside of the system). The purpose of process mining is to help you understand rather than document this complex reality. You will be able to navigate and gradually break down that complexity . By looking at the actual process reality, process mining helps you to understand what is going on and to analyze the process for improvement potential and compliance problems.
Process mining and process modeling are complementary. While documented process maps do not typically seek to cover every little exception, they can provide you with valuable information about how the process is expected to run. In return, if your organization has not yet documented their processes at all but would like to do so, you can analyze and simplify the mined process to a point, where you can export it into a process modeling environment for further documentation (see also Exporting Process Maps as XML Files).
New Improvement Methodology¶
There is a methodology about how to run a process mining project and this handbook will teach you the basic concepts that you need to understand to use process mining successfully. For example, you need to understand how you can detect data quality problems, how you should interpret the results from your process mining analysis, and which questions you can ask in the first place. But process mining does not replace any of the process improvement methodologies that you may already be using.
Process mining can be used for many different use cases (see Process Mining Use Cases). Each of these use cases have their own methods and procedures. For example, a Lean Six Sigma practitioner may be using the DMAIC framework to run their projects. And an auditor is following the protocols and methods they have been taught in their audit education. Process mining does not replace any of these methodologies.
Instead, process mining fits into these approaches like a piece of a puzzle. In fact, you often need an overarching project approach, like taught in a Lean Six Sigma education, to decide how to prioritize possible improvement ideas, how to deal with resistance to change, and how to actually implement the process improvement to realize the benefits from your process mining analysis. So, having a traditional business analyst on your team next to the process mining expert can be a great advantage.
Process mining is strictly complementary to existing process improvement methods. It makes it possible to perform a more detailed and a more objective ‘As-is’ process analysis compared to traditional (manual) process analysis methods. But it does not replace the overall improvement methods themselves. As a part of your process mining journey, you should think about your own use case, the framework you are currently using, and how process mining fits into the current approach of your organization.
Contrary to a new information system or BI-tool, process mining does not need an implementation project. You only need a data export and you can start.
For example, BI-dashboards are IT projects, where a configurator in the IT department sets up the views that the consumers want to see every day. A particular process is connected in a fixed manner. In setting up these views, many decisions are made that require a detailed understanding of the data and the underlying processes.
With a process mining tool like Disco, such a distinction between configurator and consumer does not exist. The process mining analyst is independent and in full control. The IT department extracts the data for you (see Data Requirements). As soon as you have received the data, you can immediately start analyzing your process without any programming skills or IT knowledge. And you can look at the process from many different perspectives.
Process mining is not an IT project. It’s not about buying a tool and then, once it is “implemented”, the tool runs by itself, producing reports. Instead, process mining is a discipline that needs time to be built up as an area of expertise within the organization. You need a human analyst to interpret the outcomes of the process mining analysis, to suggest improvement options in the context of the domain knowledge, and to actually do something with these insights that will lead to the benefits of your process mining project.
There may be IT projects that are spawned from your process mining initiatives after all. For example, you may be implementing ETL routines that prepare and pre-process your data for repeated analyses. You may decide to improve the data warehouse history infrastructure because of data quality problems that you detected. Or you might request to add a new KPI to your monitoring environment. But process mining itself is not an IT topic, it is a method to look at your data from a process perspective with the help of the process mining tool.
Data Mining, AI, or Machine Learning Tool¶
Because of the name ‘Process Mining’ people often think that process mining is a sub field of the data mining area. You could see it that way, but historically process mining is not part of the data mining research field. Instead, process mining emerged from the business process management research area.
This means that process mining techniques cannot be found in classical data mining and statistical tools. Process mining offers additional analyses and focuses on the process perpective. Although some of the data mining and machine learning approaches analyze process patterns, they do not offer a full end-to-end process discovery (see also How Process Mining Compares to Data Mining). At the same time, process mining tools do not replace the data mining tools that your company may already be using.
While the usage of data mining requires an expert who can select the right algorithms, tune the parameters, and train models for a specific problem, process mining is a generic tool that can be learned and successfully used by a process practitioner without a data science PhD.
Process mining and data mining are complementary and can be used together. For example, if your data set only contains an unstructured text field that is filled by the employees with arbitrary notes and comments, then this field is not suitable to be used as the activity name for your process mining analysis. You can use text mining tools to extract common activity names from this free-text information field in a pre-processing step and then apply process mining afterwards. Furthermore, after you have used process mining to discover and visualize process problems, you may be using data mining algorithms such as decision trees to detect correlations to underlying data fields or to make predictions.
People who see the Process Animation sometimes confuse process mining with simulation. The reason is that many of the process modeling and simulation tools have a “play out” functionality that visualizes the process flow in a dynamic way, so it looks similar. But the animation in process mining is different, because it re-plays the actual process that took place (see also How Process Mining Compares To Simulation).
Simulation tools are typically based on manually created models to play out various ‘what if’ scenarios. The idea is that you want to test alternative process improvement options in a simulation environment before investing a lot of money to implement them in the real world. However, to use simulation successfully, you need to have a good model reflecting the real world as a starting point. Otherwise, all of the analyses that you are doing in the simulation are worthless. The difficulty to come up with a good model to start with may be one of the reasons why simulation - although commercial simulation tools have been available for several decades - is not more widely used today.
The fact that simulation starts with a model while process mining starts with the data leads to two advantages that process mining has over simulation:
- The usefulness of simulation stands and falls with the validity of the model. This means that all relevant influences on the process behavior need to be known and captured. For simple and stable processes this can work, but for many complex processes it comes close to “modeling the world”.
In process mining, bottlenecks and problems do not need to be known in advance. They can be observed and investigated based on factual data. “Why is work always accumulating before activity X?” The root causes may lie in the incentive structure, people issues, overload, or the weather.
- In simulation, everything needs to be captured in a single model. In addition to the requirement of being “complete” this adds to the complexity because it is always easier to model different aspects of a process in isolation instead of all the interdependencies.
In process mining, multiple models can be generated to gain insights into different perspectives of the process, such as process flow, organizational, data flow, and so on (see also Change in Perspective with Process Mining). These models can be separate and just as detailed as they need to be to better understand the problem.
Process mining is complementary to simulation and can help to create better simulation models as a starting point. You can apply process mining to discover the actual process flows and use them as the base model in your simulation tool. Process mining can then help you to fill in the parameters (execution times, waiting times, utilization levels, distribution of arriving new cases, etc.) in the simulation model based on factual information. However, modeling - and thus simulating - human behavior remains hard. If you want to use simulation, we recommend to read Prof. Wil van der Aalst’s Business Process Simulation Survival Guide for a research overview about the state of the art in process mining and simulation.
Just for Some Processes or IT Systems¶
The beauty of process mining is that it is applicable to any process where you can find a case ID, an activity name, and timestamp information in the data of the supporting IT system (see also Data Requirements).
Sometimes, people think that process mining can only be used to analyze BPM or ERP systems, where the processes are fairly standardized. But this is not the case. Process mining can be particularly valuable for the analysis of systems that are not process-aware like, for example, click streams on a website or legacy systems.
In fact, data from all of the following types of systems (and many more) have been analyzed with process mining in the past: Workflow systems, IT Service Management (ITSM) systems, Customer Relationship Management (CRM) systems, Enterprise Resource Planning (ERP) systems, Manufacturing Execution Systems (MES), Product Lifecycle Management (PLM) systems, Business Process Management (BPM) systems, Warehouse management systems, Custom-built legacy systems, Click-streams from websites, technical logs from API calls, Data Warehouses, etc. Even data that was recorded in Excel, or manually collected by scanning barcodes from a piece of paper, can be analyzed with process mining.
Typically, the data is not provided by the IT system in a ready-made “log file”. Instead, you will need to request the data from your IT department (see Checklist: Prepare Your Own Data Extraction). Sometimes, the data comes back in a format that that you can use right away. And sometimes it is not so easy and additional data preparation steps are necessary.
However, once you start looking for process mining data, you will find it in many places around you!
Whenever there is a new technique, and a certain hype around it, people tend to over-promise what it can do.
Certainly, process mining seems like magic the first time you see it. You just import some data, and it magically constructs the process map for you! That’s great, but it is also important that you understand the limitations for your own projects (and manage the expectations when you explain process mining to others).
Unrealistic expectations typically come in at the two ends of process mining:
- The input side: Process mining does not automatically identify and collect the data from the IT system for you.
Sometimes, people think that process mining will “crawl” through your IT system and automatically find all the processes that are supported, and all the data that is related to these processes.
This is not what process mining is about. You do need an understanding of the process and the IT system to extract the data from it in the right way. Process mining happens in two phases: The first phase is about the data extraction and data preparation (see Data Requirements). The second phase is about the analysis of the data. Process mining tools cover only the second phase, the analysis phase.
- The output side: Process mining does not automatically identify improvements and suggestions for your process.
This is not possible, because you need an understanding of the process and domain knowledge to interpret the process mining results correctly. For example, sometimes a loop pattern in the process is an indication of excessive rework, sometimes it is a good thing, and sometimes it does not mean anything. Process mining is a discipline and only the process mining analyst can make these distinctions and derive the right actions from the analysis.
To think that an AI algorithm can make those decisions for you is an illusion. Don’t believe the self-proclaimed “thought leaders” who claim otherwise and make sure you build up your own understanding of what process mining can and cannot do.
|||In fact, there are even more meanings of process discovery, which makes matters even more confusing, as discussed in the following article: https://fluxicon.com/blog/2010/10/pitfalls-1/|
|||Get an overview of nine powerful strategies to handle complexity in process mining in the following article: http://fluxicon.com/blog/2015/03/managing-complexity-in-process-mining-quick-simplification-methods/|