Process Engine Architecture For Background Activities

Problem

I have an Appway installation with 10,000+ active process instances. I have the impression that the process engine is at full capacity. How can I learn more about the architecture of the process engine?

Solution

Simply put, the process engine makes sure that process tokens are moved from one element to the next element in a timely manner, following the rules specified in the BPMN specification.

This recipe covers the architecture of the process engine, and explains each part of the engine illustrated in Figure 1.

I'll go through each of the components in turn, starting with the process threads (marked as number 1 in Figure 1).

Process threads (Number 1)

Process threads take care of updating process instances. By default, the "process thread" thread pool is set to a size of 5.

In the Process Threads section of the Studio Console, you can see:

• The process threads in the thread pool
• What the process threads are currently executing, if anything
• Thread pool statistics

In the example shown below, there are 5 threads, 4 of which are not active. The thread with ID "3" is active, and currently updating a process instance. The entry for each thread also shows the execution time for that specific job, and the total job count for that thread.

Process Job Queue (Numbers 2 and 3)

Process threads wait on new process jobs being added to the process job queue. As soon as there is a new process job in the queue, one of the threads is activated and handles the process instance.

The process job queue is filled from different sources. The most important source is the background job called "Process Engine". This job is executed every 5 minutes, and adds 1 process job for every existing process instance to the queue. This way, every process instance gets reviewed at least once every 5 minutes.

Process Job Triggers (Numbers 4 and 5)

In some cases Appway knows that a process instance has to be updated at some point in the future. For example, a process token reaches an intermediate timer event and gets blocked there for a period of time (e.g. 7 days). In this case, the process engine creates a process job trigger: a process job with an associated execution date in the future.

You can see the list of process job triggers in the section Process Job Triggers in the Studio Console. In the screenshot below, 5 job triggers are waiting to be processed.

You can monitor the queue under 'Process Jobs' in the Studio Console. However, since most process instances do not need to be modified, and the execution of a process job takes only a couple of milliseconds, the queue should be empty most of the time.

Process Job Scheduler Thread (Number 6)

A single background thread, called the process job scheduler thread, is responsible for finding process job triggers for which the execution date has been reached. The scheduler thread then moves the process job into the process job queue, from where it gets accepted and executed by a process thread.