brief introduction

Look again at the XXL job architecture diagram:

The dispatching center mainly provides two functions: system management and task scheduling. The rest are auxiliary functions.

• As shown in the figure, system management includes task management, actuator management and log management. It also provides a management interface.
• Task scheduling is responsible for pulling tasks from the data center and delivering them to the executor according to the execution time.

Composition and structure of scheduler

Two core threads

After the dispatching center is started, the following two threads will be started:

1. schedulerThread
   Schedulerthread mainly does the following two things:

• From the data center (db), i.e. XXL_ job_ The info table scans the tasks that meet condition 1. The restrictions of condition 1 are as follows:
  • Task execution time is less than (current time + 5 s)
  • Limit the number of scans. This value is dynamic and will be related to the number of threads in the "fast and slow thread pool" mentioned later.

  count = treadpool-size * trigger-qps  (each trigger cost 50ms, qps = 1000/50 = 20) 
  
  treadpool-size = (getTriggerPoolFastMax() + getTriggerPoolSlowMax()) * 20
  // After reading the introduction of fast and slow thread pool, it will be easier to understand when you look back here

• The scanned tasks are divided into the following three categories:
• 

1. ringThread

The function of ringThread is to constantly read the tasks that need to be executed at the current time point from the "container", and the read tasks will be handed over to a thing called "fast and slow thread pool" to pass the tasks to the scheduler for execution.

Time wheel

The above ringThread and container together form a time wheel.

In short, the time wheel realizes the function of "delaying execution". Its function in XXL job is to send the tasks that have not reached the execution time "to the" actuator "one by one through the" fast and slow thread pool "according to the expected time.

• The data structure of time wheel is generally {array + linked list, and jdk1 HashMap in 7 is a truth. Each node in the linked list is a task to be executed.

• The time wheel in XXL job can be vividly described as the following figure, just like a clock with only a second hand.

• During the running process of the ringThread thread, a scale will be swept every second. Assuming that there is a job linked list at the current scale position, all jobs in the linked list will be taken out and finally thrown to the {fast and slow thread pool.

• Of course, in order to avoid taking too long to process, XXL job will cross the scale and check one more scale forward; That is, when the pointer points to 2s, the tasks at 1s and 2s will be read out at the same time.
  

Fast and slow thread pool

After the tasks mentioned above are scanned from the data center, they will be thrown into the fast and slow thread pool. The definition of fast and slow thread pool is as follows:

fastTriggerPool = new ThreadPoolExecutor(
                10,
                XxlJobAdminConfig.getAdminConfig().getTriggerPoolFastMax(),
                60L,
                TimeUnit.SECONDS,
                new LinkedBlockingQueue<Runnable>(1000),
                new ThreadFactory() {
                    @Override
                    public Thread newThread(Runnable r) {
                        return new Thread(r, "xxl-job, admin JobTriggerPoolHelper-fastTriggerPool-" + r.hashCode());
                    }
                });

        slowTriggerPool = new ThreadPoolExecutor(
                10,
                XxlJobAdminConfig.getAdminConfig().getTriggerPoolSlowMax(),
                60L,
                TimeUnit.SECONDS,
                new LinkedBlockingQueue<Runnable>(2000),
                new ThreadFactory() {
                    @Override
                    public Thread newThread(Runnable r) {
                        return new Thread(r, "xxl-job, admin JobTriggerPoolHelper-slowTriggerPool-" + r.hashCode());
                    }
                });

 

It can be seen from the above that the fast and slow thread pool includes two thread pools fast and slow. When a job is submitted to the fast and slow thread pool, the fast and slow thread pool will select one of them to perform subsequent operations according to some conditions.

The functions of fast and slow thread pool are as follows:

Realize thread pool isolation: the scheduling thread pool is isolated and split, and Slow tasks are automatically degraded into the "Slow" thread pool to avoid exhaustion of scheduling threads and improve system stability;

What is a slow task?

If a task is executed more than 10 times in one minute, it is classified as a slow task

When a specific fast or slow thread pool receives a scheduled task, it will trigger the {executor} to complete the execution logic of the task through RPC remote call.

Source code entry

com.xxl.job.admin.core.thread.JobScheduleHelper#start

com.xxl.job.admin.core.thread.JobTriggerPoolHelper#addTrigger

com.xxl.job.core.biz.client.ExecutorBizClient#run

summary

This paper is based on XXL job v2 The source code of x analyzes the composition structure of XXL job scheduler and how the scheduling center triggers tasks.

The scheduler mainly includes the following modules:

• schedulerThread: responsible for scanning the tasks to be performed from the data center
• ringThread: responsible for accurately controlling the tasks expected to be performed
• Fast and slow thread pool: the two thread pools are packaged to perform the scheduling process of fast tasks and slow tasks respectively.

Time wheel appendix