Introduction of thread pool and sharing of practice cases

One 、 The context of this knowledge

With Java give an example , What pain points are encountered during the use of threads ？ Introduction of the idea of pooling ,Java How to use thread pool to solve this problem ？Java Thread pool in use . How does the company practice ？

The highlight of this article is , Pool threads in Java How to introduce 、 practice 、 The process of evolution , Systematic elaboration . Whether at work or in an interview Can assist students to explain the problem of thread pool .

1.1、 Thread pain points

With the computer multi-core CPU The evolution of , To make full use of thread concurrency , Developers started using multithreading to improve system performance , Give full play to multi-core CPU Convenience brought by .

Whenever a task needs to be performed , We need to new A thread to run . And what we know is , The creation and destruction of threads requires overhead , Whether it's memory usage or CPU Use , Unlimited thread creation , The end result is running out of memory and CPU, As a result, the system cannot receive new requests , Thus the system is not available . This is a disaster we do not want to see . So we can think about a problem ？ Whether we need to create a thread that can provide request service every time ？ Is there any waste of resources here , And how we can reasonably control the load of the system to ensure the normal water level ？

1.2、 Pool thought

Pool definition , Refer to Baidu Encyclopedia

For storage , The concept of pooling is no stranger . so to speak , The concept of storage pooling does not begin with storage virtualization technology , When the storage is directly connected from the server to the server, the storage is in the form of SAN perhaps NAS As represented by the development of networked storage , The concept of pooling is proposed .

We can simply understand it as , In order to improve the utilization of certain resources , An abstract expression of ideas that can be used in physics or virtual .

For example, we are now talking about threads , How should we apply the idea of pooling ？ We know from the pain point of thread , The main problems we hope to solve are 2 spot ：

• Waste of resources
• Uncontrollable use of resources

We create the thread in advance , And put it in a pool ( In a queue container ） When you need it , I will provide a thread in the container pool to you , When the entire request link is completed , The thread is put back into the container . In this way, I only need to maintain the threads in the pool , You no longer need to create and recycle threads every time . Does it increase the resource reusability of threads , Improve the utilization of resources .

The above has indeed solved the problem of resource waste , What about the uncontrollable use of resources ？ For example, how do I set a reasonable number of threads in the pool ？ If the external traffic request thread far exceeds the load capacity of our system , How to deal with it ？ Here we mentioned that our multithreading is based on the multi-core capability of the server . So our first idea is multi-core capability based on server , Is it possible to calculate a reasonable number of threads ？

The formula comes from 《Java Concurrent programming practice 》

But there are also some problems here , If you use this formula in practice, you will still find that it does not conform to ？ So what problems will we encounter in practice ？ You can continue to look at the company's practices ( Based on the practice scene of meituan ), Here, we will first understand the principle of the thread pool ？

1.3、 The principle of thread pooling

Let's see first ThreadPoolExecutor The principle of the operation process , Here's the picture ：

Based on what we mentioned earlier 2 A pain points

• Waste of resources
• Uncontrollable use of resources

1.3.1、【 Pain points 1】 Waste of resources

give an example ： Suppose you want to place an order to buy a product .

• I want to buy clothes （ Task submitted ）
• A treasure jumps to the payment page （ Task assignment ）-> Call alipay pay （ Thread allocation A）
• Your payment is complete （ Task assignment ）-> Deducting the inventory （ Thread allocation B）
• You can view the order logistics information （ Task assignment ）-> Rookies show that they are in transit （ Thread allocation A）
• You close a treasure , Waiting for the clothes to come

The above simplified process is mainly described as "red" 【 Thread allocation A】 Reuse 2 Time （ It may be used many times in practice /1 Time , Here is a simple example to better illustrate ）, This avoids the need to create 3 Secondary threads and recycling 3 The cost of secondary threads . If we now have multiple users completing the above steps , Then we can give full play to the role of thread pool , Improved reusability .

1.3.2、【 Pain points 2】 Uncontrollable use of resources

From the picture above , We see a buffer execution and a task rejection . In fact, these two represent the solutions to the uncontrollable pain points of resource use we just mentioned . If the current user request load exceeds the real-time processing capacity of the thread pool , Then we can arrange it to enter the cache blocking queue , Just tell it , We can only accommodate at most 1000 people , accord with 1000 Human condition , Can enter and wait . Then it exceeds the maximum capacity 1000 people , What to do ？ To protect the availability of our system , sorry , We refused to provide service . In fact, this is also easier to understand , For example, when you sell goods in seconds , You sometimes find that ," Pro - , Server busy , Please operate later !". In fact, it is the self-protection of the processing capacity of the current system services , Refused to provide service .

1.3.3、 The principle of thread pool

First, the thread pool actually builds a producer consumer model internally , Decouple thread and task , Not directly related , So a good buffer task , Reuse threads .

The running of thread pool is mainly divided into two parts ： task management 、 Thread management .

The task management part acts as the producer , When the task is submitted , The thread pool will judge the subsequent flow of the task ：

（1） Directly request the thread to perform the task ;

（2） Buffer to queue for thread execution ;

（3） Reject the task .

The thread management part is the consumer , They are uniformly maintained in the process pool , According to the task request for the allocation of threads , When the thread finishes executing tasks, it will continue to acquire new tasks to execute , Finally, when the thread cannot get the task , Threads will be recycled .

1.4、Java Practice in

We can look at it Java Thread pool creation class provided in , And its core parameters . There is no explanation here Executors Because it is better for more custom thread pools in practice , In particular, it involves the customization of core parameters .

1.4.1、 Parameter Introduction

The above English has a good explanation , In view of the friendly explanation , Further explanation here ：

• corePoolSize( Number of core threads )
• queueCapacity( Number of task queues )
• MaxPoolSize( Maximum number of thread pools )
• KeepAliveTime（ Thread idle time ）
• allowCoreThreadTimeout( Allow core threads to time out )
• rejectedExecutionHandler( Task reject processor )

1.4.2、 Thread pool default value

• corePoolSize = 1

• queueCapacity = Integer.MAX_VALUE

• maxPoolSize = Integer.MAX_VALUE

• keepAliveTime = 60 second

• allowCoreThreadTimeout = false

• rejectedExecutionHandler = AbortPolicy()

We can initialize and set the custom thread pool according to the thread pool parameters provided above .

1.5、 Company practice

1.5.1、 Problems in company practice

Refer to meituan's 2 Accident cases ：

What problems can we see from the accident cases ？

• It is mainly about the thread rejection policy and the setting of the number of core threads in practice . So how can we reasonably set thread pool parameters ？

What we said from the point of view of the plan , At present, we can have these three schemes , But the above problem is also obvious . In fact, no silver bullet can solve this problem , But we can think about whether there is a compromise to dynamically make thread pools according to different scenarios ？

1.5.2、 Solutions provided by the company

Here we still refer to meituan's solution , Meituan's pain points based on thread pool , It is proposed that thread pool parameters can be dynamically modified + Observability manual processing + Alarm mechanism prevention .

1.5.2.1、 How to set core parameters

• It needs to be determined according to the following values

           - tasks ： Number of tasks per second , Assuming that 500~1000

           - taskcost： Each task takes time , Assuming that 0.1s

           - responsetime： The maximum response time allowed by the system , Assuming that 1s

• Do some calculations

           - corePoolSize = How many threads are needed per second ？

               * threadcount = tasks/(1/taskcost) =tasks*taskcout =  (500~1000)*0.1 = 50~100 Threads .corePoolSize The setting should be greater than 50

               * according to 8020 principle , If 80% The number of tasks per second is less than 800, that corePoolSize Set to 80 that will do

           - queueCapacity = (coreSizePool/taskcost)*responsetime

               * By calculation, we can get queueCapacity = 80/0.1*1 = 80. It means that the thread in the queue can wait 1s, It needs a new thread to execute

               * Remember that it cannot be set to Integer.MAX_VALUE, So the queue will be large , The number of threads will only remain at corePoolSize size , When the mission increases sharply , Cannot open a new thread to execute , The response time will increase sharply .

           - maxPoolSize = (max(tasks)- queueCapacity)/(1/taskcost)

               * By calculation, we can get maxPoolSize = (1000-80)/10 = 92

               * （ Maximum number of tasks - Queue capacity ）/ Processing capacity per second per thread = Maximum number of threads

           - rejectedExecutionHandler： Decide according to the specific situation , The task is not important and can be discarded , If the task is important, some buffer mechanisms should be used to deal with

           - keepAliveTime and allowCoreThreadTimeout Using default can usually meet

1.5.2.2、 How to dynamically set parameters to take effect

Reference resources ：Spring Of ThreadPoolTaskExecutor class （ Yes JDK ThreadPoolExecutor A layer of packaging , Can be understood as decorator mode ） Of setCorePoolSize Method .

That is, native JDK It provides a method to dynamically modify thread pool parameters . Do you find yourself not good at learning at this time , The source code is the best book .

Two 、 summary

I hope this article can help you understand how thread pools are introduced ？ What problem of multithreading has been solved ？ The principle of thread pool is how to solve the pain point ？ What problems will the company encounter in practice ？ And how the company meituan solved the problem ？ I hope this article will help you , Systematize the knowledge .

Reference material ：https://tech.meituan.com/2020/04/02/java-pooling-pratice-in-meituan.html