How to reasonably estimate the size of thread pool

CPU intensive

CPU Intensive is also called computing intensive , Refers to the hard disk of the system 、 Memory performance is relative CPU Is much better , here , Most of the operation of the system is CPU Loading 100%,CPU Read / Write I/O( Hard disk / Memory ),I/O It can be finished in a very short time , and CPU There's a lot more to do ,CPU Loading Very high . Wrote a cpu Dense examples , Always perform auto increment CPU bound In general, the procedure of CPU The occupancy rate is quite high . This may be because the task itself does not need much access I/O equipment , It may also be that the program is multithreaded, so the wait is blocked I/O If yes CPU Intensive application , Then the thread pool size is set to N+1;（ For computing intensive tasks , In a N On a processor system , When the thread pool size is N+1 when , Usually the best efficiency can be achieved .( Even when compute intensive threads occasionally pause due to missing faults or other reasons , This extra thread also ensures that CPU The clock cycles of are not wasted . Excerpt from 《Java Concurrency In Practise》)

Below we write a cpu intensive demo;  Try the time-consuming results of different thread numbers

public class SrcTest {
    static int threadNum = 20;
    final static int taskNum = 200;
    static ExecutorService executorService = Executors.newFixedThreadPool(threadNum);
    static CountDownLatch endGate = new CountDownLatch(taskNum);


    public static void main(String[] args) throws InterruptedException {

        long start = System.currentTimeMillis();

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;for&nbsp;(int&nbsp;i&nbsp;=&nbsp;0;&nbsp;i<&nbsp;taskNum;&nbsp;i++){
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;executorService.submit(cpuCal());
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;endGate.await();
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;long&nbsp;end&nbsp;=&nbsp;System.currentTimeMillis();

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(&quot; Number of thread pools ：&quot;+threadNum+&quot;&nbsp;CPU Check the number ：&quot;+Runtime.getRuntime().availableProcessors()+&quot; It's all over :time:&quot;+(end-start));
&nbsp;&nbsp;&nbsp;&nbsp;}

&nbsp;&nbsp;&nbsp;&nbsp;public&nbsp;static&nbsp;Thread&nbsp;cpuCal(){
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return&nbsp;new&nbsp;Thread(()->{
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;long&nbsp;start&nbsp;=&nbsp;System.currentTimeMillis();
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;// Write something about it cpu Time consuming operations
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;for&nbsp;(int&nbsp;i&nbsp;=&nbsp;0;i<10000000;i++){
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;StringBuffer&nbsp;sb&nbsp;=&nbsp;new&nbsp;StringBuffer();
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;sb.append(i).append(&quot;,&quot;);
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;long&nbsp;end&nbsp;=&nbsp;System.currentTimeMillis();
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(&quot; Threads ID:&quot;+Thread.currentThread().getId()+&quot;;&nbsp; Elapsed time :&quot;+(end-start));
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;endGate.countDown();
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;});
&nbsp;&nbsp;&nbsp;&nbsp;}
}



Execution results

threadNum=1

threadNum=2

threadNum=4

You can see that the number of threads is 4 When , My this 8 In the nuclear machine 4 individual cpu soaring

threadNum=8

open cpu Usage rate

You can see that the number of threads is 8 When , My this 8 In the nuclear machine 8 individual cpu Run at full load

threadNum=14

threadNum=20

Icon results

Experimental system configuration :

Physics cpu Number of cores ：4 

sysctl hw.physicalcpu

Logic cpu Number of cores ：8 

sysctl hw.logicalcpu

Because it's on   hyperthreading   And then there is 4 nucleus 8 Threads

Come to the conclusion

Through the above experimental data , Our analysis shows that in CPU In dense scenes ;  When the number of threads =CPU Logic auditing   When ,  The total time spent is the least ; also   When the number of threads   As you get older ,  The average time of a single task will be larger and larger , This is because the more threads , There will be more thread context switches , Consume some performance ; When   Number of threads  > CPU When counting logical cores ,  The overall time consumption will not be significantly reduced , Instead, it will rise slightly , also   The time-consuming of single task is indeed increasing ;

So the final conclusion :  When CPU In dense scenes ;  Number of threads  =CPU When counting logical cores ,  The overall efficiency is the highest ;

Of course, we can generally set it to  CPU Logic auditing +1 ;  This 1  The reason is that ： Even when compute intensive threads occasionally pause due to missing faults or other reasons , This extra thread also ensures that CPU The clock cycles of are not wasted

IO intensive

If it is IO Intensive application , Then the thread pool size is set to 2N+1

If only one application is deployed on a server and there is only one thread pool , So this estimate may be reasonable , It needs to be tested and verified by itself .

Next, in this document ： Server performance IO Optimize   An estimation formula is found in ：

The best number of threads  = （（ Thread wait time + Threads CPU Time ）/ Threads CPU Time  ）* CPU Number further converts the optimal number of threads  = （ Thread waiting time and thread CPU The ratio of time  + 1）* CPU number

We can come to a conclusion ：

The higher the percentage of thread waiting time , More threads needed . Threads CPU The higher the proportion of time , The fewer threads needed

experiment ( A little )

mixed type

Hybrid tasks   Tasks can be divided into IO Intensive and CPU Intensive task , Then use different thread pools to process .  As long as the execution time of the two tasks is not much different , Then it will be more efficient than serial execution .  Because if the execution time of the two tasks after division is very different , Then the tasks to be executed first will have to wait for the tasks to be executed later , The final time still depends on the task performed after , But also add the cost of task splitting and merging , Do more harm than good .

Why does thread context switching cost performance

The concept of context switching

Let's explain what context switching is (context switch). In a multitasking system , The number of jobs is usually greater than CPU Count . In order to make users feel that these tasks are going on at the same time ,CPU Give each task a certain amount of time , Save the current task status , The current running task becomes ready （ Or hang up 、 Delete ） state , Another selected ready task becomes the current task . after CPU You can go back and deal with the suspended tasks . Context switching is such a process , It allows the CPU Record and restore the status of various running programs , Enable it to complete the switching operation . In the process ,CPU Will stop processing the currently running program , And save the specific location where the current program runs so that it can continue to run later

Loss caused by context switching

Context switching can result in CPU Running back and forth between registers and running queues . This consumption can be divided into two kinds

Reference documents

What is? CPU intensive 、IO intensive ？

Calculate the optimal number of concurrent thread pools

How to estimate the size of thread pool reasonably ？