Performance test summary (I) -- basic theory

With the rapid development of software industry , Modern software systems are becoming more and more complex , More and more functions , In addition to ensuring the basic function test quality, testers need to , More and more people pay attention to the performance . But when it comes to performance testing , A lot of people think directly of Loadrunner. Think LR It's performance testing , It's not right .LR Just a tool for performance testing , But performance testing is more than LR. This article will introduce the basic performance testing theory from the following aspects , The related articles will be updated in the future , Try to combine theory with practice , Let performance test learning is not tool learning .

Catalog ：

One 、  What is software performance

Two 、 Performance in the eyes of different groups

3、 ... and 、 Type of performance test

Four 、 Performance test application scenario

5、 ... and 、 Basic concepts of performance testing

6、 ... and 、 do Knowledge required for performance testing

Text ：

One 、  What is software performance

       Definition ： Software performance is a kind of software Non functional features , It's not about whether the software can perform specific functions , It's what it shows when it's done timeliness .

　　 By definition, performance is concerned with the non functional features of software , So generally speaking, the time for performance test intervention is after the completion of function test . in addition , According to the timeliness in the definition, performance is also an indicator , It can be measured by time or other indicators , Usually we will use some tools or means to check whether some indicators of the software meet the requirements , This is the performance test . 

　　 Performance test definition ： It refers to simulating many kinds of normal functions through automatic testing tools 、 Peak and abnormal load conditions to test the performance of the system .

Two 、 Performance in the eyes of different groups

       Different people because of their outlook on life 、 World view 、 Values and educational background 、 Knowledge system 、 Different experiences in life , Opinions on the same thing or problem may differ . The same is true for software performance , Different people have different perspectives , The points of concern may also be different . Let's take a look at the performance in different people's eyes .

• Performance in the eyes of users

　　

• Performance in the eyes of development

　　

• Performance in the eyes of system administrator

　　

      

• What is the performance in the eyes of the test ？

             

　　　　 Testers are usually a part of software quality control , It's not just about finding bug, Need to be responsible for the quality of the entire software , Performance is also part of quality , So in the eyes of testers, the performance should be comprehensive , Things to consider also need to be comprehensive ：

              1、 Testers need to consider full performance , Include users 、 Development 、 Administrators and other perspectives of performance .

              2、 The tester should pay attention to the surface phenomena such as response time when doing performance test , We also need to pay attention to the essence , For example, the utilization rate of server data that users can't see , Is the architecture design reasonable ？ Whether the code is reasonable and so on .

3、 ... and 、 Type of performance test

1. The benchmark ： When applying a lower pressure to the system , Check the operation of the system and record the relevant data as the basic reference
2. The load test ： It refers to the continuous increase of pressure on the system or the duration under a certain pressure , Until one or more performance indexes of the system reach the safety critical value , For example, some resources have reached saturation state .
3. Pressure test ： Stress test is to evaluate the operation of the system when the system is at or exceeds the expected load , The focus is on the handling capacity of the system at peak load or beyond the maximum load .
4. Stability test ： In the case of loading certain business pressure on the system , Run the system for a period of time , To check whether the system is stable .
5. Concurrent test ： Test multiple users accessing the same application at the same time 、 Whether there are deadlock or other performance problems in the same module or data recording ,

Four 、 Performance test application scenario （ field ）　

1、 Performance test application scenario （ field ） There are mainly ： Capability verification 、 Planning capacity 、 performance tuning 、 Defect discovery 、 Performance benchmarking , The following table briefly introduces and compares the respective uses and features of these scenarios ： 

	Main uses	Typical scenario	characteristic	Common performance The test method
Capability verification	Focus on given hardware and software conditions , Can the system perform as expected	When the required average response time is less than 2 Seconds , How to judge whether the system can support 50 Million users / Days of visits ？	a) It is required to operate in a defined environment b) We need to design test plans and use cases according to typical scenarios , Including operation sequence and concurrent users , Need clear performance goals .	a) The load test b) Pressure test c) Stability test
Planning capacity	Focus on how to make the system have the performance capabilities we require	XX system plans to receive customers in a year xxx ten thousand , Will the system be able to support so many users ？ If you can't adjust the configuration of the system ？	a) It's an exploratory test b) It is often used to understand the system performance and obtain the extended performance	a) The load test b) Pressure test c) Configuration testing
performance tuning	It is mainly used to optimize the system performance	The response speed of a system is getting slower and slower after it has been on line for some time , What to do at this time ？	Change only one configuration at a time , Don't have nothing Stop tuning	a) Concurrent test b) Pressure test c) Configuration testing
Defect discovery	To discover a defect or recurrence of a problem 、 Positioning means	Some defects can only be exposed under high load , Such as thread lock 、 Resource contention or memory leak .	As a supplement to system testing , Used to find concurrent problems , Or to reproduce and locate the existing problems of the system	a) Concurrent test b) Pressure test
Performance benchmarking	Often used in agile development , Agile development process is characterized by small steps and fast steps , Quick trial and error , Short iteration period , Demand changes frequently . It's hard to define perfect performance test objectives , There is no time for detailed performance testing in each iteration , By establishing a performance baseline , By comparing the performance changes in each iteration , Judge whether the iteration has achieved the goal .

2、 Usually in a performance scenario （ field ） We need to use a variety of performance testing methods to test the performance together , The following table shows the correlation between performance test application fields and test methods ：

	Capability verification	Planning capacity	performance tuning	Defect discovery	Performance benchmarking
The benchmark	√		√
The load test	√	√	√
Pressure test	√	√	√	√	√
Concurrent test				√	√
Stability test	√

5、 ... and 、 Basic concepts of performance testing

　　1、 response time

　　　　a） Definition ： The time from the user sending a request to the user receiving the response data returned by the server is the response time

　　　　b) critical path ： The figure below shows once http The path of the request , The request will be sent over the network to web The server does the processing , If you need to operate DB, And then forwarded to the database by the network for processing , Then return the value to web The server ,web The server finally returns the result data to the client through the network .

　　　　

　　　　c) computing method ：Response time = (N1+N2+N3+N4)+ (A1+A2+a3), namely ：（ Network time + Application processing time ）

　　　　d) response time - Load correspondence ：

　　　　　 The inflection point in the picture ：

　　　　　　1、 The response time suddenly increased

　　　　　　2、 It means the limit of one or more resources utilization of the system

　　　　　　3、 In general, inflection point can be used for performance test analysis and positioning

　　2、 throughput

　　　　a） Definition ： The number of client requests processed by the system per unit time

　　　　b） Unit of account ： Generally, the number of requests ／ Second is the unit of throughput , It can be used   Number of pages ／ The stopwatch says .

　　　　　　 in addition , You can also use... From a business perspective   Number of visitors  / God or Page visits ／ God As a unit .

　　　　c） computing method ：Throughput = (number of requests) / (total time).

　　　　d） throughput - Load correspondence ：

　　　　　　

　　　　　 The inflection point in the picture ：

　　　　　　1、 Throughput gradually reaches saturation

　　　　　　2、 It means the limit of one or more resources utilization of the system

　　　　　　3、 In general, inflection point can be used for performance test analysis and positioning  

　　3、 Concurrency number ：

　　　　 Number of concurrent users ： Number of users submitting requests to the system at the same time at a certain physical time , The submitted request may be the same scenario or function , It can also be different scenes or functions .

　　　　 Number of online users ： Number of users accessing the system in a certain period of time , These users do not necessarily submit requests to the system at the same time

　　　　 Number of system users ： Total user data registered by the system

　　　　 The relationship between the three ： Number of system users >= Number of online users  >= Number of concurrent users

　　4、 Resource utilization

　　　　a)  Definition ： It refers to the use of different system resources , Usually measured as a percentage of the maximum occupancy

　　　　b) The server resources that usually need attention are as follows ：

　　　　　　1、CPU： It's like a human brain , It is mainly responsible for the judgment of relevant matters and the mechanism of actual handling

　　　　　　2、 Memory ： A memory block in the brain , Put your eyes , The place where the information collected, such as skin, is recorded , In order to offer cpu Judge , But it's temporary , Fast access , If you shut down or power off, the data here will disappear .

　　　　　　3、 disk IO： Memory blocks in the brain , Save important data （ Keep forever , Shut down or power-off will not be lost , Slow speed ）, In order to use the data again in the future .

　　　　　　4、 The Internet ：

　　　　c） Resource utilization - Load correspondence ：

　　　　　　

　　　　　 The inflection point in the picture ：

　　　　　　1、 The use of a recommendation resource on the server is gradually saturated

　　　　　　2、 In general, inflection point can be used for performance test analysis and positioning

　　5、 Other commonly used concepts ：

　　　　a) TPS：Transactions Per Second, Transactions per second

　　　　b)  Thinking time ： Pause time after each operation of the user , Or the interval between operations , There is no pressure on the server during this time

　　　　c)  clicks ： Every second the user goes to WEB Server submitted HTTP Number of requests . This indicator is WEB Apply a unique indicator :WEB The application is " request - Respond to " Pattern , The user sends out an application , The server has to deal with it once , So clicking on it is WEB The smallest unit of transactions that can be processed by an application . If you define each click as a transaction , Click through rate and TPS It's a concept . Easy to see , The more hits , The more pressure on the server . Click through rate is just a performance reference , It's important to analyze the impact of clicking . It should be noted that , The click here does not refer to a single click operation of the mouse , Because in a single click operation , The client may issue multiple... To the server HTTP request .

　　　　d）PV： Visit one URL, Produce a PV（Page View, Page visits ）, The total of every website every day PV Quantity is to describe a An important indicator of the size of the website .

　　　　　  UV： As an independent user , All pages that visit the site count as one UV（Unique Visitor, User access ）

6、 ... and 、 do Knowledge required for performance testing ：

• Master a programming language
• Master computer principles and operating system knowledge
• Good network foundation
• Master database knowledge
• middleware （apache,tomcat）
• Common bag grabbing tools
• Performance testing tools  

Thank everyone who reads my article carefully , Watching the rise and attention of fans all the way , Reciprocity is always necessary , Although it's not very valuable , If you can use it, you can take it ：

①　2000 Multiple copies Python e-book （ There should be both mainstream and classic books ）

②　Python Standard library information （ The most complete Chinese version ）

③　 Project source code （ Forty or fifty interesting and classic training projects and source code ）

④　Python Basic introduction 、 Reptiles 、web Development 、 Big data analysis video （ Suitable for Xiaobai to learn ）

 

⑤ Python Learning Roadmap （ Farewell to bad learning ）

The information above   In my QQ In the technology exchange group （ Technical exchange and resource sharing , The ad came in and broke your leg ）

You can take it by yourself , Group number 953306497（ remarks “csdn111”） The free information in the group is the essence of my more than 10 year test career . There is also a great God of the same trade to exchange technology .