Continuous testing | key to efficient testing in Devops Era

The author of this article ： Cheng Shengcong - CODING The product manager

The challenge of the times to testing

Testing has always been an important guarantee for the quality of software development process , And in the traditional R & D model , Testing activities are always lagging behind in the software life cycle , I believe we will not be unfamiliar with the following dialogue scenes ：

1. After planning the needs ： “ This version of X Function is key , The test must not be missed ！” “ well , We've made notes in the requirements list .”
2. Development process ： “ Is the function finished this time ？” “ The development is complete , But it's not finished yet .”
3. It's near the release date ： “ Is the function online ？” “ It's time for the pre release environment , Regression testing is still in progress 、 After that, comparative analysis will take time .”

The problems mentioned above ：“ What was measured ”、“ Have you finished the test ”、 and “ Is it fast ”, It can be said that the team often faces “ Three tortures of the soul ”. and With agile & DevOps Mode in the software industry promotion landing , More frequent delivery has exacerbated the industry's concerns about testing , Inefficient testing is often the leading cause of delivery delays , The test link also becomes the enterprise to carry on DevOps The biggest bottleneck of transformation .

In order to meet such a challenge ,“ Keep testing ”（ perhaps “ Agile testing ”） The concept has been put forward and gradually become the inevitable pursuit of the industry . Continuous testing can meet the following core requirements ：

1. What was measured ： Can't achieve 100% Under the premise of regression test coverage , Divide test subsets based on business value .
2. Have you finished the test ： In the process of continuous delivery , Test on demand and provide quick feedback .
3. Is it fast ： Make the test run fast enough .

What is continuous testing ？

The definition from Wikipedia ： The process of performing automated testing in a software delivery pipeline , The goal is to get immediate feedback on the business risks of pre release software . As shown in the figure below, the Mobius ring of continuous delivery ：

indeed , The above definition fully emphasizes the importance of automated testing , This is the foundation of continuous testing . But back to “ Improve efficiency through continuous testing ” On our ultimate goal , Only improve the single point efficiency of test execution , It is not enough to reflect the essence of the change of testing concept brought by continuous testing . From the perspective of overall test efficiency ,DevOps Another twin concept “ agile ” What the pattern describes “ Test within iteration “（in-sprint testing） perhaps “ Agile testing ” It becomes a better supplement ： Continuous testing should be a basic and continuous activity 、 Throughout the software delivery cycle . come from Jenkins The picture of the community better reflects this concept ：

How to realize continuous testing ？

Continuous testing changes the traditional post testing mode , Extend testing activities to every stage of the software development life cycle .

1. Demand planning phase , Plan tests early , And strategically define test subsets . First , Start planning tests in advance from the requirements analysis stage 、 Write test cases , To achieve appropriate demand coverage . This needs to be practiced helpfully, including ATDD、BDD, In especial TDD Hard to land teams can try ATDD. secondly , Be aware of optimizing test coverage . Testing should not be blindly pursued 100% Cover , It's a testing strategy based on business risk and value （Risk-based Testing）,“100% Covering high priority requirements ” Far ratio “80% Covering all requirements ” It's worth it .

2. Iteration in progress , Push the test to the left （Shift-left）, Realize the parallel work of test and development . Test execution should be carried out early in the software development life cycle , A variety of engineering practices can help teams move left ： such as Attach importance to test review , Basic support through unit testing , Development and automated testing based on interface definition , Code scanning is introduced to judge whether the coding specifications and engineering standards are met . So in the iteration cycle , We can continuously write integration test cases around requirements , And keep pace with development , Provide necessary test support for development , Make the test and development work synchronous .

3. Iteration in progress , Provide complete test environment and correct test data in a convenient way . all the time , Creating a test environment close to production and quickly preparing desensitized data are two important challenges for the team . Now As cloud native technology matures , In especial Docker Technological development , Make it possible to build and destroy environments on demand . But the management of test data is still a problem , Basic data such as account information 、 Environmental information, which is easy to standardize, has a better solution in the industry , It's a big step forward . However, business data is lack of good enough business abstraction due to the variety of scenarios , It's still in the basic stage of relying on the framework for process specification , Development based on interface definition to achieve Mock Service can also improve process efficiency .

4. After application deployment , Focus on testing right （Shift-right）. The traditional waterfall model takes deployment as the next phase of testing , It means that the application is launched 、 The end of the test is after the quick verification of the function . and Continuous testing doesn't mean that the release is complete and the test exits , The emphasis is that after the version goes online 、 Continue to pay attention to data monitoring and early warning of production environment , Find problems in time and follow up to solve them , Minimize the impact . And using production data can bring real value to the development process ： For example, copy production data for desensitization to prepare test data , The results of service access data analysis can also provide optimization guidance for testing in the development process 、 So as to adjust the test and form a better smoking and regression test strategy and so on . The practice of moving right includes data analysis 、 Grayscale / Release of canary 、 Online real-time monitoring 、 User feedback tracking process and so on .

Besides , We should pay attention to the precipitation of data in the process of continuous testing , And then we constantly optimize our behavior based on data indicators , So as to achieve DevOps The team culture of continuous improvement . With the development of agility and DevOps The constant spread of culture , R & D teams are bound to expect shorter iterations , Improve the quality of software development more effectively , Deliver business value faster .

CODING Adhering to the principle of providing one-stop service for enterprise R & D team DevOps The idea of a solution , Through the powerful test management function , Assist the R & D team to take testing as a basic activity throughout the whole process of software delivery , Greatly shorten the software delivery cycle , Let testing and R & D iterate simultaneously , Implement continuous testing , Help the team return to high quality delivery .