Summary of software testing tools in 2021 - fuzzy testing tools

~ What is fuzzy testing ？~

Fuzzy testing （Fuzz Testing） It is an automatic software testing technology , Originally by Barton of the University of Wisconsin · Miller at 1989 Year development , Usually used to identify potential vulnerabilities in programs . The core of fuzzy testing is to automatically or semi automatically generate random data and input it into the application program , At the same time, monitor the abnormal conditions of the program , Like a crash 、 Code assertion failed , To find possible program errors , Such as memory leak .

Fuzziness refers to the automatic generation and execution of tests , The random data input in the fuzzy test is called “Fuzz”, The types of random data include ： Super long string ; Random numbers, such as negative numbers , Floating point numbers , Super large number 、 Special characters such as [email protected]#$% And other characters with special meaning , As an input, it may cause an error ;unicode code , Because some programs don't support unicode Of .

Fuzzy testing includes several basic test steps **： Determine the system under test -> Confirm the input -> Generate fuzzy data -> Perform tests using fuzzy data -> Monitor and analyze the behavior of the system -> Output log .**
At present, there are three main fuzzy testing techniques ：

Black box random blur , Random variation of input data in the correct format , Then run the program with the input of these variations , See if you can trigger an exception . It's a simple hack, If an application has never been fuzzy tested , The application of this technology can effectively find vulnerabilities .

Grammar based fuzziness , It is an alternative to fuzzy complex format input , You need to specify the input syntax of the input format , It also specifies which input parts to blur and how . Syntax based fuzzy generators generate many new inputs , Each input satisfies the constraint of syntax coding . Grammar based fuzzing Through the creativity and expertise of the users of the fuzzy generator fuzzing.
White box blur , By Microsoft Research in 2008 First in , This method includes ： Dynamically execute the program under test , And collect input constraints from conditional branches encountered during execution . then , Systematically negate all these constraints one by one , And use the constraint solver to solve , Its solution is mapped to new inputs that execute different program execution paths . Repeat this process using system search technology , Try to scan all possible execution paths of the program . Compared with black box random blur , White box blur is usually more accurate , More code can be run , So as to find more bug.

Fuzzy testing belongs to dynamic testing , It is a cost-effective testing technology to automatically find software security vulnerabilities , Very serious security failures or defects are often found in the software security development life cycle , for example ： collapse 、 Memory leak , Unhandled exceptions, etc . If a software product needs to handle untrusted input or has large 、 Complex data analysis function , Fuzzy testing is very effective . Once a blur generator starts and runs , It can start looking for defects by itself , No manual operation required / Artificial intervention . in addition , Fuzzy testing helps to find defects that cannot be detected by traditional testing methods or manual audit .

Of course , Fuzzy testing cannot provide an overall assessment of a software application in terms of security threats or vulnerabilities , When dealing with security threats that will not lead to application crash, the effect is poor , For example, some viruses 、 worm 、 Trojans, etc , Therefore, it is necessary to combine other security testing methods to ensure the security of software system .

~ Fuzzy test tools ~

The core of fuzzy testing is an effective fuzzy generator （Fuzzer）, Like Barton · The assignment that Professor Miller gave his students ：

The goal of this project is to evaluate the robustness of various UNIX utility programs, given an unpredictable input stream. […] First, you will build a fuzz generator. This is a program that will output a random character stream. Second, you will take the fuzz generator and use it to attack as many UNIX utilities as possible, with the goal of trying to break them.

The goal of the project is to evaluate various factors given an unpredictable input stream UNIX The robustness of the utility .[…] First , You will build a blur generator . This is a program that will output a random character stream . secondly , You will use the blur generator , And use it to attack as many UNIX Utilities , To try to destroy them .

Most randomly generated inputs are grammatically untenable , It will soon be rejected by the tested application . In order to make the fuzzy test go on effectively , Need to increase the probability of obtaining valid input . Data generated by a good fuzzy generator , Will not deviate too far from the expected input , The data they provide has a high probability of being accepted by the application , But it was unexpected , Will trigger the abnormal behavior of the application .

At present, there are many open source fuzzy testing tools to choose from ：

American Fuzzy Lop（AFL）
Radamsa
Honggfuzz
libufuzzer
OSS-Fuzz
boofuzz
Bfuzz
PeachTech Peach Fuzzer
Microsoft Security Risk Detection
ClusterFuzz
Synopsys Defensics Fuzz Testing
Fuzzbuzz

1. American Fuzzy Lop（AFL）
   AFL An open source testing tool , Is the most widely used Fuzzer, This tool inserts the program source code before the program is executed （instrumentation）, In order to obtain the execution of the program in real time during the execution of the program .AFL Using genetic algorithm to mutate the input of the program can inject its own code when the program is running , Then automatically generate test cases for fuzzy testing .

Code coverage is the main indicator used by fuzzy testing tools to increase the likelihood of finding the code path that leads to errors . In the process of execution ,AFL Enter... Into the program under test input, Then get the coverage of the program , Will cover large input Keep it for variation , Then, in the next round of test, output these mutated results to the tested program input, Until the coverage of the program cannot continue to increase for a long time .

AFL The advantage is that it can be easily deployed , Simple configuration , The biggest advantage is the test efficiency . This tool is based on a lot of research on the operation mechanism and most useful results of the best fuzzy Tester , Designed to minimize the time it takes to compile the results returned by the query , Minimize the impact on the system .AFL At present, many branches have been derived .

Official website address ：http://lcamtuf.coredump.cx/afl/

WinAFL：Windows Version of AFL, Use DynamoRIO To insert the pile closed source program to obtain the code coverage information , Support hardware at the same time PT Get coverage information , but PT In fact, the coverage rate is not fully achieved by inserting piles , But it may be faster .

Address ：http://github.com/googleprojectzero/winafl

AFLFast： The accelerated version of AFL,Fuzzing It will be faster than the original .

Address ：http://github.com/mboehme/aflfast

2.Radamsa

Radamsa Is an open source fuzzy testing tool , It is usually used to test the tolerance of programs to format errors and potentially malicious input .Radamsa The biggest selling point is its accuracy .GitLab The developer page on lists the real vulnerabilities found by this fuzzy tester in popular software . It's easy to script , And easy to start and run .

Address ：http://github.com/Hwangtaewon/radamsa

3. Honggfuzz

Honggfuzz It was developed by Google , and AFL similar , Using genetic algorithm to compile , Is a multi process and multi-threaded fuzzy generator , So use Honggfuzz Conduct fuzzing Very fast , The performance in security vulnerability discovery is very outstanding . According to the developer of the tool , It's the only way to find OpenSSL Fuzzy testing tools for key vulnerabilities that led to the release of global security patches .

Honggfuzz Not just for Windows. It can be tested in Linux、Mac even to the extent that Android Applications running in the environment . Because it can work on multiple platforms ,Honggfuzz Provides a complete list of examples and test cases , Developers can use these examples and test cases , You can also modify it according to your own needs .

Address ：http://github.com/google/honggfuzz

4.LibfuzzerLibFuzzer Link to the library being tested , And through a specific fuzzy entry point （ Also known as “ Objective function ”） Provide fuzzy input to the library being tested . And then , The blur generator keeps track of the code areas it touches , And generate variation in the corpus of input data , To expand code coverage . Address ：http://llvm.org/docs/LibFuzzer.html

5. OSS-Fuzz
OSS-Fuzz Continuous fuzzy testing for open source software , Its purpose is to combine the updated fuzzy testing technology with scalable distributed execution , Improve the security and stability of general software infrastructure .OSS-Fuzz Combined with a variety of fuzzy testing techniques / Vulnerability capture technology （ That is, the original libfuzzer） And cleaning technology （ That is, the original AddressSanitizer）, And through ClusterFuzz It provides a test environment for large-scale distributed execution .

Help the open source community create safer applications ,OSS-Fuzz It has been quite successful in achieving this goal .OSS-Fuzz stay 200 More than... Were found in open source programs 14000 A loophole .

Address ：http://github.com/google/oss-fuzz

6.boofuzz

boofuzz The fuzzy testing framework is based on a system that is no longer maintained Sulley Developed . The tool uses Sulley Core code , But committed to continuous improvement .boofuzz As a Python Library installed . Developers added online documentation 、 More support for communication media 、 Scalable fault detection and an easy-to-use interface . The tool will also be used for serial fuzzy testing 、 Ethernet and UDP Broadcasting has been added to the default function .boofuzz The test results can also be expressed in CSV File format export , So that when repairing the detected fault, you can first study the complete list of all triggered problems .

Address ：http://github.com/jtpereyda/boofuzz

7. Bfuzz One of the popular fuzzy testing tools , Found out, forced out Epiphany Web Patch vulnerabilities and can cause Mozilla Firefox A vulnerability that triggers a buffer overflow .

BFuzz It is an input based fuzzy tester , With URL And browser as its input method . In this sense , The tool is much like DAST Tools , Suitable for companies that rely heavily on these things , because BFuzz Use a similar test method , But look for different types of mistakes .

Address ：http://github.com/RootUp/BFuzz

8.PeachTech Peach Fuzzer

PeachTech Peach Fuzzer yes PeachTech The commercial fuzzy testing tool produced by the company , A lot of cumbersome things don't need testers to do it by themselves . Testers only need to use what the company calls Peach Pit Load and configure the fuzzy test engine .

Peach Pit Is a pre written test definition , Covering a range of different platforms .PeachTech Express , Every Pit Contains specifications that apply to specific objectives , For example, the data structure of target intake , The way data flows into and out of the tested device or application, etc . In this way, testers can hardly set , Can focus on their fuzzy test . The user to use PeachTech Create your own Pit It's also very simple. , Make Peach Fuzzer It can also be used in special systems .

because Peach Fuzzer The engine can Peach Pit The reason for programming , Almost all systems are within the test scope of the tool ：Mac、Windows、Linux, It can also be used for fuzzy test network protocol 、 Embedded system 、 drive 、 Internet of things devices , As long as it is a system that can accept commands and is vulnerable to fuzzy input , Can use Peach Fuzzer To test .

Address ：http://www.peach.tech/products/peach-fuzzer/

AI Fuzzy testing uses machine learning and similar techniques to find vulnerabilities in applications or systems . Intelligent constraint algorithm and genetic algorithm are two mainstream intelligent fuzzy testing algorithms , As mentioned above AFL Is to use genetic algorithm . Besides , Some more AI Fuzzy test tools ：

9. Developed by Microsoft Security Risk Detection (MSRD)MSRD It is a dynamic application security testing service driven by artificial intelligence , Can be optimized web Application development cycle , In order to be in bug And security risks are identified and corrected when they are introduced into the code base .

Address ：http://www.microsoft.com/en-us/security-risk-detection/

10.ClusterFuzz

American Fuzzy Lop (AFL) Using genetic algorithms . This toolset is a new cloud based tool Fuzzbuzz At the heart of , It's also Google ClusterFuzz Part of the project .ClusterFuzz Is a highly scalable fuzzy testing infrastructure ,Google Use ClusterFuzz For all Google Fuzzify the product , And use it as OSS-fuzz Backend .

Address ：http://security.googleblog.com/2019/02/open-sourcing-clusterfuzz.html

11.Synopsys Defensics Fuzz Testing

Synopsys Provided Defensics It's a comprehensive 、 Universal 、 Automated fuzzy testing framework , It can help enterprises to be efficient 、 Effectively discover and repair security vulnerabilities in software . This fuzzer based on format generation uses targeted intelligent methods for negative testing ; The advanced file and protocol template obfuscator enables users to build their own test cases .SDK Support professional users to use Defensics Framework Develop your own test cases .

Address ：http://www.synopsys.com/software-integrity/security-testing/fuzz-testing.html

12.Fuzzbuzz

Fuzzbuzz Is a fuzzy test platform , Through automated complex infrastructure management and setup and connectivity to tools , Integrate fuzziness into DevOps In workflow . The platform alerts when defects are found , Data De duplication , And classify them , To eliminate noise and false positives .

Address ：http://www.fuzzbuzz.io/

The purpose of fuzzy testing is to find software security vulnerabilities , It has become the mainstream testing technology of software security testing , so far , This technology has helped people find thousands of security vulnerabilities in various software . and , With “ Software defines everything ” The coming of the age of interconnection with all things , Software security will become increasingly important . therefore , As a software tester , It is necessary to master the techniques and tools of fuzzy testing .

Finally, thank everyone who reads my article carefully , The following online link is also a very comprehensive one that I spent a few days sorting out , I hope it can also help you in need ！

These materials , For those who want to change careers 【 software test 】 For our friends, it should be the most comprehensive and complete war preparation warehouse , This warehouse also accompanied me through the most difficult journey , I hope it can help you ！ Everything should be done as soon as possible , Especially in the technology industry , We must improve our technical skills . I hope that's helpful ……

If you don't want to grow up alone , Unable to find the information of the system , The problem is not helped , If you insist on giving up after a few days , You can click the small card below to join our group , We can discuss and exchange , There will be various software testing materials and technical exchanges .

Click the small card at the end of the document to receive it

Typing is not easy , If this article is helpful to you , Click a like, collect a hide and pay attention , Give the author an encouragement . It's also convenient for you to find it quickly next time .

Self study recommendation B Stop video ：

Zero basis transition software testing ：25 Days from zero basis to software testing post , I finished today , Employment tomorrow .【 Include features / Interface / automation /python automated testing / performance / Test Development 】

Advanced automation testing ：2022B The first station is super detailed python Practical course of automated software testing , Prepare for the golden, silver and four job hopping season , After advanced learning, it soared 20K