Using qtest for data set test performance test GUI test

The previous article described how to use QTest Unit test , Actually QT The test framework function of is more than that , We can also go through QTest Implement some more complex tests , Here is how to pass QTest Implement dataset testing 、 Performance testing 、GUI test .

Dataset testing

For many algorithm modules and business logic modules , The logic of the test case is the same , The difference is only that the input data is different . To reuse logic modules , Avoid a lot of redundant test code , We can use QTest The data set module is provided to test the corresponding program . A dataset is a two-dimensional data table , Each row of the table is a test , The table contains the name of the test 、 Test data input 、 And the expected output . The structure of the test table is as follows :

index	name	number1	number2	result
0	two_positive	655	655	1310
1	two_negative	-60	-60	-120
2	zero_negative	0	-60	-60
3	zero_positive	0	80	80
4	zero_zero	0	0	0
5	positive_negative	-60	80	20

All in the test case class private Slot functions are treated as test cases , We add the following test cases :

class MyPlus
{
    
public:
    MyPlus() = default;
    ~MyPlus() = default;

    int calculate_tow_number(int number1, int number2)
    {
    
        return number1 + number2;
    }
};
class CustomTest : public QObject
{
    
...
private slots:
    // structure testPlus Data set of 
    void testPlus_data();
    
    //testPlus test 
    void testPlus();
...
};
#endif // CUSTOMTEST_H

testPlus_data() Function is responsible for giving test cases testPlus Construct data set , The names of the data set constructors corresponding to the test cases are : Use case function name _data. For example, the name of your test function is A() Then the corresponding dataset constructor name is A_data(). The corresponding implementation of the function is as follows :

void CustomTest::testPlus_data()
{
    
    QTest::addColumn<int>("number1");
    QTest::addColumn<int>("number2");
    QTest::addColumn<int>("result");

    QTest::newRow("two_positive") << 655 << 655 << 1310;
    QTest::newRow("two_negative") << -60 << -60 << -120;
    QTest::newRow("zero_negative") << 0 << -60 << -60;
    QTest::newRow("zero_positive") << 0 << 80 << 80;
    QTest::newRow("zero_zero") << 0 << 0 << 0;
    QTest::newRow("positive_negative") << -60 << 80 << 20;
}

void CustomTest::testPlus()
{
    
    QFETCH(int, number1);
    QFETCH(int, number2);
    QFETCH(int, result);
    MyPlus plus;
    int ret = plus.calculate_tow_number(number1,number2);
    QCOMPARE(ret, result);
}

First we pass QTest::addColumn Add columns to the data table and specify the name of each column and the corresponding data type . After specifying the column , We can go through QTest::newRow, Add test data to the corresponding data table . Note the name of each test case , The test framework is used instead of the test case , Therefore, when creating a table, you do not need to specify the corresponding columns separately .

In test cases , We can go through QFETCH The macro will test the data in the column according to its name and type . The frames are automatically added in the order they are added , Take out the data in turn for testing and automatically complete the iterative process . The output of the test case is as follows :

PASS : CustomTest::initTestCase()
PASS : CustomTest::testPlus(two_positive)
PASS : CustomTest::testPlus(two_negative)
PASS : CustomTest::testPlus(zero_negative)
PASS : CustomTest::testPlus(zero_positive)
PASS : CustomTest::testPlus(zero_zero)
PASS : CustomTest::testPlus(positive_negative)

Performance testing

For some algorithm modules , We need to test whether the performance is consistent under different data scales , Will there be performance degradation . This requires using the framework to test the execution speed of the corresponding module under different data scales , Is it linear . Let's take a simple sorting algorithm as an example to illustrate the usage of performance testing :

class NumberSort
{
    
public:
    NumberSort() = default;
    ~NumberSort() = default;
    void sort_number(QList<int> number_list)
    {
    
        qSort(number_list.begin(),number_list.end());
    }

};
class CustomTest : public QObject
{
    
...
private slots:
    // Performance testing constructs data sets 
    void perform_test_data();
    // Performance testing 
    void perform_test();
...	
};
#endif // CUSTOMTEST_H

Performance testing , We also need to construct the corresponding data set according to the business .

void CustomTest::perform_test_data()
{
    
    QTest::addColumn<int>("loopCount");
    QTest::newRow("loopCount: 10") << 10;
    QTest::newRow("loopCount: 100") << 100;
    QTest::newRow("loopCount: 1000") << 1000;
    QTest::newRow("loopCount: 10000") << 10000;
    QTest::newRow("loopCount: 100000") << 100000;
}
// Test the performance of the sorting algorithm in different scales 
void CustomTest::perform_test()
{
    
    QFETCH(int,loopCount);
    qsrand(QDateTime::currentDateTime().toTime_t());
    QList<int> number_list;
    for(int index=0; index<loopCount; ++index)
    {
    
        number_list << qrand() % 10000;
    }
    NumberSort sort;
    QBENCHMARK {
    
        sort.sort_number(number_list);
    }
}

The corresponding execution result is as follows :

PASS : CustomTest::perform_test(loopCount: 10)
RESULT : CustomTest::perform_test():"loopCount: 10":
0.0013 msecs per iteration (total: 87, iterations: 65536)
PASS : CustomTest::perform_test(loopCount: 100)
RESULT : CustomTest::perform_test():"loopCount: 100":
0.018 msecs per iteration (total: 74, iterations: 4096)
PASS : CustomTest::perform_test(loopCount: 1000)
RESULT : CustomTest::perform_test():"loopCount: 1000":
0.24 msecs per iteration (total: 63, iterations: 256)
PASS : CustomTest::perform_test(loopCount: 10000)
RESULT : CustomTest::perform_test():"loopCount: 10000":
3.1 msecs per iteration (total: 51, iterations: 16)
PASS : CustomTest::perform_test(loopCount: 100000)
RESULT : CustomTest::perform_test():"loopCount: 100000":
39 msecs per iteration (total: 78, iterations: 2)
PASS : CustomTest::cleanupTestCase()

QBENCHMARK The macro will automatically test the specified procedure of the function . There is one caveat , Because the deviation of single run time of each test is relatively large , In order to prevent deviation of test results ,QBENCHMARK Will automatically specify a number of repetitions , Take the average value of repeated running events . The number of repetitions is related to the time of a single test case run .

GUI test

about QT programmatic GUI test , The main process is to simulate the control corresponding to the mouse and keyboard operation , Then check whether the control state and properties have changed and whether the property state of its member variables has changed . Take the following interface as the test data for testing ：

//mywidget.h
#ifndef MYWIDGET_H
#define MYWIDGET_H

#include <QWidget>
#include <QPushButton>

namespace Ui {
    
class MyWidget;
}

class MyWidget : public QWidget
{
    
    Q_OBJECT

public:
    explicit MyWidget(QWidget *parent = 0);
    ~MyWidget();
    bool eventFilter(QObject* watched, QEvent* event) override;
protected:
    void mousePressEvent(QMouseEvent *event);
private:
    Ui::MyWidget *ui;
};

#endif // MYWIDGET_H

//mywidget.cpp
#include "mywidget.h"
#include "ui_mywidget.h"
#include <QPushButton>
#include <QMouseEvent>

MyWidget::MyWidget(QWidget *parent) :
    QWidget(parent),
    ui(new Ui::MyWidget)
{
    
    ui->setupUi(this);
    installEventFilter(this);

    connect(ui->start_btn,&QPushButton::clicked,this,[&](){
    
        ui->start_btn->setEnabled(false);
        ui->stop_btn->setEnabled(true);
        ui->reset_btn->setEnabled(true);
    });
}

MyWidget::~MyWidget()
{
    
    delete ui;
}

void MyWidget::mousePressEvent(QMouseEvent *event)
{
    
    if(event->button() == Qt::RightButton)
    {
    
        if(!ui->start_btn->isEnabled())
        {
    
            ui->start_btn->setEnabled(true);
        }
    }
}

bool MyWidget::eventFilter(QObject* watched, QEvent* event)
{
    
    if (event->type() == QEvent::KeyPress)
    {
    
        QKeyEvent* keyEvent = static_cast<QKeyEvent*>(event);
        Qt::KeyboardModifiers modifier = keyEvent->modifiers();
        if(keyEvent->key() == Qt::Key_S && modifier == Qt::NoModifier)
        {
    
            ui->start_btn->setEnabled(true);
        }
        else if(keyEvent->key() == Qt::Key_T && modifier == Qt::NoModifier)
        {
    
            ui->stop_btn->setEnabled(true);
        }
        else if(keyEvent->key() == Qt::Key_R && modifier == Qt::NoModifier)
        {
    
            ui->reset_btn->setEnabled(true);
        }
        else if(keyEvent->key() == Qt::Key_P && modifier == Qt::ControlModifier)
        {
    
            ui->reset_btn->setEnabled(false);
            ui->start_btn->setEnabled(false);
            ui->stop_btn->setEnabled(false);
        }
    }
    return QObject::eventFilter(watched, event);
}

The corresponding interface effect is as follows :

The interface mainly contains three buttons start_btn,stop_btn and reset_btn, Press down start_btn Then the status of other buttons can be affected , At the same time, we can change the status of these three buttons through mouse events and keyboard events .

After the test data is set up , We can test the corresponding interface by simulating mouse and keyboard events , The test case implementation is as follows :

//TestGui.h
#ifndef TESTGUI_H
#define TESTGUI_H

#include <QTest>
#include "mywidget.h"

class TestGui : public QObject
{
    
    Q_OBJECT
public:
    TestGui(QObject* parent = nullptr);

private slots:
    // Simulate mouse events for testing 
    void simulateMouseClick();
    
    // Simulate keyboard events to test 
    void simulateKeyPress();
private:
    MyWidget mMainWindow;
};

#endif // TESTGUI_H

//TestGui.cpp
#include "TestGui.h"
#include <QPushButton>
#include <QtTest/QtTest>
#include <qtestmouse.h>

TestGui::TestGui(QObject* parent) :
    QObject(parent),
    mMainWindow()
{
    
    //1 Seconds later, enter the event loop , Prevent the interface from not initializing 
    QTestEventLoop::instance().enterLoop(1);
}

void TestGui::simulateKeyPress()
{
    
    // according to ObjectName Find the corresponding control 
    QPushButton* startButton = mMainWindow.findChild<QPushButton*>("start_btn");
    QPushButton* stopButton = mMainWindow.findChild<QPushButton*>("stop_btn");
    QPushButton* resetButton = mMainWindow.findChild<QPushButton*>("reset_btn");
    startButton->setEnabled(false);
    stopButton->setEnabled(false);
    resetButton->setEnabled(false);

    // Simulate pressing a single key 
    QTest::keyClick(&mMainWindow,Qt::Key_S);
    QCOMPARE(startButton->isEnabled(), true);

    QTest::keyClick(&mMainWindow,Qt::Key_T);
    QCOMPARE(stopButton->isEnabled(), true);

    QTest::keyClick(&mMainWindow,Qt::Key_R);
    QCOMPARE(resetButton->isEnabled(), true);

    // Simulate pressing the key combination 
    QTest::keyClick(&mMainWindow,Qt::Key_P,Qt::ControlModifier);
    QCOMPARE(resetButton->isEnabled(), false);
    QCOMPARE(startButton->isEnabled(), false);
    QCOMPARE(stopButton->isEnabled(), false);
}

void TestGui::simulateMouseClick()
{
    
    // Simulated mouse click 
    QPushButton* startButton = mMainWindow.findChild<QPushButton*>("start_btn");
    QPushButton* stopButton = mMainWindow.findChild<QPushButton*>("stop_btn");
    QPushButton* resetButton = mMainWindow.findChild<QPushButton*>("reset_btn");

    // Click the left mouse button 
    QTest::mouseClick(startButton, Qt::LeftButton);
    QCOMPARE(stopButton->isEnabled(), true);
    QCOMPARE(startButton->isEnabled(), false);
    QCOMPARE(resetButton->isEnabled(), true);

    // Right click 
    QTest::mouseClick(&mMainWindow,Qt::RightButton);
    QCOMPARE(startButton->isEnabled(),true);
    // Middle mouse button 
    QTest::mouseClick(&mMainWindow,Qt::MiddleButton);

    // Simulate mouse double clicking 
    QTest::mouseDClick(&mMainWindow,Qt::LeftButton);
    // Simulated combo Click 
    QTest::mouseClick(&mMainWindow,Qt::LeftButton,Qt::ControlModifier);

}

The output of the test is as follows :

PASS : TestGui::initTestCase()
PASS : TestGui::simulateMouseClick()
PASS : TestGui::simulateKeyPress()
PASS : TestGui::cleanupTestCase()

There are usually multiple test classes in a test project , Many times we don't need to run all the tests , At this point, we can specify the tests to be executed through the command line parameters . At the same time, we can also configure the test framework through command line parameters , So that the test results can be output in different formats , The corresponding implementation is as follows :

#include <map>
#include <QCoreApplication>
#include <QTest>
#include <memory>
#include "customtest.h"
#include "TestGui.h"
using namespace std;
int main(int argc, char *argv[])
{
    
    QApplication app(argc, argv);
    QStringList arguments = QCoreApplication::arguments();

    map<QString, unique_ptr<QObject>> tests;
    // need C++14 Support 
    tests.emplace("customeTest",  std::make_unique<CustomTest>());
    tests.emplace("guiTest",  std::make_unique<TestGui>());

    // Execute the corresponding test according to the command line parameters 
    if (arguments.size() >= 3 && arguments[1] == "-select") {
    
        QString testName = arguments[2];
        auto iter = tests.begin();
        while(iter != tests.end()) {
    
            if (iter->first != testName) {
    
                iter = tests.erase(iter);
            } else {
    
                ++iter;
            }
        }
        arguments.removeOne("-select");
        arguments.removeOne(testName);
    }

    int status = 0;
    for(auto& test : tests) {
    
        status |= QTest::qExec(test.second.get(), arguments);
    }
    return status;
}

By combining unit tests 、 Dataset testing 、 Performance testing and GUI test , Our tests can cover more software usage scenarios , So as to improve the robustness of the system , Strangle many problems in the cradle , Prevent damage to the system .