Sikuli automatic testing technology based on pattern recognition

As a tester , During the test process, if the application interface structure is huge , Page design changes frequently , It is difficult to locate page elements , It also requires a lot of repeated tests , What quick solutions do we have ？ Maybe Sikuli Can be one of your choices , It allows you to get rid of the controls API Dependence , By retrieving the image of the current screen in real time , Get actionable object , Simulate user behavior , Verify the real screen display results .

What is? Sikuli？

Sikuli automated testing , Is the content displayed on the screen , Through image recognition , Used to locate the element , And operate GUI Components , Finally, you can judge whether the operation is successful by identifying the content in the picture . This allows testers to use screenshots instead of writing code , So as to reduce the test cost , Simplify the process of writing test scripts .

Sikuli It's in the Mexican vicho Indian language ” The eye of God ” It means , The goal is to make computers look at the real world like people , And interact with it .

Sikuli The developer of the

Sikuli Is an open source research project of the original user interface design organization at the Massachusetts Institute of technology , It is now time to maintain and further coordinate development with the open source community Sikuli The laboratory is located at the University of Boulder, Colorado . This is supportive , Part of it is awarded by the National Science Foundation IIS-0447800, Part of quanta computer TParty project .Sikuli Of MIT Issued under license .

Download and install

edition 1.1.3 The following official download page is

https://launchpad.net/sikuli/sikulix

The new version of the 1.1.4 Above in

https://raiman.github.io/SikuliX1/downloads.html

If you use 1.1.3 Under version , Double click directly in the download file to open sikulix.jar The document is sikuli Provide IDE, It can be written in a drag and drop mode .

As shown below ：

893×1073 103 KB

This way of running depends on IDE, It is not convenient to use script startup to execute tests , So there are two other ways , Direct writing python Script to execute .

01

pure python Environmental Science -Lackey library

Lackey yes Python An automation library for , Designed for ease of use and Sikuli Cross compatibility of automated scripts . It provides a simple but powerful API, Used to find pictures on the screen , And basic user input using the mouse and keyboard .

√

Lackey Dependency Library

①

numpy

②

pillow

③

opencv

④

keyboard

Installation command ：pip install Lackey

Import method in script ：from lackey import *

The library supports screenshots of .bmp, .pbm, .ras, .jpg, .tiff, and .png

find("***.png"): Look for the element , There is no throw exception

exists("***.png"): Look for the element , return true or false

wait("***.png",5): Wait for the specified number of seconds , Look for the element

click("***.png"): Click on the element

dubleClick("***.png"): Double click on the element

input_(“text”): Input text

……

Implemented a similar monkey An example script for the tool is as follows ( Put all the screenshots in the project btn_pictrues Under the directory )：

02

Write python Script call sikulixapi.jar Medium API

** ** Prepare the environment

1

install jython: pip install JPype1

2

install VCForPython27

3

install JDK, And configuration %JAVA_HOME%

Possible pit

call getDefaultJVMPath, Can't find JAVA_HOME; Or start java Virtual opportunity crash . It may be due to java and python The number of digits in the installation version is different , Both are 32 Bits or 64 Bit software can run normally .

notes ： Use java -d32 perhaps -d64 see 32 still 64 position ; Input... In the terminal python, As shown in the following figure 32 still 64 position .

summary

Sikuli The advantage of is simple and easy to use , Support for multiple programming languages （python/java）, But the disadvantages are obvious . The resolution, color and size of the picture have a great impact on the results of the program , A successful script executed on one device may not succeed on the other device , Need to re screenshot . And can only detect the content displayed on the current desktop , The background process cannot operate ,pc All user operations on the end are not fully supported ,sikuli Itself is not perfect , It is still in the development and upgrading stage . So use sikuli Complete complex large-scale automated tests , Not the best choice yet , However, its small and beautiful features can be applied to many test scenarios

Get more related information stamps ：https://qrcode.ceba.ceshiren.com/link?name=article&project_id=qrcode&from=jianshu&timestamp=1656224665&author=wuyue