[cann document express issue 04] unveiling the development of shengteng cann operator

The developer of shengteng is in the process of neural network model training or reasoning , You will often encounter the following scenarios ：

• Transform the third-party open source framework network model into an adaptive shengteng AI The model of the processor , Yes CANN Operator not yet supported .
• The performance of existing operators cannot meet the requirements of developers .
• Developers want to modify the calculation logic of existing operators .

At this point, we need to consider the development of user-defined operators , In this issue, we mainly show you CANN Type of custom operator , compile 、 Run logical architecture , And the development process of operators , Let you be right CANN Operators have a macroscopic understanding .

CANN Custom operator type

CANN There are two types of custom operators ：TBE Operators and AI CPU operator , The difference between the two operators is as follows .

among AI CPU The implementation of the operator is relatively simple ,TBE The implementation of the operator is complex , But the performance is better . If the user wants to quickly get through the model execution process , You can choose to customize AI CPU operator , So as to improve the commissioning efficiency . After function adjustment , In the subsequent performance testing process, we will AI CPU Operator switches to TBE Operator implementation .

Operator compilation

Let's take a look at CANN The compilation process of the operator , As shown in the figure below ：

1. The third-party framework network model goes through Parser After the parsing , Transitioning to an intermediate state IR (Intermediate Representation) Graph.
2. GE receive IR Graph Then prepare the diagram 、 Split 、 Subgraph optimization and other operations .
3. In the process of subgraph optimization , Can match and select operators , Priority by FE be based on TBE The operator information base judges the operator support , if TBE I won't support it , By AI CPU Engine be based on AI CPU The operator information base determines whether it supports .
4. GE Merge the split subgraphs 、 Optimize , And compile , Operators will be generated during compilation Task Information , Finally, executable om Model .

Operator operation

First GE Issue an operator execution request to Runtime, then Runtime Can judge operators Task type , if TBE operator , Then send the operator execution request to AI Core On the implementation ; if AI CPU operator , Then send the operator execution request to AI CPU On the implementation .

Operator development process

Light purple shading for operator development deliverables , The deliverables are described below ：

•   Operator prototype ： The operator is external API, Define operator input 、 Output 、 Properties and operators shape Derivation logic .
•   Operator implementation code ： Describe the operational logic of an operator . in the light of TBE operator , by python file ; in the light of AI CPU operator , by C++ file .
•   Operator information base ： The description operator is rising AI Implementation limitations on processors , Contains the input and output of the operator data type、format as well as shape Information .
•   Operator adapter plug-in ： The operators of the third-party framework are mapped to adaptive lifting AI Processor operators .

Operator project creation method

The developer can choose the operator development method by himself , And develop the deliverables under the corresponding directory based on the above rules .

More about

Learn more about , You can log in to shengteng community https://www.hiascend.com/, Read the relevant documents ：

Rise CANN The document center is dedicated to providing developers with better content and more convenient development experience , help CANN Developers co build AI ecology . Any comments and suggestions can be fed back in shengteng community , Your every attention is our driving force .