GNN (graph neural network) introduction vernacular

function ： Node classification And graph classification

airspace ： Space A model considering graph structure , That is, consider the geometric relationship between the target node and other nodes ( Is there a connection ).

The model represents ：GAT（Graph Attention Networks） Figure attention model

use Attention mechanism Weighted sum of adjacent node features . The weight of adjacent node features depends entirely on the node features , Independent of graph structure .

（ take Convolutional neural networks Pooling in is regarded as a special average weighted Attention mechanism , In other words, attention mechanism is a general pooling method with preference for input allocation ( Pooling method with parameters ）

chart 1： Figure attention network diagram and update formula

Some explanations of the above formula ：

• The formula （1） Yes l Layer node embedding

We did a linear transformation ,W^((l)) Is the trainable parameter of the transformation

• The formula （2） The original attention scores between paired nodes are calculated . It first splices the two nodes z The embedded , Be careful || It means splicing here ; Then we embed the spliced weight vector and a learnable weight vector Make some product ; Finally, a LeakyReLU Activation function . This form of attention mechanism is often referred to as additive attention , The difference in Transformer Focus on the points in the .

• The formula （3） For the original attention score obtained by all the edges of a node, a softmax operation , Got the attention weight .

• The formula （4） Shaped like a GCN Node feature update rule of , The weighted sum based on attention is made for the features of all adjacent nodes .

frequency domain ：

The model represents ：GCN（Graph Convolutional Network ） Figure convolution network

advantage ： Provincial parameters

shortcoming ： Not easy to work on dynamic graphs

( For neighborhoods of the same order, the weights assigned to different neighbors are exactly the same （ It is not allowed to assign different weights to different nodes in the neighborhood )

A graph convolution operation Contains the Normalized sum of adjacent node features ：

among N(i) Is for the node i A distance of 1 A collection of adjacent nodes . We usually add a connection node i And its own edges make i Itself is also included in N(i) in .

Is a normalized constant based on graph structure ;

σ It's an activation function （GCN Used ReLU）;

W^((l)) Is the weight matrix of node feature transformation , Shared by all nodes .

because c_ij Related to the mechanism of the figure , Make what you learn from a picture GCN It is difficult to directly apply the model to another diagram .

 

Common steps ：

1. machining Graph adjacency matrix
2. Yes Graph adjacency matrix Characteristics of decomposition , Get the eigenvalue ,
3. The core difference between （ how Collect and accumulate a distance of 1 Characteristic representation of neighbor nodes of ）
4.   Treat the eigenvector as a constant , And the convolution kernel acts on the eigenvalues

GAT The fixed standardized operation in graph convolution is replaced by attention mechanism , Will be the original Normalized constant Replace with Neighborhood node feature aggregation function using attention weight .

Long attention (Multi-head attention)

It resembles the multi-channel in convolutional neural network ,GAT Multi head attention is introduced to enrich the ability of the model and stabilize the training process . Every attention head has its own parameters . There are generally two ways to integrate the output of multiple attention mechanisms ：

In the above formula K It's the number of attention heads . The authors suggest using stitching for the middle layer and averaging for the last layer .