Tensor and tensor network background and significance - basic knowledge

tensor （Tensor） It can be understood as a generalized matrix , Its main feature is that the digital matrix is represented in a graphical way , This allows us to abstract a large matrix operation into a tensor graph with good properties . The operational network graph composed of tensors , It's called tensor network （Tensor Network）. Let's use some common graphs to see what tensor networks look like （ The picture below is reproduced from Reference link 1）：

The graph above shows from left to right ： First order tensor 、 Second order tensors and third order tensors , We can see that , The order of a tensor is abstractly called the order of tensor in the image representation leg The number of , The square or circle in the middle represents the tensor itself . actually , A vector represented by a tensor of first order , For example, we usually use python An array variable defined by ：

1
2
3
x = [1, 0]
y = [0, 1, 0]
z = [1, 2, 3, 4]

So here x,y,z They're all first-order tensors . The second-order tensor is a two-dimensional matrix , As we often see python Multidimensional arrays ：

1
2
M = [[1, -1], [-1, 1]]
N = [[1, 3], [2, 4], [5, 6]]

What is defined here M, N They're all second order tensors . By looking at the first and second order tensors in these examples, we can get a rule ： Can be used in the form of var[i] Read and traverse in the form of var The scalar elements in can be called first-order tensors , Can be used in the form of var[i][j] Read and traverse in the form of var The scalar elements in can be called second order tensors . obviously , A tensor of several orders , It has nothing to do with the number of elements contained in the tensor . Then according to this objective law , We can extend it to zero order tensors and higher order tensors ：

1
2
3
pi = 3.14
P = [[[1]]]
Q = [[[1, 1, 1], [1, 1, 1], [1, 1, 1]]]

In the above python Variable definition ,pi It's a zero order tensor , A tensor of order zero is actually equivalent to a scalar , and P, Q They're all third-order tensors . It should be noted that , Although tensor P There's only one element , But if we need to read this scalar element , We have to use the following python Instruction to execute ：

print (P[0][0][0])

therefore P It's also one with Three legs Tensor . While using tensor form to represent a single matrix , We need to consider if there are multiple matrix multiplication operations , How should we express ？ Let's start with two forms of python Matrix operation to illustrate the expression of tensor calculation ：

1
2
3
4
5
6
7
import numpy as np
M = np.random.rand(2, 2)
v = np.random.rand(2)
w = np.dot(M, v)
print (M)
print (v)
print (w)

This string python The calculation process of code representation is ：w2×1=M2×2⋅v2×1w2×1=M2×2⋅v2×1, In order not to lose widespread effectiveness , Here we use random tensors to calculate , there M It's a second-order tensor ,v,w It's a tensor of first order . If we understand it from the perspective of matrix operation , It's actually a 2×22×2 Multiply by a matrix of 2×12×1 Vector of , And got a new 2×12×1 Vector of . The results are as follows ：

1
2
3
4
[[0.09660039 0.55849787]
 [0.93007524 0.32329559]]
[0.74966152 0.59573188]
[0.40513259 0.88983912]

At the same time, we also consider another tensor operation scenario , One tensor of higher order operates with another tensor of higher order ：

1
2
3
4
5
6
7
import numpy as np
A = np.random.rand(1, 2, 2, 2)
B = np.random.rand(2, 2, 2)
C = np.einsum('ijkl,klm->ijm', A, B)
print ('A:', A)
print ('B:', B)
print ('C:', C)

This string python The calculation process of code representation is ：C1×2×2=A1×2×2×2⋅B2×2×2C1×2×2=A1×2×2×2⋅B2×2×2, Because the multi-dimensional tensor operation here can no longer use the ordinary numpy.dot To deal with it , So we still apply the professional tensor calculation function numpy.einsum To process , The calculation results are as follows ：

1
2
3
4
5
6
7
8
9
10
11
12
A: [[[[0.85939221 0.43684494]
   [0.71895754 0.31222944]]

  [[0.49276976 0.13639093]
   [0.04176578 0.14400289]]]]
B: [[[0.21157005 0.58052674]
  [0.59166167 0.21243451]]

 [[0.11420572 0.98995349]
  [0.1145634  0.97101076]]]
C: [[[0.5581652  1.60661377]
  [0.20621996 0.49621469]]]

The above two cases , From the perspective of tensor theory , It's equivalent to the tensor w And tensor C It is expressed as the result of several tensor combination operations . A combinatorial operation consisting of multiple tensors , We can use tensor networks to represent ：

The picture above shows (a)(a) and (b)(b) I mean tensors w And tensor C Tensor network diagram of . And this computes all the tensors of the tensor network , Finally, we get one or a series of new tensor matrix multiplication and addition processes , We call it Tensors are reduced and merged , English name Tensor Contraction