[notes of Wu Enda] convolutional neural network

Convolutional neural networks

Computer vision

Computer vision problems such as image recognition , Image fusion , Object detection, etc

cat Pictures of the 64 * 64 * 3

• front 64 Represents the height and width of pixels ,3 representative RGB3 layer

Such pictures are small , If for 1000 * 1000 * 3, Using neural networks

• The input of the first layer , an X_1,X_2…X_N, Altogether 1000 * 1000 * 3 individual x
• The output of the first layer ,1000 Nodes
• The parameters of the first layer
  • w (1000,3m)
  • b (1000,1)

boundary detection

The larger the number, the greater the brightness , The smaller the number, the darker

• Input is 6 * 6 Matrix
• filter 3 * 3 perhaps kernel
• Output 4 * 4

More boundary detection

There are different values for filters , Perform boundary detection differently

Padding

Boundary expansion , The input matrix is filtered by a filter , The matrix dimension of the output is reduced , To keep the dimensions of the matrix constant , We can fill the original matrix boundary with numerical values .

• Input matrix : n * n eg: 6 * 6
• Fill in the boundary : pading P=1
• filter : f * f eg: 3 *3
• The output matrix : ( n+2P-f+1 ) * ( n+2P-f+1 )
• The output matrix : 6+2-3+1 =6

Two convolution methods

• valid: No filling
• Same:Pad so that output size is the same as the input size.

Convolution step

Strided Convolutions

Picture dimension : n × n

Filter dimension : f × f

fill Padding ： p

step stride : s

Three dimensional convolution

(Convolutions overvolumes)

Think of the 3D filter as a small square

Then move on the three-dimensional matrix Multiply and add with the corresponding plane

here 27 Multiplication of two Add it all up

Single layer convolution network

One layer of a convolutional network

• Input a_{0} by 6 × 6 × 3
• The filter is equivalent to a parameter matrix w_{1}
• z_{1}=w_{1}a_{0}+b_{1}
• a_{1}=g(z_{1}) Use nonlinear functions Relu
• g Represents nonlinear transformation
• Two filters So finally 4 × 4 × 2

If you have 10 filters that are 3 x 3 x 3 in one layer of a neural network, how many parameters does that layer have?

If you have a layer in a neural network 10 individual 3x3x3 Filter , How many parameters does this layer have ？

3 × 3 × 3 ×10+bias( 10 )=280 parameters

Simple convolution network case

A simple convolution network example

first floor Output results

The second floor Output results
The third level Output results

7 × 7 × 40=1960 Features

After logical regression , Output the last y value

The type of one layer in convolutional networks :

• Convolution CONV Convolution
• Pooling POOL Pooling
• Fully connected FC Full connection

Pooling layer

Pooling layer:Max pooling

here f=2 s=2

Move the maximum value in the marquee

Hyperparameters:

• f : filter size
• s : stride
• Max or average pooling

Examples of convolutional neural networks

Convolutional neural network example

Be careful : here CONV + POOL Count the first floor , Because pooling does not require parameters

Handwritten digital pictures → layer_1(Conv1 PooL1) →layer_2(CONV2 POOL2) → Fc3 →FC4→softmax →10 outputs

Parameters required for each layer

Maybe not : layer_1(Conv1 PooL1)：5 × 5 × 3 × 6 +bias （6）

…

Fully connected layer FC3: input 400 output 120

• Every input Participate in all output Generation
• Parameters :1 * 120 * 400+bias( 1 )

Why convolutions?

Too many parameters are required for direct full connection