Deep learning final review

 

2. perceptron

2.1 Structure diagram of perceptron ,

2.2 Gradient descent method of perceptron , Algorithm flow

1. Determine initialization parameters w and b.
2. Build a perceptron model .
3. Using reverse algorithm , Complete the adjustment of weight coefficient .
   notes ： Initialization parameters can be set arbitrarily , Finally, the convergence will be completed according to the reverse algorithm .

 

 

 

 3.2 What are the activation functions , Draw their pictures , And their respective application scenarios and characteristics

 

 

 

 

 

 3.3 Neural network training process , Write out your ideas

 

 

 3.4 Loss function , What are the commonly used loss functions , Form of expression （ The formula ）, Applicable occasions

 

 3.5 In the process of neural network training , Over fitting （ The model is too complex （ Too many parameters included ）, So that the model fits the training set very well , But the prediction of unknown data is very poor （ Poor generalization ability ）.）

 （1） Increase the amount of training data
① Collect more data ;
② Data augmentation (image augmentation)： Make a series of random changes to the existing data , To produce similar but different training samples , So as to expand the scale of the training data set .

（2） Reduce model complexity
① Reduce hidden layer
② Reduce the number of neurons

（3） Add regular items
tf.keras.regularizers.l2
Parameters used ：
l： Penalty item , The default is 0.01.

（4） Early termination （Early Stopping） In the process of training , Record the best validation set results to date , As the continuous 10 individual Epoch（ Or more times ） When the best results are not exceeded , It can be considered that the accuracy is no longer improved .

（5）Dropout

Training phase ： Every step With a certain probability p Stop the neurons .
Prediction stage ： All neurons work , But the ownership value needs to be multiplied by (1-p).

4. Deep learning framework PyTorch

 

 

5. Convolutional neural networks

5.1 Give you a convolutional neural network code , You can draw a structure diagram

 

 

 

 

 

 

5.2 What are the commonly used convolutional neural networks , And the course of development

 

6. Cyclic neural network

Draw a graph of a recurrent neural network , Here you are , You can explain how it works

Text classification , You need to know the basic principle