Chapter 2 Regression

Linear regression

Gaussian distribution

Gaussian distribution is the most common kind of random distribution , Also known as normal distribution .
If random variable X Obey a mathematical expectation of μ、 The variance of σ² Is a normal distribution , Write it down as N(μ,σ²). The probability density function is the expected value of the normal distribution μ It determines its position , The standard deviation σ It determines the magnitude of the distribution . When μ = 0,σ = 1 The normal distribution of time is the standard normal distribution .

Maximum likelihood estimation MLE

Maximum likelihood estimation is a statistical method , It is used to find the parameters of the correlation probability density function of a sample set .

Least square method

Use maximum likelihood estimation to explain least squares .

• Machine learning modeling is full of assumptions , The assumption also has the following three properties
  （1） Connotative ： Assumptions are often correct , But not all of them are right .
  （2） Simplicity ： The assumption is only close to reality , Often need to do some simplification .
  （3） Divergence ： A conclusion derived under a simplified assumption , The conclusion is not necessarily true only when the hypothesis is true .

θ The analytical solution process of

The parameter optimal solution in the sense of least square

Join in λ After disturbance

If irreversible , You need to add disturbance λ.

The complexity penalty factor of linear regression

The regularization

Why add a regularization ： To prevent over fitting

Logistic Return to

The preferred algorithm for classification problems

• Classification problem is a supervised learning algorithm for modeling or predicting discrete random variables . The preferred algorithm is Logistic Return to , Solve the dichotomy problem .
• Regression problem is a supervised learning algorithm for modeling or predicting continuous random variables .

Gradient descent algorithm

• Definition of gradient descent method

• The direction of the gradient
  
• Classification of gradient descent method
  （1） Batch gradient descent algorithm
  
  （2） Stochastic gradient descent algorithm
  
  ps: actually , It is better to choose random gradient descent algorithm . Random gradient decline. Generally speaking, the loss value has a downward trend .
  （3）mini-batch Gradient descent algorithm
  The average gradient of several samples is used as the update direction .