Classification model - logistic regression, Fisher linear discriminant (SPSS)

This article draws on mathematical modeling Teacher Qingfeng's courseware and ideas , You can click the view link to view the video explanation of teacher Qingfeng ： Qingfeng mathematical modeling ：https://www.bilibili.com/video/BV1DW411s7wi

One 、 Dichotomous problem

1.0 Case study

1.0.1 Case background

 1.0.2 Case data

1.1 Logical regression

With the help of SPPSS The steps to realize logical regression are as follows ：

Import data into SPSS, Here's the picture ：

  Then create a dummy variable for the dependent variable type （01 Variable ）, Here's the picture ：

  After creation, a new 3 Column , Framed for their differences , Here's the picture ：

Now delete the two unnecessary columns . Because we want to divide apples and oranges into 0 and 1, And in these three columns , All apples in the second column are 1, All oranges and samples to be predicted are 0, So delete the first and third columns here , And put the... Of the sample to be predicted 0 Delete . The effect after deletion is as follows ：

  Then perform the following operations ：

When there are classified indicators in the indicators （ Such as gender ） when , Put this variable into the classification covariate ：

  Now we can get the result ：

  According to the table of logistic regression coefficients, the calculation formula of the model can be written ：

 

When the model results are poor ：

  After adding the square term, the result is as follows ：

  You can see , At this time, the prediction accuracy is 100%, May have produced Over fitting （ The prediction effect on the training set is good , The effect of prediction set is poor ） Here's the picture ：

The solution to over fitting ：

1.2Fisher Linear discrimination

The previous operation is consistent with logistic regression , First, create virtual variables for classification indicators （01）.

And then with the help of SPSS Realization Fisher Linear discriminant classification , among , The definition range is the category of classification indicators , In this case , There are two kinds , So it's 0-1, Here's the picture ：

  The classification results are as follows , The coefficient of the canonical discriminant function can be put into the paper , Here's the picture ：

Two 、 Multiple classification problem

2.0 Case data

2.1 Logical regression

stay Excel Create virtual variables for classification indicators 1-4, After creation, see the following figure ：

  Then do the following ：

 

  The following results can be obtained ：

 

2.2Fisher Linear discrimination

stay Excel Create virtual variables for classification indicators 1-4, After creation, see the following figure ：

Fisher Linear discriminant realizes the multi classification problem, which is similar to the two classification , Just modify the scope of the definition （ The categories classified here are 4 class , So the scope is 1-4）, Here are two pictures ：

 

  The result of the classification , In the classification result table , The above is the number of categories , The following is the probability of classification , Here's the picture ：

The following figure shows the results ,  The first one in the box is the classification result , The second in the box is 4 The probability of two kinds , For example, in the first sample of prediction , It is predicted to be No 3 The probability of a class is 0.87121, Therefore, this sample is predicted as the third kind .